---
title: "Interpreter SparkR for Apache Zeppelin"
canonical: "https://onesaitplatform.refined.site/space/DOCT/4982702092/Interpreter%20SparkR%20for%20Apache%20Zeppelin"
format: markdown
---
> Macro (toc)

# Introduction

**SparkR** is an R package that provides a lightweight front end for using Apache Spark from within R. SparkR provides an implementation of distributed data frameworks that supports operations such as selection, filtering, aggregation, etc. (similar to R's data frameworks, dplyr) but on large datasets. SparkR also supports distributed machine learning via MLlib.

[https://spark.apache.org/docs/latest/sparkr.html](https://spark.apache.org/docs/latest/sparkr.html)

Onesait Platform Notebooks include the **SparkR interpreter** by default, so it is not necessary to install anything beforehand, just include the `%spark.r` header.

Below are some examples of how to use SparkR to calculate and represent results.

# Examples of use

## Hello World

The first example consists of displaying a few variables of different types in the results window. To do this, the following code will be written in a paragraph:

```
%spark.r

foo <- TRUE
print(foo)
bare <- c(1, 2.5, 4)
print(bare)
double <- 15.0
print(double)
```

When the paragraph is executed, the result will be displayed.

![image-20250228-092259.png](media://5a735865-503d-49c5-b455-9103961b5927)

## Loading R Libraries

It is possible to load R elements as libraries, which allows you to extend the functionality of the interpreter. For example, it is possible to load the table library to display data in tabular form:

```
%spark.r

library(data.table)
dt <- data.table(Cabecera = 1:3)
print(dt)
for (i in 1:5) {
  print(i*2)
}
print(1:50)
```

In this case, the following will be displayed when executing the paragraph:

![image-20250228-093031.png](media://a673ffa4-d0f0-44bf-abe2-a31cf84ab2f6)

## Loading and interacting with Datasets

It is possible to load datasets in order to interact with them. We will take for example the ‘iris’ dataset, a dataset that comes integrated with R, and that contains information about iris flower species.

To display the data header of the dataset, the following code will be executed:

```
%spark.r

colnames(iris)
iris$Petal.Length
iris$Sepal.Length
```

After executing the above paragraph, the indicated information of the set will be displayed:

![image-20250228-093245.png](media://3c94d498-a897-4fbf-9a27-8bf33f6b1684)

## Creating a Spark DataFrame

As with other libraries, it is possible to create a DataFrame in which to perform complex operations. In the following example we are going to read information from a CSV, process the original data and then display it.

```
%spark

import org.apache.commons.io.IOUtils
import java.net.URL
import java.nio.charset.Charset

val bankText = sc.parallelize(
    IOUtils.toString(
        new URL("https://raw.githubusercontent.com/apache/zeppelin/master/testing/resources/bank.csv"),
        Charset.forName("utf8")).split("\n"))

case class Bank(age: Integer, job: String, marital: String, education: String, balance: Integer)

val bank = bankText.map(s => s.split(";")).filter(s => s(0) != "\"age\"").map(
    s => Bank(s(0).toInt, 
            s(1).replaceAll("\"", ""),
            s(2).replaceAll("\"", ""),
            s(3).replaceAll("\"", ""),
            s(5).replaceAll("\"", "").toInt
        )
).toDF()
bank.registerTempTable("bank")
```

> ⚠️ In this case `%spark` and not `%spark.r` is used as the interpreter.

## Read Spark DataFrame from R

In this example we are going to read the previously created DataFrame. To do so, this paragraph will be generated:

```
%spark.r

df <- sql("select count(*) from bank")
printSchema(df)
SparkR::head(df)
```

The result of executing the paragraph will be as follows:

![image-20250228-101854.png](media://042ace38-0c68-410c-8f21-af6c66314cbf)

## Creating an R DataFrame

It is also possible to create an R DataFrame. This will be done as follows:

```
%spark.r 

localNames <- data.frame(name=c("John", "Smith", "Sarah"), budget=c(19, 53, 18))
names <- createDataFrame(localNames)
printSchema(names)
registerTempTable(names, "names")
```

The result of executing this paragraph will show the schema of the DataFrame:

![image-20250228-102615.png](media://b33e6fd1-c700-4d95-bcb8-09c02b432cc3)

## Read R DataFrame from Spark (Scala)

It is possible to access and read this DataFrame from Spark. To do so, this paragraph will be created:

```
%spark

sqlContext.sql("select * from names").head
```

After executing it, the following will be displayed:

![image-20250228-102817.png](media://12cc0ff2-b69d-4df2-8f57-bc973e4ed9ed)

## Querying the R DataFrame with SQL

It is also possible to analyse the DataFrame using SQL, which will allow you to visualise the results:

```
%spark.sql

select * from names
```

The result will look like this:

![image-20250228-103222.png](media://e1bbc9d1-25f6-4663-9f29-c6584a203048)

> ℹ️ Depending on the display method last chosen, it will be displayed in one form or another: table, bar chart, pie chart, etc. This can be selected in the representation type bar:
> ℹ️ 
> ℹ️ ![image-20250228-103408.png](media://6ed03979-b52b-4b73-a98f-e398a8c9cb63)

## Pair chart

To analyse the distribution of data, a pair chart can be drawn in a simple way. Thus, taking the data of the ‘iris’ set as a starting point:

```
%spark.r

pairs(iris)
```

The result would be:

![image-20250228-103633.png](media://e7c81ca8-1c8b-469a-9afb-67f7838a8caf)

It is possible to represent this graph with colour ranges by slightly modifying the code. Thus, for a range of three colours it would be specified as follows:

```
%spark.r

plot(iris, col = heat.colors(3))
```

The result would look like this:

![image-20250228-103802.png](media://edf48322-b838-45a6-8c62-f86c06c84e45)

## Heat map

Another typical graphic that can be represented is the heat map. It can be generated using the following code:

```
%spark.r

library(ggplot2)
pres_rating <- data.frame(
  rating = as.numeric(presidents),
  year = as.numeric(floor(time(presidents))),
  quarter = as.numeric(cycle(presidents))
)
p <- ggplot(pres_rating, aes(x=year, y=quarter, fill=rating))
p + geom_raster()
```

![image-20250228-104540.png](media://656e5bbf-b7c7-4cc4-bfbe-07b2577e9a23)

## Visualisation of Graphs

There are also different ways to represent data in graphical form. One of them is using the **Google Charts API** (googleVis).

[https://cran.r-project.org/web/packages/googleVis/vignettes/googleVis_intro.html](https://cran.r-project.org/web/packages/googleVis/vignettes/googleVis_intro.html)

Examples include:

### Bar Chart

Defining a DataFrame with the values of coordinates and abscissae, as shown in the following code:

```
%spark.r

library(googleVis)
df=data.frame(country=c("US", "GB", "BR"), 
              val1=c(10,13,14), 
              val2=c(23,12,32))
Bar <- gvisBarChart(df)
print(Bar, tag = 'chart')
```

The resulting graph will look like the following:

![image-20250228-104811.png](media://c49d2546-4e70-4492-98d5-a1276e28e5de)

### Candle Diagrams

This type of diagram can be generated with the following code:

```
%spark.r

library(googleVis)

Candle <- gvisCandlestickChart(OpenClose, 
  options=list(legend='none'))

print(Candle, tag = 'chart')
```

The result of executing the paragraph would be as follows:

![image-20250228-104938.png](media://e007e5e0-8479-4ab0-b296-736a935cc214)

### Line Chart

In a very similar way to the bar chart, a line chart can be generated with the following code:

```
%spark.r

library(googleVis)
df=data.frame(country=c("US", "GB", "BR"), 
  val1=c(10,13,14), 
  val2=c(23,12,32))

Line <- gvisLineChart(df)

print(Line, tag = 'chart')
```

After running it, it will look like this:

![image-20250228-105056.png](media://1fdbae5f-e7cb-404f-8fba-c8aedd08b24f)

### Bubble Diagram

The information can also be visualised in the form of a bubble diagram. Taking the ‘fruits’ dataset, the diagram would be generated using the following code:

```
%spark.r

library(googleVis)
bubble <- gvisBubbleChart(Fruits, idvar="Fruit", 
  xvar="Sales", yvar="Expenses",
  colorvar="Year", sizevar="Profit",
  options=list(
  hAxis='{minValue:75, maxValue:125}'))
print(bubble, tag = 'chart')
```

The result would be the following graph:

![image-20250228-105214.png](media://64654ba0-85a7-4ce6-9a5a-73d89fba5ce2)

### Maps

Last but not least, it is possible to display projected maps. To do so, the following code must be defined:

```
%spark.r

library(googleVis)
geo = gvisGeoChart(Exports, locationvar = "Country", colorvar="Profit", options=list(Projection = "kavrayskiy-vii"))
print(geo, tag = 'chart')
```

In the example shown, a map will then be displayed with the countries coloured according to the ‘exports’ dataset:

![image-20250228-105332.png](media://5c748cdb-e4fc-400a-86ee-4d2b05103e66)

# Conclusions

As can be seen, it is possible to interact with data in different ways using SparkR Notebooks. In the case of more advanced representations, using graphs and diagrams, the different APIs, such as the Google one used in this case, are useful and very practical.

This tutorial has shown basic representations as an example, so it is recommended to read the documentation of both SparkR and the APIs that you want to use in order to configure the available options in more detail.

# Download example

The example used in this tutorial is available for download below:







a