Spark SQL

In this tutorial, you will learn

 

1.  How to read a JSON file in Spark?

2.  How to create a local temporary view in Spark SQL?

3.  How to create a global temporary view in Spark SQL?

4.  How to join views in Spark SQL?

1. Read the data in Spark

df=spark.read.json("data2.json")
df.show()
Dataset for spark dataframe

2. Creating a Temporary view in Spark SQL

df.createOrReplaceTempView('temp_view_example')
temp_view_example=spark.sql("select * from temp_view_example where age=25")
temp_view_example.show()
How to create a temporary view in spark dataframe?

3. Creating a Global Temporary View in Spark SQL                                                  

df.createGlobalTempView("global_temp_view_example")
global_temp_view_example=spark.sql("select * from temp_view_example where age<25")
global_temp_view_example.show()
How to create a global temporary view in spark dataframe?

4. Joining views in Spark SQL                                                  

join_table_example=spark.sql("select * from temp_view_example \
inner join temp_view_example2 on temp_view_example.age=temp_view_example2.age")
join_table_example.show()

Summary

 

1.  spark.read.json( ) : To read a JSON file in Spark.

2.  createOrReplaceTempView( ) : To create a local temporary view in Spark SQL.

3.  createGlobalTempView( )To create a global temporary view in Spark SQL.

4.  inner join : To join views in Spark SQL.

 You can find the Github link here.