Sample for using SQL and CSV files

Author: luisbelloch

README.md

@@ -57,6 +57,7 @@ To be added soon, stay tuned!
     - [compras_top_ten_countries.py](spark/compras_top_ten_countries.py)
     - [helpers.py](spark/helpers.py) - basic parse functions to get started quickly
 - Spark SQL
+    - [compras_sql.py](spark/compras_sql.py)
     - [container.py](spark/container.py)
     - [container_convertir_a_parquet.py](spark/container_convertir_a_parquet.py)
     - [container_rdd_to_dataset.py](spark/container_rdd_to_dataset.py)

spark/compras_sql.py

@@ -0,0 +1,12 @@
+from os import walk
+
+from pyspark.sql import SparkSession
+
+spark = SparkSession.builder.master("local").appName("SQL").getOrCreate()
+
+df = spark.read.option("delimiter", "|").option("header", "true").csv('./data/compras_tiny.csv')
+df.printSchema()
+df.show()
+
+df.createOrReplaceTempView("compras")
+spark.sql("SELECT tx_id, SUM(item_price) as tx_total FROM compras GROUP BY tx_id").show()