SparkSession object has no attribute sqlContext
This error occurs when you try to access the sqlContext attribute on a SparkSession object. In Spark 2.x, the sqlContext is no longer a separate object, and instead, you can access the SQL functionality directly through the SparkSession object.
To use SQL in Spark 2.x, you can use the following code:
from pyspark.sql import SparkSession
spark = SparkSession.builder.appName("myApp").getOrCreate()
# create a DataFrame from a CSV file
df = spark.read.csv("path/to/file.csv", header=True)
# perform SQL operations on the DataFrame
df.createOrReplaceTempView("my_table")
result = spark.sql("SELECT * FROM my_table WHERE column1 > 10")
In this example, we create a SparkSession object and use it to read a CSV file into a DataFrame. We then create a temporary view of the DataFrame and perform a SQL query on it using the spark.sql() method.
Note that you should use the SparkSession object instead of the sqlContext object, which has been deprecated in Spark 2.x
原文地址: http://www.cveoy.top/t/topic/hvRI 著作权归作者所有。请勿转载和采集!