This error occurs when you try to access the sqlContext attribute on a SparkSession object. In Spark 2.x, the sqlContext is no longer a separate object, and instead, you can access the SQL functionality directly through the SparkSession object.

To use SQL in Spark 2.x, you can use the following code:

from pyspark.sql import SparkSession

spark = SparkSession.builder.appName("myApp").getOrCreate()

# create a DataFrame from a CSV file
df = spark.read.csv("path/to/file.csv", header=True)

# perform SQL operations on the DataFrame
df.createOrReplaceTempView("my_table")
result = spark.sql("SELECT * FROM my_table WHERE column1 > 10")

In this example, we create a SparkSession object and use it to read a CSV file into a DataFrame. We then create a temporary view of the DataFrame and perform a SQL query on it using the spark.sql() method.

Note that you should use the SparkSession object instead of the sqlContext object, which has been deprecated in Spark 2.x

SparkSession object has no attribute sqlContext

原文地址: http://www.cveoy.top/t/topic/hvRI 著作权归作者所有。请勿转载和采集!

免费AI点我,无需注册和登录