出5道关于Spark RDD相关的几道程序设计题并给出python答案

统计一个RDD中每个元素出现的次数，并按照出现次数从大到小排序输出。

from pyspark import SparkContext

sc = SparkContext("local", "RDD Count")

rdd = sc.parallelize([1, 2, 3, 4, 5, 1, 2, 3, 1, 2, 1])

counts = rdd.countByValue()

sorted_counts = sorted(counts.items(), key=lambda x: x[1], reverse=True)

for item in sorted_counts:
    print(item[0], item[1])

统计一个RDD中所有元素的平均值。

from pyspark import SparkContext

sc = SparkContext("local", "RDD Average")

rdd = sc.parallelize([1, 2, 3, 4, 5])

sum = rdd.sum()
count = rdd.count()

average = sum / count

print(average)

将一个RDD中的所有元素转换成大写字母并输出。

from pyspark import SparkContext

sc = SparkContext("local", "RDD Uppercase")

rdd = sc.parallelize(["hello", "world", "spark", "rdd"])

uppercase_rdd = rdd.map(lambda x: x.upper())

for item in uppercase_rdd.collect():
    print(item)

将一个RDD中的元素按照长度从小到大排序输出。

from pyspark import SparkContext

sc = SparkContext("local", "RDD Sort")

rdd = sc.parallelize(["hello", "world", "spark", "rdd"])

sorted_rdd = rdd.sortBy(lambda x: len(x))

for item in sorted_rdd.collect():
    print(item)

统计一个RDD中每个元素的长度，并按照长度从小到大排序输出。

from pyspark import SparkContext

sc = SparkContext("local", "RDD Length")

rdd = sc.parallelize(["hello", "world", "spark", "rdd"])

length_rdd = rdd.map(lambda x: (x, len(x)))
sorted_length_rdd = length_rdd.sortBy(lambda x: x[1])

for item in sorted_length_rdd.collect():
    print(item[0], item[1])
``