Hive on Spark: 提高数据仓库查询效率的利器

Hive on Spark \u662f\u4e00\u79cd\u5c06 Hive \u4e0e Spark \u96c6\u6210\u8d77\u6765\u7684\u65b9\u6cd5\uff0c\u53ef\u4ee5\u52a0\u901f Hive \u67e5\u8be2\u7684\u6267\u884c\u6548\u7387\u3002Hive \u662f\u4e00\u4e2a\u57fa\u4e8e Hadoop \u7684\u6570\u636e\u4ed3\u5e93\u5de5\u5177\uff0c\u5e38\u5e38\u4f7f\u7528 MapReduce \u4f5c\u4e3a\u6267\u884c\u5f00\u59cb\u5668\uff0c\u800c Spark \u662f\u4e00\u4e2a\u5feb\u901f\u7684\u5927\u6570\u636e\u5904\u7406\u6846\u67b6\uff0c\u5177\u6709\u5185\u5b58\u8ba1\u7b97\u548c\u5e76\u884c\u5904\u7406\u7684\u80fd\u529b\u3002\n\n\u901a\u8fc7\u5c06 Hive \u4e0e Spark \u96c6\u6210\uff0c\u53ef\u4ee5\u5229\u7528 Spark \u7684\u5185\u5b58\u8ba1\u7b97\u548c\u5e76\u884c\u5904\u7406\u7684\u7279\u6027\uff0c\u63d0\u9ad8 Hive \u67e5\u8be2\u7684\u6267\u884c\u6548\u7387\u3002\u5177\u4f53\u800c\u8a00\uff0cHive on Spark \u53ef\u4ee5\u5e26\u6765\u4ee5\u4e0b\u7684\u597d\u5904\uff1a\n\n1\u3002\u66f4\u5feb\u7684\u67e5\u8be2\u901f\u5ea6\uff1aSpark \u7684\u5185\u5b58\u8ba1\u7b97\u548c\u5e76\u884c\u5904\u7406\u80fd\u529b\u53ef\u4ee5\u52a0\u901f\u67e5\u8be2\u7684\u6267\u884c\u901f\u5ea6\uff0c\u7279\u522b\u662f\u5bf9\u4e8e\u590d\u6742\u7684\u67e5\u8be2\u6216\u5927\u578b\u6570\u636e\u96c6\u3002\n\n2\u3002\u66f4\u9ad8\u7684\u8d44\u6e90\u5229\u7528\u7387\uff1aSpark \u53ef\u4ee5\u66f4\u6709\u6548\u7684\u5229\u7528\u96c6\u7fa4\u8d44\u6e90\uff0c\u63d0\u9ad8\u8d44\u6e90\u5229\u7528\u7387\u3002\u4e0e\u4f20\u7edf\u7684 MapReduce \u76f8\u6bd4\uff0cSpark \u901a\u8fc7\u5c06\u6570\u636e\u7f13\u5b58\u5728\u5185\u5b58\u4e2d\uff0c\u51cf\u5c11\u4e86\u78c1\u76d8\u8bfb\u5199\u7684\u6295\u8d44\uff0c\u56e0\u6b64\u63d0\u9ad8\u4e86\u8d44\u6e90\u5229\u7528\u7387\u3002\n\n3\u3002\u66f4\u597d\u7684\u4ea4\u4e92\u6027\uff1aHive on Spark \u652f\u6301\u4ea4\u4e92\u5f0f\u67e5\u8be2\uff0c\u53ef\u4ee5\u5b9e\u65f6\u54cd\u5e94\u7528\u6237\u7684\u67e5\u8be2\u8bf7\u6c42\uff0c\u63d0\u4f9b\u66f4\u597d\u7684\u4ea4\u4e92\u6027\u3002\n\n4\u3002\u66f4\u597d\u7684\u6269\u5c55\u6027\uff1aSpark \u7684\u5f39\u6027\u5206\u5e03\u5f0f\u6570\u636e\u96c6\uff08RDD\uff09\u6a21\u578b\u53ef\u4ee5\u5f88\u597d\u7684\u5904\u7406\u5927\u89c4\u6a21\u6570\u636e\uff0c\u5e76\u4e14\u53ef\u4ee5\u4e0e\u5176\u4ed6 Spark \u751f\u6态\u7cfb\u7edf\u4e2d\u7684\u7ec4\u4ef6\uff08\u5982 Spark Streaming \u548c Spark MLlib\uff09\u96c6\u6210\uff0c\u63d0\u4f9b\u66f4\u9ad8\u7ea7\u7684\u6570\u636e\u5904\u7406\u548c\u5206\u6790\u529f\u80fd\u3002\n\n\u8981\u4f7f\u7528 Hive on Spark\uff0c\u9700\u8981\u5728 Hive \u914d\u7f6e\u4e2d\u542f\u7528 Spark \u4f5c\u4e3a\u6267\u884c\u5f00\u59cb\u5668\uff0c\u5e76\u4e14\u914d\u7f6e Spark \u7684\u76f8\u5173\u53c2\u6570\u3002\u5916\u52a0\uff0c\u8fd8\u9700\u8981\u5c06 Spark \u5b89\u88c5\u5728\u96c6\u7fa4\u4e2d\uff0c\u5e76\u4e14\u786e\u4fdd Hive \u548c Spark \u7684\u7248\u672c\u517c\u5408\u3002\n\n\u603b\u4e48\u6765\u8bf4\uff0cHive on Spark \u53ef\u4ee5\u901a\u8fc7\u5229\u7528 Spark \u7684\u5185\u5b58\u8ba1\u7b97\u548c\u5e76\u884c\u5904\u7406\u80fd\u529b\uff0c\u52a0\u901f Hive \u67e5\u8be2\u7684\u6267\u884c\u6548\u7387\uff0c\u63d0\u9ad8\u6570\u636e\u5904\u7406\u7684\u6027\u80fd\u548c\u6548\u7387\u3002