打开文本文件：

text_file = open('online review data.txt','r',encoding='utf-8')

读取数据：

text = text_file.read()

读取的数据类型：

print (type(text)) print(' ')

打印文本：

print(text) print(' ')

导入所需库：

import jieba

按句子对文本进行分词：

sentences = list(jieba.cut(text, cut_all=False)) print('精确模式: ' + '/ '.join(sentences))

打印分词结果：

#print(sentences) print(sentences)

import jieba.posseg as pseg tag_filter = ['n'] words_pair = pseg.cut(text) result = [] for word, flag in words_pair: if flag in tag_filter: result.append(word) print('词性过滤完成') print(result)

导入停用词表：

stopwords_file1 = open('baidu_stopwords.txt','r',encoding='utf-8') stopwords_file2 = open('hit_stopwords.txt','r',encoding='utf-8')

读取停用词表：

stopwords1 = stopwords_file1.read().splitlines() stopwords2 = stopwords_file2.read().splitlines()

将停用词表合并：

stopwords = stopwords1 + stopwords2

去除停用词和标点符号：

filtered_words = [] for word in sentences: if word not in stopwords and word != ' ' and word != '\n' and word != '\t' and word != '\r\n' and word != '\u3000': filtered_words.append(word)

打印过滤后的词语：

print(filtered_words)