Python爬虫实战:抓取豆瓣电影《穿靴子的猫2》影评数据
import requests from bs4 import BeautifulSoup
请求头
headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'}
穿靴子的猫2的豆瓣电影地址
url = 'https://movie.douban.com/subject/25868125/comments?start={}&limit=20&sort=new_score&status=P'
爬取所有影评
comments = [] for i in range(0, 400, 20): r = requests.get(url.format(i), headers=headers) soup = BeautifulSoup(r.text, 'lxml') comment_list = soup.find_all('div', class_='comment') for item in comment_list: comments.append(item.find('p').text)
输出影评
for comment in comments: print(comment)
原文地址: https://www.cveoy.top/t/topic/oA1n 著作权归作者所有。请勿转载和采集!