前言
马上就要过年了,大家都在屯年货了网络上商品信息太多,不知道如何选择,今天,我们就用python爬取商品信息,并做可视化。
环境使用
- python 3.9
- pycharm
模块使用
- requests
- selenium
- time
- 谷歌驱动
说明
一、谷歌驱动安装
二、selenium模块
代码实现
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Chrome()
# r'C:\Users\YY\AppData\Local\Programs\Python\Python39\chromedriver.exe'
driver.get('https://www.jd.com/')
def GetInfo():
input_tag = driver.find_element(By.ID, "key")
input_tag.send_keys('笔记本')
time.sleep(5)
input_tag.send_keys(Keys.ENTER)
spider_jd()
def spider_jd():
goods = driver.find_elements(By.CLASS_NAME,"gl-item")
for good in goods:
name = good.find_element(By.CSS_SELECTOR,".p-name").text.replace('\n','')
price = good.find_element(By.CSS_SELECTOR,".p-price").text
link = good.find_element(By.TAG_NAME,"a").get_attribute('href')
msg = '''
商品:%s
价格:%s
链接:%s
'''%(name,price,link)
print(msg)
# GetInfo()
for page in range(1,11):
print(f'爬取{page}页')
GetInfo()
driver.find_element(By.CSS_SELECTOR,'.pn-next').click()
time.sleep(2)