每日一个爬虫练习：爬取喜马拉雅音频

前言

本文的文字及图片仅供学习、交流使用,不具有任何商业用途,版权归原作者所有,如有问题请及时联系我们以作处理。

本次目标

爬取喜马拉雅音频

https://www.ximalaya.com/

开发工具

• python 3.6.5
• pycharm

爬虫代码

导入工具

import requests
import re
import time

请求网页

headers = {
    'cookie': 'device_id=xm_1596531699133_kdfpr35pt5o0on; _xmLog=h5&b145d793-85e1-4aec-8cf3-25643943c990&2.1.2; x_xmly_traffic=utm_source%253A%2526utm_medium%253A%2526utm_campaign%253A%2526utm_content%253A%2526utm_term%253A%2526utm_from%253A; Hm_lvt_4a7d8ec50cfd6af753c4f8aee3425070=1600235340,1600499992,1602060323,1602060364; Hm_lpvt_4a7d8ec50cfd6af753c4f8aee3425070=1602060571',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'
url = 'https://www.ximalaya.com/youshengshu/2684034/p{}/'.format(page)
response = requests.get(url=url, headers=headers)

解析网页数据

lis = re.findall('<a title="(.*?)" href="(.*?)">', response.text, re.S)[4:-1]
    for i in lis:
        title = i[0]
        num_id = i[1].split('/')[-1]
        mp3_url = 'https://www.ximalaya.com/revision/play/v1/audio?id={}&ptype=1'.format(num_id)
        response_2 = requests.get(url=mp3_url, headers=headers)
        data = response_2.json()

保存数据

def download(url, title):
    filename = 'D:\\python\\demo\\喜马拉雅\\FM\\' + title + '.mp3'
    response = requests.get(url=url, headers=headers)
    with open(filename, mode='wb') as f:
        f.write(response.content)
        print('{}下载完成'.format(title))

运行代码，效果如下图