Python 爬虫-爬取京东手机页面的图片
2024-08-29 22:55:40
具体代码如下:
__author__ = 'Fred Zhao' import requests
from bs4 import BeautifulSoup
import os
from urllib.request import urlretrieve class Picture(): def __init__(self):
self.headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36'}
self.base_url = 'https://list.jd.com/list.html?cat=9987,653,655&page='
self.base_path = os.path.dirname(__file__) def makedir(self, name):
path = os.path.join(self.base_path, name)
isExist = os.path.exists(path)
if not isExist:
os.makedirs(path)
print("File has been created.")
else:
print('OK!The file is existed. You do not need create a new one.')
os.chdir(path) def request(self, url):
r = requests.get(url, headers=self.headers)
return r def get_img(self, page):
r = self.request(self.base_url + str(page))
plist = BeautifulSoup(r.text, 'lxml').find('div', id='plist')
item = plist.find_all('li', class_='gl-item')
print(len(item))
self.makedir('pictures')
num = 0
for i in item:
num += 1
imglist = i.find('div', class_='p-img')
print(num)
img = imglist.find('img')
print('This is %s picture' %num)
if img.get('src'):
url = 'https:' + img.get('src')
fileName = img.get('src').split('/')[-1]
urlretrieve(url, filename=fileName) elif img.get('data-lazy-img'):
url = 'https:' + img.get('data-lazy-img')
fileName = img.get('data-lazy-img').split('/')[-1]
urlretrieve(url, filename=fileName) if __name__ == '__main__':
picture = Picture()
for i in range(2): #控制爬取的页数
picture.get_img(i+1)
最新文章
- SVN 搭建
- OA项目之弹出层中再弹出层
- Android 学习笔记之网络通信基础+WebView....
- C++-new操作符
- EMVTag系列1《数据分组》
- 转载【ViewPager+Fragment】ViewPager中切换界面Fragment被销毁的问题分析
- 常用JavaScript字符串方法简述
- 1、Http概述
- LOCAL_EXPORT_××用法
- UML之顺序图
- 前端入门3-CSS基础
- iOS UICollectionView 在滚动时停在某个item位置上
- Redis学习-sorted set数据类型
- 【学习笔记】深入理解async/await
- Luogu2469 SDOI2010 星际竞速 费用流
- 步步为营-68-asp.net简单练习(get set)
- java单例类的几种实现
- Android学习笔记——从源码看Handler的处理机制
- [精彩] 关于DB2的内存分配
- 数据库Mysql的学习(八)-储存过程和事务和导入导出
热门文章
- idea 增量包配置
- vue 表格 多选 换页保存前一页的状态
- NOIP模拟测试19
- [Catalan数三连]网格&;有趣的数列&;树屋阶梯
- (转)ubuntu下如何安装使用SSH?
- anaconda里的python版本回退, requirements
- PHP中文无乱码截取
- 【Java多线程系列五】列表类
- Ubuntu12.04下Encountered a section with no Package: header错误解决方案
- 解压lzma格式的img文件报“Filename has an unknown suffix, skipping”怎么办