一个抓取智联招聘数据并存入表格的python爬虫
2024-08-24 12:52:16
talk is cheap...show you the code.....
import requests
import lxml,time,os
from bs4 import BeautifulSoup as sb
from xlwt import *
import sys
reload(sys)
sys.setdefaultencoding('utf-8')
print sys.getdefaultencoding() book = Workbook(encoding = "utf-8")
table = book.add_sheet("test1")
table.write(0,0,'number')
table.write(0,1,'position')
table.write(0,2,'feedback')
table.write(0,3,'company')
table.write(0,4,'salary')
table.write(0,5,'address ')
table.write(0,6,"updatetime")
table.write(0,7,"details")
headers = {"User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"} for num in range(90):
url = 'http://sou.zhaopin.com/jobs/searchresult.ashx?jl=%E4%B8%9C%E8%8E%9E&p='+str(num)
print url
res = requests.get(url,headers = headers)
html = sb(res.text,'lxml')
zwmc = html.find_all('td',class_="zwmc")
fk_lv = html.find_all('td',class_="fk_lv")
gsmc = html.find_all('td',class_="gsmc")
zwyx = html.find_all('td',class_="zwyx")
gzdd = html.find_all('td',class_="gzdd")
gxsj = html.find_all('td',class_="gxsj")
details = html.find_all('li',class_="newlist_deatil_last")
row = num*len(zwmc)
for i in range(1,len(zwmc)):
print zwmc[i].text.strip()+"---"+fk_lv[i].text.strip()+"---"+ gsmc[i].text.strip()+"---"+ zwyx[i].text.strip()+"---"+ gzdd[i].text.strip()+"---"+gxsj[i].text.strip()
table.write(row+i,0,row+i)
table.write(row+i,1,zwmc[i].text.strip())
table.write(row+i,2,fk_lv[i].text.strip())
table.write(row+i,3,gsmc[i].text.strip())
table.write(row+i,4,zwyx[i].text.strip())
table.write(row+i,5,gzdd[i].text.strip())
table.write(row+i,6,gxsj[i].text.strip())
table.write(row+i,7,details[i].text.strip())
book.save('result.xls')
最新文章
- Java Native Interface 四--JNI中引用类型
- PAT MOOC dataStructure 4-1
- java.util.List接口的方法subList()的使用注意事项
- 定向转发和重定向实现 <;select >;下拉表单数据传送
- LNMP系列网站零基础开发记录(三)
- linux命令行操作快捷键
- SQL Server批量更新数据
- eclipse报错 com/genuitec/eclipse/j2eedt/core/J2EEProjectUtil 转
- poj 1556 zoj1721 BellmanFord 最短路+推断直线相交
- YII2.0 ——安装yii2项目
- 十一招让Ubuntu 16.04用起来更得心应手(转)
- 怎么从一台电脑的浏览器输入地址访问另一台电脑服务器(WAMP服务器已搭建,PHPSTORM装好了)
- Markdown的基本语法记录
- 唯一约束(UNIQUE_KEY)
- pygame 笔记-10 摩擦力与屏幕环绕
- layer loading层 的设置
- 精练代码:一次Java函数式编程的重构之旅
- loadrunner&#160;脚本开发-定义全局变量
- 二进制枚举例题|poj1222,poj3279,poj1753
- log4js_Node.js中的日志管理模块使用