分享web开发知识

注册/登录|最近发布|今日推荐

主页 IT知识网页技术软件开发前端开发代码编程运营维护技术分享教程案例
当前位置:首页 > 技术分享

第一次完美完成xpath 构建不完全href 猎聘网

发布时间:2023-09-06 01:46责任编辑:沈小雨关键词:暂无标签
import requestsfrom lxml import etreefrom multiprocessing import Poolheaders = { ???‘User-Agent‘:‘Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36‘}def getLinks(url): ???wb_data = requests.get(url,headers=headers) ???soup = etree.HTML(wb_data.text) ???links = soup.xpath(‘//div[@class="job-info"]‘) ???for link in links: ???????href = link.xpath(‘h3/a/@href‘)[0] ???????getinfo(href)def getinfo(url): ???try: ???????wb_data = requests.get(url,headers=headers) ???????soup = etree.HTML(wb_data.text) ???????requires = soup.xpath(‘//div[@class="job-title-left"]‘) ???????for require in requires: ???????????a = require.xpath(‘div[1]/span[3]/text()‘) ???????????print(a) ???except: ???????href = ‘https://www.liepin.com‘ + url ???????getinfo(href)if __name__ == ‘__main__‘: ???urls = [‘https://www.liepin.com/zhaopin/?pubTime=&ckid=e49b6fdfb6a698ed&fromSearchBtn=2&compkind=&isAnalysis=&init=-1&searchType=1&dqs=&industryType=&jobKind=&sortFlag=15&degradeFlag=0&industries=&salary=&compscale=&clean_condition=&key=python%E7%88%AC%E8%99%AB&headckid=e49b6fdfb6a698ed&d_pageSize=40&siTag=tF7woi2y6F2s2RnHW3wfUw~fA9rXquZc5IkJpXC-Ycixw&d_headId=ca30a54749a469a7967ac218f6204031&d_ckId=ca30a54749a469a7967ac218f6204031&d_sfrom=search_prime&d_curPage=2&curPage={0}‘.format(str(i)) ???????????for i in range(0,4)] ???for url in urls: ???????getLinks(url)

第一次完美完成xpath 构建不完全href 猎聘网

原文地址:https://www.cnblogs.com/mayunji/p/8644087.html

知识推荐

我的编程学习网——分享web前端后端开发技术知识。 垃圾信息处理邮箱 tousu563@163.com 网站地图
icp备案号 闽ICP备2023006418号-8 不良信息举报平台 互联网安全管理备案 Copyright 2023 www.wodecom.cn All Rights Reserved