httplib模块是一个底层基础模块,可以看到建立HTTP请求的每一步,但是实际的功能比较少。在python爬虫开发中基本用不到
下面详细介绍httplib提供的常用类型和方法:
- httplib.HTTPConnection ( host [ , port [ , strict [ , timeout ]]] )
- HTTPConnection类的构造函数,表示一次与服务器之间的交互,即请求/响应。
- HTTPConnection.request ( method , url [ , body [ , headers ]] )
- 调用request 方法会向服务器发送一次请求,method 表示请求的方法,常用有方法有get 和post ;url 表示请求的资源的url ;body 表示提交到服务器的数据,必须是字符串(如果method 是"post" ,则可以把body 理解为html 表单中的数据);headers 表示请求的http 头。
HTTPConnection.getresponse ()
- 获取Http 响应。返回的对象是HTTPResponse 的实例,关于HTTPResponse 在下面会讲解。
HTTPConnection.connect () 连接到Http 服务器。
HTTPConnection.close () 关闭与服务器的连接。
httplib.HTTPResponse HTTPResponse表示服务器对客户端请求的响应。往往通过调用HTTPConnection.getresponse()来创建,它有如下方法和属性:HTTPResponse.read([amt]) 获取响应的消息体。如果请求的是一个普通的网页,那么该方法返回的是页面的html。可选参数amt表示从响应流中读取指定字节的数据。HTTPResponse.getheader(name[, default]) 获取响应头。Name表示头域(header field)吊,可选参数default在头域吊上存在的情况下作为默认值返回。HTTPResponse.getheaders() 以列表的形式返回所有的头信息。HTTPResponse.msg 获取所有的响应头信息。HTTPResponse.version 获取服务器所使用的http协议版本。11表示http/1.1;10表示http/1.0。HTTPResponse.status 获取响应的状态码。如:200表示请求成功。HTTPResponse.reason 返回服务器处理请求的结果说明。一般为”OK”
接下来使用GET和POST请求发送,
- Get请求:
import httplibconn =Nonetry: ???conn = httplib.HTTPConnection("www.zhihu.com") ???conn.request("GET", "/") ???response = conn.getresponse() ???print response.status, response.reason ???print ‘-‘ * 40 ???headers = response.getheaders() ???for h in headers: ???????print h ???print ‘-‘ * 40 ???print response.msgexcept Exception,e: ???print efinally: ???if conn: ???????conn.close()
-运行结果:
C:\Python27\python.exe F:/python_scrapy/ch03/3.2.2_Httplib.py301 Moved Permanently----------------------------------------(‘content-length‘, ‘182‘)(‘set-cookie‘, ‘tgw_l7_route=23ddf1acd85bb5988efef95d7382daa0; Expires=Wed, 01-Aug-2018 13:12:46 GMT; Path=/, _xsrf=1tvVcqOibo5QEOMGIxD3uSM36Hn3ms4J; path=/; domain=zhihu.com; expires=Sun, 17-Jan-21 12:57:46 GMT‘)(‘vary‘, ‘Accept-Encoding‘)(‘server‘, ‘ZWS‘)(‘connection‘, ‘keep-alive‘)(‘location‘, ‘https://www.zhihu.com/‘)(‘date‘, ‘Wed, 01 Aug 2018 12:57:46 GMT‘)(‘content-type‘, ‘text/html‘)----------------------------------------Date: Wed, 01 Aug 2018 12:57:46 GMTContent-Type: text/htmlContent-Length: 182Connection: keep-aliveSet-Cookie: tgw_l7_route=23ddf1acd85bb5988efef95d7382daa0; Expires=Wed, 01-Aug-2018 13:12:46 GMT; Path=/Location: https://www.zhihu.com/Server: ZWSVary: Accept-EncodingSet-Cookie: _xsrf=1tvVcqOibo5QEOMGIxD3uSM36Hn3ms4J; path=/; domain=zhihu.com; expires=Sun, 17-Jan-21 12:57:46 GMTProcess finished with exit code 0
- 下面是POST请求:
#!coding:utf-8import httplib, urllibconn = Nonetry: ???params = urllib.urlencode({‘name‘: ‘qiye‘, ‘age‘: 22}) ???headers = {"Content-type": "application/x-www-form-urlencoded" ???, "Accept": "text/plain"} ???conn = httplib.HTTPConnection("www.zhihu.com", 80, timeout=3) ???conn.request("POST", "/login", params, headers) ???response = conn.getresponse() ???print response.getheaders() #获取头信息 ???print response.status ???print response.read()except Exception, e: ???print efinally: ???if conn: ???????conn.close()
httplib/urllib实现
原文地址:https://www.cnblogs.com/guguobao/p/9403903.html