《The Python Standard Library》——http模块阅读笔记1
官方文档:https://docs.python.org/3.5/library/http.html
偷个懒,截图如下:
即,http客户端编程一般用urllib.request库(主要用于“在这复杂的世界里打开各种url”,包括:authentication、redirections、cookies and more.)。
1. urllib.request—— Extensible library for opening URLs
使用手册,结合代码写的很详细:HOW TO Fetch Internet Resources Using The urllib Package
该模块提供的函数:
urllib.request.
urlopen
(url, data=None, [timeout, ]*, cafile=None, capath=None, cadefault=False, context=None)
urllib.request.
install_opener
(opener)
urllib.request.
build_opener
([handler, ...])
urllib.request.
pathname2url
(path)
urllib.request.
url2pathname
(path)
urllib.request.
getproxies
()
该模块提供的类:
class urllib.request.
Request
(url, data=None, headers={}, origin_req_host=None, unverifiable=False, method=None)
class urllib.request.
OpenerDirector
class urllib.request.
BaseHandler
class urllib.request.
HTTPDefaultErrorHandler
class urllib.request.
HTTPRedirectHandler
class urllib.request.
HTTPCookieProcessor
(cookiejar=None)
class urllib.request.
ProxyHandler
(proxies=None)
class urllib.request.
HTTPPasswordMgr
还有很多,不一一列出了。。。
1.2 Request对象
下面的方法是Request提供的公共接口,所以它们可以被子类重写。同时,也提供了一些客户端可以查阅解析的请求的公共属性。
Request.
full_url
Request.
type
Request.
host
Request.
origin_req_host #不包含端口号
Request.
selector
Request.
data
Request.
unverifiable
Request.
method
Request.
get_method
() Request.
add_header(key, val)
Request.
add_unredirected_header
(key, header) Request.
has_header
(header) Request.
remove_header
(header)
Request.
get_full_url
() Request.
set_proxy
(host, type) Request.
get_header
(header_name, default=None) Request.
header_items
()
1.3 OpenerDirector Objects
有以下方法:
OpenerDirector.
add_handler
(handler)
OpenerDirector.
open
(url, data=None[, timeout])
OpenerDirector.
error
(proto, *args)
1.4 BaseHandler Objects
1.5 HTTPRedirectHandler Objects
1.6 HTTPCookieProcessor Objects
它只有一个属性:HTTPCookieProcessor.
cookiejar ,所有的cookies都保存在http.cookiejar.CookeiJar中。
1.x 还有太多类,需要用时直接查看官方文档吧。。
EXamples
打开url读取数据:
>>> import urllib.request
>>> with urllib.request.urlopen('http://www.python.org/') as f:
... print(f.read(300))
...
b'<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n\n\n<html
xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">\n\n<head>\n
<meta http-equiv="content-type" content="text/html; charset=utf-8" />\n
<title>Python Programming '
注意:urlopen返回一个bytes object(字节对象)。
>>> with urllib.request.urlopen('http://www.python.org/') as f:
... print(f.read(100).decode('utf-8'))
...
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtm
向CGI的stdin发送数据流:
>>> import urllib.request
>>> req = urllib.request.Request(url='https://localhost/cgi-bin/test.cgi',
... data=b'This data is passed to stdin of the CGI')
>>> with urllib.request.urlopen(req) as f:
... print(f.read().decode('utf-8'))
...
Got Data: "This data is passed to stdin of the CGI"
CGI的另一端通过stdin接收数据:
#!/usr/bin/env python
import sys
data = sys.stdin.read()
print('Content-type: text/plain\n\nGot Data: "%s"' % data)
Use of Basic HTTP Authentication:
import urllib.request
# Create an OpenerDirector with support for Basic HTTP Authentication...
auth_handler = urllib.request.HTTPBasicAuthHandler()
auth_handler.add_password(realm='PDQ Application',
uri='https://mahler:8092/site-updates.py',
user='klem',
passwd='kadidd!ehopper')
opener = urllib.request.build_opener(auth_handler)
# ...and install it globally so it can be used with urlopen.
urllib.request.install_opener(opener)
urllib.request.urlopen('http://www.example.com/login.html')
添加HTTP头部:
import urllib.request
req = urllib.request.Request('http://www.example.com/')
req.add_header('Referer', 'http://www.python.org/')
# Customize the default User-Agent header value:
req.add_header('User-Agent', 'urllib-example/0.1 (Contact: . . .)')
r = urllib.request.urlopen(req)
OpenerDirector
automatically adds a User-Agent header to every Request
. To change this:
import urllib.request
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
opener.open('http://www.example.com/')
Also, remember that a few standard headers (Content-Length, Content-Type and Host) are added when the Request
is passed to urlopen()
(or OpenerDirector.open()
).
GET:
>>> import urllib.request
>>> import urllib.parse
>>> params = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
>>> url = "http://www.musi-cal.com/cgi-bin/query?%s" % params
>>> with urllib.request.urlopen(url) as f:
... print(f.read().decode('utf-8'))
POST:
>>> import urllib.request
>>> import urllib.parse
>>> data = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
>>> data = data.encode('ascii')
>>> with urllib.request.urlopen("http://requestb.in/xrbl82xr", data) as f:
... print(f.read().decode('utf-8'))
The following example uses an explicitly specified HTTP proxy, overriding environment settings:
>>> import urllib.request
>>> proxies = {'http': 'http://proxy.example.com:8080/'}
>>> opener = urllib.request.FancyURLopener(proxies)
>>> with opener.open("http://www.python.org") as f:
... f.read().decode('utf-8'
The following example uses no proxies at all, overriding environment settings:
>>> import urllib.request
>>> opener = urllib.request.FancyURLopener({})
>>> with opener.open("http://www.python.org/") as f:
... f.read().decode('utf-8')
最新文章
- 我们为什么不能只用O记号来谈论算法?
- javascript类型系统——正则表达式RegExp类型
- winform 程序制作自己的数字签名(续)
- springMVC的注解@RequestParam与@PathVariable的区别
- Cmap的使用
- jquery获得图片的真实大小
- 【转】【WPF】WriteableBitmap应用及图片数据格式转换
- 常用的Linux操作
- monkey测试(转)
- (转)iOS应用程序生命周期(前后台切换,应用的各种状态)详解
- 30.怎样在Swift中添加运行时属性?
- [leetcode] Path sum路径之和
- VB.net 字符串 分割 及 重新倒序组装
- DEDECMS中,channelartlist标签
- *[codility]ArrayInversionCount
- 关于css中z-index 的应用
- POJ 1778 All Discs Considered(拓扑排序)
- 【转】 jni.h头文件详解(二)
- 查询SQL中字符串里有多少个逗号
- eclipse-java开发实用快捷键