1. libcurl 的参考文档如下 CURLOPT_HEADERFUNCTION Pass a pointer to a function that matches the following prototype: size_t function( void *ptr, size_t size, size_t nmemb, void *userdata);. This function gets called by libcurl as soon as it has received head
1. 利用 CURLOPT_WRITEFUNCTION 设置回调函数, 利用 CURLOPT_WRITEDATA 获取数据指针 官网文档如下 CALLBACK OPTIONS CURLOPT_WRITEFUNCTION Pass a pointer to a function that matches the following prototype: size_t function( char *ptr, size_t size, size_t nmemb, void *userdata);
网上的爬虫不能用,还是先表达谢意,不过我比较懒不喜欢重复写别人写的教程,只贴出修改,怎么用自己看教程吧. 我自己改了一版可以正常爬: #!/usr/bin/env python #coding=utf-8 # # Openwrt Package Grabber # # Copyright (C) 2016 sohobloo.me # import urllib2 import re import os import time # the url of package list page, end