C#小爬虫,通过URL进行模拟发送接收数据
2024-08-31 21:16:51
public async Task<string> SendDataAsync(HttpMethod httpMethod, string requestUrl, HttpContent postContent = null, string cookies = "")
{
HttpClientHandler httpHandler = new HttpClientHandler()
{
AllowAutoRedirect = true,
//CookieContainer = new CookieContainer(),
UseCookies = false,
AutomaticDecompression = DecompressionMethods.GZip
}; using (HttpClient httpClient = new HttpClient(httpHandler))
{
httpClient.DefaultRequestHeaders.ExpectContinue = false;
httpClient.DefaultRequestHeaders.Add("Accept", "*/*");
httpClient.DefaultRequestHeaders.Add("Accept-Encoding", "gzip, deflate");
httpClient.DefaultRequestHeaders.Add("Accept-Language", " zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3");
httpClient.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0");
//var response = await httpClient.GetAsync(sUrl);
//return (await response.Content.ReadAsAsync<AvResult>());
string sResult;
try
{
HttpRequestMessage request = new HttpRequestMessage(httpMethod, requestUrl);
bool isNullCookie = string.IsNullOrEmpty(cookies);
if (!isNullCookie)
{
//httpHandler.CookieContainer.SetCookies(new Uri(requestUrl), cookies);
request.Headers.Add("Cookie", cookies);
}
if (httpMethod.Method == "POST")
request.Content = postContent;
var response = await httpClient.SendAsync(request);
sResult = await response.Content.ReadAsStringAsync();
}
catch (Exception ex)
{
sResult = ex.Message;
}
return sResult;
}
}
次上面的代码只是爬虫的模拟数据发送的部分代码,即自定义模拟浏览器客户端的,设置请求头,没有用Cookies进行登陆保存设置
最新文章
- RabbitMQ学习系列(三): C# 如何使用 RabbitMQ
- 利用html5的画布canvas进行图片压缩处理
- EntityFramework系列:SQLite.CodeFirst自动生成数据库
- 2、android Service 详细用法
- LDA(Linear discriminate analysis)线性判别分析
- android ping网络是否成功
- samba服务设置,Linux系统和Windows文件共享
- Failed to load resource: net::ERR_CACHE_MISS
- SGU 194. Reactor Cooling(无源汇有上下界的网络流)
- tp5 点击刷新验证码
- git报错:&#39;fatal:remote origin already exists
- OpenResty 操作cookies
- 基于tcp的套接字编程
- VS Code 1.18版本更新内容整理(2017年10月 October 2017)
- 51Nod1317 相似字符串对 容斥原理 动态规划
- Flutter常用组件(Widget)解析-Text
- (转)fabric 一个链码如何调用另一个链码
- 第43章 RTC—实时时钟
- appium+python自动化53-adb logcat查看日志
- Html5 中获取镜像图像 - 解决 WebGL 中纹理倒置问题