获取网页源码

最新推荐文章于 2024-06-27 20:15:18 发布

转载最新推荐文章于 2024-06-27 20:15:18 发布 · 55 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/jesn/articles/2860510.html

本文介绍了一种使用C#语言从指定URL获取网页源码的方法。通过创建HttpWebRequest对象并设置必要的请求参数，如超时时间和缓存策略等，最终读取并返回网页内容。此外还介绍了如何根据响应内容的字符集来选择合适的编码。

 1 　　　　　/// <summary>
 2         /// 获取对应url地址的源码
 3         /// </summary>
 4         /// <param name="url">传入的url地址</param>
 5         /// <returns></returns>
 6         public string GetWebContents(string url)
 7         {
 8             string webContent = "";
 9             Encoding encoding = Encoding.GetEncoding("GB2312");
10             try
11             {
12                 //申明一个HttpWebRequest请求
13                 HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
14                 //设置连接超时时间
15                 request.Timeout = 6000;
16                 request.Headers.Set("Pragma", "no-cache");
17 
18                 HttpWebResponse response = (HttpWebResponse)request.GetResponse();
19                 System.IO.Stream streamReceive = response.GetResponseStream();
20                 //获取对应url的编码
21                 if (response.CharacterSet.Contains("gbk"))
22                 {
23                     encoding = Encoding.GetEncoding("GB2312");
24                 }
25                 else if (response.CharacterSet.Contains("utf"))
26                 {
27                     encoding = Encoding.GetEncoding("UTF-8");
28                 }
29                 System.IO.StreamReader streamReader = new System.IO.StreamReader(streamReceive, encoding);
30                 //源码
31                 webContent = streamReader.ReadToEnd();
32                 //关闭流
33                 streamReceive.Close();
34                 streamReader.Close();
35             }
36             catch (Exception ex)
37             {
38                 MessageBox.Show(ex.Message);
39             }
40             return webContent;
41         }

转载于:https://www.cnblogs.com/jesn/articles/2860510.html