【二】C# 基于WPF 使用CefSharp来做万能爬虫

目录

    【一】C# 基于WPF 使用CefSharp来做万能爬虫

    【二】C# 基于WPF 使用CefSharp来做万能爬虫

    【三】C# 基于WPF 使用CefSharp来做万能爬虫

    【四】C# 基于WPF 使用CefSharp来做万能爬虫---扩展 一个httpClient访问辅助类可设置代理​​​​​​​

一.对浏览器进行操作

    1.刷新

         chrome.GetBrowser().Reload();

    2.后退
                if (chrome.GetBrowser().CanGoBack)
                {
                    chrome.GetBrowser().GoBack();
                }

     3.前进

               if (chrome.GetBrowser().CanGoForward)
                {
                    chrome.GetBrowser().GoForward();
                }

       4.打开调试窗体

              chrome.GetBrowser().ShowDevTools();

      5.给CefSharp设置代理

            CefSharp.CefSharpSettings.Proxy = new CefSharp.ProxyOptions("IP", "Port");
            CefSharp.Cef.Initialize(settings);

        注意：设置代理必须在Browser使用前 进行设置

二.对Cookie进行操作

 1.新建类CookieMonster 继承ICookieVisitor，实现如下方法

---

    /// <summary>
    /// CefSharp的cookie的操作类
    /// </summary>
    public class CookieMonster : ICookieVisitor
    {
        public List<CefSharp.Cookie> cookies = new List<CefSharp.Cookie>();
        readonly ManualResetEvent gotAllCookies = new ManualResetEvent(false);


        /// <summary>
        /// 获取cookie
        /// </summary>
        /// <param name="DomainStr">根据域名获取，如果DomainStr为空则获取所有的cookie</param>
        /// <returns></returns>
        public List<CefSharp.Cookie> GetCookieList(string DomainStr = "")
        {
            var visitor = new CookieMonster();
            if (DomainStr.Length > 0)
            {
                var cookieManager = CefSharp.Cef.GetGlobalCookieManager();
                if (cookieManager.VisitAllCookies(visitor))
                {
                    visitor.WaitForAllCookies();
                    return visitor.cookies.Where(p => p.Domain == DomainStr || p.Domain == "." + DomainStr).ToList();
                }
                else
                {
                    return visitor.cookies;
                }
            }
            else
            {
                var cookieManager = CefSharp.Cef.GetGlobalCookieManager();
                if (cookieManager.VisitAllCookies(visitor))
                    visitor.WaitForAllCookies();
                return visitor.cookies;
            }
        }

        /// <summary>
        /// 给浏览器设置cookie
        /// </summary>
        /// <returns></returns>
        private async Task<bool> setCookie(string domainStr, string nameStr, string valueStr, bool ishttps)
        {
            string httpStr = "http";
            if (ishttps)
            {
                httpStr = "https";
            }
            var cookieManager = CefSharp.Cef.GetGlobalCookieManager();
            var bol = await cookieManager.SetCookieAsync(httpStr + "://" + domainStr, new CefSharp.Cookie()
            {
                Domain = domainStr,
                Name = nameStr,
                Value = valueStr,
                Path = "/",
                HttpOnly = true,
                Expires = DateTime.Now.AddMinutes(1)
            });
            return bol;
        }

        public bool Visit(CefSharp.Cookie cookie, int count, int total, ref bool deleteCookie)
        {

            cookies.Add(cookie);
            if (count == total - 1)
                gotAllCookies.Set();
            return true;
        }

        public void WaitForAllCookies()
        {
            gotAllCookies.WaitOne();
        }

        public void Dispose()
        {
            //cookies.Clear();
        }
    }  

2.该类直接使用即可，不需要对webbrowser来设置。

 使用如下： 

var visitor = new CookieMonster();
var list = visitor.GetCookieList("www.baidu.com");
var isSucess = visitor.setCookie("www.baidu.com","ceshi","123456",true);

 注：Cookie拿到后，想干什么就可以去干什么了。后续就不需要解释了。