<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[babi4202的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/babi4202</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; babi4202]]></copyright><item><title><![CDATA[爬取淘宝商品信息selenium+pyquery+mongodb]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784680</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784680</guid><author>babi4202</author><pubDate>Wed, 01 Aug 2018 14:37:00 +0800</pubDate><description><![CDATA['''
爬取淘宝商品信息,通过selenium获得渲染后的源码,pyquery解析,mongodb存储
'''

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as E...]]></description><category></category></item><item><title><![CDATA[requests库+ajax数据分析+多线程爬取头条图集]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784678</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784678</guid><author>babi4202</author><pubDate>Sun, 29 Jul 2018 09:12:00 +0800</pubDate><description><![CDATA['''
根据关键字获取今日头条图集
'''


import os
import requests
from urllib.parse import quote
from hashlib import md5
from multiprocessing.pool import Pool

baseurl = 'https://www.toutiao.com/search_content/?...]]></description><category></category></item><item><title><![CDATA[requests微博爬取Ajax数据+mongoDB存储]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784679</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784679</guid><author>babi4202</author><pubDate>Sat, 28 Jul 2018 15:04:00 +0800</pubDate><description><![CDATA[import requests
from pyquery import PyQuery as pq
from pymongo import MongoClient

headers = {
    'Referer': 'https://m.weibo.cn/u/2830678474',
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64)...]]></description><category></category></item><item><title><![CDATA[redis存储]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784676</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784676</guid><author>babi4202</author><pubDate>Sat, 28 Jul 2018 08:58:00 +0800</pubDate><description><![CDATA[目录
    
        
        redis存储
        安装,链接与配置
        edis
        python库redis
        RedisDump
        
        redis和StrictRedis
        连接Redis
        键操作
        字符串操作
        列表操...]]></description><category></category></item><item><title><![CDATA[pymongo操作MongoDB]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784677</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784677</guid><author>babi4202</author><pubDate>Thu, 26 Jul 2018 15:04:00 +0800</pubDate><description><![CDATA[目录
    
        
        pymongo操作MongoDB
        安装,启动及链接
        MongoDB
        pymongo
        
        连接MongoDB,指定数据库,指定集合
        插入数据
        查询
        普通查询
        条件查询
        计数
...]]></description><category></category></item><item><title><![CDATA[用pymongo连接mongodb,操作时报错:pymongo.errors.OperationFailure: not authorized on test to execute command...]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784674</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784674</guid><author>babi4202</author><pubDate>Thu, 26 Jul 2018 12:50:00 +0800</pubDate><description><![CDATA[创建连接对象时,用client = pymongo.MongoClient('mongodb://user:password@localhost:27017/')这样的方式,增加权限认证


转载于:https://www.cnblogs.com/Wang-Y/p/9371143.html]]></description><category></category></item><item><title><![CDATA[Ubuntu16.04下安装mongodb运行失败解决]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784672</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784672</guid><author>babi4202</author><pubDate>Wed, 25 Jul 2018 19:44:00 +0800</pubDate><description><![CDATA[原答案:&lt;https://askubuntu.com/questions/770054/mongodb-3-2-doesnt-start-on-lubuntu-16-04-lts-as-a-service/770133#770133
亲测有效,2018.7.25

创建mongodb.service文件:sudo nano /etc/systemd/system/mongodb.s...]]></description><category></category></item><item><title><![CDATA[pymysql操作mysql]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784675</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784675</guid><author>babi4202</author><pubDate>Wed, 25 Jul 2018 16:47:00 +0800</pubDate><description><![CDATA[目录
    
        
        pymysql操作mysql
        安装,启动及链接
        MySQL
        pymysql
        
        链接数据库
        创建表
        事务性ACID
        插入数据
        更新数据
        删除数据
        查询操作
...]]></description><category></category></item><item><title><![CDATA[ubuntu重装mysql]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784670</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784670</guid><author>babi4202</author><pubDate>Wed, 25 Jul 2018 15:01:00 +0800</pubDate><description><![CDATA[目录
    
        
        重装mysql
        首先删除mysql:
        默认安装：
        安装5.6（因为5.7版本据说经常出错）
        安装完成后查看mysql状态：
        
        
    

重装mysql
首先删除mysql:
sudo apt-get remove mysql-*
...]]></description><category></category></item><item><title><![CDATA[文件存储]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784667</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784667</guid><author>babi4202</author><pubDate>Wed, 25 Jul 2018 12:24:00 +0800</pubDate><description><![CDATA[目录
    
        
        文件存储
        TXT文本存储
        文件打开模式
        
        JSON文件存储
        对象和数组构成的json形式
        读写操作
        
        CSV文件存储
        写入
        读取
        
        
  ...]]></description><category></category></item><item><title><![CDATA[CSV转化为markdown表格形式]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784673</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784673</guid><author>babi4202</author><pubDate>Tue, 24 Jul 2018 17:29:00 +0800</pubDate><description><![CDATA[符号分割文本转化为markdown表格形式

写博客,做笔记的时候,有时需要复制表格到markdown文档,数量一多一个个添加格式还是很麻烦,然后就写了一个小工具,先将表格复制粘贴到文本,用工具转换,再复制粘贴到markdown文档..

def to_table(filename='test', rept=':'):
    '''将符号分割文本转化为markdown表格形式
    ...]]></description><category></category></item><item><title><![CDATA[爬取知乎发现页面pyquery+文件存储]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784665</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784665</guid><author>babi4202</author><pubDate>Tue, 24 Jul 2018 16:44:00 +0800</pubDate><description><![CDATA[# 爬取知乎发现页面,用文件存储

import requests
from pyquery import PyQuery as pq


def get_html(url):
    '''根据网址获取html页面
    
    parameter:url目标网址
    return:html页面
    '''
    
    headers = {
            ...]]></description><category></category></item><item><title><![CDATA[pyquery库]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784671</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784671</guid><author>babi4202</author><pubDate>Tue, 24 Jul 2018 14:09:00 +0800</pubDate><description><![CDATA[目录
    
        
        pyquery库
        初始化
        基本CSS选择器
        函数操作
        查找节点
        获取信息
        DOM节点操作
        伪类选择器
        
        
        
    

pyquery库
侧重于css选择器,JQuery...]]></description><category></category></item><item><title><![CDATA[BeautifulSoup库]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784666</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784666</guid><author>babi4202</author><pubDate>Tue, 24 Jul 2018 10:15:00 +0800</pubDate><description><![CDATA[目录
    
        
        Beautiful Soup库
        解析器
        基本用法
        节点选择器(速度快)
        方法选择器
        find_all()
        其他方法
        
        CSS选择器
        
        
    



Beautiful...]]></description><category></category></item><item><title><![CDATA[xpath和lxml库]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784669</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784669</guid><author>babi4202</author><pubDate>Tue, 24 Jul 2018 08:14:00 +0800</pubDate><description><![CDATA[目录
    
        
        XPath及lxml解析
        xpath语法
        1.XPath常用路径表达式
        2.xpath谓词
        3.选取未知节点
        4.选取若干路径
        5.XPath 轴
        
        lxml
        1.实例引入
      ...]]></description><category></category></item><item><title><![CDATA[pipenv和autoenv]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784663</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784663</guid><author>babi4202</author><pubDate>Mon, 23 Jul 2018 10:24:00 +0800</pubDate><description><![CDATA[目录
    
        
        pipenv
        安装
        创建环境
        常用命令
        换源
        
        
    


pipenv
官方文档:https://docs.pipenv.org/#pipenv-usage
安装
pip3 install pipenv
创建环境

创建环境:
...]]></description><category></category></item><item><title><![CDATA[正则表达式]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784664</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784664</guid><author>babi4202</author><pubDate>Sun, 22 Jul 2018 17:25:00 +0800</pubDate><description><![CDATA[正则表达式
在线正则表达式测试工具:http://tool.oschina.net/regex/
1.常见匹配模式




模式描述








\w
匹配字母数字及下划线







\W
匹配非字母数字下划线







\s
匹配任意空白字符，等价于 [\t\n\r\f].






...]]></description><category></category></item><item><title><![CDATA[requests库]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784661</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784661</guid><author>babi4202</author><pubDate>Sun, 22 Jul 2018 14:59:00 +0800</pubDate><description><![CDATA[目录
    
        
        request库
        1.七个主要方法
        2.Request对象（其它方法参数，用法差不多）
        3.Response对象的属性
        4.requests异常
        5.requests其他方法和属性
        requests.codes
        req...]]></description><category></category></item><item><title><![CDATA[返回码和相应的查询条件]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784662</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784662</guid><author>babi4202</author><pubDate>Sun, 22 Jul 2018 10:42:00 +0800</pubDate><description><![CDATA[# 信息性状态码
100: ('continue',),
101: ('switching_protocols',),
102: ('processing',),
103: ('checkpoint',),
122: ('uri_too_long', 'request_uri_too_long'),

# 成功状态码
200: ('ok', 'okay', 'all_ok', 'all_...]]></description><category></category></item><item><title><![CDATA[格式化文件转对象]]></title><link>https://blog.youkuaiyun.com/babi4202/article/details/101784659</link><guid>https://blog.youkuaiyun.com/babi4202/article/details/101784659</guid><author>babi4202</author><pubDate>Sat, 21 Jul 2018 13:05:00 +0800</pubDate><description><![CDATA[格式化文件转对象
爬虫构造请求头时,需要用到字典,在浏览器F12查看请求头,发现是类字典冒号分割文本,将其直接复制到本地文件,转化为字典对象,可拿来直接使用.代码是具有可扩展性的,需要其他格式转化为对象时,只用增加转化函数,自动提供文件名处理,持久化导入导出功能.
代码如下:
'''
文本格式化转换,并提供将其持久化的导入导出功能.将 (键:值/n 键/值/n)类型文本转化为字典对象
'...]]></description><category></category></item></channel></rss>