<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[Refrain__WG的博客]]></title><description><![CDATA[谁家游冶少年郎,三三五五映垂杨]]></description><link>https://blog.youkuaiyun.com/Refrain__WG</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; Refrain__WG]]></copyright><item><title><![CDATA[Python 写入excel 使用windows打开乱码问题解决方案 utf-8-sig]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/113344938</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/113344938</guid><author>Refrain__WG</author><pubDate>Tue, 02 Feb 2021 16:07:07 +0800</pubDate><description><![CDATA[将数据写入csv文件时，在windows电脑中打开使用乱码错行，mac电脑打开没有任何问题。

所以问题应该是编码不同导致的。文本中包含了中文、表情字符集等，无论使用 utf-8 还是 gkb 编码都有问题。

最终使用 utf-8-sig 编码完美解决问题。


with open(file=file, mode=mode, encoding='utf-8-sig', newline='', errors='ignore') as f:
    f.write(datas)



案例：将8W...]]></description><category></category></item><item><title><![CDATA[Python 正则替换忽略大小写 re.sub()]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/113343877</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/113343877</guid><author>Refrain__WG</author><pubDate>Thu, 28 Jan 2021 15:49:07 +0800</pubDate><description><![CDATA[re.sub('需要替换字符', '用于替换的字符', '需要替换的文本', flags=re.I)


import re

# 正则替换 不区分大小写
re.sub('world', 'Python', 'world I love World.', flags=re.I)

# 输出结果
'Python I love Python.'


]]></description><category></category></item><item><title><![CDATA[[twitter spider] Python 使用推特开发者账号应用+tweepy api，采集获取推特数据]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/112303560</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/112303560</guid><author>Refrain__WG</author><pubDate>Thu, 07 Jan 2021 10:37:10 +0800</pubDate><description><![CDATA[# 申请 twitter 开发者帐号 &amp; 应用 token
consumer_key = "b5n...W1c"
consumer_secret = "RGOGO...rbGvh"
access_token = "10177...9Gwc5"
access_token_secret = "0ugoL...WsgxW"

# 填写twitter提供的开发Key和secret + 提交你的Key和secret
auth = tweepy.OAuthHandler(consumer_key, co...]]></description><category></category></item><item><title><![CDATA[金融市场 基础知识]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/89228167</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/89228167</guid><author>Refrain__WG</author><pubDate>Thu, 07 Jan 2021 10:06:33 +0800</pubDate><description><![CDATA[1. 金融市场基础知识 思维导图Xmind





2. 股票数据处理








]]></description><category></category></item><item><title><![CDATA[scrapyd 批量清除pending任务]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/111830637</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/111830637</guid><author>Refrain__WG</author><pubDate>Mon, 28 Dec 2020 09:54:24 +0800</pubDate><description><![CDATA[一、问题

由于爬虫报错/生产环境更换等原因造成定时的scrapy任务队列pending 等待的太多

（通过http://127.0.0.1:6800/listjobs.json?project=myproject查询到 pending 的爬虫队列竟然有600+....）



二、解决方案

进入 用户目录下--&gt;dbs 目录，删除 pending 的scrapy项目db文件即可

因为 dbs 目录下存储着项目数据（包括爬虫任务队列）

如下图（scrapy项目为MyToken, 此...]]></description><category></category></item><item><title><![CDATA[Docker push 镜像到 Docker Hub]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/110948914</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/110948914</guid><author>Refrain__WG</author><pubDate>Fri, 18 Dec 2020 17:16:47 +0800</pubDate><description><![CDATA[363  docker build -t selenium_python3 .
  365  docker run -it --name selenium_python3 selenium_python3 /bin/bash
  
  378  docker login
  379  docker ps -a
  380  docker ps -l
  387  docker commit 5b231292928b doujiang2020/selenium_python3
  
  38...]]></description><category></category></item><item><title><![CDATA[Docker Ubuntu pip install lxml 报错：error: command ‘x86_64-linux-gnu-gcc‘ failed with exit status 1]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/110802530</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/110802530</guid><author>Refrain__WG</author><pubDate>Mon, 07 Dec 2020 11:36:26 +0800</pubDate><description><![CDATA[一、报错描述

docker 中 使用 pip install lxml, 报错：error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

报错图如下：



二、解决方案

在构建Docker中，使用apt-get install python3-lxml 代替 pip install lxml,可以正常安装（解决安装依赖报错问题）


apt-get install python3-lxml



参考文章：https://...]]></description><category></category></item><item><title><![CDATA[Git 常用命令]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/109779926</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/109779926</guid><author>Refrain__WG</author><pubDate>Wed, 18 Nov 2020 18:09:28 +0800</pubDate><description><![CDATA[通过git init命令把这个目录变成Git可以管理的仓库

git init

查看仓库当前的状态

git status



把文件添加到仓库

git add file1        # 添加一个文件
git add file1 file2  # 添加多个文件
git add .            # 添加目录所有变更文件

把文件提交到仓库

git commit -m "wrote a readme file"



查看工作区和版本库里面最新版本的区别

git d...]]></description><category></category></item><item><title><![CDATA[ERROR: command ‘x86_64-linux-gnu-gcc‘ failed with exit status 1 (ubuntu docker 安装 xxtea-py模块)]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/108651225</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/108651225</guid><author>Refrain__WG</author><pubDate>Wed, 14 Oct 2020 10:01:38 +0800</pubDate><description><![CDATA[参考文章：

https://stackoverflow.com/questions/26053982/setup-script-exited-with-error-command-x86-64-linux-gnu-gcc-failed-with-exit/35164888
























]]></description><category></category></item><item><title><![CDATA[Linux Centos 安装 Docker]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/96993213</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/96993213</guid><author>Refrain__WG</author><pubDate>Thu, 08 Oct 2020 15:22:01 +0800</pubDate><description><![CDATA[1. 安装 Docker


yum -y install docker

查看是否安装成功

docker version或者docker info





2. 启动 Docker


systemctl start docker
# 或者
service docker start

设置开机自启动： systemctlenable docker



3. 查找docker...]]></description><category></category></item><item><title><![CDATA[pip install ... ERROR: UnicodeDecodeError: ‘gbk‘ codec can‘t decode/ python setup.py egg_info Check]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/108758520</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/108758520</guid><author>Refrain__WG</author><pubDate>Wed, 30 Sep 2020 17:25:57 +0800</pubDate><description><![CDATA[1. pip install 安装包报错如下

ERROR: UnicodeDecodeError: 'gbk' codec can't decode/ python setup.py egg_info Check


(base) C:\Users\user&gt;pip download igramscraper
Collecting igramscraper
  Using cached igramscraper-0.3.5.tar.gz (24 kB)
    ERROR: Command erro]]></description><category></category></item><item><title><![CDATA[[weibo spider] 微博 id 和 mid 互相转换 Python]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/94560954</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/94560954</guid><author>Refrain__WG</author><pubDate>Thu, 17 Sep 2020 21:07:31 +0800</pubDate><description><![CDATA[参考资料：https://www.cnblogs.com/qiernonstop/p/3634354.html

分析思路：





实际代码：


ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"

def base62_encode(num, alphabet=ALPHABET):
   ...]]></description><category></category></item><item><title><![CDATA[自测 python 翻译的几种方法 （谷歌/百度/有道翻译）]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/96837534</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/96837534</guid><author>Refrain__WG</author><pubDate>Thu, 17 Sep 2020 21:04:43 +0800</pubDate><description><![CDATA[前言：工作需要，翻译的文本经常带有 emoji 表情。



1. 谷歌翻译（ translate&amp; googletranspython库）

优点：方便，简单

缺点：需要翻墙，translate 库有次数限制，googletrans 库不能翻译带有 emoji 表情的文本



2. 百度翻译

注册百度翻译开发者帐号，申请 appID &amp; 密码, 免费 200万字符...]]></description><category></category></item><item><title><![CDATA[node.js  从入门到？]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/102952411</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/102952411</guid><author>Refrain__WG</author><pubDate>Fri, 08 Nov 2019 15:14:33 +0800</pubDate><description><![CDATA[1. 安装 node.js 环境

node.js 官网:http://nodejs.cn/download/

从node.js官网下载安装包，打开一直执行下一步下一步，不要太简单～。



2. 安装编辑器IDE

首推 vscode, 免费、轻便且强大。

VSCode 官网：https://code.visualstudio.com/



3. 使用 node.js 写api...]]></description><category></category></item><item><title><![CDATA[mysql 进阶 -- 将查询结果集写入新数据表 & 备份当前表数据到新数据表（快照）]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/101354978</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/101354978</guid><author>Refrain__WG</author><pubDate>Wed, 25 Sep 2019 13:51:47 +0800</pubDate><description><![CDATA[备份当前表数据到新数据表（快照）
	
如果想要对一个表进行快照，即复制一份当前表的数据到一个新表，可以结合CREATE TABLE和SELECT：


-- 对class_id=1的记录进行快照，并存储为新表students_of_class1:
CREATE TABLE students_of_class1 SELECT * FROM students WHERE class_id=1...]]></description><category></category></item><item><title><![CDATA[mysql 进阶 --  添加或更新 / 添加或替换 / 添加或忽略 （数据是否存在）]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/101354093</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/101354093</guid><author>Refrain__WG</author><pubDate>Wed, 25 Sep 2019 11:58:23 +0800</pubDate><description><![CDATA[添加或更新

如果我们希望插入一条新记录（INSERT），但如果记录已经存在，就更新该记录，此时，可以使用INSERT INTO ... ON DUPLICATE KEY UPDATE ...语句：


INSERT INTO students (id, class_id, name, gender, score) VALUES (1, 1, '小明', 'F', 99) ON DUPLIC...]]></description><category></category></item><item><title><![CDATA[centos / linux 创建用户/添加用户/用户授权sudo权限]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/83858075</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/83858075</guid><author>Refrain__WG</author><pubDate>Wed, 07 Aug 2019 16:36:09 +0800</pubDate><description><![CDATA[一、Linux Centos 添加用户密码

adduser [用户名]

passwd [用户名]





二、用户授权sudo

1.添加sudoers文件可写权限

查看 sudoers文件只有可读权限



需要添加写权限


chmod -v u+w /etc/sudoers

修改之后的 sudoer 文件权限





2. 编辑 sudoer 文件，添加用户权限

vi...]]></description><category></category></item><item><title><![CDATA[Python Hash分表]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/96305616</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/96305616</guid><author>Refrain__WG</author><pubDate>Mon, 22 Jul 2019 11:25:17 +0800</pubDate><description><![CDATA[1. 创建数据表（10张分表为例）

创建 10 张分表：eth_special_addr_trx_0,eth_special_addr_trx_1,......, eth_special_addr_trx_9


    # 创建 10 个Hash分表
    for i in range(10):
        sql_create_table = '''
            ...]]></description><category></category></item><item><title><![CDATA[Centos Linux系统 安装 python3 (Anacada)]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/95073056</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/95073056</guid><author>Refrain__WG</author><pubDate>Wed, 17 Jul 2019 13:57:34 +0800</pubDate><description><![CDATA[1. 下载 Anaconda/Minconda 安装包

官网下载太慢。建议选择 国内镜像。


# Anaconda &amp; Minconda 二选一
# Anaconda
wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda3-5.0.1-Linux-x86_64.sh

# Minconda
wge...]]></description><category></category></item><item><title><![CDATA[Aliyun Code & Git -- 配置公钥 SSH Key (Linux/Centos & Windows)]]></title><link>https://blog.youkuaiyun.com/Refrain__WG/article/details/82837360</link><guid>https://blog.youkuaiyun.com/Refrain__WG/article/details/82837360</guid><author>Refrain__WG</author><pubDate>Mon, 08 Jul 2019 13:53:58 +0800</pubDate><description><![CDATA[前期准备：申请 自己的 Aliyun Code 帐号



一、Linux/Centos 系统 配置 Git

1. 登录Centos服务器, 安装 Git 服务


yum install git

安装完成可以输入`git --version`，查看Git安装版本，是否安装成功

安装成功即可在本地使用git仓库管理文件、clone 公共仓库代码等

安装完成后，设置用户名和邮箱，在命令行...]]></description><category></category></item></channel></rss>