<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[pcengineercn的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/pcengineercn</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; pcengineercn]]></copyright><item><title><![CDATA[CentOS7安装ElasticSearch-Head]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/104598832</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/104598832</guid><author>pcengineercn</author><pubDate>Sun, 01 Mar 2020 20:21:21 +0800</pubDate><description><![CDATA[下载 Nodejs


wgethttps://nodejs.org/dist/v12.16.1/node-v12.16.1-linux-x64.tar.xz


创建 nodejs 目录


mkdir /usr/local/nodejs


将node-v12.16.1-linux-x64.tar.xz解压到 /usr/local/nodejs 中


tar -xvfnode-v...]]></description><category></category></item><item><title><![CDATA[ElasticSearch集群搭建]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/104580212</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/104580212</guid><author>pcengineercn</author><pubDate>Sat, 29 Feb 2020 19:58:34 +0800</pubDate><description><![CDATA[安装elasticsearch

集群搭建

修改 elasticsearch主目录/conf/elasticsearch.yml文件，每个节点的名称保持唯一，集群名称保持一致

cluster.name: &lt;集群名称&gt;

node.name: &lt;节点名称&gt;

node.master: true # 参与主节点选举

node.data: true # 是否作为数据节点

...]]></description><category></category></item><item><title><![CDATA[解决scrpay-redis空爬问题]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/104513577</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/104513577</guid><author>pcengineercn</author><pubDate>Wed, 26 Feb 2020 11:39:37 +0800</pubDate><description><![CDATA[scrapy-redis在爬取页面完成后会处于空爬状态，不会自动结束



可以通过配置scrapy扩展来解决这个问题

在项目目录新建extensions.py


import time

from scrapy import signals
from scrapy.exceptions import NotConfigured


class ScrapyRedisAutoClose(ob...]]></description><category></category></item><item><title><![CDATA[Scrapy配置Redis集群]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/104502846</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/104502846</guid><author>pcengineercn</author><pubDate>Tue, 25 Feb 2020 18:49:07 +0800</pubDate><description><![CDATA[安装第三方库


pip install scrapy-redis-sentinel


修改 setting.py文件


ITEM_PIPELINES = {
'scrapy_redis_sentinel.pipelines.RedisPipeline': 543,
}

# Bloomfilter 配置
# 使用的哈希函数数，默认为 6
BLOOMFILTER_HASH_NUMBER = ...]]></description><category></category></item><item><title><![CDATA[Redis5.0.7集群搭建]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/104502061</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/104502061</guid><author>pcengineercn</author><pubDate>Tue, 25 Feb 2020 18:32:44 +0800</pubDate><description><![CDATA[环境

centos7.3
	3台服务器，ip分别为192.168.56.30、192.168.56.31、192.168.56.32
	
	redis5.0.7
	三主三从集群
		主：端口 9000、从：端口 9001
	
安装redis，可以参照我的另一篇博客Redis安装并开启远程访问

在centos7根目录创建redis-cluster文件夹，然后在redis-cluster里面创建两...]]></description><category></category></item><item><title><![CDATA[pyppeteer隐藏window.navigator.webdriver]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/104384470</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/104384470</guid><author>pcengineercn</author><pubDate>Tue, 18 Feb 2020 22:48:40 +0800</pubDate><description><![CDATA[由于一些网站的反爬机制，会识别window.navigator.webdriver，对爬虫进行封禁

通过阅读pyppeteer的源码，可以通过修改源码来达到隐藏window.navigator.webdriver的效果

步骤

找到pyppeteer的launcher.py

一般在 Python的home目录\Lib\site-packages\pyppeteer\

定位到60行，找到参数...]]></description><category></category></item><item><title><![CDATA[Elasticsearch5.2.2安装elasticsearch-analysis-hanlp-5.2.2插件]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/104300953</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/104300953</guid><author>pcengineercn</author><pubDate>Thu, 13 Feb 2020 19:30:37 +0800</pubDate><description><![CDATA[下载elasticsearch-analysis-hanlp-5.2.2.zip（es版本要和hanlp版本保持一致）
	
	https://github.com/KennFalcon/elasticsearch-analysis-hanlp.git
	
	进入es目录/bin输入以下命令
	
	elasticsearch-plugin install file:///路径/elasticse...]]></description><category></category></item><item><title><![CDATA[CentOS、Ubuntu等linux操作系统取消用户登录密码]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/104242046</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/104242046</guid><author>pcengineercn</author><pubDate>Sun, 09 Feb 2020 22:43:42 +0800</pubDate><description><![CDATA[passwd -d 用户名
]]></description><category></category></item><item><title><![CDATA[Pyppeteer的安装和使用]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/104221392</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/104221392</guid><author>pcengineercn</author><pubDate>Sat, 08 Feb 2020 12:48:22 +0800</pubDate><description><![CDATA[安装

通过pip命令安装

pip install pyppeteer

对应的chromium在pyppeteer首次启动时会自动下载，不需要手动配置

常见报错信息

/root/.local/share/pyppeteer/local-chromium/575458/chrome-linux/chrome: error while loading shared libraries: li...]]></description><category></category></item><item><title><![CDATA[Splash安装]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/103898700</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/103898700</guid><author>pcengineercn</author><pubDate>Wed, 08 Jan 2020 21:54:00 +0800</pubDate><description><![CDATA[目录

系统环境

安装Docker

安装splash


	系统环境
	
centos7.3


	安装Docker
	
更新yum包

yum update

以yum方式安装    yum -y install docker

启动docker    service docker start

测试运行docker    docker run hello-worl...]]></description><category></category></item><item><title><![CDATA[Scrapy和Splash爬取iframe标签里的内容]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/100796342</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/100796342</guid><author>pcengineercn</author><pubDate>Fri, 13 Sep 2019 08:44:12 +0800</pubDate><description><![CDATA[Lua脚本


function main(splash, args)
  splash:go(args.url)
  splash:wait(args.wait)
  splash:runjs("iframe = function(){ 
    var f = document.getElementById('g_iframe'); 
    return f.contentDocument....]]></description><category></category></item><item><title><![CDATA[Ubuntu18.04搭建Hadoop3.2分布式集群]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/95229428</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/95229428</guid><author>pcengineercn</author><pubDate>Tue, 09 Jul 2019 20:42:04 +0800</pubDate><description><![CDATA[目录
	
运行环境

修改每台服务器的主机名

修改每台服务器的hosts文件

配置免密登录

安装jdk1.8

安装hadoop3.2

修改hadoop配置文件

hadoop常用命令


	运行环境
	
	ubuntu18.04
	hadoop3.2
	jdk1.8



	修改每台服务器的主机名

	
vim /etc/hostname
	
4台服务器的主机名分别为 Master...]]></description><category></category></item><item><title><![CDATA[CentOS7安装python虚拟环境]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/93869359</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/93869359</guid><author>pcengineercn</author><pubDate>Thu, 27 Jun 2019 13:32:48 +0800</pubDate><description><![CDATA[目录

安装虚拟环境

创建 virtualenv 软连接

设置虚拟环境目录

virtualenvwrapper常用命令

安装虚拟环境

采用豆瓣源安装

pip3 install -i https://pypi.douban.com/simple virtualenv

pip3 install -i https://pypi.douban.com/simple virtualenvwra...]]></description><category></category></item><item><title><![CDATA[Nginx+uwsgi+Python3.6部署Flask项目]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/93867261</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/93867261</guid><author>pcengineercn</author><pubDate>Thu, 27 Jun 2019 13:15:53 +0800</pubDate><description><![CDATA[目录

安装python3.6.6

安装flask和uwsgi

安装nginx




	安装python3.6.6
	
	由于是编译安装，所以需要确认相应的依赖包已经安

yum install -y gcc zlib* sqlite-devel libXcomposite libXcursor libXi libXtst libXScrnSaver libXrandr atk at-sp...]]></description><category></category></item><item><title><![CDATA[Redis安装并开启远程访问]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/93865784</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/93865784</guid><author>pcengineercn</author><pubDate>Thu, 27 Jun 2019 12:37:58 +0800</pubDate><description><![CDATA[官网下载 Redis安装包

https://redis.io/

解压Redis安装包（当前版本为 redis-5.0.5 ）并进入解压文件夹

tar zxvfredis-5.0.5.tar &amp;&amp; cdredis-5.0.5

编译

make &amp;&amp; make install

进入 redis-5.0.5 文件夹中的 utils文件夹，执行脚本

cd ...]]></description><category></category></item><item><title><![CDATA[Nginx+uwsgi+Python3.6部署Django项目]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/89513687</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/89513687</guid><author>pcengineercn</author><pubDate>Thu, 25 Apr 2019 13:52:48 +0800</pubDate><description><![CDATA[目录

系统环境

安装python3.6.6

安装nginx及配置

配置uwsgi


	系统环境

	
		centos7.3
		
		
		python3.6.6
		
		
		django2.2
		
		
		uwsgi2.0.18
		
		
		nginx1.14.2
		
	
	
	安装python3.6.6

	下载python安装包，wget https://www.p...]]></description><category></category></item><item><title><![CDATA[Ubuntu Server 18.04修改主机名]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/88806836</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/88806836</guid><author>pcengineercn</author><pubDate>Mon, 25 Mar 2019 21:41:40 +0800</pubDate><description><![CDATA[Ubuntu Server 18.04直接进入/etc/hostname修改主机名的时候，重启后发现没有修改成功，这是因为Ubuntu18.04与之前的版本有些不同，需要先做一些额外的操作

sudo vim /etc/cloud/cloud.cfg，将preserve_hostname设置为true
	sudo vim /etc/hostname，修改主机名
	重启服务器
...]]></description><category></category></item><item><title><![CDATA[搭建FTP服务器]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/82827439</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/82827439</guid><author>pcengineercn</author><pubDate>Mon, 24 Sep 2018 10:36:52 +0800</pubDate><description><![CDATA[初次搭建ftp的过程中经过多次尝试，总结了一个较为简单的ftp搭建步骤

1.首先购买一个服务器，个人使用的是阿里云的ECS云服务器，系统选64位的CentOS7.2版本

阿里云官网 https://www.aliyun.com/

2.安装vsftp

yum install vsftpd

遇到问号直接按回车

看到Complete！表示已安装完成

设置vsftpd系统自启动 

syst...]]></description><category></category></item><item><title><![CDATA[JDBC Driver连接MySQL8.0 失败]]></title><link>https://blog.youkuaiyun.com/pcengineercn/article/details/80376215</link><guid>https://blog.youkuaiyun.com/pcengineercn/article/details/80376215</guid><author>pcengineercn</author><pubDate>Sat, 19 May 2018 17:44:15 +0800</pubDate><description><![CDATA[错误提示java.sql.SQLException:The server time zone value 'ÖÐ¹ú±ê×¼Ê±¼ä' is unrecognized or represents more than one time zone. You must configure either the server or JDBC driver (via the serverTimezone c...]]></description><category></category></item></channel></rss>