<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[jm88621的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/jm88621</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; jm88621]]></copyright><item><title><![CDATA[sqoop导入报错 Error: java.io.IOException: SQLException in nextKeyValue]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/119421440</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/119421440</guid><author>jm88621</author><pubDate>Thu, 05 Aug 2021 16:36:12 +0800</pubDate><description><![CDATA[原因：导入日期数据为0000-00-00 00:00:00时造成的
Error: java.io.IOException: SQLException in nextKeyValue
	at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
	at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapT]]></description><category></category></item><item><title><![CDATA[Ubuntu18 主分区扩容]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/119414104</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/119414104</guid><author>jm88621</author><pubDate>Thu, 05 Aug 2021 13:29:53 +0800</pubDate><description><![CDATA[查看当前磁盘情况
可以看到只有一个500g的磁盘，但是实际分配了2T的硬盘
root@cdh4:/etc/apt# df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sda1       477G  7.3G  445G   2% /
...

查看物理磁盘情况
可以看到实际磁盘有2T，主分区只用了500G
root@cdh4:~# fdisk -l
...
Disk /dev/sda: 2 TiB, 2199023255552 ]]></description><category></category></item><item><title><![CDATA[rsync同步脚本示例，带有exclude参数说明]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/119240259</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/119240259</guid><author>jm88621</author><pubDate>Fri, 30 Jul 2021 10:45:43 +0800</pubDate><description><![CDATA[同步数据脚本
#!/bin/bash
rsync -vae  "ssh -o StrictHostKeyChecking=no" --exclude "*.gz"  root@xxx.xxx.xx.xx:/root/log/access.log-* /root/cmooc_log/

说明

将IP=xxx.xxx.xx.xx的主机上的/root/log/access.log-*文件同步到本机的/root/cmooc_log/路径下
但是去除.gz后缀的文件，代码如下：

--exclude "*.gz"
]]></description><category></category></item><item><title><![CDATA[使用shell脚本重启java服务]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/115210762</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/115210762</guid><author>jm88621</author><pubDate>Thu, 25 Mar 2021 16:02:14 +0800</pubDate><description><![CDATA[1 脚本内容
#!/bin/bash
  
ids=`ps -ef | grep "xxx-1.0-SNAPSHOT.jar" | grep -v "grep" | awk '{print $2}'`
echo "当前服务id:" $ids


for id in $ids
do
    kill -9 $id
    echo "killed $id"  
done

sleep 5s

cd /xxx/xxx

echo "开始启动新进程..."
nohup java -jar -Xms8192m -X]]></description><category></category></item><item><title><![CDATA[centos7，sftp创建账号，配置权限和目录]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/113984003</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/113984003</guid><author>jm88621</author><pubDate>Tue, 23 Feb 2021 13:58:08 +0800</pubDate><description><![CDATA[目录1 创建用户组sftp2 创建用户3 设置密码4 创建并修改sftp根目录owner为root5 设置sshd_config6 在ChrootDirectory 下创建owner为sftpuser的子目录（重点！！！）7 重启sshd服务8 测试sftp服务9 搞定！有问题请留言！
1 创建用户组sftp
(base) [root@localhost ~]# groupadd sftp

2 创建用户
useradd参数说明
-g 加入到sftp组
-s 禁止登录
-d 指定sftp用户的登入目录
(b]]></description><category></category></item><item><title><![CDATA[CDH配置sqoop]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/112931980</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/112931980</guid><author>jm88621</author><pubDate>Thu, 21 Jan 2021 13:38:48 +0800</pubDate><description><![CDATA[添加sqoop服务

打开cluster

添加服务

选择sqoop2，然后一路继续即可



4. 安装完成后查看服务

配置mysql驱动

下载mysql驱动到cdh的jars路径下

mysql驱动下载地址
root@cdh1:/opt/cloudera/parcels/CDH/jars# ll mysql-connector-java-5.1.49-bin.jar 
-rw-r--r-- 1 root root 1006906 1月  21 09:17 mysql-connector-java-]]></description><category></category></item><item><title><![CDATA[CDH配置hive.exec.dynamic.partition.mode]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/112171137</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/112171137</guid><author>jm88621</author><pubDate>Mon, 04 Jan 2021 11:24:46 +0800</pubDate><description><![CDATA[cdh配置hive.exec.dynamic.partition.mode
看图说话，我是用的server2，所以根据自己情况配置


]]></description><category></category></item><item><title><![CDATA[CDH配置lzo压缩实践]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/111310013</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/111310013</guid><author>jm88621</author><pubDate>Thu, 17 Dec 2020 10:52:13 +0800</pubDate><description><![CDATA[下载安装文件
1 下载文件
点击 cdh安装包地址
下载与你的系统以及cdh版本相匹配的包，例如我的cdh版本是5.14.2，系统是ubuntu，我下载三个文件如图所示：

2 上传文件
将文件上传到parcel-repo路径下，如图

3 配置cloudera manager
检查新Parcel-&gt;分配-&gt;激活

查看lzo的jar包位置
/opt/cloudera/parcels/GPLEXTRAS/lib/hadoop/lib/


修改yarn的mapreduce.application]]></description><category></category></item><item><title><![CDATA[centos环境下nginx配置https]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/111033557</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/111033557</guid><author>jm88621</author><pubDate>Fri, 11 Dec 2020 13:54:57 +0800</pubDate><description><![CDATA[原因
网站被攻击灌入广告
1 nginx配置https
先从/etc/nginx/nginx.conf文件中查找.conf的存放路径。

然后查看server的具体配置文件
/etc/nginx/conf.d/xxx.conf

提前准备好证书文件.crt和.key，然后在xxx.conf中添加443端口的相关配置
server
{
    listen 80;
    server_name xxx.xxx.xxx.cn;

    return 301 https://$host$request_uri]]></description><category></category></item><item><title><![CDATA[CDH-5.14.2 flume采用TAILDIR SOURCE实践]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/110926492</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/110926492</guid><author>jm88621</author><pubDate>Wed, 09 Dec 2020 15:10:36 +0800</pubDate><description><![CDATA[报错
java.io.FileNotFoundException: /opt/flume-position/log_position.json (Permission denied)

解决
提前创建目录
root@cdh7:/opt# mkdir flume-posoiton


创建json文件
root@cdh7:/opt/flume-position# touch log_position.json


赋权限
root@cdh7:/opt/flume-position# chmod 777 log]]></description><category></category></item><item><title><![CDATA[备份mysql数据库shell脚本]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/110630271</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/110630271</guid><author>jm88621</author><pubDate>Fri, 04 Dec 2020 10:52:21 +0800</pubDate><description><![CDATA[目录导出数据库设置定时导出同步数据文件
导出数据库
创建shell脚本
root@cdh4:~/dataBak# vi backupdata.sh

脚本内容如下
backupdir=/root/dataBak/mydb

# 时间
time=_` date +%Y_%m_%d_%H_%M_%S `
# 需要备份的数据库名称
db_name=mydb
# 导出数据库，数据文件后缀为当前日期
/usr/bin/mysqldump  $db_name | gzip &gt; $backupdir/$db_nam]]></description><category></category></item><item><title><![CDATA[fighting]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/110630159</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/110630159</guid><author>jm88621</author><pubDate>Fri, 04 Dec 2020 10:31:57 +0800</pubDate><description><![CDATA[fighting
零零散散做了好多事，都忘记总结一下了，现在开始慢慢把自己做的事情记录起来，fighting！

]]></description><category></category></item><item><title><![CDATA[使用PhpSpreadsheet生成excel]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/110473597</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/110473597</guid><author>jm88621</author><pubDate>Wed, 02 Dec 2020 11:27:37 +0800</pubDate><description><![CDATA[目录phpspreadsheet资料安装phpspreadsheet导出excel工程结构数据格式表头变量调用方法
phpspreadsheet资料
PhpSpreadsheet’s documentation: link.
安装phpspreadsheet
在项目根目录下安装运行
// An highlighted block
composer require phpoffice/phpspreadsheet

安装完成后生成vendor目录，如下图所示：

导出excel
工程结构

数据格式
$lis]]></description><category></category></item><item><title><![CDATA[kafka manager监测不到consumer]]></title><link>https://blog.youkuaiyun.com/jm88621/article/details/100736645</link><guid>https://blog.youkuaiyun.com/jm88621/article/details/100736645</guid><author>jm88621</author><pubDate>Wed, 11 Sep 2019 15:03:48 +0800</pubDate><description><![CDATA[第一步，首先勾选poll

第二步，在创建consumer时必须设置group_id，否则manager监测不到
def consume(topic):
    # 必须设置group_id，否则kafka manager监测不到
    consumer = KafkaConsumer(topic,
                             bootstrap_servers=s...]]></description><category></category></item></channel></rss>