<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[Marsm的博客]]></title><description><![CDATA[每天进步一点点]]></description><link>https://blog.youkuaiyun.com/Mars_sock</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; Mars_sock]]></copyright><item><title><![CDATA[odps sql创建日期维表]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/121476447</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/121476447</guid><author>Mars_sock</author><pubDate>Mon, 22 Nov 2021 17:34:48 +0800</pubDate><description><![CDATA[在odps中使用sql创建日期维表

表结构

create table if not exists dim_date ( 
 date_id                   string         comment '日期(yyyymmdd)' 
,datestr                   string         comment '日期(yyyy-mm-dd)'
,date_name				   string 		  comment '日期名称中文'
,weekid        ]]></description><category></category></item><item><title><![CDATA[pandas.read_csv读取csv文件报错]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/106551101</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/106551101</guid><author>Mars_sock</author><pubDate>Thu, 04 Jun 2020 17:19:46 +0800</pubDate><description><![CDATA[文章目录pandas读取文件报错读取csv文件报错OSError: Initializing from file failed
pandas读取文件报错
读取csv文件报错OSError: Initializing from file failed

使用pandas读取csv文件的时候报错：OSError: Initializing from file failed，其原因是读取的文件名包含中文

根据文章参考得知，pandas的read_csv()方法，默认使用C engine作为parser engi]]></description><category></category></item><item><title><![CDATA[java反编译jar包]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/106473156</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/106473156</guid><author>Mars_sock</author><pubDate>Mon, 01 Jun 2020 15:36:08 +0800</pubDate><description><![CDATA[java反编译jar包

在日常开发中，由于之前的系统中UDF函数源码并没有接手，要想把现有UDF函数一直到其他平台上使用，因此需要对现有jar包进行反编译，并在其他平台上调试使用。
反编译方式分两种：

GUI工具（JD-GUI）
idea插件：java-decompiler.jar



GUI工具
- 使用反编译GUI工具，这种工具可以直接打开jar包，并将反编译之后的结果导出



反编译的结果文件入下(附带一下其他东西，还得手工处理一下)：

/*    */ package com.alibab]]></description><category></category></item><item><title><![CDATA[centos7 yum源安装mysql5.7]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/106251979</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/106251979</guid><author>Mars_sock</author><pubDate>Thu, 21 May 2020 18:45:57 +0800</pubDate><description><![CDATA[【代码】centos7 yum源安装mysql5.7。]]></description><category></category></item><item><title><![CDATA[ITerm2+on-my-zsh配置终端主题]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/106065437</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/106065437</guid><author>Mars_sock</author><pubDate>Mon, 11 May 2020 23:43:11 +0800</pubDate><description><![CDATA[文章目录ITerm2+on-my-zsh配置终端主题1.设置主题2.配置vim高亮3.命令高亮插件4.命令提示插件
ITerm2+on-my-zsh配置终端主题
1.设置主题


安装on-my-zsh


curl安装：

sh -c “$(curl -fsSL https://raw.github.com/robbyrussell/oh-my-zsh/master/tools/install.sh)”



wget安装：

sh -c “$(wget https://raw.github.com/ro]]></description><category></category></item><item><title><![CDATA[Linux常见报错处理]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/106047645</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/106047645</guid><author>Mars_sock</author><pubDate>Mon, 11 May 2020 09:52:42 +0800</pubDate><description><![CDATA[这里写自定义目录标题Linux常见报错处理远程执行命令报错
Linux常见报错处理
远程执行命令报错

报错场景：在使用ssh远程命令执行远程主机上的脚本是，报如下错误

FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/root']

原因是根目录下的这些目录没有可用的空间，df -h查看磁盘空间，发现被占用完了，清理磁盘空间问题解决


...]]></description><category></category></item><item><title><![CDATA[编译spark 2.1.0源码]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/103908257</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/103908257</guid><author>Mars_sock</author><pubDate>Thu, 09 Jan 2020 14:49:30 +0800</pubDate><description><![CDATA[编译spark 2.1.0源码

准备环境：

准备spark源码包:https://archive.apache.org/dist/spark/spark-2.1.0/spark-2.1.0.tgz
准备maven-3.3.9以上
scala-2.11.8:https://downloads.lightbend.com/scala/2.11.8/scala-2.11.8.tgz


设置mavn...]]></description><category></category></item><item><title><![CDATA[CDH 5.16.1集群环境搭建]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/103631211</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/103631211</guid><author>Mars_sock</author><pubDate>Fri, 20 Dec 2019 13:50:25 +0800</pubDate><description><![CDATA[CDH 5.16.1集群环境搭建
集群节点
192.168.10.1	hadoop01	235G+4T
192.168.10.2	hadoop02	235G+4T
192.168.10.3	hadoop03	235G+4T
192.168.10.4 hadoop04	235G+4T
集群节点初始化
1.配置节点hosts
192.168.10.1    hadoop01.office.gdapi....]]></description><category></category></item><item><title><![CDATA[Centos7 Python3安装]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/103487916</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/103487916</guid><author>Mars_sock</author><pubDate>Wed, 11 Dec 2019 10:27:01 +0800</pubDate><description><![CDATA[Centos7 Python3安装

安装python3环境包：

yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gcc make


下载python3安装包：

wget https://www.python.org/ftp/pytho...]]></description><category></category></item><item><title><![CDATA[Linux磁盘挂载\卸载]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/103427305</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/103427305</guid><author>Mars_sock</author><pubDate>Fri, 06 Dec 2019 17:53:22 +0800</pubDate><description><![CDATA[Linux磁盘挂载\卸载
挂载磁盘

使用 fdisk -l查看可挂载的磁盘

执行命令后，如果不存在/dev/vdb，表示没有数据盘。确认数据盘是否已挂载。
执行 fdisk -u /dev/vdb 创建分区



[root@ecshost~ ]# fdisk -u /dev/vdb
Welcome to fdisk (util-linux 2.23.2).
Changes will rema...]]></description><category></category></item><item><title><![CDATA[python包和模块的打包]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/103425752</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/103425752</guid><author>Mars_sock</author><pubDate>Fri, 06 Dec 2019 16:58:04 +0800</pubDate><description><![CDATA[python包和模块的打包

在需要打包的包目录下创建 setup.py 文件 和 MANIFEST.in（无静态文件可以不用创建） 文件
setup.py内容

# 引入构建包信息的模块
from distutils.core import setup

# 定义发布的包文件的信息
setup(
    name="hive_udf_test",  # 发布的包的名称
    version="...]]></description><category></category></item><item><title><![CDATA[Linux误卸载自带python的解决方法]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/103292930</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/103292930</guid><author>Mars_sock</author><pubDate>Thu, 28 Nov 2019 14:48:44 +0800</pubDate><description><![CDATA[Linux误卸载自带python的解决方法
1.卸载python（防止未卸载干净）

rpm -qa|grep python|xargs rpm -ev --allmatches --nodeps #强制删除已安装程序及其关联
whereis python|xargs rm -frv #删除所有残余文件 #xargs，允许你对输出执行其他某些命令

2.卸载yum

rpm -qa|grep yu...]]></description><category></category></item><item><title><![CDATA[shell脚本常用命令]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/102503920</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/102503920</guid><author>Mars_sock</author><pubDate>Fri, 11 Oct 2019 16:45:29 +0800</pubDate><description><![CDATA[脚本目录

获取当前目录

current_dir=$(cd `dirname $0`; pwd)

日期操作

获取当前日期

current_day=$(date '+%Y%m%d' )


获取前一天

yest_day=$(date '+%Y%m%d'  -d '-1 day')


获取下一天

next_day=$(date '+%Y%m%d' -d '+1 day')

定时任务(分...]]></description><category></category></item><item><title><![CDATA[windows+pycharm搭建spark开发环境]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/89709629</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/89709629</guid><author>Mars_sock</author><pubDate>Tue, 30 Apr 2019 15:00:41 +0800</pubDate><description><![CDATA[windows+pycharm搭建spark开发环境

创建python文件
点击 file &gt;&gt; setting &gt;&gt; Project:PythonProject &gt;&gt; Prohject Structure

添加pyspark.zip和py4j包到项目中（这两个文件在spark安装包的D:\apps\spark-2.3.2-bin-hadoop2.7\pyt...]]></description><category></category></item><item><title><![CDATA[hive表中array类型数据处理（行列转换）]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/84400677</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/84400677</guid><author>Mars_sock</author><pubDate>Fri, 23 Nov 2018 17:17:55 +0800</pubDate><description><![CDATA[hive表中array类型数据处理
hive表中数据如下：
1       [&quot;20170101&quot;,&quot;20170102&quot;,&quot;20170201&quot;,&quot;20170203&quot;]
2       [&quot;20170102&quot;,&quot;20170102&quot;,&quot;20170201&quot;]
3       [&quot;20170104&quot;]
2       [&quot;20170201&quot]]></description><category></category></item><item><title><![CDATA[Numpy基本用法]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/83418379</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/83418379</guid><author>Mars_sock</author><pubDate>Fri, 26 Oct 2018 17:31:10 +0800</pubDate><description><![CDATA[文章目录一、Numpy基本用法二、Numpy创建数组1.使用np.array()由python list创建2.使用np的routines函数创建2.1 np.ones()2.2 zeros()2.3 np.full()2.4 np.eye()2.5 np.linspace()2.6 arange()2.7 randint()2.8 randn()  正太分布2.9 normal() 正太分布2....]]></description><category></category></item><item><title><![CDATA[Numpy基础入门]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/82788370</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/82788370</guid><author>Mars_sock</author><pubDate>Thu, 20 Sep 2018 15:38:55 +0800</pubDate><description><![CDATA[文章目录Numpy基础入门Numpy创建数组Numpy查看数组属性数组元素个数数组形状数组维度数组元素类型快速创建N维数组Numpy的ndarray：多维数组对象创建随机数组np.randomNumpy计算条件运算统计运算指定轴最大值amax（参数1：数组；参数2：axis=0/1;0表示列1表示行）指定轴最小值amin指定轴平均值mean方差std数组运算数组与数的运算矩阵运算np.dot()
...]]></description><category></category></item><item><title><![CDATA[9、Python字典]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/82746806</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/82746806</guid><author>Mars_sock</author><pubDate>Mon, 17 Sep 2018 18:16:10 +0800</pubDate><description><![CDATA[

Python字典
访问字典里的值
修改字典
删除字典元素
字典键的特性
字典内置函数&amp;amp;amp;方法









Python字典


字典是另一种可变容器模型，且可存储任意类型对象。
字典的每个键值（key=&amp;amp;gt;value）对用冒号（:）分割，每个对之间用逗号（,）分割，整个字典包括在花括号（{}）中，格式如下所示： 




d = {key1 : value1, key2 : v...]]></description><category></category></item><item><title><![CDATA[8、Python元组]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/82746401</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/82746401</guid><author>Mars_sock</author><pubDate>Mon, 17 Sep 2018 17:47:20 +0800</pubDate><description><![CDATA[

Python元组
访问元组
修改元组
删除元组
元组运算符



  Python 表达式
  结果
  描述



  len((1, 2, 3))
  3
  计算元素个数


  (1, 2, 3) + (4, 5, 6)
  (1, 2, 3, 4, 5, 6)
  连接


  [‘Hi!’] * 4
  [‘Hi!’, ‘Hi!’, ‘Hi!’, ‘Hi!’]
  复制


  3...]]></description><category></category></item><item><title><![CDATA[7、Python列表]]></title><link>https://blog.youkuaiyun.com/Mars_sock/article/details/82746372</link><guid>https://blog.youkuaiyun.com/Mars_sock/article/details/82746372</guid><author>Mars_sock</author><pubDate>Mon, 17 Sep 2018 17:45:58 +0800</pubDate><description><![CDATA[Python列表


Python包括了大量的复合数据类型，用于组织其他数值。最有用的是列表，即写在方括号之间、用逗号分隔开的数值列表。列表内的项目不必全是相同的类型。




&amp;amp;gt;&amp;amp;gt;&amp;amp;gt; a = ['spam', 'eggs', 100, 1234]
&amp;amp;gt;&amp;amp;gt;&amp;amp;gt; a
['spam', 'eggs', 100, 1234]
&amp;amp;gt;&amp;amp;gt;&amp;am]]></description><category></category></item></channel></rss>