<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[sanjiang521的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/sanjiang521</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; sanjiang521]]></copyright><item><title><![CDATA[mongoDB安装与配置]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/114825172</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/114825172</guid><author>sanjiang521</author><pubDate>Mon, 15 Mar 2021 11:48:44 +0800</pubDate><description><![CDATA[mongoDB安装与配置配置yum源通过yum进行安装配置开启启动验证查看版本和启动状态启动
配置yum源
[hadoop@hadoop101 ~]$ vi /etc/yum.repos.d/mongodb.repo

[MongoDB]
name=MongoDB Repository
baseurl=http://mirrors.aliyun.com/mongodb/yum/redhat/7Server/mongodb-org/4.0/x86_64/
gpgcheck=0
enabled=1

通过yum]]></description><category></category></item><item><title><![CDATA[hadoop无法启动namenode解决方法]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/114363321</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/114363321</guid><author>sanjiang521</author><pubDate>Thu, 04 Mar 2021 10:56:14 +0800</pubDate><description><![CDATA[hadoop无法启动namenode解决方法
删除Hadoop下的/logs和/tmp后格式化namenode

]]></description><category></category></item><item><title><![CDATA[docker启动tomcat容器访问端口显示404]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/114005743</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/114005743</guid><author>sanjiang521</author><pubDate>Wed, 24 Feb 2021 00:02:35 +0800</pubDate><description><![CDATA[docker启动tomcat容器访问端口显示404

        
        
                
启动Tomcat容器 
docker run -d -p 8081:8080 tomcat
 
查看已经启动的容器 
docker ps
 
使用8081端口访问Tomcat失败 先查看防火墙状况 
service firewalld status
 
出现下图所示代码，表示已经关闭防火墙 如果防火墙没关闭，先关闭防火墙 
# 关闭防火墙
systemctl stop firewall]]></description><category></category></item><item><title><![CDATA[Ubuntu数据仓库环境部署(docker、tomcat、jdk、mysql、hadoop、hive、sqoop)]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/114004982</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/114004982</guid><author>sanjiang521</author><pubDate>Tue, 23 Feb 2021 23:47:31 +0800</pubDate><description><![CDATA[Ubuntu数据仓库环境部署一、Docker安装（本文主要讲Ubuntu环境）1.1 Centos Docker安装1.2 Ubuntu Docker安装【推荐】1.3 MacOs Docker安装1.4 Windows Docker安装【不推荐】二、容器准备2.1 拉取镜像2.2 启动并创建容器2.3 进入容器三、环境准备3.1 安装必要软件3.2 配置SSH免密登录3.3 设置时区3.4 关闭防火墙3.5 安装包准备四、MySQL安装4.1 上传解压安装包4.2 安装必要依赖4.3 安装服务端和客户端4]]></description><category></category></item><item><title><![CDATA[MobaXterm连接Ubuntu]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/114004929</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/114004929</guid><author>sanjiang521</author><pubDate>Tue, 23 Feb 2021 22:51:31 +0800</pubDate><description><![CDATA[（1）更新源
sudo apt-get update

（2）安装openssh-server
sudo apt-get install openssh-server

可能出现以下错误

解决方法如下：
sudo dpkg-divert --local --rename --add /sbin/initctl Adding ‘local diversion of /sbin/initctl to /sbin/initctl.distrib’
sudo ln -s /bin/true /sbin/initc]]></description><category></category></item><item><title><![CDATA[VMware Workstation15.5安装ubuntu-16.04.6]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/114001454</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/114001454</guid><author>sanjiang521</author><pubDate>Tue, 23 Feb 2021 22:41:29 +0800</pubDate><description><![CDATA[VMware Workstation15.5安装ubuntu-16.04.6安装包新建虚拟机编辑虚拟机设置虚拟机上安装Ubuntu系统
安装包
ubuntu-16.04.6
提取码: rtym
新建虚拟机
打开VMware→文件→新建虚拟机→默认下一步至选择客户机操作系统为Ubuntu64位

继续下一步根据需求选择直至新建完成
编辑虚拟机设置

选择Ubuntu的镜像文件

虚拟机上安装Ubuntu系统
开启虚拟机

选择中文简体，并安装

第一个选项可选可不选，第二个不选

选择“清除整个磁盘并安装Ub]]></description><category></category></item><item><title><![CDATA[hive执行sql报错：FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLoca]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/113280998</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/113280998</guid><author>sanjiang521</author><pubDate>Wed, 27 Jan 2021 23:00:58 +0800</pubDate><description><![CDATA[hive执行sql报错：FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask

解决措施：
hive&gt; set hive.auto.convert.join=false;关闭自动mapjoin

]]></description><category></category></item><item><title><![CDATA[启动Hbase shell后执行list命令报错： Master is initializing]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/112301335</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/112301335</guid><author>sanjiang521</author><pubDate>Thu, 07 Jan 2021 01:21:05 +0800</pubDate><description><![CDATA[启动Hbase shell后执行list命令报错： Master is initializing

        
                
                    &nbsp;问题出现原因： 
可能由于集群的机器时间不同步时，盲目的开启hbase造成的。 
此外，集群机器时间不同步，还会导致集群开启异常，比如，开启的节点个数变少以及Hmaster节点启动一段时间后自动关闭等问题，解决方法如下。 
 
 hbase(main):001:0&gt; list_namespace NA]]></description><category></category></item><item><title><![CDATA[Hbase伪分布式安装]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/112300118</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/112300118</guid><author>sanjiang521</author><pubDate>Thu, 07 Jan 2021 00:39:17 +0800</pubDate><description><![CDATA[Hbase伪分布式安装安装与配置修改Hbase配置文件
#安装包
hbase-1.2.0-cdh5.14.2.tar
提取码: bukx
安装与配置
解压到指定路径
[root@hadoop30 software]# tar -zvxf hbase-1.2.0-cdh5.14.2.tar.gz -C /opt/

对hbase重命名
[root@hadoop30 opt]# mv hbase-1.2.0-cdh5.14.2/ hbase120/

配置环境变量
vim /etc/profile

expor]]></description><category></category></item><item><title><![CDATA[基于Spark GraphX的图形数据分析(一)]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109997850</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109997850</guid><author>sanjiang521</author><pubDate>Wed, 25 Nov 2020 08:57:39 +0800</pubDate><description><![CDATA[基于Spark GraphX的图形数据分析（一）
为什么需要图计算图（Graph）的基本概念图的术语顶点（Vertex）边（Edge）有向图无向图有环图无环图度图的经典表示法邻接矩阵Spark GraphX 简介GraphX核心抽象GraphX API创建Graph属性图应用示例构建用户合作关系属性图构建用户社交网络关系查看图信息
)
为什么需要图计算

许多大数据以大规模图或网络的形式呈现
许多非图结构的大数据，常会被转换为图模型进行分析
图数据结构很好地表达了数据之间的关联性

图（Graph）的基本概]]></description><category></category></item><item><title><![CDATA[spark清洗过滤json格式]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109968332</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109968332</guid><author>sanjiang521</author><pubDate>Mon, 23 Nov 2020 00:10:02 +0800</pubDate><description><![CDATA[spark清洗过滤json格式
待处理数据内容如下

将该文件上传至hdfs
[root@hadoop30 kb09file]# hdfs dfs -put /opt/kb09file/op.log /kb09file


验证是否上传成功
[root@hadoop30 kb09file]# hdfs dfs -ls /kb09file


启动spark-shell
[root@hadoop30 kb09file]# spark-shell

从hdfs查看op.log文件内容
sc.textFile("]]></description><category></category></item><item><title><![CDATA[Exception in thread “main“ java.lang.RuntimeException: java.net.ConnectException: Call From hadoop30]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109737176</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109737176</guid><author>sanjiang521</author><pubDate>Tue, 17 Nov 2020 10:25:23 +0800</pubDate><description><![CDATA[Exception in thread "main" java.lang.RuntimeException: java.net.ConnectException: Call From hadoop30/192.168.29.30 to hadoop30:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org]]></description><category></category></item><item><title><![CDATA[cannot access /opt/spark/lib/spark-assembly-*.jar: No such file or directory]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109736887</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109736887</guid><author>sanjiang521</author><pubDate>Tue, 17 Nov 2020 10:19:10 +0800</pubDate><description><![CDATA[cannot access /opt/spark/lib/spark-assembly-*.jar: No such file or directory问题叙述原因解决办法
问题叙述
安装好spark后，启动hive报错，内容如下：

原因
spark版本升级到2.x以后,原有lib目录下的大JAR包被分散成多个小JAR包，原来的spark-assembly-*.jar已经不存在，所以hive没有办法找到这个JAR包。
解决办法
打开hive下面的bin目录,找到hive文件,编辑hive文件,找到如下一行]]></description><category></category></item><item><title><![CDATA[Linux下scala与spark安装与配置]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109387785</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109387785</guid><author>sanjiang521</author><pubDate>Thu, 05 Nov 2020 20:13:21 +0800</pubDate><description><![CDATA[spark安装与配置安装包具体安装与配置
安装包
scala-2.11.12
提取码: yrrt
spark-2.4.5-bin-hadoop2.6
提取码: sn75
具体安装与配置
新建一个
将安装包解压到/opt/下

]]></description><category></category></item><item><title><![CDATA[Scala集合函数方法（三）]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109363272</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109363272</guid><author>sanjiang521</author><pubDate>Thu, 29 Oct 2020 17:28:32 +0800</pubDate><description><![CDATA[Scala集合函数方法（三）函数名具体函数方法
函数名




slice			    
sliding			
sortBy			    
sortWith		    
sorted			    
span			    




splitAt			
startsWith		    
stringPrefix	    
sum				
tail			    
tails			    


take			    
takeRight		    
takeWhile		    
to				    
toArr]]></description><category></category></item><item><title><![CDATA[idea的jar包导入]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109362808</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109362808</guid><author>sanjiang521</author><pubDate>Thu, 29 Oct 2020 16:20:59 +0800</pubDate><description><![CDATA[idea的jar包导入
File–&gt;Project Structure

Modules–&gt;Dependencies–&gt;+选择想导入的jar包即可


]]></description><category></category></item><item><title><![CDATA[Scala集合函数方法（二）]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109332254</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109332254</guid><author>sanjiang521</author><pubDate>Wed, 28 Oct 2020 17:38:30 +0800</pubDate><description><![CDATA[Scala集合函数方法（二）函数名具体函数方法
函数名




indexOfSlice		
indexWhere			
indices			
init				
inits				
intersect			




isDefinedAt		
isEmpty				
isTraversableAgain	
iterator			
last				
lastIndexOf		


lastIndexOfSlice	
lastIndexWhere		
lastOption			
length				
lengt]]></description><category></category></item><item><title><![CDATA[Scala集合函数方法（一）]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109318593</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109318593</guid><author>sanjiang521</author><pubDate>Wed, 28 Oct 2020 00:44:51 +0800</pubDate><description><![CDATA[Scala函数（一）函数具体用法




++
++:
+:
:+
/:
:\ 




addString
aggregate
andThen
apply
applyOrElse
array


canEqual
charAt
clone
collect
collectFirst
combinations


companion
compose
contains
containsSlice
copyToArray
copyToBuffer


corresponds
count
deep
diff
dis]]></description><category></category></item><item><title><![CDATA[Scala函数之部分应用函数与偏函数]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109292430</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109292430</guid><author>sanjiang521</author><pubDate>Mon, 26 Oct 2020 17:33:57 +0800</pubDate><description><![CDATA[Scala函数之部分应用函数与偏函数部分应用函数偏函数
部分应用函数
部分应用函数（Partial Applied Function)是缺少部分参数的函数。
例：
object Function{
def main(args:Array[String]):Unit={
def add(a:Int,b:Int,c:Int):Int={
a+b+c
		}
def sumadd=add(1,_:Int,_:Int)		
println(sumadd(2,3))		
	}
}

运行结果
6

偏函数

偏函数是]]></description><category></category></item><item><title><![CDATA[Scala模式匹配与隐式参数]]></title><link>https://blog.youkuaiyun.com/sanjiang521/article/details/109222619</link><guid>https://blog.youkuaiyun.com/sanjiang521/article/details/109222619</guid><author>sanjiang521</author><pubDate>Thu, 22 Oct 2020 16:32:38 +0800</pubDate><description><![CDATA[Scala模式匹配模式匹配的基础语法（案例：成绩评价）在模式匹配中使用if守卫在模式匹配中进行变量赋值对类型进行模式匹配（案例：异常处理）对Array和List的元素进行模式匹配（案例：对朋友打招呼）case class与模式匹配（案例：学校门禁）Option与模式匹配（案例：成绩查询）
Scala的模式匹配类似于Java中的swich case语法，即对一个值进行条件判断，然后针对不同的条件，进行不同的处理。
但是Scala的模式匹配的功能比Java的swich case语法的功能要强大地多，Java的s]]></description><category></category></item></channel></rss>