qq_53827620的博客

原创 Spark的搭建及实现单词统计

安装spark-2.4.4-bin-hadoop2.7版本 匹配Hadoop2.7版本上传，解压（一）修改环境变量：vi /etc/profileexport SPARK_HOME=/opt/spark export PATH=$PATH:$SPARK_HOME/binwqsource /etc/profile（二）修改配置文件：cd /opt/spark/confcp spark-env.sh.template spark-env.shvi spark-env.shex

原创 ①hadoop的搭建配置(详细步骤)

原创 MapReduce –WordCount实现(详细步骤)

hadoop fs -mkdir /input新建a1.txt a2.txt a3.txtvi a1.txthadoop map reducevi a2.txthadoop hdfs reducevi a3.txthadoop map hadop上传到hdfs上的input目录上:hadoop fs -put a*.txt /input运行jar包:hadoop jar /opt/hadoop/share/hadoop/mapreduce/hadoop-mapre

原创 Hbase伪分布式搭建(详细步骤)

vi /etc/profilesource /etc/profilecd /opt/hbase/confvi hbase-env.shexport JAVA_HOME=/opt/jdk1.8(删除＃号)export HBASE_CLASSPATH=/opt/hadoop/etc/hadoopexport HBASE_PID_DIR=/var/hadoop/pids (删除＃号)export HBASE_MANAGES_ZK=TRUE(用自带的zookeeper)(以上代码

原创 Zookeeper搭建与启动(详细步骤)

cd /opt/zookeeper/confcp zoo_sample.cfg zoo-1.cfglsvi zoo-1.cfgdataDir=/tmp/zoo-1#theclinetPort=2181server.1=127.0.0.1:2888:3888server.2=127.0.0.1:2889:3889server.3=127.0.0.1:2890:3890cp zoo-1.cfg zoo-2.cfgvi zoo-2.cfgdataDir=/tm.