Linux下安装配置Hadoop2.7.6

最新推荐文章于 2024-01-18 07:48:29 发布

weixin_34077371

最新推荐文章于 2024-01-18 07:48:29 发布

阅读量664

点赞数

CC 4.0 BY-SA版权

文章标签：大数据开发工具 java

原文链接：https://my.oschina.net/mengzhang6/blog/1831772

博客介绍了安装JDK和Hadoop的步骤，包括下载、解压和配置环境变量。通过修改配置文件，重新加载文件完成环境配置。还给出了Hadoop的两个官方案例，如使用grep和wordcount命令进行操作，帮助读者了解Hadoop在Java环境下的使用。

2019独角兽企业重金招聘Python工程师标准>>>

前提
安装jdk

下载
wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.7.6/hadoop-2.7.6.tar.gz

解压

配置

vim /etc/profile

# 配置java环境变量
export JAVA_HOME=/opt/jdk1.8.0_162
export PATH=$PATH:$JAVA_HOME/bin

# 配置hadoop环境变量
export HADOOP_HOME=/usr/local/src/hadoop-2.7.6
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin

重新加载文件
source /etc/profile

[root@localhost hadoop-2.7.6]# hadoop version
Hadoop 2.7.6
Subversion https://shv@git-wip-us.apache.org/repos/asf/hadoop.git -r 085099c66cf28be31604560c376fa282e69282b8
Compiled by kshvachk on 2018-04-18T01:33Z
Compiled with protoc 2.5.0
From source with checksum 71e2695531cb3360ab74598755d036
This command was run using /usr/local/src/hadoop-2.7.6/share/hadoop/common/hadoop-common-2.7.6.jar

vim etc/hadoop/hadoop-env.sh

将
export JAVA_HOME=${JAVA_HOME}
改为
export JAVA_HOME=/opt/jdk1.8.0_162

hadoop官方案例
http://hadoop.apache.org/docs/r2.7.6/hadoop-project-dist/hadoop-common/SingleCluster.html

mkdir input
cp etc/hadoop/*.xml input
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.6.jar grep input output 'dfs[a-z.]+'
cat output/*

hadoop官方案例2
mkdir wcinput
cd wcinput
vim wc.input

hello hello
world
go
java java
c c c
hadoop

cd ..
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.6.jar wordcount wcinput wcoutput
cat wcoutput/*

转载于:https://my.oschina.net/mengzhang6/blog/1831772