伪分布搭建

最新推荐文章于 2024-08-02 11:10:24 发布

原创最新推荐文章于 2024-08-02 11:10:24 发布

· 287 阅读

2 ·

版权

1.将jdk上传解压到linux目录下

tar -zxvf jdk -C /usr/local

2.配置jdk文件


vi /etc/profile
JAVA_HOME=/usr/local/jdk1.8.0_192
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME PATH
刷新 :source /etc/profile

3.安装hdfs

tar -zxvf hadoop.2.8.5.tar.gz -C /usr/local

4.将hadoop-env.sh 中的JAVA_HOME路径更改

vi /usr/local/hadoop-2.8.5/etc/hadoop/hadoop-env.sh

export JAVA_HOME=/usr/local/jdk1.8.0_192

5.更改/usr/local/hadoop-2.8.5/etc/hadoop下core-site.xml中的配置

vi core-site.xml

此标签

加入


<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://hdp1/</value>
</property>
</configuration>

6.更改/usr/local/hadoop-2.8.5/etc/hadoop下hdfs-site.xml中的配置

vi hdfs-site.xml

此标签


<configuration>

<property>
<name>dfs.name.dir</name>
<value>/usr/local/hdpdata/name</value>
</property>

<property>
<name>dfs.data.dir</name>
<value>/usr/local/hdpdata/data</value>
</property>



<property>
  <name>dfs.namenode.checkpoint.dir</name>
  <value>/usr/local/hdpdata/scondary</value>
</property>

</configuration>

7.修改/usr/local/hadoop-2.8.5/etc/hadoop下slaves

vi slaves

写入

hdp?

8.配置 vi /etc/profile

JAVA_HOME=/usr/local/jdk1.8.0_192
HADOOP_HOME=/usr/local/hadoop-2.8.5
PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
export JAVA_HOME HADOOP_HOME PATH

刷新 :source /etc/profile

9.修改网络映射

vi /etc/hosts
192.168.56.101 hdp1

10./usr/local/hadoop-2.8.5/etc/hadoop格式化

hdfs namenode -format

免密登陆
ssh-keygen -t rsa(回车3次)
ssh-copy-id 192.168.56.10?(需要免密的IP)

11.启动 start-dfs.sh

在这里插入图片描述

安装伪分布模式的yarn

1.在/usr/local/hadoop-2.8.5/etc/hadoop目录下

	vi yarn-site.xml

<configuration>
<property>
  <name>yarn.resourcemanager.hostname</name>
  <value>hdp4</value>
</property>

<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>

<property>
  <name>yarn.nodemanager.resource.memory-mb</name>
  <value>4096</value>
</property>

<property>
  <name>yarn.nodemanager.resource.cpu-vcores</name>
  <value>2</value>
</property>

</configuration>

	cp mapred-site.xml.template mapred-site.xml
	vi mapred-site.xml

<configuration>

<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>

</configuration>

2.启动yarn

	start-yarn.sh