hadoop与hbase伪分布式的基本配制文件设置

最新推荐文章于 2024-12-03 13:00:47 发布

原创最新推荐文章于 2024-12-03 13:00:47 发布 · 5.8k 阅读

0 ·

CC 4.0 BY-SA版权

大数据生态系统技术同时被 2 个专栏收录

60 篇文章

订阅专栏

大数据与云计算

50 篇文章

订阅专栏

本文详细介绍了Hadoop集群的配置过程，包括core-site.xml、hdfs-site.xml等关键文件的具体设置，并深入探讨了HBase的相关配置，如hbase-site.xml的参数调整，为读者提供了实践指导。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Hadoop

0.hbase-env.sh

export JAVA_HOME=/software/jdk1.7.0_80

1.core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>file:/software/hadoop-2.6.4/tmp</value>
</property>
<property>
<name>hadoop.native.lib</name>
<value>false</value>
</property>
</configuration>

2.hdfs-site.xml

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/software/hadoop-2.6.4/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/software/hadoop-2.6.4/dfs/data</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
</configuration>

3.mapred-site.xml

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>

4.yarn-site.xml

<configuration>
<property>
<description>The hostname of the RM.</description>
<name>yarn.resourcemanager.hostname</name>
<value>master</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>

5.slaves

master

Hbase:

0.hbase-env.sh
export JAVA_HOME=/software/jdk1.7.0_80
export HBASE_CLASSPATH=/software/hadoop-2.6.4/etc/hadoop
export HBASE_MANAGES_ZK=true
export HBASE_LOG_DIR=/software/hbase-1.2.1/logs

1.hbase-site.xml

<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://master:9000/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.tmp.dir</name>
<value>file:/software/hbase-1.2.1/tmp</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>master</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>file:/software/hbase-1.2.1/zookeeper</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>