ganglia cdh 配置

hbase

# for Ganglia 3.7
*.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31  
*.sink.ganglia.period=10  
hbase.sink.ganglia.period=10  
hbase.sink.ganglia.servers=239.2.11.71:8649

hadoop

# for Ganglia 3.7
*.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31  
*.sink.ganglia.period=10
# default for supportsparse is false
*.sink.ganglia.supportsparse=true
*.sink.ganglia.slope=jvm.metrics.gcCount=zero,jvm.metrics.memHeapUsedM=both  
*.sink.ganglia.dmax=jvm.metrics.threadsBlocked=70,jvm.metrics.memHeapUsedM=40  
namenode.sink.ganglia.servers=239.2.11.71:8649  
resourcemanager.sink.ganglia.servers=239.2.11.7102:8649  
datanode.sink.ganglia.servers=239.2.11.71:8649   
nodemanager.sink.ganglia.servers=239.2.11.71:8649 
maptask.sink.ganglia.servers=239.2.11.71:8649   
reducetask.sink.ganglia.servers=239.2.11.71:8649

spark

       <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-ganglia-lgpl_2.10</artifactId>
            <version>1.6.1</version>
        </dependency>
    *.sink.ganglia.class=org.apache.spark.metrics.sink.GangliaSink 
    *.sink.ganglia.host=239.2.11.71
    *.sink.ganglia.port=5080
    *.sink.ganglia.period=10 
    *.sink.ganglia.unit=seconds 
    *.sink.ganglia.ttl=1 
    *.sink.ganglia.mode=multicast 
安装CDH6.3需要几个步骤: 1. 创建华为云ECS实例:在华为云控制台创建ECS实例,选择操作系统为CentOS 7.6 64位。 2. 安装CDH依赖包:在ECS实例中运行以下命令安装CDH依赖包。 ``` sudo yum install -y wget curl openssh-server ntp ntpdate unzip lsof net-tools bind-utils ``` 3. 安装JDK:CDH需要Java环境,可以通过以下命令安装Oracle JDK 8。 ``` sudo wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" \ http://download.oracle.com/otn-pub/java/jdk/8u131-b11/d54c1d3a095b4ff2b6607d096fa80163/jdk-8u131-linux-x64.rpm sudo yum localinstall -y jdk-8u131-linux-x64.rpm ``` 4. 安装CDH:通过以下步骤安装CDH。 - 添加CDH6仓库 ``` sudo wget https://archive.cloudera.com/cdh6/6.3.0/redhat7/yum/cloudera-cdh6.repo -P /etc/yum.repos.d/ ``` - 安装CDH ``` sudo yum install -y hadoop hadoop-hdfs-namenode hadoop-hdfs-datanode hadoop-mapreduce \ hadoop-yarn-nodemanager hadoop-yarn-resourcemanager hadoop-hdfs-journalnode hadoop-hdfs-zkfc \ hive hive-metastore hive-server2 impala impala-server impala-shell impala-catalog \ impala-state-store impala-shell solr-server solr-doc zookeeper-server \ zookeeper-devel hue hue-plugins flume-ng-agent oozie oozie-client sqoop \ sqoop2-server sqoop2-client spark-core spark-python spark-worker spark-history-server \ spark-thrift-server spark-yarn-shuffle spark-master spark-driver spark-examples \ spark-r spark-sql spark-streaming spark-ganglia-monitoring spark-ganglia-shuffle-monitoring \ spark-javadoc spark-doc python-devel python-pip ``` 5. 配置Hadoop:进行以下配置。 - 配置Hadoop ``` sudo /usr/bin/hadoop namenode -format ``` - 启动Hadoop ``` sudo systemctl start hadoop-hdfs-namenode sudo systemctl start hadoop-hdfs-datanode sudo systemctl start hadoop-yarn-resourcemanager sudo systemctl start hadoop-yarn-nodemanager sudo systemctl start hadoop-hdfs-journalnode sudo systemctl start hadoop-hdfs-zkfc ``` 6. 配置Hue:通过以下步骤配置Hue。 - 修改hue配置文件 ``` sudo cp /usr/lib/hue/desktop/conf.dist/hue.ini /usr/lib/hue/desktop/conf/hue.ini sudo sed -i 's/localhost/0.0.0.0/g' /usr/lib/hue/desktop/conf/hue.ini ``` - 启动Hue ``` sudo systemctl start hue ``` 7. 配置Impala:通过以下步骤配置Impala。 - 启动Impala ``` sudo systemctl start impala-server sudo systemctl start impala-catalog sudo systemctl start impala-state-store ``` - 配置Impala daemon ``` sudo cp /etc/impala/conf.dist/impalad_flags /etc/impala/conf/ sudo sed -i 's/--num_scan_ranges_per_disk=0/--num_scan_ranges_per_disk=1/g' /etc/impala/conf/impalad_flags sudo sed -i 's/--use_local_catalog/--use_local_catalog --enable_rm=false/g' /etc/impala/conf/impalad_flags ``` 8. 配置Oozie:通过以下步骤配置Oozie。 - 修改oozie配置文件 ``` sudo cp /usr/lib/oozie/conf.dist/oozie-site.xml /usr/lib/oozie/conf/ sudo sed -i 's/localhost/your_hostname/g' /usr/lib/oozie/conf/oozie-site.xml ``` - 启动Oozie ``` sudo systemctl start oozie ``` 9. 完成安装:至此,CDH6.3已经安装完成。 注意:以上步骤仅供参考,具体操作可能与实际情况略有不同。在实际操作中,请根据实际情况进行相应的修改。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值