安装ganglia及监控haoop

最新推荐文章于 2019-09-11 23:07:09 发布

PenPenApple

最新推荐文章于 2019-09-11 23:07:09 发布

阅读量509

点赞数

CC 4.0 BY-SA版权

分类专栏： hadoop ganglia 文章标签： ganglia hadoop

本文链接：https://blog.youkuaiyun.com/qq_27078095/article/details/52174572

hadoop 同时被 2 个专栏收录

4 篇文章

订阅专栏

ganglia

3 篇文章

订阅专栏

本文介绍如何在Hadoop集群中部署Ganglia监控系统，包括安装和配置Apache HTTP Server、PHP、Ganglia相关组件，以及配置Ganglia监控Hadoop的NameNode和DataNode节点的具体步骤。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

ganglia的web功能需要php的支持
192.168.78.129 master
192.168.78.130 slave1

在下master安装

安装httpd和php

# yum -y install httpd php php-ZendFramework php-gd rrdtool php-xml

安装ganglia

# rpm -ivh ganglia-3.7.2-2.el7.x86_64.rpm ganglia-gmetad-3.7.2-2.el7.x86_64.rpm ganglia-gmond-3.7.2-2.el7.x86_64.rpm ganglia-web-3.7.1-2.el7.x86_64.rpm libconfuse-2.7-7.el7.x86_64.rpm libevent-2.0.21-4.el7.x86_64.rpm libmemcached-1.0.16-5.el7.x86_64.rpm

配置httpd

# cat /etc/httpd/conf.d/ganglia.conf

Alias /ganglia /usr/share/ganglia

<Location /ganglia>
  Require all granted
</Location>

配置gmentad

# cat /etc/ganglia/gmetad.conf 
data_source "my cluster" 192.168.78.129

配置gmond

# cat /etc/ganglia/gmond.conf
udp_send_channel {
  #bind_hostname = yes # Highly recommended, soon to be default.
                       # This option tells gmond to use a source address
                       # that resolves to the machine's hostname.  Without
                       # this, the metrics may appear to come from any
                       # interface and the DNS names associated with
                       # those IPs will be used to create the RRDs.
  mcast_join = 192.168.78.129
  port = 8649
  ttl = 1
}

/* You can specify as many udp_recv_channels as you like as well. */
udp_recv_channel {
#  mcast_join = 239.2.11.71
  port = 8649
#  bind = 239.2.11.71
  retry_bind = true
  # Size of the UDP buffer. If you are handling lots of metrics you really
  # should bump it up to e.g. 10MB or even higher.
  # buffer = 10485760
}

启动服务

# service httpd start
# service gmetad start
# service gmond start

在slave1下安装

安装依赖

# yum -y install apr

安装gmond

# rpm -ivh ganglia-3.7.2-2.el7.x86_64.rpm ganglia-gmond-3.7.2-2.el7.x86_64.rpm libconfuse-2.7-7.el7.x86_64.rpm

配置gmond

# cat /etc/ganglia/gmond.conf
udp_send_channel {
  #bind_hostname = yes # Highly recommended, soon to be default.
                       # This option tells gmond to use a source address
                       # that resolves to the machine's hostname.  Without
                       # this, the metrics may appear to come from any
                       # interface and the DNS names associated with
                       # those IPs will be used to create the RRDs.
  mcast_join = 192.168.78.129
  port = 8649
  ttl = 1
}

/* You can specify as many udp_recv_channels as you like as well. */
udp_recv_channel {
#  mcast_join = 239.2.11.71
  port = 8649
#  bind = 239.2.11.71
  retry_bind = true
  # Size of the UDP buffer. If you are handling lots of metrics you really
  # should bump it up to e.g. 10MB or even higher.
  # buffer = 10485760
}

启动服务

# service gmond start

查看：

192.168.78.129/ganglia

监控hadoop集群

监控namenode和datanode

$ cat hadoop-2.x/etc/hadoop/hadoop-metrics2.properties

*.sink.ganglia.class=org.apache.hadoop.metrics2.sink.ganglia.GangliaSink31
*.sink.ganglia.period=10
*.sink.ganglia.supportsparse=true
namenode.sink.ganglia.servers=192.168.78.129:8649
datanode.sink.ganglia.servers=192.168.78.129:8649

重启namenode和datanode及gmond

$ cd hadoop-2.x/
$ sbin/hadoop-daemon.sh stop namenode                             
$ sbin/hadoop-daemon.sh stop datanode
$ sbin/hadoop-daemon.sh start namenode                            
$ sbin/hadoop-daemon.sh start datanode  
# service gmond restart

查看界面