CM5.11.0安装笔记

最新推荐文章于 2024-12-03 11:37:09 发布

原创最新推荐文章于 2024-12-03 11:37:09 发布 · 3.9k 阅读

2 ·

CC 4.0 BY-SA版权

文章标签：

#cdh #cm5-11-0

hadoop 专栏收录该内容

24 篇文章

订阅专栏

1.配置网络

[root@server1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0 

DEVICE=eth0
HWADDR=00:0C:29:2b:f8:e0
TYPE=Ethernet
UUID=c2618c28-e1b5-46da-9124-2af371aa5df1
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.80.165
NETMASK=255.255.255.0
GATEWAY=192.168.80.2

2.安装jdk
略
3.配置hosts


[root@server1 ~]# vi /etc/hosts

192.168.80.165 server1
192.168.80.166 server2
192.168.80.167 server3

4.关闭防火墙和selinux

chkconfig iptables off

[root@server1 ~]# vi /etc/selinux/config 
SELINUX=disabled

5.ssh免密码

[root@server1 ~]# ssh-keygen -t rsa
每台都要执行

每台机器运行命令
ssh-copy-id server1
ssh-copy-id server2
ssh-copy-id server3

6.安装ntp

设置开机启动
yum install –y ntp
chkconfig ntpd on
设置时间同步

7.配置安装mysql

[root@server1 mysource]# yum install -y mysql mysql-server mysql-devel

[root@server1 mysource]# chkconfig mysqld on

[root@server1 mysource] # service mysqld start

[root@server1 mysource] # mysql

mysql> use mysql;

mysql> grant all privileges on *.* to 'root'@'%' identified by '111111' with grant option;
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> delete from user where host !='%';

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

8 安装依赖包 (3台机器都要装)

yum install -y chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb

9.安装cm server和angent

上传cm 安装包和 cdh的文件到server1的 /usr/java/mysource目录

其他台机器也要创建 mkdir /usr/java/mysource 目录

并且把cm安装包上传到该目录

10.建立目录(默认的目录) 三台都要创建并解压

mkdir /opt/cloudera-manager
tar xzvf cloudera-manager*.tar.gz -C /opt/cloudera-manager

11.创建用户 cloudera-scm

[root@server1 mysource]# useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

12.配置cm agent

修改文件 /opt/cloudera-manager/cm-5.11.0/etc/cloudera-scm-agent/config.ini
修改: server_host=server1 (指定server是server1 3台都改)

13.拷贝mysql的驱动包到 /usr/share/java (名称改为mysql-connector-java.jar,没有文件夹就创建)
14.Mysql 创建用户

mysql> grant all on *.* to 'cmmanager'@'%' identified by 'cmmanager' with grant option;
mysql> flush privileges;

15.执行脚本(只在server1)

切换目录: [root@server1 java]# cd /opt/cloudera-manager/cm-5.11.0/share/cmf/schema/
执行:
[root@server1 schema]# ./scm_prepare_database.sh mysql cmmanager -h server1 -ucmmanager -pcmmanager --scm-host server1 scm scm scm

16.创建parcel目录:(注意如果不赋予权限安装cm不会出现本地源的parcel)

server: 
[root@server1 schema]# mkdir -p /opt/cloudera/parcel-repo
[root@server1 schema]# chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/*

agent:
[root@server1 schema]# mkdir -p /opt/cloudera/parcels
[root@server1 schema]# chown cloudera-scm:cloudera-scm /opt/cloudera/parcels/*

17.copy parcel文件

[root@server1 parcel-repo]# cp /usr/java/mysource/CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel* ./
[root@server1 parcel-repo]# cp /usr/java/mysource/manifest.json ./

18.启动cm

三台机器都切换到: cd /opt/cloudera-manager/cm-5.11.0/etc/init.d/
启动server: [root@server1 init.d]# ./cloudera-scm-server start
启动agent(3台都执行): ./cloudera-scm-agent start

[root@server1 init.d]# ./cloudera-scm-server start
Starting cloudera-scm-server:                              [  OK  ]
[root@server1 init.d]# ./cloudera-scm-agent start
Starting cloudera-scm-agent:                               [  OK  ]
[root@server1 init.d]#

19.观察log:

/opt/cloudera-manager/cm-5.11.0/log
执行成功:log为:
2017-05-24 21:13:21,335 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

20.访问cm

http://192.168.80.165:7180/cmf/login

这里写图片描述

22.登录

账号密码默认admin

这里写图片描述

选择所有主机,下一步:

这里写图片描述

没有本地的5.11的源,查看原来是
这里写图片描述

所有节点执行: chown cloudera-scm:cloudera-scm /opt/cloudera/*
然后重启server和agent,然后在访问cm的web界面重复上面的操作

不知道为啥… 还是没有cm5.11的源…..

注意:

后面终于发现,原来是CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel.sha1文件的后缀应为sha,修改了CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel.sha1为CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel.sha 然后又重启服务,发现cm5.11 终于出来了!!!然后点击下一步

这里写图片描述

执行:
[root@server1 vm]# echo 10 > /proc/sys/vm/swappiness
[root@server1 vm]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@server1 vm]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

vi /etc/rc.local 并保存

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

自定义安装(只安装hdfs yarn zk):
这里写图片描述