Cloudera Manager Server安装

Cloudera Manager Server安装

 

核心命令备份

Cloudera Manager Server安装

创建/opt/cloudera-manager安装目录并解压安装介质

mkdir /opt/cloudera-manager
tar xzf cloudera-manager*.tar.gz -C /opt/cloudera-manager

重新配置的话需要删除

rm -f /home/hadoop/cloudera-manager/cm-5.15.1/lib/cloudera-scm-agent/cm_guid

创建系统用户cloudera-scm

useradd --system --home=/home/hadoop/cloudera-manager/cm-5.15.1/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

验证用户是否创建成功：

cat /etc/passwd | grep cloudera-scm

 

 

grant all privileges on *.* to 'root'@'localhost' identified by 'abcd1234' with grant option;
 
grant all privileges on *.* to 'root'@'nginx' identified by 'abcd1234' with grant option;
 
grant all privileges on *.* to 'root'@'127.0.0.1' identified by 'abcd1234' with grant option;
 
grant all privileges on *.* to 'root'@'%' identified by 'abcd1234' with grant option;
 
 
grant all privileges on *.* to 'scm'@'localhost' identified by 'scm' with grant option;
 
grant all privileges on *.* to 'scm'@'hadoop1' identified by 'scm' with grant option;
 
grant all privileges on *.* to 'scm'@'127.0.0.1' identified by 'scm' with grant option;
 
grant all privileges on *.* to 'scm'@'%' identified by 'scm' with grant option;



grant all on amon.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';
grant all on rman.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';
grant all on metastore.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';
grant all on sentry.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';
grant all on nav.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';
grant all on navms.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';

create database oozie DEFAULT CHARACTER SET utf8;
grant all on oozie.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';

create database scm DEFAULT CHARACTER SET utf8;
grant all on scm.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';



 sudo /home/hadoop/cloudera-manager/cm-5.15.1/share/cmf/schema/scm_prepare_database.sh mysql scm hadoop

创建server存储目录

mkdir /var/lib/cloudera-scm-server
chown cloudera-scm:cloudera-scm /var/lib/cloudera-scm-server

创建hadoop离线安装包存储目录

mkdir -p /opt/cloudera/parcels
chown cloudera-scm:cloudera-scm /opt/cloudera/parcels

配置agent的server指向

vi /home/hadoop/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini

将server_host修改为cloudera manager server的主机名,对于本示例而言，也就是server主机ns1。

部署CDH离线安装包

mkdir -p /opt/cloudera/parcel-repo
chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo
mv CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha manifest.json /opt/cloudera/parcel-repo/

启动Cloudera Manager Server

/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-server start

启动Cloudera Manager Agent

/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-agent start
 

Cloudera Manager Agent安装

安装数据库驱动

mkdir -p /usr/share/java
cp mysql-connector-java-8.0.12.jar /usr/share/java/mysql-connector-java.jar

创建安装目录并解压安装介质

mkdir /home/hadoop/cloudera-manager
tar xzf cloudera-manager*.tar.gz -C /home/hadoop/cloudera-manager

建系统用户cloudera-scm

useradd --system --home=/home/hadoop/cloudera-manager/cm-5.15.1/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

验证用户是否创建成功：

cat /etc/passwd | grep cloudera-scm

创建hadoop离线安装包存储目录

mkdir -p /opt/cloudera/parcels
chown cloudera-scm:cloudera-scm /opt/cloudera/parcels

配置agent的server指向

vim  /home/hadoop/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini

将server_host修改为cloudera manager server的主机名,对于本示例而言，也就是server主机ns1。
启动Cloudera Manager Agent

/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-agent start

服务均显示启动成功后，才进行下一步操作。若服务没有启动成功，则需要查看相关日志文件，解决了问题之后方可进行下一步。
 

 

 

 

 

 

server的安装就只在server主机上执行：
1、上传安装包

Cloudera Manager 安装包：cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz

MySQL驱动包：mysql-connector-java-8.0.12.tar.gz

大数据离线安装库：

CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel

CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha

manifest.json
2、创建相关安装目录并解压安装介质
创建//home/hadoop/cloudera-manager安装目录并解压安装介质

    mkdir //home/hadoop/cloudera-manager
    tar xzf cloudera-manager*.tar.gz -C /opt/cloudera-manager

安装数据库驱动并初始化数据库

安装数据库驱动

    #解压数据库驱动包，得到一个mysql-connector-java-8.0.12文件夹，里面有一个mysql-connector-java-8.0.12.jar文件
    tar xzf mysql-connector-java-8.0.12.tar.gz
     
    mkdir -p /usr/share/java
    cp mysql-connector-java-8.0.12.jar s

初始化数据库
创建系统用户cloudera-scm

useradd --system --home=/opt/cloudera-manager/cm-5.15.1/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

useradd --system --home=/home/hadoop/cloudera-manager/cm-5.15.1/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

 

验证用户是否创建成功：

cat /etc/passwd | grep cloudera-scm

    #设置CM数据库的权限
    mysql -uroot -p    #进入mysql数据库
    mysql> grant all privileges on *.*  to 'root'@'%' identified by 'abcd1234' with grant option;
    mysql> grant all privileges on *.* to 'root'@'nginx' identified by 'abcd1234' with grant option;
    mysql> flush privileges;
    mysql> use  mysql;
    mysql> select host ,user from user;    
     

当运行初始化数据库脚本语句：cm-5.13.0/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -p123456 --scm-host localhost scm scm scm

之前，进入mysql：mysql -hlocalhost -uroot -p，执行一下sql语句，否则有可能报错：
---------------------  

grant all privileges on *.* to 'root'@'localhost' identified by 'abcd1234' with grant option;

grant all privileges on *.* to 'root'@'nginx' identified by 'abcd1234' with grant option;

grant all privileges on *.* to 'root'@'127.0.0.1' identified by 'abcd1234' with grant option;

grant all privileges on *.* to 'root'@'%' identified by 'abcd1234' with grant option;


grant all privileges on *.* to 'scm'@'localhost' identified by 'scm' with grant option;

grant all privileges on *.* to 'scm'@'hadoop1' identified by 'scm' with grant option;

grant all privileges on *.* to 'scm'@'127.0.0.1' identified by 'scm' with grant option;

grant all privileges on *.* to 'scm'@'%' identified by 'scm' with grant option;
 

 

 

 

grant all on amon.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';
grant all on rman.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';
grant all on metastore.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';
grant all on sentry.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';
grant all on nav.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';
grant all on navms.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';

create database oozie DEFAULT CHARACTER SET utf8;
grant all on oozie.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';

create database scm DEFAULT CHARACTER SET utf8;
grant all on scm.* TO 'hadoop'@'%' IDENTIFIED BY 'abcd1234';





 sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm hadoop

 

 

 

    #进入ns1服务器以下文件夹，执行脚本命令，事先确认已经将mysql-connector-java.jar放入/usr/share/java文件夹中
    cd /opt/cloudera-manager/cm-5.15.1/share/cmf/schema
    ./scm_prepare_database.sh mysql -h ns1 -utemp -ptemp --scm-host ns1 scm scm scm    

./scm_prepare_database.sh mysql -h nginx -utemp -ptemp --scm-host nginx scm scm scm    

创建server存储目录

    mkdir /var/lib/cloudera-scm-server
    chown cloudera-scm:cloudera-scm /var/lib/cloudera-scm-server

创建hadoop离线安装包存储目录

    mkdir -p /opt/cloudera/parcels
    chown cloudera-scm:cloudera-scm /opt/cloudera/parcels

配置agent的server指向

vi /home/hadoop/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini

将server_host修改为cloudera manager server的主机名,对于本示例而言，也就是server主机ns1。
部署CDH离线安装包

    mkdir -p /opt/cloudera/parcel-repo
    chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo
    mv CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha manifest.json /c

启动Cloudera Manager Server

/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-server start

启动Cloudera Manager Agent

/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-agent start

四、Cloudera Manager Agent安装

在除了server服务器外的其他的服务器上面执行以下步骤进行对agent的部署：
1、上传安装包

对于agent的安装我们只需要以下的两个安装介质：

Cloudera Manager 安装包：cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz

MySQL驱动包：mysql-connector-java-8.0.12.tar.gz
2、创建相关安装目录并解压安装介质
安装数据库驱动

    mkdir -p /usr/share/java
    cp mysql-connector-java-8.0.12.jar /usr/share/java/mysql-connector-java.jar

创建安装目录并解压安装介质

    mkdir /opt/cloudera-manager
    tar xzf cloudera-manager*.tar.gz -C /opt/cloudera-manager

建系统用户cloudera-scm

useradd --system --home=/opt/cloudera-manager/cm-5.15.1/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

验证用户是否创建成功：

cat /etc/passwd | grep cloudera-scm

创建hadoop离线安装包存储目录

    mkdir -p /opt/cloudera/parcels
    chown cloudera-scm:cloudera-scm /opt/cloudera/parcels

配置agent的server指向

vim /home/hadoop/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini

将server_host修改为cloudera manager server的主机名,对于本示例而言，也就是server主机ns1。
启动Cloudera Manager Agent

/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-agent start

服务均显示启动成功后，才进行下一步操作。若服务没有启动成功，则需要查看相关日志文件，解决了问题之后方可进行下一步。
五、Cloudera ManagerMent Service集群的安装

我们部署成功CDH的server和agent之后，我们的其他一切操作都在网页上进行操作。在浏览器中输入http://10.66.241.9:7180进入web页面：

用户名、密码均为admin。