Impala学习笔记（一）CDH5.4.0安装

Impala是Cloudra公司发布的实时查询开源项目，基于Hive但使用内存进行计算，是使用CDH的首选PB级大数据实时查询分析引擎。


Impala的安装方式有两种，CM方式和手动安装，手动安装比较麻烦，需要额外的步骤验证与其他组件的交互，这里采用CM的方式安装。

1、安装准备
MySQL5.6.17版本
[code="sql"]
# rpm -ivh MySQL-server-5.6.17-1.el6.x86_64.rpm
[/code]
[img]http://dl2.iteye.com/upload/attachment/0114/4363/7c3ca960-a10f-3e0d-abbb-d30de7050e2d.jpg[/img]

由于版本冲突，卸载之前的MySQL5.1.71版本
[code="java"]
# yum -y remove mysql-libs-5.1.71*
[/code]
再次安装MySQL就可以了。
[code="java"]
# rpm -ivh MySQL-server-5.6.17-1.el6.x86_64.rpm
# rpm -ivh MySQL-client-5.6.17-1.el6.x86_64.rpm
[/code]
[table]
|目录|说明|
|/usr/bin|Client programs and scripts|
|/usr/sbin|The mysqld server|
|/var/lib/mysq|数据文件|
|/usr/my.cnf|配置文件|
[/table]

2、下载CM和CDH
CM当前最新版本为5.5.1，这里安装CM5.4.0和CDH5.4.0。

[url]http://www.cloudera.com/content/www/en-us/downloads.html[/url]

在该网站能够找到CM和CDH的下载链接，下载时选定了相应的版本后，会有安装的帮助信息。


安装CM可以使用在线安装和离线安装两种方式，在线安装也需要将基础安装包下载到本地再进行安装，安装过程中自动下载需要安装的软件，但对于带宽要求较高，由于国内的网络环境下载境外的资源时容易受到限制，影响安装速度，可以通过在本地配置yum源进行安装。

[url]http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.4.0/[/url]

实际需要使用的文件为RPMS/x86_64目录下的文件：
[img]http://dl2.iteye.com/upload/attachment/0114/4365/a8c79f01-a784-3291-965b-98909a70b9e8.jpg[/img]

[url]http://archive.cloudera.com/cdh5/parcels/[/url]
[img]http://dl2.iteye.com/upload/attachment/0114/4367/b1e32de6-f587-3b4f-a74e-c6230f46fd50.jpg[/img]
说明：我的虚拟机环境为Centos6.5，这里只需要下载el6版本的即可。

3、将软件发布成web服务
在nnode节点上启动HTTP服务，CM和CDH的介质需要放在HTTP服务器的root目录下。
[code="java"]
# service httpd start
[/code]

验证Web服务：
[img]http://dl2.iteye.com/upload/attachment/0114/4369/a5ea0ba1-c7da-39a1-b041-bcf415679586.jpg[/img]

查找httpd的根目录：

[hadoop@nnode html]$ cat /etc/httpd/conf/httpd.conf |grep var/www/html
DocumentRoot "/var/www/html"
<Directory "/var/www/html">
[hadoop@nnode html]$ ll /var/www/html
total 0
[hadoop@nnode html]$

将下载的cm及cdh目录下的文件拷贝到该根目录下：
[code="java"]
# mv /lucl/Cloudera/cm5 .
# mv /lucl/Cloudera/cdh5 .
[/code]

RPM-GPG-KEY-cloudera的下载地址为：
[url]https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera[/url]

目录结构：
[code="java"]
# pwd
/var/www/html
# ll
total 12
drwxrwxr-x 3 hadoop hadoop 4096 Jan 10 04:27 cdh5
drwxrwxr-x 4 hadoop hadoop 4096 Jan 10 06:18 cm5
-rwxr-xr-x 1 root root 1690 Jan 10 04:15 RPM-GPG-KEY-cloudera
#
[/code]

cm5的目录结构为：
[code="java"]
# pwd
/var/www/html/cm5/RPMS/x86_64
# ls
cloudera-manager-agent-5.4.0-1.cm540.p0.165.el6.x86_64.rpm
cloudera-manager-daemons-5.4.0-1.cm540.p0.165.el6.x86_64.rpm
cloudera-manager-server-5.4.0-1.cm540.p0.165.el6.x86_64.rpm
cloudera-manager-server-db-2-5.4.0-1.cm540.p0.165.el6.x86_64.rpm
enterprise-debuginfo-5.4.0-1.cm540.p0.165.el6.x86_64.rpm
jdk-6u31-linux-amd64.rpm
oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
#
[/code]
[img]http://dl2.iteye.com/upload/attachment/0114/4394/83ca9a0a-fea7-3a7e-9861-4142a69d95f9.jpg[/img]

cdh5的目录结构为：
[code="java"]
# pwd
/var/www/html/cdh5/parcels/5.4.0
# ll -h
total 1.2G
-rwxrwxr-x 1 hadoop hadoop 1.2G Jan 10 02:54 CDH-5.4.0-1.cdh5.4.0.p0.27-el6.parcel
-rwxrwxr-x 1 hadoop hadoop 41 Jan 10 02:54 CDH-5.4.0-1.cdh5.4.0.p0.27-el6.parcel.sha1
-rwxrwxr-x 1 hadoop hadoop 42K Jan 10 02:54 manifest.json
#
[/code]
[img]http://dl2.iteye.com/upload/attachment/0114/4396/1fcfb019-4303-3ae7-a744-7042df88d763.jpg[/img]

分别在cm5和parcels目录下执行createrepo .命令，为rpm建立索引
[code="java"]
# createrepo .
Spawning worker 0 with 7 pkgs
Workers Finished
Gathering worker results

Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
#
# 执行后repodata目录会生成相应的文件。

# createrepo .

Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete


[/code]

4、配置yum源
通过cloudera网站下载repo资源文件
cloudera-manager的repo文件
[url]https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/cloudera-manager.repo[/url]

cloudera-cdh的repo文件
[url]https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/cloudera-cdh5.repo[/url]

说明：
[url]https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/RPM-GPG-KEY-cloudera[url]
和
[url]https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera[url]
内容实际是一样的，这里就使用了一份。

[code="java"]
# cat cloudera-cdh5.repo
[cloudera-cdh5]
# Packages for Cloudera's Distribution for Hadoop, Version 5, on RedHat or CentOS 6 x86_64
name=Cloudera's Distribution for Hadoop, Version 5
baseurl=http://nnode/cdh5/parcels/
gpgkey=http://nnode/RPM-GPG-KEY-cloudera
gpgcheck = 1

# cat cloudera-manager.repo
[cloudera-manager]
# Packages for Cloudera Manager, Version 5, on RedHat or CentOS 6 x86_64
name=Cloudera Manager
baseurl=http://nnode/cm5/
gpgkey=http://nnode/RPM-GPG-KEY-cloudera
gpgcheck = 1

#
[/code]

5、下载安装脚本
[url]http://archive.cloudera.com/cm5/installer/5.4.0/cloudera-manager-installer.bin[/url]

6、执行安装

chmod u+x cloudera-manager-installer.bin

在图形界面下执行./cloudera-manager-installer.bin

[img]http://dl2.iteye.com/upload/attachment/0114/4410/027f5379-126d-39f9-8e0d-227c40d46046.jpg[/img]

[img]http://dl2.iteye.com/upload/attachment/0114/4412/4b7c26fd-1cd8-3b95-aee8-4f315619cf06.jpg[/img]

7、访问验证
http://nnode:7180
[img]http://dl2.iteye.com/upload/attachment/0114/4414/cbfd5295-feac-3084-876a-a04c33df24fd.jpg[/img]
[img]http://dl2.iteye.com/upload/attachment/0114/4416/5cac8bee-7796-3e0b-b0b4-768594a60914.jpg[/img]