centos7下的 impala安装及与hive集成

1.环境：

      1.首先得先装好的环境：centos7, hadoop2.7， jdk1.8 ，hive ,mysql；

      2. 三台机子名称映射为：

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

127.0.0.1 localhost
192.168.174.132 master
192.168.174.133 slave2
192.168.174.134 slave1

      3. 下载该地址下的所有impala包，并将它传到环境下(存放目录看个人)，若环境有差异，在这个网址找下适合的包：

            http://archive.cloudera.com/beta/impala-kudu/redhat/7/x86_64/impala-kudu/0/RPMS/x86_64/ 

 2. 下载安装依赖包(这个我没装，同步时间的)：yum -y install cyrus-sasl-plain lsb ntp

3. 安装impala rpm包(刚下的),注意主从机：

       3.1.安装 bigtop-utils (主从机都要安装)

rpm -ivh bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el7.noarch.rpm

      3.2.安装 impala-kudu-2.7.0 (主从机都要安装)

 rpm -ivh impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm --nodeps

      3.3.安装 impala-kudu-catalog (主机安装)

rpm -ivh impala-kudu-catalog-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm

      3.4.安装 impala-kudu-state (主机安装)

 rpm -ivh impala-kudu-state-store-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm

     3.5.主机安装 impala-kudu-server (主从机都要安装)

rpm -ivh impala-kudu-server-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm

     3.6.安装impala-kudu-shell (主从机都要安装)

rpm -ivh impala-kudu-shell-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm

      3.7.安装 impala-kudu-udf-devel(主从机都要安装)

rpm -ivh impala-kudu-udf-devel-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm

4. impala配置，主从机操作，可在一台机子改好后，将修改的文件scp过去（一定别忘）：

        4.1. 添加hadoop安装目录下的core-site.xml,hdfs.xml 和 hive的hive-site.xml 到/etc/impala/conf (目录可能稍有不同),如下：

[root@slave1 conf]# cd /etc/impala/conf
[root@slave1 conf]# ls
core-site.xml  hdfs-site.xml  hive-site.xml
[root@slave1 conf]# pwd
/etc/impala/conf
[root@slave1 conf]# 

        4.2. 修改文件 /etc/default/bigtop-utils ，新增java_home路径；

        4.3. 修改文件 /etc/default/impala，只需修改前两行，改为主节点的ip地址或者hostname, 若/etc/hosts文件配置了 127.0.0.1 localhost ，也可不做修改，如下：

IMPALA_CATALOG_SERVICE_HOST=192.168.174.132
IMPALA_STATE_STORE_HOST=192.168.174.132
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/log/impala

IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} "
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}"
IMPALA_SERVER_ARGS=" \
    -log_dir=${IMPALA_LOG_DIR} \
    -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
    -state_store_port=${IMPALA_STATE_STORE_PORT} \
    -use_statestore \
    -state_store_host=${IMPALA_STATE_STORE_HOST} \
    -be_port=${IMPALA_BACKEND_PORT}"

ENABLE_CORE_DUMPS=false

        4.4 时间同步配置；

        4.5.  修改core-site.xml，新增以下几项:

<property>
        <name>dfs.client.read.shortcircuit</name>
        <value>true</value>
</property>
<property>
        <name>dfs.client.read.shortcircuit.skip.checksum</name>
        <value>false</value>
</property>
<property>
        <name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
        <value>true</value>
</property>

        4.6  修改hdfs-site.xml，新增以下几项:

<!--impala configuration -->
<property>
        <name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
        <value>true</value>
</property>
<property>
        <name>dfs.block.local-path-access.user</name>
        <value>impala</value>
</property>
<property>
        <name>dfs.client.file-block-storage-locations.timeout.millis</name>
        <value>60000</value>
</property>

        4.7 权限配置(主从机操作)：

sermod -G hdfs,hadoop impala
groups impala

         4.8 创建impala在hdfs目录，赋予权限(单节点即可)：

[root@master run]# hdfs dfs -mkdir /user/impala
[root@master run]#  hadoop fs -chown impala /user/impala

5. 启动impala 之前，先启动hadoop ,hiveserver2的服务(若配置了，否则启动hiveserver服务)：

# slave1启动hiveserver2服务：
nohup  hive --service hiveserver2 2>&1 >> /var/log.log &

可验证是否成功：
beeline  -u  jdbc:hive2://slave1:10000  -n  root

  启动impala服务,  主机节点即可，从机可以不启动impala-server服务,所示的ip为刚才配置文件所配的ip或者为ip对应的 hostname，未修改则为127.0.0.1：

[root@master run]# service impala-state-store restart --kudu_master_hosts=192.168.174.132:7051
Stopped Impala State Store Server:                         [  确定  ]
Started Impala State Store Server (statestored):           [  确定  ]
[root@master run]# service impala-catalog restart --kudu_master_hosts=192.168.174.132:7051
Stopped Impala Catalog Server:                             [  确定  ]
Started Impala Catalog Server (catalogd) :                 [  确定  ]
[root@master run]# service impala-server restart --kudu_master_hosts=192.168.174.132:7051
Stopped Impala Server:                                     [  确定  ]
Started Impala Server (impalad):                           [  确定  ]
[root@master run]# 

6. 可在从节点进行验证：

root@slave1 run]# impala-shell -i master
Starting Impala Shell without Kerberos authentication
Connected to master:21000
Server version: impalad version 2.7.0-cdh5-IMPALA_KUDU-cdh5 RELEASE (build 48f1ad385382cd90dbaed53b174965251d91d088)
***********************************************************************************
Welcome to the Impala shell. Copyright (c) 2015 Cloudera, Inc. All rights reserved.
(Impala Shell v2.7.0-cdh5-IMPALA_KUDU-cdh5 (48f1ad3) built on Thu Aug 18 12:15:44 PDT 2016)

The HISTORY command lists all shell commands in chronological order.
***********************************************************************************
[master:21000] > show databases;
Query: show databases
+------------------+----------------------------------------------+
| name             | comment                                      |
+------------------+----------------------------------------------+
| _impala_builtins | System database for Impala builtin functions |
| default          | Default Hive database                        |
| emc_pub          | EMC_PUB                                      |
| test             |                                              |
| udfs             |                                              |
+------------------+----------------------------------------------+
Fetched 5 row(s) in 0.04s
[master:21000] >