impala HA

最新推荐文章于 2024-03-13 08:00:00 发布

空中的鱼1987

最新推荐文章于 2024-03-13 08:00:00 发布

阅读量1.5k

点赞数

CC 4.0 BY-SA版权

分类专栏： imapla haproxy HA

本文链接：https://blog.youkuaiyun.com/lookqlp/article/details/52096329

imapla 同时被 3 个专栏收录

1 篇文章

订阅专栏

haproxy

1 篇文章

订阅专栏

1 篇文章

订阅专栏

本文介绍如何通过配置HAProxy为Impala JDBC提供统一接口，包括HAProxy的安装、配置文件调整、启动与关闭方法，以及Impala的相关配置。同时分享了一个实际案例，即在配置完成后impala-shell无法正常工作的问题解决办法。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

目的：
为impala jdbc提供统一的接口，作用参照http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/impala_proxy.html

步骤：
安装haproxy
选择一台非impalad的机器安装haproxy
yum install haproxy
编辑/etc/haproxy/haproxy.cfg，参考
global
    # To have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events. This is done
    # by adding the '-r' option to the SYSLOGD_OPTIONS in
    # /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    # file. A line like the following can be added to
    # /etc/sysconfig/syslog
    #
    # local2.* /var/log/haproxy.log
    #
    log 127.0.0.1 local0
    log 127.0.0.1 local1 notice
    chroot /var/lib/haproxy
    pidfile /var/run/haproxy.pid
    maxconn 4000
    user haproxy
    group haproxy
    daemon

    # turn on stats unix socket
    #stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#
# You might need to adjust timing values to prevent timeouts.
#---------------------------------------------------------------------
defaults
    mode http
    log global
    option httplog
    option dontlognull
    option http-server-close
    #option forwardfor except 127.0.0.0/8
    option redispatch
    retries 3
    maxconn 3000
    timeout connect 5000
    timeout client 50000
    timeout server 50000

#
# This sets up the admin page for HA Proxy at port 25002.
#
listen stats :25002
    balance
    mode http
    stats enable
    stats auth username:password

# This is the setup for Impala. Impala client connect to load_balancer_host:25003.
# HAProxy will balance connections among the list of servers listed below.
# The list of Impalad is listening at port 21000 for beeswax (impala-shell) or original ODBC driver.
# For JDBC or ODBC version 2.x driver, use port 21050 instead of 21000.
listen impala :25003
    mode tcp
    option tcplog
    balance leastconn
    server cdhslave1 cdhslave1.yeahmobi.com:21050 check
    server cdhslave2 cdhslave2.yeahmobi.com:21050 check
    server cdhslave3 cdhslave3.yeahmobi.com:21050 check

启动haproxy
haproxy -f /etc/haproxy/haproxy.cfg
关闭service haproxy stop
impala配置
impala daemon group->advanced->Impala Daemon Command Line Argument Advanced Configuration Snippet (Safety Valve)
-principal=impala/cdhmaster.yeahmobi.com@YEAHMOBI.COM
-be_principal=impala/_HOST@YEAHMOBI.COM
测试
private static final String IMPALAD_HOST = "cdhmaster.yeahmobi.com";//haproxy server的hostname
private static final String IMPALAD_JDBC_PORT = "25003";//端口选择haproxy的代理端口
private static final String CONNECTION_URL = "jdbc:hive2://" + IMPALAD_HOST + ':' + IMPALAD_JDBC_PORT + "/data_system;user=hive;password=111111";//ldap用户及密码

使用时发现，impala-shell不好使了，不能正常连接，去除如上粗体步骤后，impala-shell正常，jdbc HAproxy也正常使用。