hue 安装笔记

最新推荐文章于 2023-03-20 14:12:05 发布

liutizhong

最新推荐文章于 2023-03-20 14:12:05 发布

阅读量3.9k

点赞数 1

本文链接：https://blog.youkuaiyun.com/liutizhong/article/details/48265155

版权

Hadoop MapReduce 专栏收录该内容

4 篇文章

订阅专栏

本文主要记录hue的yum源下的安装与配置，Hue 集成 Hdfs、Hive、Impala、Yarn、Kerberos、LDAP、Sentry、Solr 等

集群情况：192.168.211.178(HA-active)

192.168.211.179(datanode)

192.168.211.180(HA-standby)

192.168.211.185(datanode)

192.168.211.253(datanode)

一、安装 Hue

hue安装机器：192.168.211.179

$  yum install hue hue-server

$  yum install hadoop-httpfs

二、配置hue server

修改配置文件：hue.ini

[desktop]
secret_key=test123   #任意值，越复杂越好，用来加密用的
http_host=192.168.211.179   #安装hue的机器
http_port=8888      #默认端口，可自定义
use_cherrypy_server= True  #决定启动某个web服务
server_user=hue
server_group=hue
enable_server=yes
[hadoop]
  [[hdfs_clusters]]
    # HA support by using HttpFs 
    [[[default]]]
      fs_defaultfs=hdfs://BIService  #对应core-site.xml
      webhdfs_url=http://192.168.211.178:14000/webhdfs/v1   #对应HA的HttpFs

如果集群使用MR1提交job则配置mapred_clusters，否则配置yarn_clusters（MR2），二者选其一

[[mapred_clusters]]
    # HA support by specifying multiple configs
    [[[default]]]
    submit_to=False  #标明不使用MR1提交job

[[yarn_clusters]]
    [[[default]]]
      # Enter the host on which you are runningthe ResourceManager
      #对应yarn-site.xml中的 yarn.resourcemanager.address
      resourcemanager_host=192.168.211.178
      resourcemanager_port=18040 

      # Whether to submit jobs to this cluster
       submit_to=True   #标明使用MR1提交job

      # Defaults to $HADOOP_CONF_DIR or/etc/hadoop/conf
       hadoop_conf_dir=/etc/hadoop/conf

      # URL of the ResourceManager API
      #对应yarn-site.xml中的 yarn.resourcemanager.webapp.addres
      resourcemanager_api_url=http://192.168.211.178:18088

[beeswax] #设置hive

      #Thehostname or IP that the Hive Server should bind to. By default it #binds to localhost,and therefore only serves local IPC clients.
      #对应hive-site.xml的属性hive.server2.thrift.port
      beeswax_server_host=localhost
      server_interface=hiveserver2
      beeswax_server_port=10001    
      # Host where internal metastore Thrift daemonis running.
      ## beeswax_meta_server_host=localhost
      beeswax_meta_server_host=localhost

[impala]
# Host of the Impala Server (one of theImpalad)
      server_host=localhost

如果集群式HA的则：配置方式选择如下的HttpFS模式（HA）

如果集群为非HA配置，择选 WebHDFS的配置方式

配置HttpFS模式（HA）

HttpFS: Verify that /etc/hadoop-httpfs/conf/httpfs-site.xml has the following configuration:

<!-- Hue HttpFS proxy user setting-->
  <property>
     <name>httpfs.proxyuser.hue.hosts</name>
     <value>*</value>
  </property>
  <property>
     <name>httpfs.proxyuser.hue.groups</name>
     <value>*</value>
  </property>

Verify that core-site.xml has the followingconfiguration:

  <property>
     <name>hadoop.proxyuser.httpfs.hosts</name>
     <value>*</value>
  </property>
  <property>
     <name>hadoop.proxyuser.httpfs.groups</name>
     <value>*</value>
  </property>

If the configuration is not present, add it to /etc/hadoop/conf/core-site.xml and restart Hadoop.

配置WebHDFS（非HA模式）

a) Add thefollowing property in hdfs-site.xml to enableWebHDFS in the NameNode and DataNodes:

<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>

b) Restart your HDFS cluster.

Configure Hue as a proxy userfor all other users and groups, meaning it may submit a request on behalf ofany other user:

Add to core-site.xml:

<!-- Hue WebHDFS proxy user setting -->
<property>
  <name>hadoop.proxyuser.hue.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.hue.groups</name>
  <value>*</value>
</property>

3. Verify that core-site.xml has the following configuration:

<property>
   <name>hadoop.proxyuser.httpfs.hosts</name>
   <value>*</value>
</property>
<property>
   <name>hadoop.proxyuser.httpfs.groups</name>
   <value>*</value>
</property>

If the configuration is not present, add it to /etc/hadoop/conf/core-site.xml and restartHadoop.

三、启动

[root@slave1 hue]# servicehadoop-httpfs start
[root@slave1 hue]# servicehue start

日志查看位置：/var/log/hue/error.log

访问地址：192.168.211.179:8888

四、参考文档：

1、Configuring CDH Components for Hue

http://www.cloudera.com/content/cloudera/en/documentation/cdh4/latest/CDH4-Installation-Guide/CDH4-Installation-Guide.html

2、Hue 安装及使用教程

http://blog.youkuaiyun.com/nsrainbow/article/details/43677077

3、安装和配置Hue

http://itindex.net/detail/52831-hue

4、hue 问题记录

http://m.oschina.net/blog/288336

如出现如下异常：

异常信息：

"UnicodeDecodeError:'ascii' codec can't decode byte 0xe9 in position 0: ordinal not inrange(128)"

File "/usr/share/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/django/core/handlers/base.py",line 100, in get_response

response =callback(request, *callback_args, **callback_kwargs)

File"/usr/share/hue/apps/beeswax/src/beeswax/views.py", line 60, in index

returnexecute_query(request)

File"/usr/share/hue/apps/beeswax/src/beeswax/views.py", line 382, inexecute_query

dbs =db.get_databases()

File"/usr/share/hue/apps/beeswax/src/beeswax/server/dbms.py", line 119,in get_databases

returnself.client.get_databases()

File"/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py",line 581, in get_databases

return[table[col] for table in self._client.get_databases()]

File"/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py",line 324, in get_databases

res =self.call(self._client.GetSchemas, req)

File"/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py",line 289, in call

session =self.open_session(self.user)

File"/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py",line 270, in open_session

res =self._client.OpenSession(req)

File"/usr/share/hue/desktop/core/src/desktop/lib/thrift_util.py", line289, in wrapper

raiseStructuredException('THRIFTAPPLICATION', str(e), data=None, error_code=502)

修改：

/usr/share/hue/build/env/lib/python2.6/site-packages/Django-1.2.3-py2.6.egg/django/core/handlers/base.py

/usr/share/hue/apps/beeswax/src/beeswax/views.py

/usr/share/hue/apps/beeswax/src/beeswax/server/dbms.py

/usr/share/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py

/usr/share/hue/desktop/core/src/desktop/lib/thrift_util.py

解决办法，在该python文件的前面加上如下几句，问题得到解决。

import sys
default_encoding = 'utf-8'
if sys.getdefaultencoding() != default_encoding:
   reload(sys)
   sys.setdefaultencoding(default_encoding)