运行环境:
-
CentOS 6.5
-
hadoop-2.6.0-cdh5.7.0
-
hadoop-2.6.0-cdh5.7.0
-
hive-1.1.0-cdh5.7.0
-
hue-3.9.0-cdh5.7.0
-
sqoop-1.4.6-cdh5.7.0
Hue:
官网地址:http://gethue.com/
官网安装说明:http://cloudera.github.io/hue/latest/admin-manual/manual.html#centosrhel
官网github地址: http://github.com/cloudera/hue
Hue是一个开源的Apache Hadoop UI系统,最早是由Cloudera Desktop演化而来,由Cloudera贡献给开源社区,它是基于Python Web框架Django实现的。通过使用Hue我们可以在浏览器端的Web控制台上与Hadoop集群进行交互来分析处理数据,例如操作HDFS上的数据,运行MapReduce Job,Hive
等等
支持功能:
-
数据库查询编辑器,支持 Hive, Impala, MySql, PostGres, Sqlite and Oracle
-
动态查询仪表盘,支持 Solr
-
支持 Spark 编辑器和仪表盘
-
浏览器查看状态,支持 YARN, HDFS, Hive table Metastore, HBase, ZooKeeper
-
支持 Pig Editor, Sqoop2, Oozie workflows 编辑器和仪表盘
-
将数据导入hdfs
1.CentOS6.5依赖环境:
yum install -y ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain gcc gcc-c++ krb5-devel libffi-devel libxml2-devel libxslt-devel make mysql mysql-devel openldap-devel python-devel sqlite-devel gmp-devel
2.下载CDH5.7.0平台Hue:
wget http://archive-primary.cloudera.com/cdh5/cdh/5/http://archive-primary.cloudera.com/cdh5/cdh/5/hue-3.9.0-cdh5.7.0.tar.gz
3.环境变量配置:
vim /.bash_profile export HUE_HOME=/home/hadoop/app/hue-3.9.0 export PATH=$HUE_HOME/bin:$PATH source ~/.bash_profile
4.编译源码(Hue3.9)
tar -xzvf hue-3.9.0-cdh5.7.0.tar.gz -C ~/app/ -- 检查解压后的用户和用户组 cd hue-3.9.0 make apps
编译过程需要下很多模块和包,所以根据你的网速可能会需要编译很久。
最后出现xxx post-processed说明编译成功
1190 static files copied to '/home/hadoop/app/hue-3.9.0/build/static', 1190 post-processed. make[1]: Leaving directory `/home/hadoop/app/hue-3.9.0/apps'
5.Hue配置全局文件( $HUE_HOME/desktop/conf/hue.ini)
[desktop] secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o http_host=hadoop001 time_zone=Asia/Shanghai
6.Hue集成Hadoop( $HADOOP_HOME/etc/hadoop )
hdfs-site.xml <property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property> core-site.xml <property> <name>hadoop.proxyuser.hue.hosts</name> <value>*</value> </property> <property> <name>hadoop.proxyuser.hue.groups</name> <value>*</value> </property> httpfs-site.xml <property> <name>httpfs.proxyuser.hue.hosts</name> <value>*</value> </property> <property> <name>httpfs.proxyuser.hue.groups</name> <value>*</value> </property>
6.1 Hue集成Hadoop( $HUE_HOME/desktop/conf/hue.ini )
[hadoop] # Configuration for HDFS NameNode # ------------------------------------------------------------------------ [[hdfs_clusters]] # HA support by using HttpFs [[[default]]] # Enter the filesystem uri fs_defaultfs=hdfs://localhost:8020 # NameNode logical name. ## logical_name= # Use WebHdfs/HttpFs as the communication mechanism. # Domain should be the NameNode or HttpFs host. # Default port is 14000 for HttpFs. webhdfs_url= -- 取消注释
7.Hue集成Hive( $HIVE_HOME/conf/hive-site.xml )
<property> <name>hive.server2.thrift.bind.port</name> <value>10000</value> </property> <property> <name>hive.server2.thrift.bind.host</name> <value>hadoop001</value> </property> <property> <name>hive.server2.long.polling.timeout</name> <value>5000</value> </property> <property> <name>hive.server2.authentication</name> <value>NOSASL</value> </property>
7.1.Hue集成Hive( $HUE_HOME/desktop/conf/hue.ini )
[beeswax] # Host where HiveServer2 is running. # If Kerberos security is enabled, use fully-qualified domain name (FQDN). hive_server_host=hadoop001 # Port where HiveServer2 Thrift server runs on. hive_server_port=10000 # Hive configuration directory, where hive-site.xml is located hive_conf_dir=/home/hadoop/app/apache-hive-2.2.0-bin/conf
8.启动Hue
-- 先启动Hive [hadoop@hadoop001 bin]$ ./hiveserver2 --hiveconf hive.server2.thrift.port=10000 -- 启动hue服务 [hadoop@hadoop001 hue-3.9.0]$ build/env/bin/supervisor
9.访问Hue操作hive
http://IP:8888/
由于初次创建的用户admin用户 只能权限操作HDFS上/user/admin/* 目录
如果要操作/user/hadoop/目录,hue必须创建hadoop用户操作;
遇到的权限报错:
Note: you are a Hue admin but not a HDFS superuser, "hadoop" or part of HDFS supergroup, "supergroup".
解决办法:
# This should be the hadoop cluster admin default_hdfs_superuser=hadoop -- HDFS 修改为 hadoop
来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/31441024/viewspace-2168672/,如需转载,请注明出处,否则将追究法律责任。
转载于:http://blog.itpub.net/31441024/viewspace-2168672/