Alex 的 Hadoop 菜鸟教程: 第18课用Http的方式访问HDFS - HttpFs 教程

本文介绍了如何在Centos 6.x + CDH 5.x环境下使用HttpFs通过浏览器和REST API管理HDFS文件，包括安装、配置、启动步骤及各项操作如创建文件夹、打开文件等。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

原文地址： http://blog.youkuaiyun.com/nsrainbow/article/details/43678237 最新课程请关注原作者博客，获得更好的显示体验

声明

本文基于Centos 6.x + CDH 5.x

HttpFs 有啥用

HttpFs可以干这两件事情

通过HttpFs你可以在浏览器里面管理HDFS上的文件
HttpFs还提供了一套REST 风格的API可以用来管理HDFS

其实很简单的一个东西嘛，但是很实用

安装HttpFs

在集群里面找一台可以访问hdfs的机器安装HttpFs

$ sudo yum install hadoop-httpfs

配置

编辑/etc/hadoop/conf/core-site.xml

<property>  
<name>hadoop.proxyuser.httpfs.hosts</name>  
<value>*</value>  
</property>  
<property>  
<name>hadoop.proxyuser.httpfs.groups</name>  
<value>*</value>  
</property>

这边是定义可以使用httpfs的用户组和host，写*就是不限制

配置好之后重启hadoop

启动HttpFs

$ sudo service hadoop-httpfs start

使用HttpFs

打开浏览器访问 http://host2:14000/webhdfs/v1?op=LISTSTATUS&user.name=httpfs 可以看到

{
	"FileStatuses": {
		"FileStatus": [{
			"pathSuffix": "hbase",
			"type": "DIRECTORY",
			"length": 0,
			"owner": "hbase",
			"group": "hadoop",
			"permission": "755",
			"accessTime": 0,
			"modificationTime": 1423446940595,
			"blockSize": 0,
			"replication": 0
		},
		{
			"pathSuffix": "tmp",
			"type": "DIRECTORY",
			"length": 0,
			"owner": "hdfs",
			"group": "hadoop",
			"permission": "1777",
			"accessTime": 0,
			"modificationTime": 1423122488037,
			"blockSize": 0,
			"replication": 0
		},
		{
			"pathSuffix": "user",
			"type": "DIRECTORY",
			"length": 0,
			"owner": "hdfs",
			"group": "hadoop",
			"permission": "755",
			"accessTime": 0,
			"modificationTime": 1423529997937,
			"blockSize": 0,
			"replication": 0
		},
		{
			"pathSuffix": "var",
			"type": "DIRECTORY",
			"length": 0,
			"owner": "hdfs",
			"group": "hadoop",
			"permission": "755",
			"accessTime": 0,
			"modificationTime": 1422945036465,
			"blockSize": 0,
			"replication": 0
		}]
	}
}

这个 &user.name=httpfs 表示用默认用户 httpfs 访问，默认用户是没有密码的。

webhdfs/v1 这是HttpFs的根目录

访问 http://host2:14000/webhdfs/v1/user?op=LISTSTATUS&user.name=httpfs 可以看到

{
	"FileStatuses": {
		"FileStatus": [{
			"pathSuffix": "cloudera",
			"type": "DIRECTORY",
			"length": 0,
			"owner": "root",
			"group": "hadoop",
			"permission": "755",
			"accessTime": 0,
			"modificationTime": 1423472508868,
			"blockSize": 0,
			"replication": 0
		},
		{
			"pathSuffix": "hdfs",
			"type": "DIRECTORY",
			"length": 0,
			"owner": "hdfs",
			"group": "hadoop",
			"permission": "700",
			"accessTime": 0,
			"modificationTime": 1422947019504,
			"blockSize": 0,
			"replication": 0
		},
		{
			"pathSuffix": "history",
			"type": "DIRECTORY",
			"length": 0,
			"owner": "mapred",
			"group": "hadoop",
			"permission": "1777",
			"accessTime": 0,
			"modificationTime": 1422945692887,
			"blockSize": 0,
			"replication": 0
		},
		{
			"pathSuffix": "hive",
			"type": "DIRECTORY",
			"length": 0,
			"owner": "hive",
			"group": "hadoop",
			"permission": "755",
			"accessTime": 0,
			"modificationTime": 1423123187569,
			"blockSize": 0,
			"replication": 0
		},
		{
			"pathSuffix": "hive_people",
			"type": "DIRECTORY",
			"length": 0,
			"owner": "root",
			"group": "hadoop",
			"permission": "755",
			"accessTime": 0,
			"modificationTime": 1423216966453,
			"blockSize": 0,
			"replication": 0
		},
		{
			"pathSuffix": "hive_people2",
			"type": "DIRECTORY",
			"length": 0,
			"owner": "root",
			"group": "hadoop