calvin_kui的专栏

一、hadoop命令 #键入hadoop回车，查看使用帮助:Usage: hadoop [--config confdir] COMMANDwhere COMMAND is one of:  namenode -format     format the DFS filesystem  secondarynamenode    run the DFS secondary nam

1. 在eclipse中新建一个java project, 名字为hadoop(可任意命名)2. 拷贝haoop目录下的core, hdfs, mapred三个文件夹到eclipse的src下3. 调整hadoop/src下的目录结构     右键项目hadoop -> Properties -> Java Build Path -> Source 删除hadoop

环境：CentOS6.5, Hadoop1.1.2, jdk1.6三台Linux虚拟机：192.168.190.100:hostname:vm1规划:master节点192.168.190.101hostname:vm2规划:slave节点1192.168.190.102hostname:vm3规划:slave节点2一、配置hosts

环境：CentOS 6.5, JDK 1.6, hadoop 1.1.2一、设置静态IP查看博客http://blog.youkuaiyun.com/calvin_kui/article/details/44783355二、设置HOSTNAME1. 在/etc/sysconfig/network文件中修改HOSTNAME2. 在/etc/hosts中绑定HOSTNAME

环境：CentOS 6.5, Eclipse 4.4.2, Hadoop 1.1.2任务目标：从数据源抽取指定的字段，并统计出错行数一、数据源准备在hdfs://vm1:9000/user/hadoop/in目录中上传了两个数据文件,test1.txt和test2.txt内容如下：test1.txtMAY 12:10:12 192.158.202 cal

环境：CentOS 6.5, Eclipse 4.4.2, Hadoop 1.1.2任务目标：从数据源统计ip第一位数字的记录列表一、数据源准备在hdfs://vm1:9000/user/hadoop/in目录中上传了两个数据文件,test1.txt和test2.txt内容如下：test1.txtMAY 12:10:12 192.

dfs.namenode.logging.level  info  The logging level for dfs namenode. Other values are "dir"(trace namespace mutations), "block"(trace block under/over replications and blockcreations/deletion

环境：hdoop1.1.2方法一、使用java.net.URL访问eg.import java.io.InputStream;import java.net.URL;import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;import org.apache.hadoop.io.IOUtils;public cl

hadoop.job.history.location    If job tracker is static the history files are stored   in this single well known place. If No value is set here, by default,  it is in the local file system at

环境：CentOS 6.5, Eclipse 4.4.2, Hadoop 1.1.2一、eclipse hadoop插件编译1. 查看以下博客链接编译或者点击以下链接直接下载http://download.youkuaiyun.com/detail/calvin_kui/8618045二、eclipse hadoop插件安装1.复制hadoop插件到ecli

环境：hadoop1.1.2一、 定义服务提供的对象的接口, 此接口必须extends org.apache.hadoop.ipc.VersionedProtocoleg.import org.apache.hadoop.ipc.VersionedProtocol;/** * 业务接口, must extends VersionedProtocol * @autho

hadoop.tmp.dir  /tmp/hadoop-${user.name}  A base for other temporary directories.  hadoop.native.lib  true  Should native hadoop libraries, if present, be used.  hadoop.htt

环境：CentOS 6.5, Hadoop 1.1.2, Eclipse 4.4.2, Ant 1.9.4一、 软件准备1. 在CentOS上安装好Eclipse2. 在CentOS上安装好Hadoop3. 在CentOS上安装好Ant本机安装软件目录如下：HADOOP_HOME=/home/hadoop/hadoop-1.1.2ECLIPSE_HO