hadoop02 有不少变化，直接在windows,eclipse 环境下

windows 在eclipse 下run wordcount;
需要：
1. 安装cygwin
2. 把 cygwin/bin 放到 windows 环境变量 path

hadoop start up
1。下载 ,tar
2。修改 con/hadoop-env.sh , java_home
3. bin/hadoop 显示帮助
4。运行 word count
mkdir testinput
cp conf/*.xml testinput
bin/hadoop jar hadoop-*-examples.jar grep testinput testoutput 'dfs[a-z.]+'
cat testoutput/*

bin/hadoop jar hadoop-streaming.jar -input input -output output -mapper /bin/cat -reducer /bin/wc


bin/hadoop pipes -conf src/examples/pipes/conf/word.xml -input input -output cppwordcount


hadoop contrib 开发环境
1。 装 ant:
设置 ant_home
2. svn checkout
svn checkout http://svn.apache.org/repos/asf/hadoop/core/trunk/ hadoop-core-trunk
3. ant

9. 执行unit test
ant -Dtestcase=TestConfiguration test-core
太慢


6. ssh 准备 (linux)
ssh-keygen -t rsa -P ‘’
cat ~/。ssh/id_rsa.pub >> ~/.ssh/authorized_keys

8. 启动eclipse : ,new java project

9. 把ant.jar ,搞过来

9.1 <property>
<name>hadoop.job.ugi</name>
<value>user,users</value>
</property>

10。run wordCount , program parameter : input ,output
jvm -Xms256m -Xmx256m
run


配置cluster：
配置 conf/hadoop-env.sh , java_home
conf/core-site :
8 <property>
9 <name>hadoop.tmp.dir</name>
10 <value>/home/jianqi/usep/tmp</value>
11 </property>
12 <property>
13 <name>fs.default.name</name>
14 <value>hdfs://localhost:54310</value>
15 </property>

conf/hdfs-site:
6 <configuration>
7 <property>
8 <name>dfs.replication</name>
9 <value>1</value>
10 </property>

conf/mapreduce-site
8 <property>
9 <name>mapred.job.tracker</name>
10 <value>hdfs://localhost:54311</value>
11 </property>

5. format hdfs
bin/hadoop namenode -format


linux eclipse 也放弃了。