开源云计算技术系列（四）（Cloudera安装配置hadoop 0.20最新版配置）

接上文，我们继续体验Cloudera 0.20最新版。

wgethadoop-0.20-conf-pseudo_0.20.0-1cloudera0.5.0~lenny_all.deb

wgethadoop-0.20_0.20.0-1cloudera0.5.0~lenny_all.deb

debian:~# dpkg –ihadoop-0.20-conf-pseudo_0.20.0-1cloudera0.5.0~lenny_all.deb

dpkg –ihadoop-0.20_0.20.0-1cloudera0.5.0~lenny_all.deb

就这么简单。ok

如果不清楚安装到哪里了，可以用

debian:~# dpkg -L hadoop-0.20

可以看到清晰的安装目录结构。

启动：

debian:~# cd /etc/init.d/hadoop-0.20-
hadoop-0.20-datanode hadoop-0.20-namenode hadoop-0.20-tasktracker
hadoop-0.20-jobtracker hadoop-0.20-secondarynamenode
debian:~# cd /etc/init.d/hadoop-0.20-

debian:~# /etc/init.d/hadoop-0.20-namenode start

debian:~# /etc/init.d/hadoop-0.20-namenode status
hadoop-0.20-namenode is running

debian:~# /etc/init.d/hadoop-0.20-datanode start

debian:~# /etc/init.d/hadoop-0.20-datanode status
hadoop-0.20-datanode is running

debian:~# /etc/init.d/hadoop-0.20-jobtracker start

debian:~# /etc/init.d/hadoop-0.20-jobtracker status
hadoop-0.20-jobtracker is running

debian:~# /etc/init.d/hadoop-0.20-tasktracker start

debian:~# /etc/init.d/hadoop-0.20-tasktracker status
hadoop-0.20-tasktracker is running

启动完成。

接着可以进行常规的example的测试了。

值得测试的是

debian:~# sqoop –help
Usage: hadoop sqoop.jar org.apache.hadoop.sqoop.Sqoop (options)

Database connection options:
–connect (jdbc-uri) Specify JDBC connect string
–driver (class-name) Manually specify JDBC driver class to use
–username (username) Set authentication username
–password (password) Set authentication password
–local Use local imp<wbr>ort fast path (mysql on<wbr>ly)</wbr></wbr>

Imp<wbr>ort control options:<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –table (tablename) Table to read<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –columns (col,col,col…) Columns to export from table<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –order-by (column-name) Column of the table used to order results<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –hadoop-home (dir) Override $HADOOP_HOME<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –warehouse-dir (dir) HDFS path for table destination<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –as-sequencefile Imp<wbr>orts da<wbr>ta to SequenceFiles<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –as-textfile Imp<wbr>orts da<wbr>ta as plain text (default)<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –all-tables Imp<wbr>ort all tables in database<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> (Ignores –table, –columns and –order-by)</wbr></wbr></wbr></wbr></wbr></wbr>

Co<wbr>de generation options:<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –outdir (dir) Output directory for generated co<wbr>de<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –bindir (dir) Output directory for compiled objects<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> –generate-on<wbr>ly Stop after co<wbr>de generation; do not imp<wbr>ort</wbr></wbr></wbr></wbr></wbr>

Additional commands:
–list-tables List tables in database and exit
–list-databases List all databases available and exit
–debug-sql (statement) Execute ’statement’ in SQL and exit

Generic Hadoop command-line options:
Generic options supported are
-conf <configuration file> specify an application configuration file
-D <property=value> use value for given property
-fs <local|namenode:port> specify a namenode
-jt <local|jobtracker:port> specify a job tracker
-files <comma separated list of files> specify comma separated files to be copied to the map reduce cluster
-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.

The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]

At minimum, you must specify –connect and either –table or –all-tables.
Alternatively, you can specify –generate-on<wbr>ly or on<wbr>e of the additional<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> commands.</wbr></wbr>

可以通过apt-get install mysql-server 安装debian下的mysql进行联合测试。

测试体验见前面的文章，到这里大家可以进行一个完整的体验了。Cloudera的出现的确对hadoop的配置进行了极大的简化，推动了开源云计算的发展。

转自：http://rdc.taobao.com/blog/dw/archives/417