架构之美

xxxx

kadmin.local: listprincsK/M@JACKkadmin/admin@JACKkadmin/changepw@JACKkadmin/hadoopmaster.jack@JACKkrbtgt/JACK@JACKkadmin.local: addprinc admin/admin@JACKWARNING: no policy specified fo...

[root@hadoopmaster ~]# cat /etc/hosts127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4::1 localhost localhost.localdomain localhost6 localhost6.localdomain6...

[hadoop@hadoopmaster test]$ hadoop distcp hdfs://hadoopmaster:9000/user/hive/warehouse/jacktest.db hdfs://hadoopmaster:9000/jacktest/todir15/11/18 05:39:30 INFO tools.DistCp: Input Options: DistCpO...

SQL context available as sqlContext.scala> var myVar : String = "Foo"myVar: String = Fooscala> val myVal : String = "Foo"myVal: String = Fooscala> var myVar : String = "Foo1"...

namenode（hdfs）+jobtracker（mapreduce）可以放在一台机器上，datanode+tasktracker可以在一台机器上，辅助namenode要单独放一台机器，jobtracker通常情况下分区跟datanode一样(目录最好分布在不同的磁盘上，一个目录对应一个磁盘)，namenode存储目录需要格式化，datanode存储目录不需要格式化，启动时自动创建 同一个d...

import java.sql.Connection;import java.sql.DriverManager;import java.sql.ResultSet;import java.sql.SQLException;import java.sql.Statement;public class TestHive { // hive --service hivese...

stackoverflowFollowing are the three commands which appears same but have minute differenceshadoop fs {args}hadoop dfs {args}hdfs dfs {args}hadoop fs FS relates to a generic file sys...

[hadoop@hadoopmaster ~]$ pig pig3.pig 15/08/30 01:34:26 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL15/08/30 01:34:26 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE15/08/30 01:34:2...

hadoop fsckUsage: DFSck [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]] 检查这个目录中的文件是否完整 -move 破损的文件移至/lost+found目录 ...

https://cwiki.apache.org/confluence/display/Hive/HCatalog+UsingHCat

hdfs://hadoopmaster:9000/user/hadoop/ncdc_data.txt<r 2> 71hdfs://hadoopmaster:9000/user/hadoop/user       <dir>grunt> cat ncdc_data.txt1953:122:51954:83:51955:44:51956:33:51957:50:519...

http://blog.youkuaiyun.com/zythy/article/details/18098651 记录  http://blackproof.iteye.com/blog/1791980  初学者可以参考这个博文

hadoop jar /opt/jack.jar org.apache.hadoop.t1.WordCount  /jackdemodir/wordcount/input /jackdemodir/wordcount/output1  hadoop jar /opt/jack.jar org.apache.hadoop.t1.WordCount  /jackdemodir/wordcou...

ChainMapper 支持多个map reduce参考http://blog.youkuaiyun.com/chaoping315/article/details/6221440

1. list the database sqoop list-databases --connect jdbc:mysql://192.168.1.9:3306/ -username root -password root; 2. list tables in one databasesqoop list-tables --connect jdbc:mysql://192.1...

1. prepare the virtual environment for the hadoop cluster. you can choose Virtual Box or VM Wave. There are some issues with Vitural box in my labtop. So i choose VM 10.0. then you need use the foll...