- 博客(67)
- 资源 (1)
- 收藏
- 关注
RSS订阅原创 spark snappy save text file
SQL context available as sqlContext.scala> var myVar : String = "Foo"myVar: String = Fooscala> val myVal : String = "Foo"myVal: String = Fooscala> var myVar : String = "Foo1"...
2016-08-28 21:54:38
724
原创 kerberos 5
kadmin.local: listprincsK/M@JACKkadmin/admin@JACKkadmin/changepw@JACKkadmin/hadoopmaster.jack@JACKkrbtgt/JACK@JACKkadmin.local: addprinc admin/admin@JACKWARNING: no policy specified fo...
2015-12-14 21:42:04
449
原创 hadoop network
[root@hadoopmaster ~]# cat /etc/hosts127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4::1 localhost localhost.localdomain localhost6 localhost6.localdomain6...
2015-11-21 23:09:42
248
原创 distcp
[hadoop@hadoopmaster test]$ hadoop distcp hdfs://hadoopmaster:9000/user/hive/warehouse/jacktest.db hdfs://hadoopmaster:9000/jacktest/todir15/11/18 05:39:30 INFO tools.DistCp: Input Options: DistCpO...
2015-11-18 21:47:42
634
原创 spark
http://www.tuicool.com/articles/ZfeQrq7http://spark.apache.org/docs/1.3.1/programming-guide.htmlhttp://www.imobilebbs.com/wordpress/archives/4984?variant=zh-sg
2015-11-07 22:36:06
108
原创 hadoop常用命令
namenode（hdfs）+jobtracker（mapreduce）可以放在一台机器上，datanode+tasktracker可以在一台机器上，辅助namenode要单独放一台机器，jobtracker通常情况下分区跟datanode一样(目录最好分布在不同的磁盘上，一个目录对应一个磁盘)，namenode存储目录需要格式化，datanode存储目录不需要格式化，启动时自动创建 同一个d...
2015-09-19 14:58:23
138
原创 java
log4j-1.2.16mybatis-3.2.3mybatis-generator-core-1.3.2mysql-connector-java-5.1.28-binojdbc14 classpath:application.properties http://yl-f...
2015-09-10 23:21:19
119
原创 多线程之同步
publci class Singleton{ private static Singleton instance = null; private Singleton(){} public static synchronized Singleton getInstance(){ if(instance == null){ ...
2015-09-06 18:12:30
92
原创 hive之jdbc
import java.sql.Connection;import java.sql.DriverManager;import java.sql.ResultSet;import java.sql.SQLException;import java.sql.Statement;public class TestHive { // hive --service hivese...
2015-09-06 16:08:19
137
原创 hadoop fs, hdfs dfs, hadoop dfs科普
stackoverflowFollowing are the three commands which appears same but have minute differenceshadoop fs {args}hadoop dfs {args}hdfs dfs {args}hadoop fs FS relates to a generic file sys...
2015-09-06 11:14:19
162
原创 teamcity
continues integration toolsteam cityhttp://blog.youkuaiyun.com/huwei2003/article/details/42875815
2015-09-05 17:39:39
195
原创 读取文件名，统计文件数量
[root@linuxstudy sh]# cat ./s1.sh #!/bin/bashfilelist=$(ls ~/)echo "-----------begin------------"for file in $filelistdoecho $file if [ -f ~/$file ] then echo "this is...
2015-08-31 23:19:51
391
原创 pig call hcatalog
[hadoop@hadoopmaster ~]$ pig pig3.pig 15/08/30 01:34:26 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL15/08/30 01:34:26 INFO pig.ExecTypeProvider: Trying ExecType : MAPREDUCE15/08/30 01:34:2...
2015-08-30 16:33:28
208
原创 hadoop fsck
hadoop fsckUsage: DFSck [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]] 检查这个目录中的文件是否完整 -move 破损的文件移至/lost+found目录 ...
2015-08-25 22:59:29
236
原创 hcatalog study
https://cwiki.apache.org/confluence/display/Hive/HCatalog+UsingHCat
2015-08-25 22:41:53
209
原创 git http.sslVerify=false
打开eclipse中的windows-->Preferences-->Team-->Git-->Configuration-->User Settings.然后点Add Entry新建一个键值对，输入http.sslVerify=false。public class FileName{ //static String filename ="publish_aa_bb_cc_dd...
2015-08-23 22:00:02
15199
1
原创 jaas介绍
http://lyb520320.iteye.com/blog/720478这个文章写的很清晰http://www.blogjava.net/hk2000c/archive/2006/03/23/jaas.htmlhttp://blog.youkuaiyun.com/huachao1/article/details/4290206kerberoshttp://idior.cnb...
2015-08-23 15:13:34
335
原创 pig
hdfs://hadoopmaster:9000/user/hadoop/ncdc_data.txt<r 2> 71hdfs://hadoopmaster:9000/user/hadoop/user <dir>grunt> cat ncdc_data.txt1953:122:51954:83:51955:44:51956:33:51957:50:519...
2015-08-22 15:34:15
241
原创 pig
http://blog.youkuaiyun.com/zythy/article/details/18098651 记录 http://blackproof.iteye.com/blog/1791980 初学者可以参考这个博文
2015-08-19 23:03:40
279
原创 nohup command &
[root@linuxstudy testshell]# jobs -l[1]+ 2568 Running nohup python pytest1.py &[root@linuxstudy testshell]# bg %1-bash: bg: job 1 already in background[root@linuxstudy testshell]...
2015-08-19 22:34:23
248
原创 cut sed
[root@linuxstudy testshell]# cut -d: -f 1 /etc/passwdrootbindaemonadmlpsyncshutdownhaltmailuucpoperatorgamesgopherftpnobodydbususbmuxdvcsartkitavahi-autoipdabrthaldaemongdmntpapachesaslauthpostfixpu...
2015-08-18 22:35:42
148
原创 shell
[root@linuxstudy testshell]# cat s2.sh #!/bin/shexport name=jackprintf "p0=%s \n" "$0"echo "p1="$1echo "p2="$2echo $nameecho "pid="$$echo "all param num:"$#echo "all param:"$@e
2015-08-18 22:18:21
82
原创 hadoop jar command
hadoop jar /opt/jack.jar org.apache.hadoop.t1.WordCount /jackdemodir/wordcount/input /jackdemodir/wordcount/output1 hadoop jar /opt/jack.jar org.apache.hadoop.t1.WordCount /jackdemodir/wordcou...
2015-08-16 22:57:23
165
原创 iptables
service iptables status1) 重启后生效开启： chkconfig iptables on关闭： chkconfig iptables off2) 即时生效，重启后失效开启： service iptables start关闭： service iptables stop...
2015-08-16 22:06:27
106
原创 信息查看定时任务
uname -aview all the information of the linux cat /proc/cpuinfo dmesg |grep CPUfind the cpu info file /bin/ls find the bit of the system lsb_release -afind the release ver...
2015-08-16 22:05:15
178
原创 mapreduce
ChainMapper 支持多个map reduce参考http://blog.youkuaiyun.com/chaoping315/article/details/6221440
2015-08-16 21:56:12
91
原创 ssl
keytool -genkey -alias tigerwu -keyalg RSA -keystore d:/keys/tigerwukey输入密钥库口令:jack123再次输入新口令:您的名字与姓氏是什么? [Unknown]: jacklocal您的组织单位名称是什么? [Unknown]: sandbox您的组织名称是什么? [Unknown]: sandbox您所在的城市...
2015-08-02 22:57:49
149
原创 linux mysql
enquiry mysql version in centos linuxyum list installed | grep mysqlyum -y remove mysql-libs.x86_64enquiry mysql version in yum repositoryyum list | grep mysql oryum -y list mysql*install my...
2015-07-18 09:35:38
111
原创 sqoop command
1. list the database sqoop list-databases --connect jdbc:mysql://192.168.1.9:3306/ -username root -password root; 2. list tables in one databasesqoop list-tables --connect jdbc:mysql://192.1...
2015-06-06 18:54:43
143
原创 hadoop 2.5.2安装实录
1. prepare the virtual environment for the hadoop cluster. you can choose Virtual Box or VM Wave. There are some issues with Vitural box in my labtop. So i choose VM 10.0. then you need use the foll...
2014-12-09 23:35:23
119
原创 ant tool
http://blog.youkuaiyun.com/xiaoli_feng/article/details/4316252http://www.atool.org/colorpicker.phphttp://www.qqtz.com/hao/color.htm *{margin:0; padding:0;font-size:12px;} ...
2014-11-06 14:34:00
207
原创 maven ant plugin
pom.xml jar matched----------------------- 4.0.0 com.hsbc.jack antplugin maven-plugin 0.0.1-SNAPSHOT antplugin Maven Mojo http://maven.apache.org org.apache.maven mave...
2014-10-29 22:42:34
324
原创 mybatis 和 hibernate 之我见
在中国的很多软件企业中，每天仍然在重复的制造轮子，耗时耗力还看不到效果。如果软件都模块化，还会有那么多麻烦的维护工作吗，都是前人挖坑后人跳。我也见过一些软件大国的架构师，每天在说SOA，结果他们设计的项目，耦合度那么高，看看他们的源代码，不敢直视。 一个类写了个千把行，我知道你能写，等你些到一万行的时候，你这个项目就黄了。在软件领域混了好几年了，还是最初的想法，简单而不失灵活和功能，才是实...
2014-10-26 00:26:28
804
原创 pom5
release zip false ${project.build.directory} *.war ear ${project.build.directory} **/*.xml config fals...
2014-10-22 21:28:48
233
原创 maven pom4
org.apache.maven.plugins maven-resources-plugin 2.4.2 org.apache.maven.plugins maven-compiler-plugin 3.0 1.6 1.6 ...
2014-10-21 21:46:35
210
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人