<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[ 寂空师叔]]></title><description><![CDATA[做一个安安静静的屌丝～]]></description><link>https://blog.youkuaiyun.com/guoning_scdn</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; guoning_scdn]]></copyright><item><title><![CDATA[Spark核心类介绍]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/50208565</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/50208565</guid><author>guoning_scdn</author><pubDate>Mon, 07 Dec 2015 17:24:05 +0800</pubDate><description><![CDATA[Spark核心类介绍
SparkContent 
SparkContext是Spark的入口,负责连接Spark集群，创建RDD，累积量和广播量等。 
其中有两个关键调度器会在SparkContext实例化时启动:dagScheduler: DAGScheduler
taskScheduler: TaskScheduler
DAGScheduler 
DAGScheduler主要属性:eventPr]]></description><category></category></item><item><title><![CDATA[Spark 学习笔记二开发环境]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/45394497</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/45394497</guid><author>guoning_scdn</author><pubDate>Thu, 30 Apr 2015 14:30:17 +0800</pubDate><description><![CDATA[一、在这里本码奴强烈推荐一款IDE   
intellij Idea 官网地址 http://www.jetbrains.com/idea/ 只能用一句话来概括~ 您值得拥有
二、 正题
1、插件安装
下载安装 scala 然后重启 Intellij Idea




2、新建一个 scala 项目
File -> New Project 可以找到如下图菜单




]]></description><category></category></item><item><title><![CDATA[Spark学习笔记一  环境搭建]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/45226787</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/45226787</guid><author>guoning_scdn</author><pubDate>Thu, 23 Apr 2015 20:44:03 +0800</pubDate><description><![CDATA[一、环境介绍
我在这里用的是三台虚拟机
master 192.168.1.100
slave1 192.168.1.101
slave2 192.168.1.102
二、程序下载
这里已经把程序传到网盘上这，方便大家下载
1、hadoop-2.5.2
2、scala-2.11.6
3、spark-1.3.1-bin-hadoop2.4
三、开始安装
1、安装hadoop，安装]]></description><category></category></item><item><title><![CDATA[利用Wget 直接将数据Put到HDFS 上]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/43736617</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/43736617</guid><author>guoning_scdn</author><pubDate>Wed, 11 Feb 2015 18:18:53 +0800</pubDate><description><![CDATA[如题：


wget http://117.79.226.247:8888/impala/download/1565/xls -O - | hadoop fs -put - /tmp/wget_to_hdfs


-O 重命名
-O  后面的 - 为重定向到标准输出，（这里就是控制台） -put 后面的 - 标准输入（对应前面的标准输出）

这样 就可以是数据不落地直接PUT 到]]></description><category></category></item><item><title><![CDATA[Hive Impala  Count 效率比对]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/43342661</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/43342661</guid><author>guoning_scdn</author><pubDate>Sat, 31 Jan 2015 18:09:56 +0800</pubDate><description><![CDATA[Hive 主要是跑批处理的，Impala主要是做准实时的内存查询，我拿他俩做比较～～感觉我好没意思阿～～
一、count 比较
十万数量级
Imapal
[cdh-node2:21000] > select count(1) from userinfo;
Query: select count(1) from userinfo
+----------+
| count(1) |
+----]]></description><category></category></item><item><title><![CDATA[Hive SQL  使用笔记]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/43314397</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/43314397</guid><author>guoning_scdn</author><pubDate>Fri, 30 Jan 2015 21:02:17 +0800</pubDate><description><![CDATA[一、建表语句 
 
1、普通建表，指定分隔符 
 
create table user(
    name string,
    ips string  COMMENT 'IP Address of the User')
    COMMENT 'This is the user view table'
    ROW FORMAT DELIMITED FIELDS TERMINATED]]></description><category></category></item><item><title><![CDATA[Hive 执行sql 报 2015-01-22 17:07:59,263 WARN  conf.HiveConf (HiveConf.java:initialize(1507)) - DEPRECAT]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/43314119</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/43314119</guid><author>guoning_scdn</author><pubDate>Fri, 30 Jan 2015 20:55:00 +0800</pubDate><description><![CDATA[在执行Hive的时候报
2015-01-22 17:07:59,263 WARN  conf.HiveConf (HiveConf.java:initialize(1507)) - DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a val]]></description><category></category></item><item><title><![CDATA[Cloudera Manager 使用笔记---远程访问不了Hue的管理页面]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/43313955</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/43313955</guid><author>guoning_scdn</author><pubDate>Fri, 30 Jan 2015 20:51:42 +0800</pubDate><description><![CDATA[安装好Hue 后 发现 远程访问不了Hue的管理页面，而内网可以，一番周折，发现需要勾选 
将 Hue 服务器绑定到通配符地址]]></description><category></category></item><item><title><![CDATA[Hive 研发笔记---LEFT JOIN 中的坑]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/43273751</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/43273751</guid><author>guoning_scdn</author><pubDate>Thu, 29 Jan 2015 17:19:56 +0800</pubDate><description><![CDATA[因为日志急速增长，原来放在Mysql上的统计 越来越吃力，所以公司决定把统计业务迁移到Hadoop上。
在比对数据的时候，发现了Hive中的一个坑
select a.* from  default.t_softuser a
left join
 t_softuser b on
a.hid=b.hid and a.corp=b.corp and a.softid=b.softid and a.s]]></description><category></category></item><item><title><![CDATA[Cloudera Manager Server 5.0.2 Hadoop 集群搭建笔记]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/41483377</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/41483377</guid><author>guoning_scdn</author><pubDate>Tue, 25 Nov 2014 16:04:47 +0800</pubDate><description><![CDATA[题记：现学现卖、多多交流~~~]]></description><category></category></item><item><title><![CDATA[java 获取 类的绝对路径]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/38963133</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/38963133</guid><author>guoning_scdn</author><pubDate>Sun, 31 Aug 2014 20:50:27 +0800</pubDate><description><![CDATA[package com.venus.helper;

import java.io.File;
import java.io.UnsupportedEncodingException;
import java.net.URL;

/**
 * Copyright: 版权所有 ( c ) 北京启明星辰信息安全技术有限公司 2013。保留所有权利
 * 作者: 郭宁
 * 创建时间: 14-8-31 ]]></description><category></category></item><item><title><![CDATA[JS将表单序列化成 json]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/38873669</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/38873669</guid><author>guoning_scdn</author><pubDate>Wed, 27 Aug 2014 17:16:35 +0800</pubDate><description><![CDATA[最近项目用到JqGrid。在查询的时候需要把Form表单序列花]]></description><category></category></item><item><title><![CDATA[IP 验证，IP转Long，Long转IP]]></title><link>https://blog.youkuaiyun.com/guoning_scdn/article/details/38873025</link><guid>https://blog.youkuaiyun.com/guoning_scdn/article/details/38873025</guid><author>guoning_scdn</author><pubDate>Wed, 27 Aug 2014 16:47:34 +0800</pubDate><description><![CDATA[import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 * Copyright: 版权所有 ( c ) 北京启明星辰信息安全技术有限公司 2013。保留所有权利
 * 作者: 郭宁
 * 创建时间: 14-8-1 下午上午10:32
 * 文件描述:
 * 修改描述:
 */
public class IpHelp]]></description><category></category></item></channel></rss>