<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[lovemelovemycode的专栏]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/lovemelovemycode</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; lovemelovemycode]]></copyright><item><title><![CDATA[tensorflow实现的一个三层神经网络]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/52201647</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/52201647</guid><author>lovemelovemycode</author><pubDate>Sat, 13 Aug 2016 21:08:54 +0800</pubDate><description><![CDATA[# -*- coding: utf-8 -*-
"""
Created on Sat Aug 13 16:38:38 2016
cnn of myself,today!
@author: root
"""
import tensorflow as tf
import numpy as np
'''
inputs : input x set
input_feature_size]]></description><category></category></item><item><title><![CDATA[Zookeeper  CURD的一个例子，简单记忆]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/48440715</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/48440715</guid><author>lovemelovemycode</author><pubDate>Mon, 14 Sep 2015 16:24:49 +0800</pubDate><description><![CDATA[// 创建一个与服务器的连接 需要(服务端的 ip+端口号)(session过期时间)(Watcher监听注册)
        ZooKeeper zk = null;
        try {
            zk = new ZooKeeper("192.168.30.110:2181", 3000, new Watcher() {

                pu]]></description><category></category></item><item><title><![CDATA[flume配置参考]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/46532117</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/46532117</guid><author>lovemelovemycode</author><pubDate>Wed, 17 Jun 2015 11:48:11 +0800</pubDate><description><![CDATA[1 redis source,memory channel,json intercepter and file_roll sink配置例子
agent.sources = redis redis1
agent.channels = memory
agent.sinks = fileSink


agent.sources.redis.type = com.flumeng.plugins]]></description><category></category></item><item><title><![CDATA[Redis常用命令]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/46401179</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/46401179</guid><author>lovemelovemycode</author><pubDate>Sun, 07 Jun 2015 18:13:35 +0800</pubDate><description><![CDATA[1 创建-是否存储-查看-删除
    set name maojun;exists name;get name;del name;
2 序列化记录
   set name maojun;exists name;dump name;退出客户端在进入，exists name,name依然存在。
3 给key设置生存时间-修改生存时间
   set name maojun;exists na]]></description><category></category></item><item><title><![CDATA[CentOS6.6上安装Python2.7.10]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/46231535</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/46231535</guid><author>lovemelovemycode</author><pubDate>Fri, 29 May 2015 12:59:04 +0800</pubDate><description><![CDATA[CenoOS6.6 64位系统默认的Python版本是2.6.6，工作中需要使用Python2.7.10。本文介绍一下Python2.7.10的安装方法。切记：不可覆盖Python2.6.6，小心你的系统不能用了。总的来说安装过程是：Python2.7.10源码安装---->setuptools16.0安装---->pip安装
Python2.7.10源码安装预安装

yum groupin]]></description><category></category></item><item><title><![CDATA[CDH5.2.0升级到CDH5.3.3]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/45056321</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/45056321</guid><author>lovemelovemycode</author><pubDate>Wed, 15 Apr 2015 11:31:45 +0800</pubDate><description><![CDATA[公司有一个Spark on Yarn集群，基于CM5.2.0+CDH5.2.0搭建，Spark的版本是1.1.0.为了使用Spark1.2.0的一些特性，决定将集群版本升级到CM5.3.3+CDH5.3.3。之所以升级CM，是因为CM的版本号必须大于等于CDH的版本号。下面分成两个步骤介绍升级过程：CM升级和CDH升级。
1 CM升级过程介绍
    1.1 admin用户登陆http://1]]></description><category></category></item><item><title><![CDATA[Spark优化]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/43700717</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/43700717</guid><author>lovemelovemycode</author><pubDate>Tue, 10 Feb 2015 11:13:07 +0800</pubDate><description><![CDATA[Spark 优化
·        数据序列化
·        内存优化
o   确定内存使用
o   调整数据结构
o   序列化的RDD存储
o   垃圾回收调整
·        其它注意事项
o   并行粒度
o   Reduce任务内存使用
o   广播大的变量
o   数据本地性
·        总结
基于内存的计算（当然也可以用磁盘）是Spark的一个重]]></description><category></category></item><item><title><![CDATA[spark on yarn 执行过程介绍]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/43404415</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/43404415</guid><author>lovemelovemycode</author><pubDate>Mon, 02 Feb 2015 10:34:28 +0800</pubDate><description><![CDATA[执行语句：

sudo spark-submit --master yarn--driver-memory 7G --executor-memory 5G --executor-cores 24 --num-executors 4--class spark.init.InitSpark /home/hxf/gogo.jar>/home/hxf/dddd.txt2>&1;

执行过程：]]></description><category></category></item><item><title><![CDATA[scala private private[package] protected 使用说明]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/43155811</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/43155811</guid><author>lovemelovemycode</author><pubDate>Mon, 26 Jan 2015 18:37:43 +0800</pubDate><description><![CDATA[package com.scala.idle
object PrivateDoor {


  def main(args: Array[String]): Unit = {
    val privateContainer = new PrivateContainer()
    //访问共有变量
    println(privateContainer.publicVar)
 ]]></description><category></category></item><item><title><![CDATA[scala 用trait还是用abstract class]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/43152769</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/43152769</guid><author>lovemelovemycode</author><pubDate>Mon, 26 Jan 2015 15:13:01 +0800</pubDate><description><![CDATA[package com.scala.idle


object TraitOrAbstractClass {


  def main(args: Array[String]): Unit = {
    
  }


}
/**
 * 继承两个trait，N多方便啊！
 */
class TestClass000 extends Trait000 with Trai]]></description><category></category></item><item><title><![CDATA[spark sortByKey   subtractByKey   take  takeOrdered等函数使用例子]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42933699</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42933699</guid><author>lovemelovemycode</author><pubDate>Tue, 20 Jan 2015 22:48:32 +0800</pubDate><description><![CDATA[package com.latrobe.spark

import org.apache.spark.{SparkContext, SparkConf}

/**
 * Created by spark on 15-1-19.
 * 根据key对K-V类型的RDD进行排序获得新的RDD
 */
object SortByKey {
  def main(args: Array[String]) {]]></description><category></category></item><item><title><![CDATA[spark  foldByKey]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42847057</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42847057</guid><author>lovemelovemycode</author><pubDate>Sun, 18 Jan 2015 22:58:51 +0800</pubDate><description><![CDATA[package com.latrobe.spark

import org.apache.spark.{SparkContext, SparkConf}

/**
 * Created by spark on 15-1-18.
 */
object FoldByKey {
  def main(args: Array[String]) {
    val conf = new SparkConf(]]></description><category></category></item><item><title><![CDATA[Spark       PairRDDFunctions     flatMapValues]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42845683</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42845683</guid><author>lovemelovemycode</author><pubDate>Sun, 18 Jan 2015 22:22:59 +0800</pubDate><description><![CDATA[package com.latrobe.spark

import org.apache.spark.{SparkContext, SparkConf}

/**
 * Created by spark on 15-1-18.
 */
object FlatMapValues {
  def main(args: Array[String]) {
    val conf = new SparkC]]></description><category></category></item><item><title><![CDATA[spark rdd countByValue]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42835615</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42835615</guid><author>lovemelovemycode</author><pubDate>Sun, 18 Jan 2015 14:45:24 +0800</pubDate><description><![CDATA[package com.latrobe.spark

import org.apache.spark.{SparkContext, SparkConf}

/**
 * Created by spark on 15-1-18.
 * 统计出集合中每个元素的个数
 */
object CountByValue {
  def main(args: Array[String]) {
    val c]]></description><category></category></item><item><title><![CDATA[spark rddToPairRDDFunctions  countByKey]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42835561</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42835561</guid><author>lovemelovemycode</author><pubDate>Sun, 18 Jan 2015 14:41:28 +0800</pubDate><description><![CDATA[package com.latrobe.spark

import org.apache.spark.{SparkContext, SparkConf}

/**
 * Created by spark on 15-1-18.
 * 计算出每个key对应的value的数量
 */
object CountByKey {
  def main(args: Array[String]) {
    v]]></description><category></category></item><item><title><![CDATA[spark RDD  countApproxDistinct]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42835441</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42835441</guid><author>lovemelovemycode</author><pubDate>Sun, 18 Jan 2015 14:24:11 +0800</pubDate><description><![CDATA[package com.latrobe.spark

import org.apache.spark.{SparkConf, SparkContext}

/**
 * Created by spark on 15-1-18.
 * countApproxDistinct : RDD的一个方法，作用是对RDD集合内容进行去重统计。
 * 该统计是一个大约的统计，参数relativeSD控制统计的精]]></description><category></category></item><item><title><![CDATA[scala implicit 学习简记]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42611003</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/42611003</guid><author>lovemelovemycode</author><pubDate>Sun, 11 Jan 2015 21:11:37 +0800</pubDate><description><![CDATA[scala 隐式转换氛围三种：隐式参数、隐式类和隐式对象。这篇文章会结合隐式转换在Spark中的应用来分别说明三种隐式转换的用法。]]></description><category></category></item><item><title><![CDATA[Spark RDD与共享变量简介]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/41309469</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/41309469</guid><author>lovemelovemycode</author><pubDate>Thu, 20 Nov 2014 10:58:21 +0800</pubDate><description><![CDATA[hadoop有两个东东：HDFS（存储）和MapReduce（计算）。MapReduce计算比较慢，于是Spark（速度是MR的10~100倍）出现了。Spark有两个核心的概念：弹性分布式数据集RDD与共享变量。下面进行一下简单的介绍。

弹性分布式数据集（RDD）获得方式：1
并行化驱动程序内的集合; 2
从外部数据集加载。
1 并行化驱动程序内的集合code demo
val d]]></description><category></category></item><item><title><![CDATA[sqoop使用例子]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/40920059</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/40920059</guid><author>lovemelovemycode</author><pubDate>Sat, 08 Nov 2014 13:25:39 +0800</pubDate><description><![CDATA[简介： sqoop是工具，一个可以把ORACLE、MYSQL中的数据导入到HDFS、HIVE、HBASE中（反过来也可以）的工具。下面是一些使用的例子。留下做个参考：
#从Mysql中抽取数据到HDFS.问题：文件太多，全他妈是小文件；目标目录如果已经存在会报错
sqoop import --connect jdbc:mysql://10.10.244.137/test --username]]></description><category></category></item><item><title><![CDATA[YARN架构简介]]></title><link>https://blog.youkuaiyun.com/lovemelovemycode/article/details/40751649</link><guid>https://blog.youkuaiyun.com/lovemelovemycode/article/details/40751649</guid><author>lovemelovemycode</author><pubDate>Mon, 03 Nov 2014 23:28:10 +0800</pubDate><description><![CDATA[MapReduce has undergone a complete overhaul in hadoop-0.23 and we now have, what we call, MapReduce 2.0 (MRv2) or YARN.
The fundamental idea of MRv2 is to split up the two major functionalities of th]]></description><category></category></item></channel></rss>