<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[luyllyl的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/luyllyl</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; luyllyl]]></copyright><item><title><![CDATA[JVM的 OutOfMemoryError异常]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/160603058</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/160603058</guid><author>luyllyl</author><pubDate>Tue, 28 Apr 2026 22:30:41 +0800</pubDate><description><![CDATA[本文主要探讨了Java内存溢出的几种类型及处理方法：1)堆溢出通过GC Roots引用链分析内存泄漏，使用内存分析工具定位问题；2)栈溢出分为StackOverflowError和线程创建过多导致的OOM；3)方法区溢出在JDK8后表现为Metaspace问题；4)直接内存溢出与堆内存无关。文章详细介绍了每种溢出场景的诊断方法、参数配置建议和解决方案，包括使用Eclipse Memory Analyzer分析堆转储、调整线程栈大小、监控元空间使用等实用技巧。最后还涉及字符串常量池的变化和直接内存管理注意事项]]></description><category></category></item><item><title><![CDATA[JVM 对象访问定位理解]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/160602751</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/160602751</guid><author>luyllyl</author><pubDate>Tue, 28 Apr 2026 22:22:33 +0800</pubDate><description><![CDATA[本文主要探讨Java对象访问机制，分析了两种reference访问方式：句柄池方式和直接指针方式。句柄池方式通过中间句柄间接访问对象，便于垃圾回收但效率较低；直接指针方式则直接指向堆中对象实例数据，访问速度更快（Java 5后默认方式）。]]></description><category></category></item><item><title><![CDATA[jvm对象创建及对象的内存布局]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/160602269</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/160602269</guid><author>luyllyl</author><pubDate>Tue, 28 Apr 2026 22:17:37 +0800</pubDate><description><![CDATA[本文详细解析了JVM执行new指令时的完整流程：1）首先检查类符号引用是否在常量池中定位到；2）若类未加载则触发加载、链接、初始化过程；3）类加载完成后在堆中分配内存并初始化对象头；4）执行构造方法完成对象初始化。同时介绍了对象内存布局的三部分结构（对象头、实例数据、对齐填充），特别说明了对象头中锁状态的2位标识规则。文章还解答了TLAB默认开启、构造器字节码名称等关键问题，通过案例和字节码分析展示了new指令的执行细节。]]></description><category></category></item><item><title><![CDATA[jdk历史及jvm内存区域与内存溢出异常]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/160601987</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/160601987</guid><author>luyllyl</author><pubDate>Tue, 28 Apr 2026 22:05:07 +0800</pubDate><description><![CDATA[本文主要介绍了JDK发展历程和JVM内存管理机制。在JDK历史方面，Oracle通过收购Sun和安卓侵权案获利，后将JavaEE组件移交给Eclipse基金会并更名为Jakarta EE。JVM采用&quot;解释器+JIT编译器&quot;混合模式提升性能，其中JIT负责将热点代码编译为本地机器码。内存区域方面，详细解析了程序计数器、虚拟机栈、本地方法栈、堆和方法区的作用机制，包括栈帧结构、操作数栈工作原理，以及堆内存分区管理策略。特别指出程序计数器是唯一不会OOM的区域，而方法区在JDK8后由元空间实现]]></description><category></category></item><item><title><![CDATA[spark-core_30：Executor初始化过程env.blockManager.initialize(conf.getAppId)源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80420260</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80420260</guid><author>luyllyl</author><pubDate>Wed, 23 May 2018 15:10:29 +0800</pubDate><description><![CDATA[在（spark-core_28及spark-core_29：Executor初始化过程env.blockManager.initialize(conf.getAppId)-NettyBlockTransferService.init()源码分析）分析了NettyBlockTransferService.init()做了如下四件事/**NettyBlockTransferService.init(t...]]></description><category></category></item><item><title><![CDATA[spark-core_29：blockManager.initialize（）=》NettyBlockTransferService.init()-NettyServer创建源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80420171</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80420171</guid><author>luyllyl</author><pubDate>Wed, 23 May 2018 15:06:22 +0800</pubDate><description><![CDATA[上一节（spark-core_28：Executor初始化过程env.blockManager.initialize(conf.getAppId)- NettyBlockTransferService.init()源码分析）分析了a、NettyBlockRpcServer是用于打开的上传注册在BlockManager中的Block块b,TansportConf:会通过它成员：ConfigProvi...]]></description><category></category></item><item><title><![CDATA[spark-core_28：Executor初始化过程env.blockManager.initialize(conf.getAppId)- blockTransferService.init()分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80420025</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80420025</guid><author>luyllyl</author><pubDate>Wed, 23 May 2018 15:00:57 +0800</pubDate><description><![CDATA[查看（spark-core_25:Master通知Worker启动CoarseGrainedExecutorBackend进程及CoarseGrainedExecutorBackend初始化源码分析）//sparkContext在初始化时也调用了_env.blockManager.initialize(_applicationId)执行的过程差不多 private[spark] class Blo...]]></description><category></category></item><item><title><![CDATA[spark-core_27: DAGScheduler的初始化源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80419844</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80419844</guid><author>luyllyl</author><pubDate>Wed, 23 May 2018 14:52:46 +0800</pubDate><description><![CDATA[1，SparkContext在初始完成SparkDeploySchedulerBacked和TaskSchedulerImpl,就开始初始化DAGScheduler，并将TaskSchedulerImpl、MapOutputTrackerMaster、BlockManagerMaster，设置到它的成员上  * 1，DAGScheduler面向stage的高级调度层。 它计算每个Job的DAG(一...]]></description><category></category></item><item><title><![CDATA[spark-core_25:Master通知Worker启动CoarseGrainedExecutorBackend进程及初始化源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80419117</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80419117</guid><author>luyllyl</author><pubDate>Wed, 23 May 2018 14:44:47 +0800</pubDate><description><![CDATA[承接上文（spark-core_24:AppClient的ClientEndpoint注册RegisterApplication）上文中提到：master调用launchExecutor(){worker.endpoint.send(LaunchExecutor(masterUrl,    exec.application.id, exec.id,exec.application.desc, ex...]]></description><category></category></item><item><title><![CDATA[spark-core_24:注册RegisterApplication，schedule()分配worker的资源,启动CoarseGrainedExecutorBackend源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80407612</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80407612</guid><author>luyllyl</author><pubDate>Tue, 22 May 2018 16:36:13 +0800</pubDate><description><![CDATA[基于上文（spark-core_23: TaskSchedulerImpl.start()、SparkDeploySchedulerBackend.start()、CoarseGrainedExecutorBackend.start()、AppClient.start()源码分析）11，AppClient初始化ClientEndpoint初始化时调用tryRegisterAllMasters，触发...]]></description><category></category></item><item><title><![CDATA[spark-core_23: TaskSchedulerImpl.start()、SparkDeploySchedulerBackend.start()、AppClient.start()源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80407383</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80407383</guid><author>luyllyl</author><pubDate>Tue, 22 May 2018 16:24:06 +0800</pubDate><description><![CDATA[1，在sparkContext初始化完createTaskScheduler()调用TaskSchedulerImpl.start()//在TaskSchedulerImpl在DAGScheduler的构造函数中设置DAGScheduler引用后启动TaskScheduler_taskScheduler.start()2,进入TaskSchedulerImpl.start()方法override ...]]></description><category></category></item><item><title><![CDATA[spark-core_22: SparkDeploySchedulerBackend，TaskSchedulerImpl的初始化源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80407256</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80407256</guid><author>luyllyl</author><pubDate>Tue, 22 May 2018 16:15:20 +0800</pubDate><description><![CDATA[1，  在sparkContext初始时会执行// Create and startthe scheduler，这个master是在sparkSubmit.Main方法得到：spark://luyl152:7077,luyl153:7077,luyl154:7077//如果集群管理器是standalone模式：该方法返回（SparkDeploySchedulerBackend，TaskSchedu...]]></description><category></category></item><item><title><![CDATA[spark-core_21: SortShuffleManager的初始化源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80407164</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80407164</guid><author>luyllyl</author><pubDate>Tue, 22 May 2018 16:10:31 +0800</pubDate><description><![CDATA[1，SparkEnv初始过时，通过反射的方式默认将SortShuffleManager实例化出来// Let the userspecify short names for shuffle managers//使用sort shuffle做为mapper和reducer网络传输val shortShuffleMgrNames = Map(  &quot;hash&quot; -&amp;gt; &quot;org.apache.spa...]]></description><category></category></item><item><title><![CDATA[spark-core_20: MapOutputTrackerMaster、MapOutputTracker、MapOutputTrackerMasterEndpoint等源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80406946</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80406946</guid><author>luyllyl</author><pubDate>Tue, 22 May 2018 16:05:43 +0800</pubDate><description><![CDATA[1，在SparkEnv.create()初始化了MapOutputTrackerMaster(记录ShuffleMapTask输出信息)val mapOutputTracker = if (isDriver) {  /* MapOutputTrackerMaster属于driver，这里使用TimeStampedHashMap来跟踪 map的输出信息，也可以将旧信息进行清除    * 一、MapO...]]></description><category></category></item><item><title><![CDATA[spark-core_19:ActorSystem的初始化源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80406842</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80406842</guid><author>luyllyl</author><pubDate>Tue, 22 May 2018 15:52:58 +0800</pubDate><description><![CDATA[1，在SparkEnv再创建时调用create(),里面调用初始ActorSystem注：ActorSystem在后面版本会被RpcEnv替换掉private def create(    conf: SparkConf,    executorId: String,    hostname: String,    port: Int,    isDriver: Boolean,    isLoc...]]></description><category></category></item><item><title><![CDATA[spark-core_18:Akka分布式通信案例]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80406669</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80406669</guid><author>luyllyl</author><pubDate>Tue, 22 May 2018 15:45:05 +0800</pubDate><description><![CDATA[Scala认为Java线程通过共享数据以及通过锁来维护共享数据的一致性是糟糕的做法，容易引起锁的争用，而且线程的上下文切换会带来不少开销，降低并发程序的性能，甚至会引入死锁的问题。Akka是Actor编程模型的高级类库，类似于JDK 1.5之后越来越丰富的并发工具包，简化了程序员并发编程的难度。Akka是一款提供了用于构建高并发的、分布式的、可伸缩的、基于Java虚拟机的消息驱动应用的工具集和运行...]]></description><category></category></item><item><title><![CDATA[spark-core_17:自定义RpcEnv模拟Master和Worker通信及RpcEnv、RpcEndpoint、RpcEndpointRef源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80406375</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80406375</guid><author>luyllyl</author><pubDate>Tue, 22 May 2018 15:30:51 +0800</pubDate><description><![CDATA[RpcEnv底层就是Netty实现一、自定义RpcEnv模拟Master和Worker进行通信1、自定义的RpcEndPoint,需要实现RpcEndPoint//自定义的RPC实现ThreadSafeRpcEndpoint，说明它是线程安全的class TestMathEndpoint(override val rpcEnv: RpcEnv) extends ThreadSafeRpcEndpo...]]></description><category></category></item><item><title><![CDATA[spark-core_16:初始化Driver的SparkEnv]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80406232</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80406232</guid><author>luyllyl</author><pubDate>Tue, 22 May 2018 15:24:18 +0800</pubDate><description><![CDATA[1，通过SparkContext初始化时调用createSparkEnv()==&amp;gt;然后调用SparkEnv.createDriverEnv()来初始化SparkEnv对象/**
 * Create a SparkEnv for the driver.
  * 该方法被SparkContext.createSparkEnv()方法进行调用
  * numCores:如果master是local...]]></description><category></category></item><item><title><![CDATA[spark-core_15:sparkContext初始化源码分析]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/80405965</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/80405965</guid><author>luyllyl</author><pubDate>Tue, 22 May 2018 15:16:53 +0800</pubDate><description><![CDATA[这幅图是网友提供的，非常感谢/** * Main entry point for Sparkfunctionality. A SparkContext represents the connection to a Spark cluster, andcan be used to create RDDs, accumulators and broadcast variables on thatclu...]]></description><category></category></item><item><title><![CDATA[spark-core_14:Worker源码分析2-Worker与Master通信]]></title><link>https://blog.youkuaiyun.com/luyllyl/article/details/79715087</link><guid>https://blog.youkuaiyun.com/luyllyl/article/details/79715087</guid><author>luyllyl</author><pubDate>Tue, 27 Mar 2018 16:30:54 +0800</pubDate><description><![CDATA[承接上文，在上文中Worker的onStart(){。。。。registerWithMaster() 。。。}将自己的信息注册到Master上private def registerWithMaster() {
  // onDisconnected may be triggered multiple times, so don't attempt registration
  // if the...]]></description><category></category></item></channel></rss>