sources study-part 6- remote debugging

最新推荐文章于 2022-02-27 22:03:45 发布

leibnitz09

最新推荐文章于 2022-02-27 22:03:45 发布

阅读量93

点赞数

分类专栏： hadoop sources reading 文章标签： Hadoop Socket

hadoop sources reading 专栏收录该内容

12 篇文章

订阅专栏

本文介绍在集群环境下使用Hadoop进行调试的方法。作者分享了使用JDPA调试平台及配置Hadoop选项来实现客户端阶段的调试经验，并尝试通过分离DFS和MapReduce组件进一步调试，但遇到了一些挑战。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

when run in cluster(pseudo),it is difficult to debug the code.i have tried to debug use some tricks to do it:

a use the jdpa debug platform.

add this options to hadoop opts after start and before to run a job:

HADOOP_OPTS="$HADOOP_OPTS -agentlib:jdwp=transport=dt_socket,address=8001,server=y,suspend=y"
but i foud it is useful for debuggin the client phase.when the client submited a job to JT,the client have not authority to control the job progress,so i think it as this.

b separate the dfs and mapred.

start the dfs daemons in the command ,and start your JT and TT in eclispe.and TT must be run in debug mode to debug.

but sometimes this is failed to get your purpose,why?threads,undetermined things to affect to debug?