cdh5 HA涉及的JournalNodes介绍

最新推荐文章于 2023-09-01 13:27:12 发布

weixin_34101229

最新推荐文章于 2023-09-01 13:27:12 发布

阅读量445

点赞数

CC 4.0 BY-SA版权

文章标签： java python

原文链接：https://my.oschina.net/u/3197158/blog/1625280

为了确保HDFS的高可用性和容错能力，必须配置至少三个JournalNode守护进程来实现多数写入机制。这使得系统能够容忍单一节点故障，并保持正常运行。本文详细介绍了JournalNode的配置要求及最佳实践。

2019独角兽企业重金招聘Python工程师标准>>>

官网介绍：

There must be at least three JournalNode daemons, since edit log modifications must be written to a majority of JournalNodes. This will allow the system to tolerate the failure of a single machine. You can also run more than three JournalNodes, but in order to actually increase the number of failures the system can tolerate, you should run an odd number of JournalNodes, (three, five, seven, etc.) Note that when running with N JournalNodes, the system can tolerate at most (N - 1) / 2 failures and continue to function normally. If the requisite quorum is not available, the NameNode will not format or start, and you will see an error similar to this:

12/10/01 17:34:18 WARN namenode.FSEditLog: Unable to determine input streams from QJM to [10.0.1.10:8485, 10.0.1.10:8486, 10.0.1.10:8487]. Skipping.
java.io.IOException: Timed out waiting 20000ms for a quorum of nodes to respond.

选择奇数个主机（至少三个）作为JournalNodes。 JournalNodes应托管在与NameNodes具有相似硬件规格的主机上。建议您将JournalNode分别放置在与主动和备用NameNode相同的主机上，以及类似硬件上的第三个JournalNode，例如JobTracker。
参考资料连接：
http://www.cloudera.com/documentation/manager/5-0-x/Cloudera-Manager-Managing-Clusters/cm5mc_hdfs_hi_avail.html

转载于:https://my.oschina.net/u/3197158/blog/1625280