本文介绍了一种用于帮助用户在不同应用和表之间管理HBase集群资源的工具。通过添加资源分配管理功能，多个应用程序可以在同一集群上并发运行，同时使用不同的分组进行优化配置，确保高优先级表不受低优先级表干扰。系统忙碌时，确保高优先级表不受到影响。表格和区域服务器可以轻松在分组之间移动，改变配置后，分组可以单独重启而不必重启整个集群。
The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables.
When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished.
After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group.
In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables
Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size.
Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster.
git entry : https://github.com/ICT-Ope/HBase_allocation .
更多关注一下hbase jira: https://issues.apache.org/jira/browse/HBASE-4120
另外推荐一个hbase的blog: http://hbase.iteye.com/ （BTW: 多么好的blog名字啊～）
扫一扫
483
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
