hadoop rackawareness实现

由于集群扩大，新机器安装在另一台机架上，所以需要配置机架感知。机器感知没有使用默认脚本实现，而是采用直接配置topology.node.switch.mapping.impl，实现类是java类。以下是在测试集群中的实现。

public class JavaTestBasedMapping implements DNSToSwitchMapping {

	//key:ip value:rack
	private static ConcurrentHashMap<String,String> cache = new ConcurrentHashMap<String,String>();
	
	static {
		//rack0 16
		cache.put("192.168.5.116", "/ht_dc/rack0");
		cache.put("192.168.5.117", "/ht_dc/rack0");
		cache.put("192.168.5.118", "/ht_dc/rack0");
		cache.put("192.168.5.120", "/ht_dc/rack0");
		cache.put("192.168.5.121", "/ht_dc/rack0");
		cache.put("host116", "/ht_dc/rack0");
		cache.put("host117", "/ht_dc/rack0");
		cache.put("host118", "/ht_dc/rack0");
		cache.put("host120", "/ht_dc/rack0");
		cache.put("host121", "/ht_dc/rack0");
	}
	
	@Override
	public List<String> resolve(List<String> names) {
		List<String> m = new ArrayList<String>();
		
		if (names == null || names.size() == 0) {
			m.add("/default-rack");
			return m;
		}
		
		for (String name : names) {
			String rack = cache.get(name);
			if (rack != null) {
				m.add(rack);
			}
		}
		
		return m;
	}
}

打成jar，加上执行权限。我是直接放到了$HADOOP_HOME/lib下。

core-site.xml加入 

<property>
                <name>topology.node.switch.mapping.impl</name>
                <value>com.dmp.hadoop.cluster.topology.JavaTestBasedMapping</value>

</property>

测试成功。hadoop dfsadmin -report 显示配置的RACK名称。

建议写一个配置文件。