手动配置安全的Hadoop YARN集群

小弟现在就职于一家美国的专注于Hadoop的创业公司，本着回馈国内IT社区的想法，分享一些在工作中遇到的Hadoop手记。今天，先贴一篇我记录的关于如何Hadoop YARN集群进行安全配置的文档，比较偷懒，就暂时不翻译成中文了，希望对国内同行也有所帮助。日后，再陆续分享一些其他的关于Hadoop的手记。

－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－

Enabling Kerberos Authentication for a Hadoop YARN Cluster Manually

Assumption

• OS: CentOS 6.5

• Hadoop Version: 2.5

• Hadoop Components: NameNode, DataNode, ResourceManager, NodeManager, TimelineServer

• Cluster: Single-node Cluster

Prerequisite

1. You need to setup a DNS server to get reverse lookups work, and make sure your host name is mapped to your host IP address in /etc/hosts. For example,

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 zjshen-centos ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 zjshen-centos

2. You need to install Kerberos service packages, setup proper configurations, and start the services.

• Install the packages with the command: sudo yum install krb5-server krb5-workstation.

• Config /etc/krb5.conf as follows:

[logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] default_realm = EXAMPLE.COM dns_lookup_realm = false dns_lookup_kdc = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true [realms] EXAMPLE.COM = {  kdc = localhost  admin_server = localhost  auth_to_local = RULE: [1:$1@$0](.*@EXAMPLE.COM)s/@.*// } [domain_realm] localhost = EXAMPLE.COM .localhost = EXAMPLE.COM

• Config /var/kerberos/krb5kdc/kdc.conf as follows:

[kdcdefaults] kdc_ports = 88 kdc_tcp_ports = 88 [realms] EXAMPLE.COM = {  #master_key_type = aes256-cts