【服务器篇笔记04-kafka-阿里云服务器ECS单机版部署配置】

初衷：

至于为什么写这个，因为我前面在本地虚拟机部署是完全没问题的，但是由于迁移到阿里云服务器，存在公网ip和私有ip的区别，

很多以前配置的潜在问题就暴露出来了~~主要是conf文件夹下的server.properties的配置需要注意

注意点：

############################# Server Basics #############################

# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0

集群的话，这个id会不同~~

############################# Socket Server Settings #############################

# The address the socket server listens on. It will get the value returned from 
# java.net.InetAddress.getCanonicalHostName() if not configured.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
#listeners=PLAINTEXT://:9092
listeners=PLAINTEXT://xx.xxx.xxx.xxx:9092



如果这里配置的是你买的服务器的公网ip,会直接启动不了kafka。

org.apache.kafka.common.KafkaException: Socket server failed to bind to xx.xxx.xxx.xxx:9092: Cannot assign requested address.
	at kafka.network.Acceptor.openServerSocket(SocketServer.scala:450)
	at kafka.network.Acceptor.<init>(SocketServer.scala:340)
	at kafka.network.SocketServer$$anonfun$createAcceptorAndProcessors$1.apply(SocketServer.scala:146)
	at kafka.network.SocketServer$$anonfun$createAcceptorAndProcessors$1.apply(SocketServer.scala:142)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
	at kafka.network.SocketServer.createAcceptorAndProcessors(SocketServer.scala:142)
	at kafka.network.SocketServer.startup(SocketServer.scala:91)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:250)
	at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
	at kafka.Kafka$.main(Kafka.scala:75)
	at kafka.Kafka.main(Kafka.scala)
Caused by: java.net.BindException: Cannot assign requested address
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:433)
	at sun.nio.ch.Net.bind(Net.java:425)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
	at kafka.network.Acceptor.openServerSocket(SocketServer.scala:446)
	... 11 more



但是如果是虚拟机（直接写设置的ip值）是不会出现任何问题的~~

上面需要配置成 listeners=PLAINTEXT://:9092

# Hostname and port the broker will advertise to producers and consumers. If not set, 
# it uses the value for "listeners" if configured.  Otherwise, it will use the value
# returned from java.net.InetAddress.getCanonicalHostName().
#advertised.listeners=PLAINTEXT://your.host.name:9092

然而如果只是单纯配置第二条注意的点，是启动不了消费者的~~ 因为连接不到服务，会报超时错误，见下面的客户端代码和错误描述



要配置advertised.listeners
advertised.listeners=PLAINTEXT://xx.xxx.xxx.xxx:9092

客户端代码：

var kafka = require('kafka-node'),
    Consumer = kafka.Consumer,
    client = new kafka.KafkaClient({kafkaHost: 'xx.xxx.xxx.xxx:9092'}),
    consumer = new Consumer(
        client, 
        [
            { topic: 'Processing', partition: 0 }
        ],
        {
            autoCommit: false
        }
    );
consumer.on('message', function (message) {
    console.log(message);
});
consumer.on('error', function (err) {
    console.log(err);
});

超时报错：

Request timed out after 30000ms


TimeoutError: Request timed out after 30000ms    at new TimeoutError

还有一点需要注意的~~~


启动kafka,如果是用下面类似的启动命令
在关闭ssh连接窗口的同时，kafka会停止运行~~
./kafka-server-start.sh /usr/local/java/kafka_2.11-2.1.1/config/server.properties &

参考原因：
Unix/Linux下一般比如想让某个程序在后台运行，很多都是使用& 在程序结尾来让程序自动运行。比如我们要运行mysql在后台：
/usr/local/mysql/bin/mysqld_safe --user=mysql &
但是加入我们很多程序并不象mysqld一样做成守护进程，可能我们的程序只是普通程序而已，一般这种程序使用& 结尾，但是如果终端关闭，那么程序也会被关闭。但是为了能够后台运行，那么我们就可以使用nohup这个命令，比如我们有个test.php需要在后台运行，并且希望在后台能够定期运行，那么就使用nohup：


nohup ./kafka-server-start.sh /usr/local/java/kafka_2.11-2.1.1/config/server.properties &