多tomcat启动dubbo服务的应用时，dubbo端口冲突问题记录

出现的情况

1. 当在一台服务器节点上启动多个tomcat去发布dubbo服务
2. 配置的dubbo服务发布端口为-1
3. 同时启动多个tomcat就有可能出现dubbo端口冲突导致有些dubbo服务启动失败

地址已在使用
		at com.alibaba.dubbo.rpc.protocol.dubbo.DubboProtocol.createServer(DubboProtocol.java:287)
		at com.alibaba.dubbo.rpc.protocol.dubbo.DubboProtocol.openServer(DubboProtocol.java:264)
		at com.alibaba.dubbo.rpc.protocol.dubbo.DubboProtocol.export(DubboProtocol.java:251)
		at com.alibaba.dubbo.qos.protocol.QosProtocolWrapper.export(QosProtocolWrapper.java:62)
		at com.alibaba.dubbo.rpc.protocol.ProtocolListenerWrapper.export(ProtocolListenerWrapper.java:57)
		at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper.export(ProtocolFilterWrapper.java:100)
		at com.alibaba.dubbo.rpc.Protocol$Adaptive.export(Protocol$Adaptive.java)
		at com.alibaba.dubbo.registry.integration.RegistryProtocol.doLocalExport(RegistryProtocol.java:172)
		at com.alibaba.dubbo.registry.integration.RegistryProtocol.export(RegistryProtocol.java:135)
		at com.alibaba.dubbo.qos.protocol.QosProtocolWrapper.export(QosProtocolWrapper.java:60)
		at com.alibaba.dubbo.rpc.protocol.ProtocolListenerWrapper.export(ProtocolListenerWrapper.java:55)
		at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper.export(ProtocolFilterWrapper.java:98)
		at com.alibaba.dubbo.rpc.Protocol$Adaptive.export(Protocol$Adaptive.java)
		at com.alibaba.dubbo.config.ServiceConfig.doExportUrlsFor1Protocol(ServiceConfig.java:517)
		at com.alibaba.dubbo.config.ServiceConfig.doExportUrls(ServiceConfig.java:362)
		at com.alibaba.dubbo.config.ServiceConfig.doExport(ServiceConfig.java:320)
		at com.alibaba.dubbo.config.ServiceConfig.export(ServiceConfig.java:218)
		at com.alibaba.dubbo.config.spring.ServiceBean.export(ServiceBean.java:266)
		at com.alibaba.dubbo.config.spring.ServiceBean.onApplicationEvent(ServiceBean.java:106)
		at com.alibaba.dubbo.config.spring.ServiceBean.onApplicationEvent(ServiceBean.java:53)

出现的原因

1. 如果我们为每一个dubbo服务指定的端口为-1，这种情况把端口使用交给dubbo内部去查找可以使用的端口，查找端口源码（2.6.2版本com.alibaba.dubbo.config.ServiceConfig）如下:

private Integer findConfigedPorts(ProtocolConfig protocolConfig, String name, Map<String, String> map) {
        Integer portToBind = null;

        // parse bind port from environment
        String port = getValueFromConfig(protocolConfig, Constants.DUBBO_PORT_TO_BIND);
        portToBind = parsePort(port);

        // if there's no bind port found from environment, keep looking up.
        if (portToBind == null) {
            portToBind = protocolConfig.getPort();
            if (provider != null && (portToBind == null || portToBind == 0)) {
                portToBind = provider.getPort();
            }
            final int defaultPort = ExtensionLoader.getExtensionLoader(Protocol.class).getExtension(name).getDefaultPort();
            if (portToBind == null || portToBind == 0) {
                portToBind = defaultPort;
            }
            if (portToBind == null || portToBind <= 0) {
                portToBind = getRandomPort(name);
                if (portToBind == null || portToBind < 0) {
                    portToBind = getAvailablePort(defaultPort);
                    putRandomPort(name, portToBind);
                }
                logger.warn("Use random available port(" + portToBind + ") for protocol " + name);
            }
        }

        // save bind port, used as url's key later
        map.put(Constants.BIND_PORT_KEY, String.valueOf(portToBind));

        // registry port, not used as bind port by default
        String portToRegistryStr = getValueFromConfig(protocolConfig, Constants.DUBBO_PORT_TO_REGISTRY);
        Integer portToRegistry = parsePort(portToRegistryStr);
        if (portToRegistry == null) {
            portToRegistry = portToBind;
        }

        return portToRegistry;
    }

因为配置的dubbo服务发布端口为-1，没有额外在其他配置位置指定端口最终会走到portToBind = getRandomPort(name); 这段代码去查找端口，该方法源码（com.alibaba.dubbo.common.utils.NetUtils）如下：

public static int getAvailablePort(int port) {
        if (port <= 0) {
            return getAvailablePort();
        }
        for (int i = port; i < MAX_PORT; i++) {
            ServerSocket ss = null;
            try {
                ss = new ServerSocket(i);
                return i;
            } catch (IOException e) {
                // continue
            } finally {
                if (ss != null) {
                    try {
                        ss.close();
                    } catch (IOException e) {
                    }
                }
            }
        }
        return port;
    }

从上面代码可以看到通过new ServerSocket(port) 校验端口是否可用，直到找到一个可以使用的端口，在返回端口前会调用方法ss.close(); 把端口占用进行关闭

问题就出在这个地方，比如：存在两个tomcat，一个tomcat启动获取到了一个可以使用的端口20880，由于tomcat启动比较慢，dubbo服务启动时去绑定该端口还没有来得及执行，然后另一个tomcat获取20880时发现可用，也使用该端口去发布dubbo服务就会出现两个应用使用了同一个端口去发布服务，此处出现端口冲突。

发布dubbo服务端口绑定源码（netty4）如下：

protected void doOpen() throws Throwable {
        bootstrap = new ServerBootstrap();

        bossGroup = new NioEventLoopGroup(1, new DefaultThreadFactory("NettyServerBoss", true));
        workerGroup = new NioEventLoopGroup(getUrl().getPositiveParameter(Constants.IO_THREADS_KEY, Constants.DEFAULT_IO_THREADS),
                new DefaultThreadFactory("NettyServerWorker", true));

        final NettyServerHandler nettyServerHandler = new NettyServerHandler(getUrl(), this);
        channels = nettyServerHandler.getChannels();

        bootstrap.group(bossGroup, workerGroup)
                .channel(NioServerSocketChannel.class)
                .childOption(ChannelOption.TCP_NODELAY, Boolean.TRUE)
                .childOption(ChannelOption.SO_REUSEADDR, Boolean.TRUE)
                .childOption(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT)
                .childHandler(new ChannelInitializer<NioSocketChannel>() {
                    @Override
                    protected void initChannel(NioSocketChannel ch) throws Exception {
                        NettyCodecAdapter adapter = new NettyCodecAdapter(getCodec(), getUrl(), NettyServer.this);
                        ch.pipeline()//.addLast("logging",new LoggingHandler(LogLevel.INFO))//for debug
                                .addLast("decoder", adapter.getDecoder())
                                .addLast("encoder", adapter.getEncoder())
                                .addLast("handler", nettyServerHandler);
                    }
                });
        // bind
        ChannelFuture channelFuture = bootstrap.bind(getBindAddress());
        channelFuture.syncUninterruptibly();
        channel = channelFuture.channel();

    }

绑定地址操作代码为：bootstrap.bind(getBindAddress());

综上所属：可以发现dubbo端口的获取到dubbo端口绑定是有一个过程，不是同时做的，在中间这段时间，有可能被另一个应用获取到端口。

解决方案

方案一：tomcat顺序启动，保证一个tomcat启动完成后再去启动另一个tomcat
方案二：为每一个dubbo服务绑定端口进行指定（如何指定端口，看官方文档即可）