Netty源码解析-NioEventLoop初始化

    作为Netty的核心组件之一，NioEventLoop负责处理netty中的各种事件，每个NioEventLoop上有一个selector负责轮询IO事件，有一个定时任务队列及普通任务队列负责处理注册到NioEventLoop上的非IO任务。而1组NioEventLoop右由一个NioEventLoopGroup负责管理，默认一个NioEventLoopGroup管理2倍CPU核数的NioEventLoop。

    参照官网的服务端写法：
    

public void run() {
    EventLoopGroup bossGroup = new NioEventLoopGroup();
    EventLoopGroup workGroup = new NioEventLoopGroup();

    ServerBootstrap b = new ServerBootstrap();
    b.group(bossGroup, workGroup)
            .channel(NioServerSocketChannel.class)
            .option(ChannelOption.SO_BACKLOG, 128)
            .childOption(ChannelOption.SO_KEEPALIVE, true)
            .handler(new ChannelHandler())
            .childHandler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel socketChannel) throws Exception {
                    socketChannel.pipeline().addLast(new ChannelInboundA(), new ChannelInboundB(), new ChannelInboundC());
                }
            });

    try {
        ChannelFuture future = b.bind(port).sync();
        log.info("server start running");
        future.channel().closeFuture().sync();
    } catch (InterruptedException e) {
        e.printStackTrace();
    } finally {
        bossGroup.shutdownGracefully();
        workGroup.shutdownGracefully();
    }
}

1. NioEventLoop实在初始化NioEventLoopGroup的过程中被初始化；
2. 当NioEventLoopGroup的构造参数不传值时，netty默认会为每个NioEventLoopGroup配置2倍的CPU核数个NioEventLoop：
   

   static {
           DEFAULT_EVENT_LOOP_THREADS = Math.max(1, SystemPropertyUtil.getInt(
                   "io.netty.eventLoopThreads", Runtime.getRuntime().availableProcessors() * 2));
   
           if (logger.isDebugEnabled()) {
               logger.debug("-Dio.netty.eventLoopThreads: {}", DEFAULT_EVENT_LOOP_THREADS);
           }
       }

3. 最终调用到NioEventLoopGroup的父类MultithreadEventExecutorGroup的构造方法：
   

   protected MultithreadEventExecutorGroup(int nThreads, Executor executor,
                                           EventExecutorChooserFactory chooserFactory, Object... args) {
       if (nThreads <= 0) {
           throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));
       }
   
       if (executor == null) {
           executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
       }
   
       children = new EventExecutor[nThreads];
   
       for (int i = 0; i < nThreads; i ++) {
           boolean success = false;
           try {
               children[i] = newChild(executor, args);
               success = true;
           } catch (Exception e) {
               // TODO: Think about if this is a good exception type
               throw new IllegalStateException("failed to create a child event loop", e);
           } finally {
               if (!success) {
                   for (int j = 0; j < i; j ++) {
                       children[j].shutdownGracefully();
                   }
   
                   for (int j = 0; j < i; j ++) {
                       EventExecutor e = children[j];
                       try {
                           while (!e.isTerminated()) {
                               e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS);
                           }
                       } catch (InterruptedException interrupted) {
                           // Let the caller handle the interruption.
                           Thread.currentThread().interrupt();
                           break;
                       }
                   }
               }
           }
       }
   
       chooser = chooserFactory.newChooser(children);
   
       final FutureListener<Object> terminationListener = new FutureListener<Object>() {
           @Override
           public void operationComplete(Future<Object> future) throws Exception {
               if (terminatedChildren.incrementAndGet() == children.length) {
                   terminationFuture.setSuccess(null);
               }
           }
       };
   
       for (EventExecutor e: children) {
           e.terminationFuture().addListener(terminationListener);
       }
   
       Set<EventExecutor> childrenSet = new LinkedHashSet<EventExecutor>(children.length);
       Collections.addAll(childrenSet, children);
       readonlyChildren = Collections.unmodifiableSet(childrenSet);
   }

4. 在该构造方法中主要做了3件事：
   1).创建一个线程执行器ThreadPerTaskExecutor，以便于后续NioEventLoop在处理task任务时，可直接调用io.netty.util.concurrent.ThreadPerTaskExecutor#execute该方法启动线程，异步执行任务。
   2).为NioEventLoopGroup创建EventExecutor数组，及NioEventLoop数组。
   3).为NioEventLoopGroup创建一个NioEventLoop旋转器，负责有事件事从NioEventLoop数组中选择NioEventLoop来执行任务。
5. 创建ThreadPerTaskExecutor：
   1).
   

   executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());

   2).调用newDefaultThreadFactory()设置NioEventLoop线程标识前缀：
   

   public DefaultThreadFactory(String poolName, boolean daemon, int priority, ThreadGroup threadGroup) {
       if (poolName == null) {
           throw new NullPointerException("poolName");
       }
       if (priority < Thread.MIN_PRIORITY || priority > Thread.MAX_PRIORITY) {
           throw new IllegalArgumentException(
                   "priority: " + priority + " (expected: Thread.MIN_PRIORITY <= priority <= Thread.MAX_PRIORITY)");
       }
   
       prefix = poolName + '-' + poolId.incrementAndGet() + '-';
       this.daemon = daemon;
       this.priority = priority;
       this.threadGroup = threadGroup;
   }

   其中poolName为nioEventLoopGroup，可以看到线程的前缀为nioEventLoopGroup-自增id-；
6. NioEventLoopGroup轮询创建NioEventLoop线程组：
   1).调用NioEventLoopGroup的newChild方法：
   

   protected EventLoop newChild(Executor executor, Object... args) throws Exception {
       return new NioEventLoop(this, executor, (SelectorProvider) args[0],
           ((SelectStrategyFactory) args[1]).newSelectStrategy(), (RejectedExecutionHandler) args[2]);
   }

   2).进入NioEventLoop的构造：
   

   NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider,
                SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler) {
       super(parent, executor, false, DEFAULT_MAX_PENDING_TASKS, rejectedExecutionHandler);
       if (selectorProvider == null) {
           throw new NullPointerException("selectorProvider");
       }
       if (strategy == null) {
           throw new NullPointerException("selectStrategy");
       }
       provider = selectorProvider;
       selector = openSelector();
       selectStrategy = strategy;
   }

   a.首先调用父类构造保存线程执行器，创建taskQueue有界阻塞队列：
   

   protected SingleThreadEventExecutor(EventExecutorGroup parent, Executor executor,
                                       boolean addTaskWakesUp, int maxPendingTasks,
                                       RejectedExecutionHandler rejectedHandler) {
       super(parent);
       this.addTaskWakesUp = addTaskWakesUp;
       this.maxPendingTasks = Math.max(16, maxPendingTasks);
       this.executor = ObjectUtil.checkNotNull(executor, "executor");
       taskQueue = newTaskQueue(this.maxPendingTasks);
       rejectedExecutionHandler = ObjectUtil.checkNotNull(rejectedHandler, "rejectedHandler");
   }

   b.父类构造执行结束后，再保存selectorProvider，和selectStrategy。
   c.为新创建的NioEventLoop创建一个selector：
   

   private Selector openSelector() {
       final Selector selector;
       try {
           selector = provider.openSelector();
       } catch (IOException e) {
           throw new ChannelException("failed to open a new selector", e);
       }
   
       if (DISABLE_KEYSET_OPTIMIZATION) {
           return selector;
       }
   
       final SelectedSelectionKeySet selectedKeySet = new SelectedSelectionKeySet();
   
       Object maybeSelectorImplClass = AccessController.doPrivileged(new PrivilegedAction<Object>() {
           @Override
           public Object run() {
               try {
                   return Class.forName(
                           "sun.nio.ch.SelectorImpl",
                           false,
                           PlatformDependent.getSystemClassLoader());
               } catch (ClassNotFoundException e) {
                   return e;
               } catch (SecurityException e) {
                   return e;
               }
           }
       });
   
       if (!(maybeSelectorImplClass instanceof Class) ||
               // ensure the current selector implementation is what we can instrument.
               !((Class<?>) maybeSelectorImplClass).isAssignableFrom(selector.getClass())) {
           if (maybeSelectorImplClass instanceof Exception) {
               Exception e = (Exception) maybeSelectorImplClass;
               logger.trace("failed to instrument a special java.util.Set into: {}", selector, e);
           }
           return selector;
       }
   
       final Class<?> selectorImplClass = (Class<?>) maybeSelectorImplClass;
   
       Object maybeException = AccessController.doPrivileged(new PrivilegedAction<Object>() {
           @Override
           public Object run() {
               try {
                   Field selectedKeysField = selectorImplClass.getDeclaredField("selectedKeys");
                   Field publicSelectedKeysField = selectorImplClass.getDeclaredField("publicSelectedKeys");
   
                   selectedKeysField.setAccessible(true);
                   publicSelectedKeysField.setAccessible(true);
   
                   selectedKeysField.set(selector, selectedKeySet);
                   publicSelectedKeysField.set(selector, selectedKeySet);
                   return null;
               } catch (NoSuchFieldException e) {
                   return e;
               } catch (IllegalAccessException e) {
                   return e;
               } catch (RuntimeException e) {
                   // JDK 9 can throw an inaccessible object exception here; since Netty compiles
                   // against JDK 7 and this exception was only added in JDK 9, we have to weakly
                   // check the type
                   if ("java.lang.reflect.InaccessibleObjectException".equals(e.getClass().getName())) {
                       return e;
                   } else {
                       throw e;
                   }
               }
           }
       });
   
       if (maybeException instanceof Exception) {
           selectedKeys = null;
           Exception e = (Exception) maybeException;
           logger.trace("failed to instrument a special java.util.Set into: {}", selector, e);
       } else {
           selectedKeys = selectedKeySet;
           logger.trace("instrumented a special java.util.Set into: {}", selector);
       }
   
       return selector;
   }

       再创建NioEventLoop的selector时，Netty除了调用JDK创建Selector外，还会将sun.nio.ch.SelectorImpl中保存SelectionKey的set集合利用反射用数组 SelectedSelectionKeySet替换掉该set集。
   

   Field selectedKeysField = selectorImplClass.getDeclaredField("selectedKeys");
   Field publicSelectedKeysField = selectorImplClass.getDeclaredField("publicSelectedKeys");
   
   selectedKeysField.setAccessible(true);
   publicSelectedKeysField.setAccessible(true);
   
   selectedKeysField.set(selector, selectedKeySet);
   publicSelectedKeysField.set(selector, selectedKeySet);

   SelectedSelectionKeySet添加SelectionKey时是直接在数组的最后有值位后添加，相比于set集合的add时间复杂度为O(1)，SelectedSelectionKeySet的效率更高。
   

   @Override
   public boolean add(SelectionKey o) {
       if (o == null) {
           return false;
       }
   
       if (isA) {
           int size = keysASize;
           keysA[size ++] = o;
           keysASize = size;
           if (size == keysA.length) {
               doubleCapacityA();
           }
       } else {
           int size = keysBSize;
           keysB[size ++] = o;
           keysBSize = size;
           if (size == keysB.length) {
               doubleCapacityB();
           }
       }
   
       return true;
   }        

7. 创建NioEventLoop旋转器：
   

   public EventExecutorChooser newChooser(EventExecutor[] executors) {
       if (isPowerOfTwo(executors.length)) {
           return new PowerOfTowEventExecutorChooser(executors);
       } else {
           return new GenericEventExecutorChooser(executors);
       }
   

   1).首选调用isPowerOfTwo(int val)判断NioEventLoopGroup的NioEventLoop线程组是否是2的幂:
   

   private static boolean isPowerOfTwo(int val) {
       return (val & -val) == val;
   }

   2).如果是2的幂执行PowerOfTowEventExecutorChooser的构造：
   

   private static final class PowerOfTowEventExecutorChooser implements EventExecutorChooser {
       private final AtomicInteger idx = new AtomicInteger();
       private final EventExecutor[] executors;
   
       PowerOfTowEventExecutorChooser(EventExecutor[] executors) {
           this.executors = executors;
       }
   
       @Override
       public EventExecutor next() {
           return executors[idx.getAndIncrement() & executors.length - 1];
       }
   }

           PowerOfTowEventExecutorChooser的构造保存NioEventLoop线程组，该类中还有一个next()方法，即是在有事件分配NioEventLoop时调用，下标是每次自增前&上线程数组的长度减1，利用机器码快速选择NioEventLoop；
   3).如果不是2的幂：
   

   private static final class GenericEventExecutorChooser implements EventExecutorChooser {
       private final AtomicInteger idx = new AtomicInteger();
       private final EventExecutor[] executors;
   
       GenericEventExecutorChooser(EventExecutor[] executors) {
           this.executors = executors;
       }
   
       @Override
       public EventExecutor next() {
           return executors[Math.abs(idx.getAndIncrement() % executors.length)];
       }
   }

           netty在不是2的幂时，分配NioEventLoop，调用next()获取NioEventLoop是自增前的值与线程组长度取余。
           显然与运算的效率要高于求余运算，可见Netty为了效率简直丧心病狂了。。。
   
8. 整个NioEventLoop初始化流程就是这些了。Netty既是采用这种主从多线程模型实现高效的异步处理事件。服务端启动两个NioEventLoopGroup，一个boss，一个work，boss负责接受请求分发任务，work负责异步处理任务。boss和work下又有多个 NioEventLoop具体来做事。