Netty定时任务调度器-HashedWheelTimer

Netty定时任务调度器-HashedWheelTimer

前段时间被问起定时器的设计，之前有使用过一些定时任务中间件，本文以Netty的HashedWheelTimer为例，阐述实现定时任务队列的基本思路。

1、基本数据结构

如其命名:HashedWheelTimer一样，HashedWheelTimer通过一定的hash规则将不同timeout的定时任务划分到HashedWheelBucket进行管理，而HashedWheelBucket利用双向链表结构维护了某一时刻需要执行的定时任务列表。

1.1 Worker

Worker继承自Runnable，HashedWheelTimer必须通过Worker线程操作HashedWheelTimer中的定时任务。Worker是整个HashedWheelTimer的执行流程管理者，控制了定时任务分配、全局deadline时间计算、管理未执行的定时任务、时钟计算、未执行定时任务回收处理。

1.2 HashedWheelTimeout

HashedWheelTimeout为HashedWheelTimer的执行单位，维护了其所属的HashedWheelTimer和HashedWheelBucket的引用、需要执行的任务逻辑、当前轮次以及当前任务的超时时间(不变)等，可以认为是自定义任务的一层Wrapper。

1.3 HashedWheelBucket

HashedWheelBucket维护了hash到其内的所有HashedWheelTimeout结构，是一个双向队列。

时间轮每一段格子代表一段时间，如1s,2s,…。以1s为例，当前时间轮可表示delayTimeout在[ns, 8ns]内的时刻。n表示需要转n次时间轮，n储存在每个各自对应的定时任务链表节点信息中，每过一轮该值减1。

根据任务延迟执行的时间对TimerWheel的ticksPerWheel值取模，取模算法采用java8中HashMap的hash算法思想，如：

2. HashdWheelTimer是如何工作的？

首先我们看下HashWheelTimer的构造函数都干了些啥。

• 初始化时间轮
• 创建Worker线程(一个HashedWheelTimer仅有一个Worker线程)
• 开启内存泄露检测

2.1 初始化时间轮HashedWheelTimer

初始化时间轮(HashedWheelTimer)时需要根据构造函数传入的ticksPerWheel对一轮的bucket数量做标准化：

private static int normalizeTicksPerWheel(int ticksPerWheel) {
    int normalizedTicksPerWheel = 1;
    while (normalizedTicksPerWheel < ticksPerWheel) {
        normalizedTicksPerWheel <<= 1;
    }
    return normalizedTicksPerWheel;
}

可以看到是从1逐步试探无符号左移1位来获取比ticksPerWheel大的最小数值，其实这里做了假设：ticksPerWheel不会很大。否则左移次数过多，执行效率低。Java8的HashMap的做法可以借鉴：

/**
* 理论基础：当length总是2的n次方时，h&(length-1)运算等价于对length取模，也就是 
* h%length，但是&比%具有更高的效率。
*/
private int normalizeTicksPerWheel(int ticksPerWheel) {
    int n = ticksPerWheel - 1;
    n |= n >>> 1;
    n |= n >>> 2;
    n |= n >>> 4;
    n |= n >>> 8;
    n |= n >>> 16;
    return (n < 0) ? 1 : (n >= 1073741824) ? 1073741824 : n + 1;
}

2.2 添加定时任务HashedWheelTimeout

时间轮已经有了，但我们还没有添加定时任务。此时如果调用start()方法启动时间轮，会发生空转浪费资源的情况(基于netty-all-4.1.6.Final)，没明白提供公共方法start()的用意。实际在调用newTimeout方法生成并添加定时任务，就会调用start()方法启动时间轮，这种”有需要再调用”的思路很合理！

public Timeout newTimeout(TimerTask task, long delay, TimeUnit unit) {
    if (task == null) {
        throw new NullPointerException("task");
    }
    if (unit == null) {
        throw new NullPointerException("unit");
    }
    start(); // 1

    // Add the timeout to the timeout queue which will be processed on the next tick.
    // During processing all the queued HashedWheelTimeouts will be added to the correct HashedWheelBucket.
    long deadline = System.nanoTime() + unit.toNanos(delay) - startTime;
    HashedWheelTimeout timeout = new HashedWheelTimeout(this, task, deadline);
    timeouts.add(timeout);
    return timeout;
}

public void start() {
        switch (WORKER_STATE_UPDATER.get(this)) {
            case WORKER_STATE_INIT:
                if (WORKER_STATE_UPDATER.compareAndSet(this, WORKER_STATE_INIT, WORKER_STATE_STARTED)) {
                    workerThread.start();
                }
                break;
            case WORKER_STATE_STARTED:
                break;
            case WORKER_STATE_SHUTDOWN:
                throw new IllegalStateException("cannot be started once stopped");
            default:
                throw new Error("Invalid WorkerState");
        }

        // Wait until the startTime is initialized by the worker.
        while (startTime == 0) { // 2
            try {
                startTimeInitialized.await();
            } catch (InterruptedException ignore) {
                // Ignore - it will be ready very soon.
            }
        }
    }

1，2处的代码会等待worker线程初始化完startTime后才执行后续的HashedWheelTimeout初始化工作，而在Worker执行时通过startTimeInitialized变量完成workerThread和timer线程间的通信，否则startTime将因没有被第一个加入的timeout初始化为当前系统时间影响后续时刻计算。

2.3 开始时间轮转

时刻计算、轮转以及任务执行均由Worker线程负责。

• 计算全局deadline
• 处理被取消的定时任务
• 从HashedWheelTimer维护的timeout列表中取出1W条纪录，计算出remainingRounds并放置到对应的bucket里
• 执行任务

// waitForNextTick()
long deadline = tickDuration * (tick + 1);
            for (;;) {
                final long currentTime = System.nanoTime() - startTime;
                long sleepTimeMs = (deadline - currentTime + 999999) / 1000000;

                if (sleepTimeMs <= 0) {
                    if (currentTime == Long.MIN_VALUE) {
                        return -Long.MAX_VALUE;
                    } else {
                        return currentTime;
                    }
                }

                // Check if we run on windows, as if thats the case we will need
                // to round the sleepTime as workaround for a bug that only affect
                // the JVM if it runs on windows.
                //
                // See https://github.com/netty/netty/issues/356
                if (PlatformDependent.isWindows()) {
                    sleepTimeMs = sleepTimeMs / 10 * 10;
                }

                try {
                    Thread.sleep(sleepTimeMs);
                } catch (InterruptedException ignored) {
                    if (WORKER_STATE_UPDATER.get(HashedWheelTimer.this) == WORKER_STATE_SHUTDOWN) {
                        return Long.MIN_VALUE;
                    }
                }
            }

将deadline(时间轮时刻)和当前时间对比，如果还没到deadline时刻，则sleep至deadline时刻；如果已经过了deadline时刻，则直接返回当前时间作为deadline，这里可能存在少许时间误差。

算好deadline后，遍历cancelledTimeout并remove。

取出1W条待执行的timeout纪录，并分配到正确的bucket里：

private void transferTimeoutsToBuckets() {
            // transfer only max. 100000 timeouts per tick to prevent a thread to stale the workerThread when it just
            // adds new timeouts in a loop.
            for (int i = 0; i < 100000; i++) {
                HashedWheelTimeout timeout = timeouts.poll();
                if (timeout == null) {
                    // all processed
                    break;
                }
                if (timeout.state() == HashedWheelTimeout.ST_CANCELLED) {
                    // Was cancelled in the meantime.
                    continue;
                }

                long calculated = timeout.deadline / tickDuration;
                timeout.remainingRounds = (calculated - tick) / wheel.length;

                final long ticks = Math.max(calculated, tick); // Ensure we don't schedule for past.
                int stopIndex = (int) (ticks & mask);

                HashedWheelBucket bucket = wheel[stopIndex];
                bucket.addTimeout(timeout);
            }
        }

分配好timeout后，根据当前时钟时刻获取bucket并开始执行定时任务：

public void expireTimeouts(long deadline) {
            HashedWheelTimeout timeout = head;

            // process all timeouts
            while (timeout != null) {
                boolean remove = false;
                if (timeout.remainingRounds <= 0) {
                    if (timeout.deadline <= deadline) {
                        timeout.expire();
                    } else {
                        // The timeout was placed into a wrong slot. This should never happen.
                        throw new IllegalStateException(String.format(
                                "timeout.deadline (%d) > deadline (%d)", timeout.deadline, deadline));
                    }
                    remove = true;
                } else if (timeout.isCancelled()) {
                    remove = true;
                } else {
                    timeout.remainingRounds --;
                }
                // store reference to next as we may null out timeout.next in the remove block.
                HashedWheelTimeout next = timeout.next;
                if (remove) {
                    remove(timeout);
                }
                timeout = next;
            }
        }

思路很清晰：如果remainingRounds<=0(如果timeout的deadline

关键点

• 超时计算方式
• 时钟tick流转并执行bucket下的timeout对应的task时，全局deadline(自增)的概念
• HashedWheelTimeout.deadline的含义(不变)
• bucket.expire()仅执行落在当前bucket内的timeout，否则视为异常

引用

• netty源码解读之时间轮算法实现-HashedWheelTimer