Netty学习（四）—LengthFieldBasedFrameDecoder解码器

Netty学习（四）—LengthFieldBasedFrameDecoder解码器

LengthFieldBasedFrameDecoder和LengthFieldPrepender组合是解决TCP粘包和拆包问题的最佳方案，通过将消息分为消息头和消息体记录消息长度解决读半包问题；

个人主页：tuzhenyu’s page
原文地址：Netty学习（四）—LengthFieldBasedFrameDecoder解码器

(0)使用实例

• 在发送端添加LengthFiledPrepender编码器，为发送的消息添加表示消息长度的消息头

Bootstrap b = new Bootstrap();
b.group(group).channel(NioSocketChannel.class)
        .handler(new ChannelInitializer<SocketChannel>() {
            @Override
            protected void initChannel(SocketChannel socketChannel) throws Exception {
                socketChannel.pipeline().addLast("encoder", new LengthFieldPrepender(4,false));
                socketChannel.pipeline().addLast(new ClientHandler());
            }
        });

public void channelActive(ChannelHandlerContext ctx) throws Exception {
    byte[] req = null;
    ByteBuf buffer = null;
    StringBuilder sb = new StringBuilder();
    for (int i=0;i<100;i++){
        sb.append("abcdefghijklmnopqrstuvwxyz");
    }
    req = sb.toString().getBytes();
    buffer = Unpooled.buffer(req.length);
    buffer.writeBytes(req);
    ctx.writeAndFlush(buffer);
}

• 在接收端添加LengthFieldBasedFrameDecoder解码器，根据消息头中消息总长度接收消息并去除消息头往下传递消息体数据；

ServerBootstrap b= new ServerBootstrap();
b.group(bossGroup,workerGroup).channel(NioServerSocketChannel.class)
        .option(ChannelOption.SO_BACKLOG,1024)
        .childHandler(new ChannelInitializer<SocketChannel>() {
            @Override
            protected void initChannel(SocketChannel socketChannel) throws Exception {
                socketChannel.pipeline().addLast("decoder", new LengthFieldBasedFrameDecoder(Integer.MAX_VALUE,0,4,0,4));
                socketChannel.pipeline().addLast(new ServerHandler());
            }
        });

public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
    ByteBuf buf = (ByteBuf)msg;
    byte[] bytes = new byte[buf.readableBytes()];
    buf.readBytes(bytes);
    System.out.println("receive the bytes: "+new String(bytes,"UTF-8"));
}

• 发送消息的长度远远超过链路MTU，发送过程中肯定会出现TCP拆包问题，但是添加编码解码器后解决了半包问题

receive the bytes: abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnop...

(1)LengthFiledPrepender编码器

• LengthFiledPrepender编码器继承了MessageToMessageEncoder基础编码器，将传入的消息添加消息头(仅包含消息长度)后写入Socket通道；

• LengthFiledPrepender的构造器

  • lengthFieldLength表示消息长度的字节数，一般设为4字节

  • lengthIncludesLengthFieldLength表示length长度是否包括lengthFiledLength长度的字节，一般设为false，表示长度仅为消息体长度不包含消息头长度；

public LengthFieldPrepender(int lengthFieldLength, boolean lengthIncludesLengthFieldLength) {
    this(lengthFieldLength, 0, lengthIncludesLengthFieldLength);
}

• LengthFiledPrepender对传入的消息进行编码，将消息体长度写入到ByteBuf中

protected void encode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
    int length = msg.readableBytes() + lengthAdjustment;
    if (lengthIncludesLengthFieldLength) {
        length += lengthFieldLength;
    }

    if (length < 0) {
        throw new IllegalArgumentException(
                "Adjusted frame length (" + length + ") is less than zero");
    }

    switch (lengthFieldLength) {
    case 1:
        if (length >= 256) {
            throw new IllegalArgumentException(
                    "length does not fit into a byte: " + length);
        }
        out.add(ctx.alloc().buffer(1).order(byteOrder).writeByte((byte) length));
        break;
    case 2:
        if (length >= 65536) {
            throw new IllegalArgumentException(
                    "length does not fit into a short integer: " + length);
        }
        out.add(ctx.alloc().buffer(2).order(byteOrder).writeShort((short) length));
        break;
    case 3:
        if (length >= 16777216) {
            throw new IllegalArgumentException(
                    "length does not fit into a medium integer: " + length);
        }
        out.add(ctx.alloc().buffer(3).order(byteOrder).writeMedium(length));
        break;
    case 4:
        out.add(ctx.alloc().buffer(4).order(byteOrder).writeInt(length));
        break;
    case 8:
        out.add(ctx.alloc().buffer(8).order(byteOrder).writeLong(length));
        break;
    default:
        throw new Error("should not reach here");
    }
    out.add(msg.retain());
}

(2)LengthFieldBasedFrameDecoder解码器

• LengthFieldBasedFrameDecoder解码器继承ByteToMessageDecoder基础解码器，解析出消息长度后根据长度接收后续消息数据，并将完整的去除消息头的数据往下传递；

• LengthFieldBasedFrameDecoder的构造器

  • maxFrameLength：设定包的最大长度，超出包的最大长度netty将会做一些特殊处理；

    - lengthFieldOffset：指的是长度域的偏移量，表示跳过指定长度个字节之后的才是长度域LengthField；

    - lengthFieldLength：记录该帧数据长度的字段本身的长度；

    - initialBytesToStrip：从数据帧中跳过的字节数，表示获取完一个完整的数据包之后，忽略前面的指定的位数个字节，应用解码器拿到的就是不带长度域的数据包；

    - failFast：如果为true，则表示读取到长度域，TA的值的超过maxFrameLength，就抛出一个 TooLongFrameException，而为false表示只有当真正读取完长度域的值表示的字节之后，才会抛出 TooLongFrameException，默认情况下设置为true，建议不要修改，否则可能会造成内存溢出。

public LengthFieldBasedFrameDecoder(
        int maxFrameLength, int lengthFieldOffset, int lengthFieldLength,
        int lengthAdjustment, int initialBytesToStrip, boolean failFast) {
    this(
            ByteOrder.BIG_ENDIAN, maxFrameLength, lengthFieldOffset, lengthFieldLength,
            lengthAdjustment, initialBytesToStrip, failFast);
}

• LengthFieldBasedFrameDecoder调用decode()方法对传入的消息进行解码

  - 去除消息头还原消息本身数据，传递到下一个处理器

protected Object decode(ChannelHandlerContext ctx, ByteBuf in) throws Exception {
    if (discardingTooLongFrame) {
        long bytesToDiscard = this.bytesToDiscard;
        int localBytesToDiscard = (int) Math.min(bytesToDiscard, in.readableBytes());
        in.skipBytes(localBytesToDiscard);
        bytesToDiscard -= localBytesToDiscard;
        this.bytesToDiscard = bytesToDiscard;

        failIfNecessary(false);
    }

    if (in.readableBytes() < lengthFieldEndOffset) {
        return null;
    }

    int actualLengthFieldOffset = in.readerIndex() + lengthFieldOffset;
    long frameLength = getUnadjustedFrameLength(in, actualLengthFieldOffset, lengthFieldLength, byteOrder);

    if (frameLength < 0) {
        in.skipBytes(lengthFieldEndOffset);
        throw new CorruptedFrameException(
                "negative pre-adjustment length field: " + frameLength);
    }

    frameLength += lengthAdjustment + lengthFieldEndOffset;

    if (frameLength < lengthFieldEndOffset) {
        in.skipBytes(lengthFieldEndOffset);
        throw new CorruptedFrameException(
                "Adjusted frame length (" + frameLength + ") is less " +
                "than lengthFieldEndOffset: " + lengthFieldEndOffset);
    }

    if (frameLength > maxFrameLength) {
        long discard = frameLength - in.readableBytes();
        tooLongFrameLength = frameLength;

        if (discard < 0) {
            // buffer contains more bytes then the frameLength so we can discard all now
            in.skipBytes((int) frameLength);
        } else {
            // Enter the discard mode and discard everything received so far.
            discardingTooLongFrame = true;
            bytesToDiscard = discard;
            in.skipBytes(in.readableBytes());
        }
        failIfNecessary(true);
        return null;
    }

    // never overflows because it's less than maxFrameLength
    int frameLengthInt = (int) frameLength;
    if (in.readableBytes() < frameLengthInt) {
        return null;
    }

    if (initialBytesToStrip > frameLengthInt) {
        in.skipBytes(frameLengthInt);
        throw new CorruptedFrameException(
                "Adjusted frame length (" + frameLength + ") is less " +
                "than initialBytesToStrip: " + initialBytesToStrip);
    }
    in.skipBytes(initialBytesToStrip);

    // extract frame
    int readerIndex = in.readerIndex();
    int actualFrameLength = frameLengthInt - initialBytesToStrip;
    ByteBuf frame = extractFrame(ctx, in, readerIndex, actualFrameLength);
    in.readerIndex(readerIndex + actualFrameLength);
    return frame;
}

总结

• LengthFieldPrepender编码器和LengthFieldBasedFrameDecoder解码器结合通过添加添加消息头并在消息头中定义消息长度解决TCP粘包和拆包问题；