【Netty权威指南学习笔记】Netty中的编解码技术

最新推荐文章于 2025-03-28 08:34:25 发布

a513142804

最新推荐文章于 2025-03-28 08:34:25 发布

阅读量191

点赞数

分类专栏： Netty小酒馆

本文链接：https://blog.youkuaiyun.com/a513142804/article/details/106720248

版权

Netty小酒馆专栏收录该内容

4 篇文章

订阅专栏

本文主要介绍了Netty中使用MessagePack和Protobuf进行编解码的技术。MessagePack作为一种高效的序列化方式，能减少码流大小并支持跨语言。在使用MessagePack时注意对象需使用@Message注解。Netty通过LengthFieldPrepender和LengthFieldBaseFrameDecoder解决TCP粘包和半包问题。对于Protobuf，详细讲解了从生成Java代码到实现客户端和服务端通信的全过程，特别提醒ProtobufDecoder不处理半包，需要结合其他解码器使用。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

java序列化缺点

1. 无法跨语言

目前几乎所有流行的java RPC通信框架都没有使用java序列化作为编解码框架。原因就在于无法跨语言。

2. 序列化后的码流太大

MessagePack

编码高效，性能高；

反序列化后码流小；

支持跨语言。

需要依赖的jar

<dependency>
      <groupId>org.msgpack</groupId>
      <artifactId>msgpack</artifactId>
      <version>0.6.12</version>
</dependency>

小坑：书上么有明确指出序列化对象要使用注解@Message标识，导致服务端总是收不到数据，坑的一批。

@Message
public class UserInfo

利用Netty的半包编码和解码器

LengthFieldPrepender 和 LengthFieldBaseFrameDecoder , 轻松解决TCP粘包和半包问题

.childHandler(new ChannelInitializer<SocketChannel>() {
                        protected void initChannel(SocketChannel socketChannel) throws Exception {
                            socketChannel.pipeline().addLast("frameDecoder", new LengthFieldBasedFrameDecoder(65535, 0, 2, 0, 2));
                            socketChannel.pipeline().addLast("msgpack decoder", new MsgpackDecoder());
                            socketChannel.pipeline().addLast("frameEncoder", new LengthFieldPrepender(2));
                            socketChannel.pipeline().addLast("msgpack encoder", new MsgpackEncoder());
                            socketChannel.pipeline().addLast(new EchoServerMsgHandler());
                        }
                    }

(这里半包或粘包会导致数据少读)

自定义编码器

public class MsgpackEncoder extends MessageToByteEncoder<Object> {

    @Override
    protected void encode(ChannelHandlerContext channelHandlerContext, Object o, ByteBuf byteBuf) throws Exception {

        MessagePack msgpack = new MessagePack();

        byte[] raw = msgpack.write(o);
        byteBuf.writeBytes(raw);
    }
}

自定义解码器

public class MsgpackDecoder extends MessageToMessageDecoder<ByteBuf> {

    @Override
    protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List<Object> list) throws Exception {
        final byte[] array;
        final int length = byteBuf.readableBytes();
        array = new byte[length];

        byteBuf.getBytes(byteBuf.readerIndex(), array, 0, length);
        MessagePack msgpack = new MessagePack();
        list.add(msgpack.read(array));
    }
}

Server

public class EchoMsgServer {

    public void bind(int port){

        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();

        try {

            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .option(ChannelOption.SO_BACKLOG, 100)
                    //.handler(new LoggingHandler(LogLevel.INFO))
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        protected void initChannel(SocketChannel socketChannel) throws Exception {
                            socketChannel.pipeline().addLast("frameDecoder", new LengthFieldBasedFrameDecoder(65535, 0, 2, 0, 2));
                            socketChannel.pipeline().addLast("msgpack decoder", new MsgpackDecoder());
                            socketChannel.pipeline().addLast("frameEncoder", new LengthFieldPrepender(2));
                            socketChannel.pipeline().addLast("msgpack encoder", new MsgpackEncoder());
                            socketChannel.pipeline().addLast(new EchoServerMsgHandler());
                        }
                    });

            // 绑定端口,同步等待绑定成功
            ChannelFuture f = b.bind(port).sync();
            // 等待服务监听端口关闭
            f.channel().closeFuture().sync();
        }catch (Exception e){
            e.printStackTrace();
        }finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }

    }


    public static void main(String[] args) {
        int port = 7788;
        new EchoMsgServer().bind(port);

    }
}

Client

public class EchoMsgClient {

    public void connect(int port, String host, final int sendNumber) {
        EventLoopGroup group = new NioEventLoopGroup();
        try {
            Bootstrap b = new Bootstrap();
            b.group(group).channel(NioSocketChannel.class)
                    .option(ChannelOption.TCP_NODELAY, true)
                    .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 3000)
                    .handler(new ChannelInitializer<SocketChannel>() {
                        protected void initChannel(SocketChannel socketChannel) throws Exception {
                            ByteBuf delimiter = Unpooled.copiedBuffer("$_".getBytes());
                            socketChannel.pipeline().addLast("frameDecoder", new LengthFieldBasedFrameDecoder(65535, 0, 2, 0, 2));
                            socketChannel.pipeline().addLast("msgpack decoder", new MsgpackDecoder());
                            socketChannel.pipeline().addLast("frameEncoder", new LengthFieldPrepender(2));
                            socketChannel.pipeline().addLast("msgpack encoder", new MsgpackEncoder());
                            socketChannel.pipeline().addLast(new EchoClientMsgHandler(sendNumber));

                        }
                    });

            ChannelFuture f = b.connect(host, port).sync();
            f.channel().closeFuture().sync();
        }catch (Exception e){
            e.printStackTrace();
        }finally {
            group.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        int port = 7788;
        new EchoMsgClient().connect(port, "127.0.0.1", 5);
    }
}

ClientHandler

public class EchoClientMsgHandler extends ChannelHandlerAdapter {

    private final int sendNumber;

    public EchoClientMsgHandler(int sendNumber) {
        this.sendNumber = sendNumber;
    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("客户端开始写入数据...");
        UserInfo[] infos = userInfo();
        for (UserInfo infoE: infos) {
            ctx.writeAndFlush(infoE);
            System.out.println("------> " + infoE.toString());
        }
        System.out.println("客户端写入数据完成...");
    }

    private UserInfo[] userInfo(){
        UserInfo[] userInfos = new UserInfo[sendNumber];
        UserInfo userInfo = null;
        for (int i = 0; i < sendNumber; i++) {
            userInfo = new UserInfo();
            userInfo.setName("张三"+i);
            userInfo.setAge(i);
            userInfo.setAddress("三地_" + i);
            userInfos[i] = userInfo;
        }
        return userInfos;
    }


    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        System.out.println("Client received the msgpack message : " + msg);
        //ctx.write(msg);
    }

    @Override
    public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
        ctx.flush();
    }
}

Protobuf

下载地址

安装使用可以参考这个兄弟的教程protobuf教程

准备工作

请求信息

syntax = "proto2";
package netty;
option java_package = "ibo.panghai.protocpackage.protobuf";
option java_outer_classname= "SubscribeReqProto";

message SubscribeReq{
	required int32 subReqID = 1;
	required string username = 2;
	required string productName = 3;
	required string address = 4;
}

响应信息

syntax = "proto2";
package netty;
option java_package = "ibo.panghai.protocpackage.protobuf";
option java_outer_classname= "SubscribeRespProto";

message SubscribeResp{
	required int32 subReqID = 1;
	required int32 respCode = 2;
	required string desc = 3;
}

执行protoc命令生成java代码

protoc -I=C:\Users\chenfb\Desktop\logs --java_out=C:\Users\chenfb\Desktop\logs SubscribeResp.proto

需要依赖的jar

<dependency>
      <groupId>com.google.protobuf</groupId>
      <artifactId>protobuf-java</artifactId>
      <version>3.12.2</version>
</dependency>

测试小例子

public class TestSubscribeReqProto {

    private static byte[] encode(SubscribeReqProto.SubscribeReq req){
        return req.toByteArray();
    }

    private static SubscribeReqProto.SubscribeReq decode(byte[] body) throws InvalidProtocolBufferException {
        return SubscribeReqProto.SubscribeReq.parseFrom(body);
    }

    private static SubscribeReqProto.SubscribeReq createSubscribeReq(){
        SubscribeReqProto.SubscribeReq.Builder builder = SubscribeReqProto.SubscribeReq.newBuilder();
        builder.setSubReqID(1);
        builder.setUsername("Netty Book");
        builder.setProductName("Netty");
        builder.setAddress("NanMen");
        /*List<String> address = new ArrayList<String>();
        address.add("Beijing");
        address.add("guanganmen");
        builder.getAddress(address);*/
        return builder.build();
    }


    public static void main(String[] args) throws InvalidProtocolBufferException {

        SubscribeReqProto.SubscribeReq req = createSubscribeReq();
        System.out.println("befor encode : " + req.toString());

        SubscribeReqProto.SubscribeReq req2 = decode(encode(req));
        System.out.println("after decode : " + req2.toString());

        System.out.println("assert equal : ---> " + req2.equals(req));
    }
}

升级demo

服务端

public class SubReqServer {

    public void bind(int port){
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap b = new ServerBootstrap();
            b.group(bossGroup, workerGroup).channel(NioServerSocketChannel.class)
                    .option(ChannelOption.SO_BACKLOG, 100)
                    .handler(new LoggingHandler(LogLevel.INFO))
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        protected void initChannel(SocketChannel socketChannel) throws Exception {
                            socketChannel.pipeline()
                                    .addLast(new ProtobufVarint32FrameDecoder())// 主要用于半包处理
                                    .addLast(new ProtobufDecoder(SubscribeReqProto.SubscribeReq.getDefaultInstance()))// 标识需要解码的目标类型
                                    .addLast(new ProtobufVarint32LengthFieldPrepender())
                                    .addLast(new ProtobufEncoder())
                                    .addLast(new SubReqServerHandler());
                        }
                    });


            ChannelFuture f = b.bind(port).sync();
            f.channel().closeFuture().sync();
        }catch (Exception e){
            e.printStackTrace();
        }finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }

    }


    public static void main(String[] args) {
        int port = 7788;
        new SubReqServer().bind(port);
    }
}

服务端处理类

public class SubReqServerHandler extends ChannelHandlerAdapter {

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        SubscribeReqProto.SubscribeReq req = (SubscribeReqProto.SubscribeReq) msg;
        if("Netty Book".equals(req.getUsername())){
            System.out.println("Server accept client subscribe req : [ " + req.toString() + " ]");
            ctx.writeAndFlush(resp(req.getSubReqID()));
        }
    }

    private SubscribeRespProto.SubscribeResp resp(int subReqID){
        SubscribeRespProto.SubscribeResp.Builder builder = SubscribeRespProto.SubscribeResp.newBuilder();
        builder.setSubReqID(subReqID);
        builder.setRespCode(0);
        builder.setDesc("Netty order succeed");
        return builder.build();
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        cause.printStackTrace();
        ctx.close();
    }
}

客户端

public class SubReqClient {

    public void connect(int port, String host){
        EventLoopGroup group = new NioEventLoopGroup();
        try {
            Bootstrap b = new Bootstrap();
            b.group(group).channel(NioSocketChannel.class)
                    .option(ChannelOption.TCP_NODELAY, true)
                    .handler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        protected void initChannel(SocketChannel socketChannel) throws Exception {
                            socketChannel.pipeline()
                                    .addLast(new ProtobufVarint32FrameDecoder())
                                    .addLast(new ProtobufDecoder(SubscribeRespProto.SubscribeResp.getDefaultInstance()))
                                    .addLast(new ProtobufVarint32LengthFieldPrepender())
                                    .addLast(new ProtobufEncoder())
                                    .addLast(new SubReqClientHandler());
                        }
                    });

            ChannelFuture f = b.connect(host, port).sync();
            f.channel().closeFuture().sync();
        }catch (Exception e){
            e.printStackTrace();
        }finally {
            group.shutdownGracefully();
        }
    }


    public static void main(String[] args) {
        int port = 7788;
        new SubReqClient().connect(port, "127.0.0.1");
    }
}

客户端处理类

public class SubReqClientHandler extends ChannelHandlerAdapter {


    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        for (int i = 0; i < 10; i++) {
            ctx.write(subReq(i));
        }
        ctx.flush();
    }

    private SubscribeReqProto.SubscribeReq subReq(int i){
        SubscribeReqProto.SubscribeReq.Builder builder = SubscribeReqProto.SubscribeReq.newBuilder();
        builder.setSubReqID(i);
        builder.setUsername("Netty Book");
        builder.setProductName("Netty");
        builder.setAddress("NanMen");
        return builder.build();
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        System.out.println("Receive server response : [ " + msg + " ]");
    }

    @Override
    public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
        ctx.flush();
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        cause.printStackTrace();
        ctx.close();
    }
}

注意事项：

ProtobufDecoder 仅仅负责解码，不支持读半包。因此解码前一定要有能够处理半读的解码器
1. 使用Netty提供的 ProtobufVarint32FrameDecoder

2. 继承Netty提供的通用半包解码器 LengthFieldBasedFrameDecoder

3. 继承 ByteToMessageDecoder类，自己处理半包消息