gateway 堆外内存溢出

最新推荐文章于 2023-11-15 10:34:34 发布

xiaowenligg

最新推荐文章于 2023-11-15 10:34:34 发布

阅读量3.1k

点赞数

文章标签：网关 java

本文链接：https://blog.youkuaiyun.com/asima8106/article/details/117417100

版权

本文详细记录了在Spring Cloud Gateway项目中遇到的堆外内存溢出问题，通过设置Netty内存泄漏检测、检查代码中资源释放、排查第三方库如Redis客户端Lettuce的内存使用，最终确定为Gateway版本问题。升级Spring Cloud Gateway版本至2.2.8.RELEASE并更新相关依赖，成功解决了内存溢出问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

gateway 网关运行一段时间后会堆外内存溢出

gateway 网关工程依赖版本如下：

  <properties>
        <java.version>1.8</java.version>
        <spring-cloud.version>Hoxton.SR3</spring-cloud.version>
        <spring-cloud-alibaba.version>2.2.3.RELEASE</spring-cloud-alibaba.version>
    </properties>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.6.RELEASE</version>
        <relativePath />
    </parent>

网关分配2G内存，当进程运行到2.9G时候开始所有请求报错

reactor.netty.ReactorNetty$InternalNettyException: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used: 2046820359, max: 2058354688)
Caused by: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used: 2046820359, max: 2058354688)
	at io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:775)
	at io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:730)
	at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:645)
	at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:621)
	at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:204)
	at io.netty.buffer.PoolArena.tcacheAllocateSmall(PoolArena.java:174)
	at io.netty.buffer.PoolArena.allocate(PoolArena.java:136)
	at io.netty.buffer.PoolArena.allocate(PoolArena.java:128)
	at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:378)
	at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
	at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
	at io.netty.channel.unix.PreferredDirectByteBufAllocator.ioBuffer(PreferredDirectByteBufAllocator.java:53)
	at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:114)
	at io.netty.channel.epoll.EpollRecvByteAllocatorHandle.allocate(EpollRecvByteAllocatorHandle.java:75)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:780)
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

解决问题：

1、启动加入 -Dio.netty.leakDetectionLevel=advanced 来看详细报错的代码。
2、检查代码，确认业务逻辑上是否有未关闭的堆外内存。发现只有一处使用到并且已经关闭。

public abstract class BaseFilter {
	/**
	 * 返回消息
	 * 
	 * @param exchange
	 * @param values
	 * @return
	 */
	protected Mono<Void> responseR(ServerWebExchange exchange, R r) {
		ServerHttpResponse response = exchange.getResponse();
		response.setStatusCode(HttpStatus.OK);
		response.getHeaders().setContentType(MediaType.APPLICATION_JSON);
		DataBuffer dataBuffer = response.bufferFactory().allocateBuffer()
				.write(JSON.toJSONString(r).getBytes(StandardCharsets.UTF_8));;
		try {
			return response.writeWith(Mono.just(dataBuffer));
		} finally {
			DataBufferUtils.release(dataBuffer);
		}
	}

	/**
	 * 从Flux<DataBuffer>中获取字符串的方法
	 * @return 请求体
	 */
	protected String resolveBodyFromRequest(ServerHttpRequest serverHttpRequest) {
		// 获取请求体
		Flux<DataBuffer> body = serverHttpRequest.getBody();
		AtomicReference<String> bodyRef = new AtomicReference<>();
		body.subscribe(buffer -> {
			try {
			bodyRef.set(StandardCharsets.UTF_8.decode(buffer.asByteBuffer()).toString());
			} finally {
				DataBufferUtils.release(buffer);
			}
		});
		return bodyRef.get();
	}

}

3、怀疑是网关中redis使用的Lettuce泄露，更换了redis连接池为Jedis还是一样不行。
4、打开Hoxton.SR3 发现依赖的gateway版本为2.2.2.RELEASE。网上有人说2.2.5以下的是有bug。直接更换Hoxton.SR11 的版本，依赖gateway为2.2.8.RELEASE。打包上线。解决。

  <properties>
        <java.version>1.8</java.version>
        <spring-cloud.version>Hoxton.SR11</spring-cloud.version>
        <spring-cloud-alibaba.version>2.2.3.RELEASE</spring-cloud-alibaba.version>
    </properties>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.6.RELEASE</version>
        <relativePath />
    </parent>