之前redis已经加过一次内存了,这次又加了16G,发现过了不久又爆了,这次感觉到是程序的问题了,而不是业务增长导致的。

这是使用工具看到的,43G的内存全部占满了。
查看了一些key,以为是业务的某些对象导致的,结果到开发环境排查,才发现是因为client_id_to_access 的token 导致的。
查看redis client_id_to_access :xx token 数据达到了一千万,一共3个client 。
删除后,内存瞬间下去


这是删除后,过了一两天就增长到了4万多条数据了。
查看了RedisTokenStore 发现token 会不断地往list塞值。
public void storeAccessToken(OAuth2AccessToken token, OAuth2Authentication authentication) {
byte[] serializedAccessToken = serialize(token);
byte[] serializedAuth = serialize(authentication);
byte[] accessKey = serializeKey(ACCESS + token.getValue());
byte[] authKey = serializeKey(AUTH + token.getValue());
byte[] authToAccessKey = serializeKey(AUTH_TO_ACCESS + authenticationKeyGenerator.extractKey(authentication));
byte[] approvalKey = serializeKey(UNAME_TO_ACCESS + getApprovalKey(authentication));
byte[] clientId = serializeKey(CLIENT_ID_TO_ACCESS + authentication.getOAuth2Request().getClientId());
RedisConnection conn = getConnection();
try {
conn.openPipeline();
if (springDataRedis_2_0) {
try {
this.redisConnectionSet_2_0.invoke(conn, accessKey, serializedAccessToken);
this.redisConnectionSet_2_0.invoke(conn, authKey, serializedAuth);
this.redisConnectionSet_2_0.invoke(conn, authToAccessKey, serializedAccessToken);
} catch (Exception ex) {
throw new RuntimeException(ex);
}
} else {
conn.set(accessKey, serializedAccessToken);
conn.set(authKey, serializedAuth);
conn.set(authToAccessKey, serializedAccessToken);
}
if (!authentication.isClientOnly()) {
conn.rPush(approvalKey, serializedAccessToken);
}
conn.rPush(clientId, serializedAccessToken);
if (token.getExpiration() != null) {
int seconds = token.getExpiresIn();
conn.expire(accessKey, seconds);
conn.expire(authKey, seconds);
conn.expire(authToAccessKey, seconds);
conn.expire(clientId, seconds);
conn.expire(approvalKey, seconds);
}
OAuth2RefreshToken refreshToken = token.getRefreshToken();
if (refreshToken != null && refreshToken.getValue() != null) {
byte[] refresh = serialize(token.getRefreshToken().getValue());
byte[] auth = serialize(token.getValue());
byte[] refreshToAccessKey = serializeKey(REFRESH_TO_ACCESS + token.getRefreshToken().getValue());
byte[] accessToRefreshKey = serializeKey(ACCESS_TO_REFRESH + token.getValue());
if (springDataRedis_2_0) {
try {
this.redisConnectionSet_2_0.invoke(conn, refreshToAccessKey, auth);
this.redisConnectionSet_2_0.invoke(conn, accessToRefreshKey, refresh);
} catch (Exception ex) {
throw new RuntimeException(ex);
}
} else {
conn.set(refreshToAccessKey, auth);
conn.set(accessToRefreshKey, refresh);
}
if (refreshToken instanceof ExpiringOAuth2RefreshToken) {
ExpiringOAuth2RefreshToken expiringRefreshToken = (ExpiringOAuth2RefreshToken) refreshToken;
Date expiration = expiringRefreshToken.getExpiration();
if (expiration != null) {
int seconds = Long.valueOf((expiration.getTime() - System.currentTimeMillis()) / 1000L)
.intValue();
conn.expire(refreshToAccessKey, seconds);
conn.expire(accessToRefreshKey, seconds);
}
}
}
conn.closePipeline();
} finally {
conn.close();
}
}
默认的实现方法 DefaultTokenServices,里面就写了如果失效了就会删除token,反之则不断塞值进去
public OAuth2AccessToken createAccessToken(OAuth2Authentication authentication) throws AuthenticationException {
OAuth2AccessToken existingAccessToken = tokenStore.getAccessToken(authentication);
OAuth2RefreshToken refreshToken = null;
if (existingAccessToken != null) {
if (existingAccessToken.isExpired()) {
if (existingAccessToken.getRefreshToken() != null) {
refreshToken = existingAccessToken.getRefreshToken();
// The token store could remove the refresh token when the
// access token is removed, but we want to
// be sure...
tokenStore.removeRefreshToken(refreshToken);
}
tokenStore.removeAccessToken(existingAccessToken);
}
else {
// Re-store the access token in case the authentication has changed
tokenStore.storeAccessToken(existingAccessToken, authentication);
return existingAccessToken;
}
}
// Only create a new refresh token if there wasn't an existing one
// associated with an expired access token.
// Clients might be holding existing refresh tokens, so we re-use it in
// the case that the old access token
// expired.
if (refreshToken == null) {
refreshToken = createRefreshToken(authentication);
}
// But the refresh token itself might need to be re-issued if it has
// expired.
else if (refreshToken instanceof ExpiringOAuth2RefreshToken) {
ExpiringOAuth2RefreshToken expiring = (ExpiringOAuth2RefreshToken) refreshToken;
if (System.currentTimeMillis() > expiring.getExpiration().getTime()) {
refreshToken = createRefreshToken(authentication);
}
}
OAuth2AccessToken accessToken = createAccessToken(authentication, refreshToken);
tokenStore.storeAccessToken(accessToken, authentication);
// In case it was modified
refreshToken = accessToken.getRefreshToken();
if (refreshToken != null) {
tokenStore.storeRefreshToken(refreshToken, authentication);
}
return accessToken;
}
最终原因就是 client 的token失效时间设置了永久导致的,这个坑也是够好玩的,坑了我几个星期。

本文分析了一个由于客户端token永久有效设置导致的Redis内存持续增长的问题。通过排查发现,由于token没有设置过期时间,使得每次认证都会将token存入Redis且不删除,最终导致内存占用激增。
4320





