jodd.cache.LRUCache: 小巧的本地缓存, 及其并发bug

• LRU (Least recently used) 最近最少使用，如果数据最近被访问过，那么将来被访问的几率也更高。
• LFU (Least frequently used) 最不经常使用，如果一个数据在最近一段时间内使用次数很少，那么在将来一段时间内被使用的可能性也很小。

jodd.cache.LRUCache本身使用很简单,

static LRUCache<String, String> cache = new LRUCache(500000);

直接new 一个LRUCache, 并设置缓存大小即可, 来一条数据get一下, 没有就继续走, 有就返回, 并且每条最后都再put一次.  实测LRUCache 和LFUCache缓存命中率极其接近, 但LRUCache效率稍高(其实效率都很高, 相较于qps 700左右的redisCache).

        为什么不用redisCache, 实测redisCache的qps在700左右, 不能满足线上需求, 而LRUCache/LFUCache 10w缓存qps103w/128w, 100w缓存qps52w/73w.

        为什么不设置缓存过期时间, 实测不管是本地缓存还是redis缓存, 加了过期时间后性能都下降厉害, 如LRUCache 设置缓存10w, 1h过期时间, qps只有3000, 且设置过期时间后命中率相差不大. 

        为什么用jodd.cache.LRUCache, 因为先采用的org.redisson.cache.LRUCacheMap被坑了, 也同样有验证的并发bug. org.redisson.cache.LRUCacheMap 在多线程, 缓存满时, 会导致cpu占用100%! 并且被阻塞. 

        jodd.cache.LRUCache有坑吗, 有, jodd.cache.LRUCache也有严重的并发bug, 在多线程中对LRUCache.put时, LRUCache.size会超过初始化定义的size! 然后内存占用缓慢增加直到100%,  见下面测试代码跑出来的日志和内存信息. 

import jodd.cache.LRUCache;
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.File;
import java.util.List;
import java.util.concurrent.*;

public class LruStressTest {

    private final static Logger logger = LoggerFactory.getLogger(LruStressTest.class);

    public static void main(String[] args) throws Exception {
        logger.info("test start...");
        test2();
    }

    private final static Semaphore semaphore = new Semaphore(30);
    private static long total = 0L;
    private static long hitCount = 0L;

    private static LRUCache<String, String> cache = new LRUCache(500000);
    private static ArrayBlockingQueue<String> queue = new ArrayBlockingQueue<>(10000);

    static void test2() throws Exception {
        LruStressTest lruStressTest = new LruStressTest();

        //start queue thread
        ExecutorService singleThreadExecutor = Executors.newSingleThreadExecutor();
        singleThreadExecutor.submit(() -> {
            try {
                readToQueue();
            } catch (Exception e) {
                e.printStackTrace();
            }
        });

        //consumer thread
        long startMs = System.currentTimeMillis();
        ExecutorService taskThreadExecutor = Executors.newFixedThreadPool(30);
        for (; ; ) {
            semaphore.acquire();
            total++;
            String text = queue.take();
            taskThreadExecutor.submit(() -> {
                try {
                    String v = lruStressTest.getCacheValue(text);
                    if (StringUtils.isNotBlank(v)) {
                        hitCount++;
                    } else {
                        v = String.valueOf(System.currentTimeMillis());
                        TimeUnit.MILLISECONDS.sleep(1);
                    }
                    lruStressTest.putCacheValue(text, v);
                } catch (Exception e) {
                    e.printStackTrace();
                    throw new IllegalStateException("handle error");
                } finally {
                    semaphore.release();
                }
            });
            if (total % 10000 == 0){
                logger.info("total: {}, hits: {}, queue.size: {} spendMs: {}", total, hitCount, queue.size(), System.currentTimeMillis() - startMs);
                logger.info("cache.size: {}, getHitCount: {}, getMissCount: {}, isFull: {}",
                        cache.size(), cache.getHitCount(), cache.getMissCount(), cache.isFull() );
            }

        }

    }

    public String getCacheValue(String key){
        return cache.get(key);
    }

    //must add synchronized here !
    public void putCacheValue(String key, String value){
        cache.put(key, value);
    }

    //read local text file to queue, mock prd env
    private static void readToQueue() throws Exception {
        File dir = new File("E:\\Projects\\text\\");
        File[] files = dir.listFiles();
        int i = 0;
        for (;;){
            for (File file : files) {
                List<String> lines = FileUtils.readLines(file);
                for (String line : lines) {
                    queue.add(line);
                    i++;
                    if (i>=100){
                        if (queue.size()>0) TimeUnit.MILLISECONDS.sleep(10);
                        i = 0;
                    }
                }
            }
        }
    }


}

14:16:22.105 [main] INFO LruStressTest - total: 23700000, hits: 13556179, queue.size: 0 spendMs: 3248056
14:16:22.105 [main] INFO LruStressTest - cache.size: 9734776, getHitCount: 13045869, getMissCount: 9985785, isFull: true
14:16:29.920 [main] INFO LruStressTest - total: 23710000, hits: 13566116, queue.size: 0 spendMs: 3255871
14:16:29.920 [main] INFO LruStressTest - cache.size: 9734776, getHitCount: 13055325, getMissCount: 9985785, isFull: true

 

        跑出来的日志信息 cache.size: 9734776! 内存一直飙高! 所以 cache.put上一定要加个锁, 实测加锁后正常, 且性能无差异. 

    public synchronized String getCacheValue(String key){
        return cache.get(key);
    }
    
    public synchronized void putCacheValue(String key, String value){
        cache.put(key, value);
    }

16:06:59.832 [main] INFO LruStressTest - total: 43490000, hits: 7916914, queue.size: 0 spendMs: 5100450
16:06:59.832 [main] INFO LruStressTest - cache.size: 500000, getHitCount: 7917744, getMissCount: 35572254, isFull: true
16:07:01.045 [main] INFO LruStressTest - total: 43500000, hits: 7918649, queue.size: 0 spendMs: 5101663
16:07:01.045 [main] INFO LruStressTest - cache.size: 500000, getHitCount: 7919478, getMissCount: 35580522, isFull: true

        加锁后cache.size正常, 内存最高占用2G且后续回收正常了.