主要通过测试，推理memcached的存储机制。
平台 windows7
版本 memcached-1.2.6-win32
启动日志：
E:\memcached\memcached-1.2.6-win32-bin>memcached -m 32 -p 12001 -vv
slab class 1: chunk size 88 perslab 11915
slab class 2: chunk size 112 perslab 9362
slab class 3: chunk size 144 perslab 7281
slab class 4: chunk size 184 perslab 5698
slab class 5: chunk size 232 perslab 4519
slab class 6: chunk size 296 perslab 3542
slab class 7: chunk size 376 perslab 2788
slab class 8: chunk size 472 perslab 2221
slab class 9: chunk size 592 perslab 1771
slab class 10: chunk size 744 perslab 1409
slab class 11: chunk size 936 perslab 1120
slab class 12: chunk size 1176 perslab 891
slab class 13: chunk size 1472 perslab 712
slab class 14: chunk size 1840 perslab 569
slab class 15: chunk size 2304 perslab 455
slab class 16: chunk size 2880 perslab 364
slab class 17: chunk size 3600 perslab 291
slab class 18: chunk size 4504 perslab 232
slab class 19: chunk size 5632 perslab 186
slab class 20: chunk size 7040 perslab 148
slab class 21: chunk size 8800 perslab 119
slab class 22: chunk size 11000 perslab 95
slab class 23: chunk size 13752 perslab 76
slab class 24: chunk size 17192 perslab 60
slab class 25: chunk size 21496 perslab 48
slab class 26: chunk size 26872 perslab 39
slab class 27: chunk size 33592 perslab 31
slab class 28: chunk size 41992 perslab 24
slab class 29: chunk size 52496 perslab 19
slab class 30: chunk size 65624 perslab 15
slab class 31: chunk size 82032 perslab 12
slab class 32: chunk size 102544 perslab 10
slab class 33: chunk size 128184 perslab 8
slab class 34: chunk size 160232 perslab 6
slab class 35: chunk size 200296 perslab 5
slab class 36: chunk size 250376 perslab 4
slab class 37: chunk size 312976 perslab 3
slab class 38: chunk size 391224 perslab 2
slab class 39: chunk size 489032 perslab 2
<96 server listening
<112 server listening
<116 send buffer was 8192, now 268435456
<116 server listening (udp)
<120 new client connection
<120 exit
>120 ERROR
<120 quit
<120 connection closed.
<120 new client connection
日志只是显示了将来的分配策略，并未真正分配内存。
Slab表示块大小的级别，chunk表示slab中的一个块，还有个比较重要的概念就是page，page表示同一级slab大小的块的集合。
查看数据状态：
stats
STAT pid 12212
STAT uptime 10
STAT time 1367559145
STAT version 1.2.6
STAT pointer_size 32
STAT curr_items 0
STAT total_items 0
STAT bytes 0 --已存数据大小
STAT curr_connections 3
STAT total_connections 4
STAT connection_structures 4
STAT cmd_get 0
STAT cmd_set 0
STAT get_hits 0
STAT get_misses 0
STAT evictions 0
STAT bytes_read 11
STAT bytes_written 7
STAT limit_maxbytes 33554432 ---最大容量
STAT threads 1
END
Slab的大小是怎么算出来的呢？
slab大小的计算方式： 从0M到1M，按等比数列划分，默认比例因子是1.25.例如上例中：88*1.25=112.
存数据，测试代码：
public static void main(String[] arg){
byte[] arr=new byte[1024*6]; //使用7k的块
int i=0;
int n=i+140;
for(;i<n;i++){
boolean status=MemcachedUtil.getInstance().set(String.valueOf(System.currentTimeMillis()+""+i), arr,60*60);
if(!status){
System.out.println("------------------------------");
}
System.out.println(status);
}
}
结果：
stats slabs
STAT 20:chunk_size 7040 --块大小
STAT 20:chunks_per_page 148 ----每个page中块数量
STAT 20:total_pages 1 --------只有一个page
STAT 20:total_chunks 148 ---第20个slab级别的块数量总和= total_pages* chunks_per_page
STAT 20:used_chunks 148
STAT 20:free_chunks 0
STAT 20:free_chunks_end 8
STAT active_slabs 1
STAT total_malloced 1041920 ---所有数据占用内存=140*(7040+key所占内存)
END
stats items
STAT items:20:number 140
STAT items:20:age 241
STAT items:20:evicted 0
STAT items:20:outofmemory 0
END
stats sizes
6240 140
END
使用了第20个slab的块（7k的块），使用了140个，还剩8个。
再执行8次：
int i=0;
int n=i+8;
结果：
stats slabs
STAT 20:chunk_size 7040
STAT 20:chunks_per_page 148
STAT 20:total_pages 1
STAT 20:total_chunks 148
STAT 20:used_chunks 148
STAT 20:free_chunks 0
STAT 20:free_chunks_end 0 ---第一个page中148个chunk全部占用
STAT active_slabs 1
STAT total_malloced 1041920
END
stats items
STAT items:20:number 148
STAT items:20:age 407
STAT items:20:evicted 0
STAT items:20:outofmemory 0
END
那么再放一个
7k的块会怎么分配呢？
测试：
int i=0;
int n=i+1;
结果：
stats slabs
STAT 20:chunk_size 7040
STAT 20:chunks_per_page 148
STAT 20:total_pages 2 -----新加了一个page
STAT 20:total_chunks 296
STAT 20:used_chunks 296
STAT 20:free_chunks 0
STAT 20:free_chunks_end 147
STAT active_slabs 1
STAT total_malloced 2083840
END
stats items
STAT items:20:number 149 ---总过有149个块被使用
STAT items:20:age 1053
STAT items:20:evicted 0
STAT items:20:outofmemory 0
END
那么我用7k的块，把32M内存占完，之后再存数据看会是什么情况：
先测一下多少数据可以把内存沾满，经测试，当到以下状态时，再放数据，total_chunks不会增加，所以可见4736个7k的块可以把内存放满：
stats slabs
STAT 20:chunk_size 7040
STAT 20:chunks_per_page 148
STAT 20:total_pages 32
STAT 20:total_chunks 4736
STAT 20:used_chunks 4736
STAT 20:free_chunks 0
STAT 20:free_chunks_end 0
STAT active_slabs 1
STAT total_malloced 33341440
END
修改测试程序：
byte[] arr=new byte[1024*6]; //使用7k的块
MemcachedUtil.getInstance().set("first1", arr,60*60);
MemcachedUtil.getInstance().set("first2", arr,60*60);
MemcachedUtil.getInstance().set("first3", arr,60*60);
MemcachedUtil.getInstance().set("first4", arr,60*60);
int i=1;
int n=i+4732;
for(;i<n;i++){
boolean status=MemcachedUtil.getInstance().set(String.valueOf(i), arr,60*60);
}
MemcachedUtil.getInstance().set("last1", arr,60*60);
MemcachedUtil.getInstance().set("last2", arr,60*60);
MemcachedUtil.getInstance().set("last3", arr,60*60);
MemcachedUtil.getInstance().set("last4", arr,60*60);
System.out.println(MemcachedUtil.getInstance().get("first1"));
System.out.println(MemcachedUtil.getInstance().get("first2"));
System.out.println(MemcachedUtil.getInstance().get("first3"));
System.out.println(MemcachedUtil.getInstance().get("first4"));
System.out.println(MemcachedUtil.getInstance().get("last1"));
System.out.println(MemcachedUtil.getInstance().get("last2"));
System.out.println(MemcachedUtil.getInstance().get("last3"));
System.out.println(MemcachedUtil.getInstance().get("last4"));
输出：
null
null
null
null
[B@46a5c4
[B@2d09e0
[B@e38fca
[B@1f528ab
数据状态：
stats items
STAT items:20:number 4736
STAT items:20:age 87
STAT items:20:evicted 4
STAT items:20:outofmemory 0
END
可见，
last把first全部替换出去了。
那么再放一个其他大小块的数据呢？
byte[] arr=new byte[1024*8]; //使用8k的块
MemcachedUtil.getInstance().set("new1", arr,60*60);
System.out.println(MemcachedUtil.getInstance().get("new1"));
输出：
[B@1ff0a34
数据状态：
stats slabs
=================slab20的状态没变化
STAT 20:chunk_size 7040
STAT 20:chunks_per_page 148
STAT 20:total_pages 32
STAT 20:total_chunks 4736
STAT 20:used_chunks 4736
STAT 20:free_chunks 0
STAT 20:free_chunks_end 0
============数据被放在了slab21中，新开了一个slab21的page
STAT 21:chunk_size 8800
STAT 21:chunks_per_page 119
STAT 21:total_pages 1
STAT 21:total_chunks 119
STAT 21:used_chunks 119
STAT 21:free_chunks 0
STAT 21:free_chunks_end 118
STAT active_slabs 2
STAT total_malloced 34388640
END
stats items
STAT items:20:number 4736
STAT items:20:age 283
STAT items:20:evicted 4
STAT items:20:outofmemory 0
STAT items:21:number 1
STAT items:21:age 22
STAT items:21:evicted 0
STAT items:21:outofmemory 0
END
那么把slab21的这个page放满之后，如果再放数据会是什么情况呢？是继续新增slab21的page还是做替换呢？：
byte[] arr=new byte[1024*8]; //使用8k的块
int i=1;
int n=i+118;
for(;i<n;i++){
boolean status=MemcachedUtil.getInstance().set("newnew"+String.valueOf(i), arr,60*60);
}
System.out.println(MemcachedUtil.getInstance().get("newnew118"));
输出：
[B@c0fc8e
stats slabs
STAT 20:chunk_size 7040
STAT 20:chunks_per_page 148
STAT 20:total_pages 32
STAT 20:total_chunks 4736
STAT 20:used_chunks 4736
STAT 20:free_chunks 0
STAT 20:free_chunks_end 0
STAT 21:chunk_size 8800
STAT 21:chunks_per_page 119
STAT 21:total_pages 1
STAT 21:total_chunks 119
STAT 21:used_chunks 119
STAT 21:free_chunks 0
STAT 21:free_chunks_end 0 ---已全部被占用
STAT active_slabs 2
STAT total_malloced 34388640
END
stats items
STAT items:20:number 4736
STAT items:20:age 579
STAT items:20:evicted 4
STAT items:20:outofmemory 0
STAT items:21:number 119
STAT items:21:age 318
STAT items:21:evicted 0
STAT items:21:outofmemory 0
END
再放一个：
MemcachedUtil.getInstance().set("newnewnew0", arr,60*60);
System.out.println(MemcachedUtil.getInstance().get("newnewnew0"));
输出：
[B@1ff0a34
stats slabs
STAT 20:chunk_size 7040
STAT 20:chunks_per_page 148
STAT 20:total_pages 32
STAT 20:total_chunks 4736
STAT 20:used_chunks 4736
STAT 20:free_chunks 0
STAT 20:free_chunks_end 0
STAT 21:chunk_size 8800
STAT 21:chunks_per_page 119
STAT 21:total_pages 1 -----page没有增加，还是1
STAT 21:total_chunks 119
STAT 21:used_chunks 119
STAT 21:free_chunks 0
STAT 21:free_chunks_end 0
STAT active_slabs 2
STAT total_malloced 34388640
END
stats items
STAT items:20:number 4736
STAT items:20:age 725
STAT items:20:evicted 4
STAT items:20:outofmemory 0
STAT items:21:number 119
STAT items:21:age 162
STAT items:21:evicted 1 ----替换掉了一个slab21的块
STAT items:21:outofmemory 0
END
再放一个2k的块也一样：
byte[] arr=new byte[1024*2]; //使用2k的块
MemcachedUtil.getInstance().set("newnewnewnew0", arr,60*60);
System.out.println(MemcachedUtil.getInstance().get("newnewnewnew0"));
stats slabs
STAT 15:chunk_size 2304
STAT 15:chunks_per_page 455
STAT 15:total_pages 1
STAT 15:total_chunks 455
STAT 15:used_chunks 455
STAT 15:free_chunks 0
STAT 15:free_chunks_end 454
STAT 20:chunk_size 7040
STAT 20:chunks_per_page 148
STAT 20:total_pages 32
STAT 20:total_chunks 4736
STAT 20:used_chunks 4736
STAT 20:free_chunks 0
STAT 20:free_chunks_end 0
STAT 21:chunk_size 8800
STAT 21:chunks_per_page 119
STAT 21:total_pages 1
STAT 21:total_chunks 119
STAT 21:used_chunks 119
STAT 21:free_chunks 0
STAT 21:free_chunks_end 0
STAT active_slabs 3
STAT total_malloced 35436960
END
stats items
STAT items:15:number 1
STAT items:15:age 55
STAT items:15:evicted 0
STAT items:15:outofmemory 0
STAT items:20:number 4736
STAT items:20:age 1251
STAT items:20:evicted 4
STAT items:20:outofmemory 0
STAT items:21:number 119
STAT items:21:age 688
STAT items:21:evicted 1
STAT items:21:outofmemory 0
END
可见，当内存被沾满后，如果再存放一种新的slab大小的数据，那么memcached会新开一个且只开一个这种slab大小的page，当这个page被沾满后，就会执行数据替换。
这样算下来，memcached中的实际 数据总量可能就会超出启动时设置的32M，因为当一个slab大小的数据把内存沾满后，还可能为每个其他的slab大小的数据额为分配一个page，每个page的大小是1M。例如，如果有39个slab，那么就可能多占用38M的内存。而启动时 -f 因子的大小会决定slab数量的大小，所以也会影响到最后有多少数据会超出容量。
扫一扫
1162
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
