本文深入探讨Memcached的原理与应用，包括libevent事件处理、内存存储机制、安装配置、状态监控及常见问题解决，适合数据库缓存和高性能网络应用的学习。
Slab Allocator的基本原理是按照预先规定的大小，将分配的内存分割成特定长度的块(chunk)，并把尺寸相同的块分成组，以完全解决内存碎片问题。但由于分配的是特定长度的内存，因此无法有效利用分配的内存。比如将100字节的数据缓存到128字节的chunk中，剩余的28字节就浪费了。 Page：分配给Slab的内存空间，默认是1MB。分配给Slab之后根据slab的大小切分成chunk。 Chunk：用于缓存记录的内存空间。 Slab Class：特定大小的chunk的组。 memcached根据收到的数据的大小，选择最适合数据大小的slab。 memcached中保存着slab内空闲chunk的列表，根据该列表选择chunk，然后将数据缓存于其中。 数据过期方式：Lazy Expiration + LRU Lazy Expiration memcached内部不会监视记录是否过期，而是在get时查看记录的时间戳，检查记录是否过 期。这种技术被称为lazy（惰性）expiration。因此，memcached不会在过期监视上耗费CPU时间。 LRU memcached会优先使用已超时的记录的空间，但即使如此，也会发生追加新记录时空间不 足的情况，此时就要使用名为 Least Recently Used（LRU）机制来分配空间。当memcached的内存空间不足时（无法从slab class 获取到新的空间时），就从最近未被使用的记录中搜索，并将其空间分配给新的记录。 二．memcached安装 1.安装libevent # wget http://www.monkey.org/~provos/libevent-2.0.11-stable.tar.gz # tar zxvpf libevent-2.0.11-stable.tar.gz -C ../software/ # ./configure --prefix=/usr/local # make # make install 2.安装memcached # wget http://memcached.googlecode.com/files/memcached-1.4.5.tar.gz # tar zxvf memcached-1.4.5.tar.gz -C ../software/ # ./configure --prefix=/usr/local/memcached-1.4.5 --with-libevent=/usr/local # make # make install 3.相关错误以及解决方案 错误信息： ./memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory 解决方法： # whereis libevent-2.0.so.5 libevent-2.0.so: /usr/local/lib/libevent-2.0.so.5 # ldd /usr/local/memcached-1.4.5/bin/memcached libevent-2.0.so.5 => not found libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032f2600000) libc.so.6 => /lib64/libc.so.6 (0x00000032f1a00000) /lib64/ld-linux-x86-64.so.2 (0x00000032f1600000) # LD_DEBUG=libs ./memcached -v 9873: find library=libevent-2.0.so.5 [0]; searching 9873: search cache=/etc/ld.so.cache 9873: search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64 (system search path) 9873: trying file=/lib64/tls/x86_64/libevent-2.0.so.5 9873: trying file=/lib64/tls/libevent-2.0.so.5 9873: trying file=/lib64/x86_64/libevent-2.0.so.5 9873: trying file=/lib64/libevent-2.0.so.5 9873: trying file=/usr/lib64/tls/x86_64/libevent-2.0.so.5 9873: trying file=/usr/lib64/tls/libevent-2.0.so.5 9873: trying file=/usr/lib64/x86_64/libevent-2.0.so.5 9873: trying file=/usr/lib64/libevent-2.0.so.5 9873: ./memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory 搜索路径为：/lib64 /usr/lib64 因此做个软连接到/usr/lib64目录下： # ln -s /usr/local/lib/libevent-2.0.so.5 /usr/lib64/libevent-2.0.so.5 # ldd /usr/local/memcached-1.4.5/bin/memcached libevent-2.0.so.5 => /usr/lib64/libevent-2.0.so.5 (0x00002adde8270000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032f2600000) libc.so.6 => /lib64/libc.so.6 (0x00000032f1a00000) librt.so.1 => /lib64/librt.so.1 (0x00000032f2a00000) /lib64/ld-linux-x86-64.so.2 (0x00000032f1600000) 可看出已经ok了。很多时候遇到找不到lib库的问题可以通过上面的方法来解决。 三．启动memcached 1.memcached启动选项： -p TCP监听端口 (default: 11211) -U UDP 监听端口 (default: 11211, 0 is off) -s UNIX socket监听路径，不支持网络 -a UNIX socket访问掩码, 八进制 (default: 0700) -l <ip_addr> 监听的服务器IP地址 (default: all addresses) -d 启动一个守护进程 -r 最大限度利用核心文件限制 -u 运行memcached用户 -m 最大的内存使用 (default: 64 MB) -M 内存耗尽返回错误 -c 最大并发连接 (default: 1024) -k 锁定所有分页内存 -v 输出警告和错误信息 -vv 同时打印客户端请求和返回信息 -vvv 打印内部状态转换信息 -i 打印memcached 和 libevent 版本信息 -P 设置保存pid文件, only used with -d option -f 块大小增长倍数 (default: 1.25) -n key+value+flags最小分配空间(default: 48) -L 如何有效，尝试使用大内存页。增加内存页大小可以减少失误的TLB数量，提高性能。 -D 指定key和IDs的分隔符 default is ":" (colon). 如果指定此选项，统计信息收集自动开启; -t 使用的线程数量 (default: 4) -R 每个事件的最大请求数 (default: 20) -C 禁止使用 CAS -b 设置积压队列数限制 (default: 1024) -B 绑定协议 - one of ascii, binary, or auto (default) -I 分配给每个slab页(default: 1mb, min: 1k, max: 128m) 2.启动 # ./memcached -d -m 500 -u root -l 192.168.50.10 -p 12000 -c 256 -P /tmp/memcached.pid -vvv slab class 1: chunk size 96 perslab 10922 slab class 2: chunk size 120 perslab 8738 slab class 3: chunk size 152 perslab 6898 slab class 4: chunk size 192 perslab 5461 slab class 5: chunk size 240 perslab 4369 slab class 6: chunk size 304 perslab 3449 slab class 7: chunk size 384 perslab 2730 slab class 8: chunk size 480 perslab 2184 slab class 9: chunk size 600 perslab 1747 slab class 10: chunk size 752 perslab 1394 slab class 11: chunk size 944 perslab 1110 slab class 12: chunk size 1184 perslab 885 slab class 13: chunk size 1480 perslab 708 slab class 14: chunk size 1856 perslab 564 slab class 15: chunk size 2320 perslab 451 slab class 16: chunk size 2904 perslab 361 slab class 17: chunk size 3632 perslab 288 slab class 18: chunk size 4544 perslab 230 slab class 19: chunk size 5680 perslab 184 slab class 20: chunk size 7104 perslab 147 slab class 21: chunk size 8880 perslab 118 slab class 22: chunk size 11104 perslab 94 slab class 23: chunk size 13880 perslab 75 slab class 24: chunk size 17352 perslab 60 slab class 25: chunk size 21696 perslab 48 slab class 26: chunk size 27120 perslab 38 slab class 27: chunk size 33904 perslab 30 slab class 28: chunk size 42384 perslab 24 slab class 29: chunk size 52984 perslab 19 slab class 30: chunk size 66232 perslab 15 slab class 31: chunk size 82792 perslab 12 slab class 32: chunk size 103496 perslab 10 slab class 33: chunk size 129376 perslab 8 slab class 34: chunk size 161720 perslab 6 slab class 35: chunk size 202152 perslab 5 slab class 36: chunk size 252696 perslab 4 slab class 37: chunk size 315872 perslab 3 slab class 38: chunk size 394840 perslab 2 slab class 39: chunk size 493552 perslab 2 slab class 40: chunk size 616944 perslab 1 slab class 41: chunk size 771184 perslab 1 slab class 42: chunk size 1048576 perslab 1 使用12000端口，root用户，最大使用500M内存，253个并发连接，输出详细信息，以守护进程方式运行。 从输出信息可看出memcached分配内存的过程。 四．安装php的memcache扩展 # wget http://pecl.php.net/get/memcache-2.2.6.tgz # tar zxvf memcache-2.2.6.tgz -C ../software/ # /usr/local/php5.2.13/bin/phpize # ./configure --enable-memcache --with-php-config=/usr/local/php5.2.13/bin/php-config --with-zlib-dir # make # make install 编辑php.ini # vim /usr/local/Zend/etc/php.ini extension=/usr/local/php5.2.13/lib/php/extensions/no-debug-zts-20060613/memcache.so extension=memcache.so 五．查看memcached状态信息 1.telnet 192.168.50.10 12000 2.memstat命令 需要安装libmemcached # wget http://launchpad.net/libmemcached/1.0/0.49/+download/libmemcached-0.49.tar.gz # tar zxvf libmemcached-0.49.tar.gz -C ../software/ # ./configure --with-memcached # make # make install 提供一下几个工具： memcat - Copy the value of a key to standard output memflush - Flush the contents of your servers. memrm - Remove a key(s) from the server. memstat - Dump the stats of your servers to standard output memslap - Load generation utility (benchmark!) # memstat --server=192.168.50.10:12000 Server: 192.168.50.10 (12000) pid: 26204 uptime: 8 time: 1306137602 version: 1.4.5 pointer_size: 64 rusage_user: 0.000999 rusage_system: 0.000999 curr_connections: 5 total_connections: 6 connection_structures: 6 cmd_get: 0 cmd_set: 0 cmd_flush: 0 get_hits: 0 get_misses: 0 delete_misses: 0 delete_hits: 0 incr_misses: 0 incr_hits: 0 decr_misses: 0 decr_hits: 0 cas_misses: 0 cas_hits: 0 cas_badval: 0 auth_cmds: 0 auth_errors: 0 bytes_read: 16 bytes_written: 15 limit_maxbytes: 524288000 accepting_conns: 1 listen_disabled_num: 0 threads: 4 conn_yields: 0 bytes: 0 curr_items: 0 total_items: 0 evictions: 0 reclaimed: 0 3. memcached-tool # ./memcached-tool 192.168.50.10:12000 # Item_Size Max_age 1MB_pages Count Full? 1 96 B 149761 s 1 2259 no 2 120 B 149692 s 1 318 no 3 152 B 147345 s 1 1 no 4 192 B 113194 s 1 105 no 5 240 B 82055 s 1 92 no 6 304 B 81829 s 1 195 no 7 384 B 87061 s 1 12 no 8 480 B 149783 s 1 10 no 9 600 B 127498 s 1 1 no 10 752 B 45682 s 34 39453 no 11 944 B 45684 s 17 17876 no 12 1.2 kB 53027 s 62 54796 no 13 1.4 kB 75019 s 1 246 no 14 1.8 kB 74749 s 1 406 no 15 2.3 kB 149917 s 1 21 no 16 2.8 kB 147832 s 1 24 no 17 3.5 kB 149802 s 1 7 no 18 4.4 kB 87529 s 1 6 no 19 5.5 kB 87529 s 1 27 no 20 6.9 kB 87531 s 1 6 no 21 8.7 kB 94683 s 1 5 no 22 10.8 kB 149594 s 1 7 no 23 13.6 kB 132613 s 1 5 no 24 16.9 kB 124454 s 1 1 no 25 21.2 kB 148060 s 1 3 no 26 26.5 kB 67672 s 1 3 no 27 33.1 kB 74285 s 1 3 no 28 41.4 kB 147603 s 1 1 no 29 51.7 kB 147903 s 1 1 no 含义如下： # slab class编号 Item_Size chunk大小 Max_age LRU内最旧的记录的生存时间 1MB_pages 分配给slab的页数 Count slab内的记录数 Full？ Slab内是否含有空闲chunk 六．memcached监控 Cacti监控： # wget ftp://ftp.tummy.com/pub/python-memcached/python-memcached-latest.tar.gz # tar zxvf python-memcached-latest.tar.gz -C ../software/ # cd ../software/python-memcached-1.47/ # yum install python-setuptools # python setup.py install # wget http://s1.dlnws.com/dealnews/developers/cacti-memcached-1.0.tar.gz # tar zxvf cacti-memcached-1.0.tar.gz -C ../software/ # cd ../software/cacti-memcached-1.0/ # cp memcached.py /usr/local/cacti/scripts/ 按照cacti添加节点添加新的监控主机。 Nagios监控： define command{ command_name check_memcached command_line $USER1$/check_tcp -H $ARG1$ -p $ARG2$ -t $ARG3$ -E -s 'stats\r\nquit\r\n' -e 'uptime' -M crit } define service{ use local-service host_name XXX.XXX.XXX.XXX service_description 192.168.1.1 check_command check_memcached!192.168.1.1!12000!5 }
memcached使用详解 一．memcached介绍 1.基于libevent的事件处理 libevent是一套跨平台的事件处理接口的封装，能够兼容包括这些操作系统：Windows/Linux/BSD/Solaris 等操作系统的的事件处理。 包装的接口包括：poll、select(Windows)、epoll(Linux)、kqueue(BSD)、/dev/pool(Solaris) Memcached 使用libevent来进行网络并发连接的处理，能够保持在很大并发情况下，仍旧能够保持快速的响应能力。 libevent: http://www.monkey.org/~provos/libevent/ 2.内置内存存储方式 为了提高性能，memcached中保存的数据都存储在memcached内置的内存存储空间中。由于数据仅存在于内存中，因此重启memcached、重启操作系统会导致全部数据消失。另外，内容容量达到指定值之后，就基于LRU(Least Recently Used)算法自动删除不使用的缓存。 数据存储方式：Slab Allocation 结构图如下：
Slab Allocator的基本原理是按照预先规定的大小，将分配的内存分割成特定长度的块(chunk)，并把尺寸相同的块分成组，以完全解决内存碎片问题。但由于分配的是特定长度的内存，因此无法有效利用分配的内存。比如将100字节的数据缓存到128字节的chunk中，剩余的28字节就浪费了。 Page：分配给Slab的内存空间，默认是1MB。分配给Slab之后根据slab的大小切分成chunk。 Chunk：用于缓存记录的内存空间。 Slab Class：特定大小的chunk的组。 memcached根据收到的数据的大小，选择最适合数据大小的slab。 memcached中保存着slab内空闲chunk的列表，根据该列表选择chunk，然后将数据缓存于其中。 数据过期方式：Lazy Expiration + LRU Lazy Expiration memcached内部不会监视记录是否过期，而是在get时查看记录的时间戳，检查记录是否过 期。这种技术被称为lazy（惰性）expiration。因此，memcached不会在过期监视上耗费CPU时间。 LRU memcached会优先使用已超时的记录的空间，但即使如此，也会发生追加新记录时空间不 足的情况，此时就要使用名为 Least Recently Used（LRU）机制来分配空间。当memcached的内存空间不足时（无法从slab class 获取到新的空间时），就从最近未被使用的记录中搜索，并将其空间分配给新的记录。 二．memcached安装 1.安装libevent # wget http://www.monkey.org/~provos/libevent-2.0.11-stable.tar.gz # tar zxvpf libevent-2.0.11-stable.tar.gz -C ../software/ # ./configure --prefix=/usr/local # make # make install 2.安装memcached # wget http://memcached.googlecode.com/files/memcached-1.4.5.tar.gz # tar zxvf memcached-1.4.5.tar.gz -C ../software/ # ./configure --prefix=/usr/local/memcached-1.4.5 --with-libevent=/usr/local # make # make install 3.相关错误以及解决方案 错误信息： ./memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory 解决方法： # whereis libevent-2.0.so.5 libevent-2.0.so: /usr/local/lib/libevent-2.0.so.5 # ldd /usr/local/memcached-1.4.5/bin/memcached libevent-2.0.so.5 => not found libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032f2600000) libc.so.6 => /lib64/libc.so.6 (0x00000032f1a00000) /lib64/ld-linux-x86-64.so.2 (0x00000032f1600000) # LD_DEBUG=libs ./memcached -v 9873: find library=libevent-2.0.so.5 [0]; searching 9873: search cache=/etc/ld.so.cache 9873: search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64 (system search path) 9873: trying file=/lib64/tls/x86_64/libevent-2.0.so.5 9873: trying file=/lib64/tls/libevent-2.0.so.5 9873: trying file=/lib64/x86_64/libevent-2.0.so.5 9873: trying file=/lib64/libevent-2.0.so.5 9873: trying file=/usr/lib64/tls/x86_64/libevent-2.0.so.5 9873: trying file=/usr/lib64/tls/libevent-2.0.so.5 9873: trying file=/usr/lib64/x86_64/libevent-2.0.so.5 9873: trying file=/usr/lib64/libevent-2.0.so.5 9873: ./memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory 搜索路径为：/lib64 /usr/lib64 因此做个软连接到/usr/lib64目录下： # ln -s /usr/local/lib/libevent-2.0.so.5 /usr/lib64/libevent-2.0.so.5 # ldd /usr/local/memcached-1.4.5/bin/memcached libevent-2.0.so.5 => /usr/lib64/libevent-2.0.so.5 (0x00002adde8270000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032f2600000) libc.so.6 => /lib64/libc.so.6 (0x00000032f1a00000) librt.so.1 => /lib64/librt.so.1 (0x00000032f2a00000) /lib64/ld-linux-x86-64.so.2 (0x00000032f1600000) 可看出已经ok了。很多时候遇到找不到lib库的问题可以通过上面的方法来解决。 三．启动memcached 1.memcached启动选项： -p TCP监听端口 (default: 11211) -U UDP 监听端口 (default: 11211, 0 is off) -s UNIX socket监听路径，不支持网络 -a UNIX socket访问掩码, 八进制 (default: 0700) -l <ip_addr> 监听的服务器IP地址 (default: all addresses) -d 启动一个守护进程 -r 最大限度利用核心文件限制 -u 运行memcached用户 -m 最大的内存使用 (default: 64 MB) -M 内存耗尽返回错误 -c 最大并发连接 (default: 1024) -k 锁定所有分页内存 -v 输出警告和错误信息 -vv 同时打印客户端请求和返回信息 -vvv 打印内部状态转换信息 -i 打印memcached 和 libevent 版本信息 -P 设置保存pid文件, only used with -d option -f 块大小增长倍数 (default: 1.25) -n key+value+flags最小分配空间(default: 48) -L 如何有效，尝试使用大内存页。增加内存页大小可以减少失误的TLB数量，提高性能。 -D 指定key和IDs的分隔符 default is ":" (colon). 如果指定此选项，统计信息收集自动开启; -t 使用的线程数量 (default: 4) -R 每个事件的最大请求数 (default: 20) -C 禁止使用 CAS -b 设置积压队列数限制 (default: 1024) -B 绑定协议 - one of ascii, binary, or auto (default) -I 分配给每个slab页(default: 1mb, min: 1k, max: 128m) 2.启动 # ./memcached -d -m 500 -u root -l 192.168.50.10 -p 12000 -c 256 -P /tmp/memcached.pid -vvv slab class 1: chunk size 96 perslab 10922 slab class 2: chunk size 120 perslab 8738 slab class 3: chunk size 152 perslab 6898 slab class 4: chunk size 192 perslab 5461 slab class 5: chunk size 240 perslab 4369 slab class 6: chunk size 304 perslab 3449 slab class 7: chunk size 384 perslab 2730 slab class 8: chunk size 480 perslab 2184 slab class 9: chunk size 600 perslab 1747 slab class 10: chunk size 752 perslab 1394 slab class 11: chunk size 944 perslab 1110 slab class 12: chunk size 1184 perslab 885 slab class 13: chunk size 1480 perslab 708 slab class 14: chunk size 1856 perslab 564 slab class 15: chunk size 2320 perslab 451 slab class 16: chunk size 2904 perslab 361 slab class 17: chunk size 3632 perslab 288 slab class 18: chunk size 4544 perslab 230 slab class 19: chunk size 5680 perslab 184 slab class 20: chunk size 7104 perslab 147 slab class 21: chunk size 8880 perslab 118 slab class 22: chunk size 11104 perslab 94 slab class 23: chunk size 13880 perslab 75 slab class 24: chunk size 17352 perslab 60 slab class 25: chunk size 21696 perslab 48 slab class 26: chunk size 27120 perslab 38 slab class 27: chunk size 33904 perslab 30 slab class 28: chunk size 42384 perslab 24 slab class 29: chunk size 52984 perslab 19 slab class 30: chunk size 66232 perslab 15 slab class 31: chunk size 82792 perslab 12 slab class 32: chunk size 103496 perslab 10 slab class 33: chunk size 129376 perslab 8 slab class 34: chunk size 161720 perslab 6 slab class 35: chunk size 202152 perslab 5 slab class 36: chunk size 252696 perslab 4 slab class 37: chunk size 315872 perslab 3 slab class 38: chunk size 394840 perslab 2 slab class 39: chunk size 493552 perslab 2 slab class 40: chunk size 616944 perslab 1 slab class 41: chunk size 771184 perslab 1 slab class 42: chunk size 1048576 perslab 1 使用12000端口，root用户，最大使用500M内存，253个并发连接，输出详细信息，以守护进程方式运行。 从输出信息可看出memcached分配内存的过程。 四．安装php的memcache扩展 # wget http://pecl.php.net/get/memcache-2.2.6.tgz # tar zxvf memcache-2.2.6.tgz -C ../software/ # /usr/local/php5.2.13/bin/phpize # ./configure --enable-memcache --with-php-config=/usr/local/php5.2.13/bin/php-config --with-zlib-dir # make # make install 编辑php.ini # vim /usr/local/Zend/etc/php.ini extension=/usr/local/php5.2.13/lib/php/extensions/no-debug-zts-20060613/memcache.so extension=memcache.so 五．查看memcached状态信息 1.telnet 192.168.50.10 12000 2.memstat命令 需要安装libmemcached # wget http://launchpad.net/libmemcached/1.0/0.49/+download/libmemcached-0.49.tar.gz # tar zxvf libmemcached-0.49.tar.gz -C ../software/ # ./configure --with-memcached # make # make install 提供一下几个工具： memcat - Copy the value of a key to standard output memflush - Flush the contents of your servers. memrm - Remove a key(s) from the server. memstat - Dump the stats of your servers to standard output memslap - Load generation utility (benchmark!) # memstat --server=192.168.50.10:12000 Server: 192.168.50.10 (12000) pid: 26204 uptime: 8 time: 1306137602 version: 1.4.5 pointer_size: 64 rusage_user: 0.000999 rusage_system: 0.000999 curr_connections: 5 total_connections: 6 connection_structures: 6 cmd_get: 0 cmd_set: 0 cmd_flush: 0 get_hits: 0 get_misses: 0 delete_misses: 0 delete_hits: 0 incr_misses: 0 incr_hits: 0 decr_misses: 0 decr_hits: 0 cas_misses: 0 cas_hits: 0 cas_badval: 0 auth_cmds: 0 auth_errors: 0 bytes_read: 16 bytes_written: 15 limit_maxbytes: 524288000 accepting_conns: 1 listen_disabled_num: 0 threads: 4 conn_yields: 0 bytes: 0 curr_items: 0 total_items: 0 evictions: 0 reclaimed: 0 3. memcached-tool # ./memcached-tool 192.168.50.10:12000 # Item_Size Max_age 1MB_pages Count Full? 1 96 B 149761 s 1 2259 no 2 120 B 149692 s 1 318 no 3 152 B 147345 s 1 1 no 4 192 B 113194 s 1 105 no 5 240 B 82055 s 1 92 no 6 304 B 81829 s 1 195 no 7 384 B 87061 s 1 12 no 8 480 B 149783 s 1 10 no 9 600 B 127498 s 1 1 no 10 752 B 45682 s 34 39453 no 11 944 B 45684 s 17 17876 no 12 1.2 kB 53027 s 62 54796 no 13 1.4 kB 75019 s 1 246 no 14 1.8 kB 74749 s 1 406 no 15 2.3 kB 149917 s 1 21 no 16 2.8 kB 147832 s 1 24 no 17 3.5 kB 149802 s 1 7 no 18 4.4 kB 87529 s 1 6 no 19 5.5 kB 87529 s 1 27 no 20 6.9 kB 87531 s 1 6 no 21 8.7 kB 94683 s 1 5 no 22 10.8 kB 149594 s 1 7 no 23 13.6 kB 132613 s 1 5 no 24 16.9 kB 124454 s 1 1 no 25 21.2 kB 148060 s 1 3 no 26 26.5 kB 67672 s 1 3 no 27 33.1 kB 74285 s 1 3 no 28 41.4 kB 147603 s 1 1 no 29 51.7 kB 147903 s 1 1 no 含义如下： # slab class编号 Item_Size chunk大小 Max_age LRU内最旧的记录的生存时间 1MB_pages 分配给slab的页数 Count slab内的记录数 Full？ Slab内是否含有空闲chunk 六．memcached监控 Cacti监控： # wget ftp://ftp.tummy.com/pub/python-memcached/python-memcached-latest.tar.gz # tar zxvf python-memcached-latest.tar.gz -C ../software/ # cd ../software/python-memcached-1.47/ # yum install python-setuptools # python setup.py install # wget http://s1.dlnws.com/dealnews/developers/cacti-memcached-1.0.tar.gz # tar zxvf cacti-memcached-1.0.tar.gz -C ../software/ # cd ../software/cacti-memcached-1.0/ # cp memcached.py /usr/local/cacti/scripts/ 按照cacti添加节点添加新的监控主机。 Nagios监控： define command{ command_name check_memcached command_line $USER1$/check_tcp -H $ARG1$ -p $ARG2$ -t $ARG3$ -E -s 'stats\r\nquit\r\n' -e 'uptime' -M crit } define service{ use local-service host_name XXX.XXX.XXX.XXX service_description 192.168.1.1 check_command check_memcached!192.168.1.1!12000!5 }
Slab Allocator的基本原理是按照预先规定的大小，将分配的内存分割成特定长度的块(chunk)，并把尺寸相同的块分成组，以完全解决内存碎片问题。但由于分配的是特定长度的内存，因此无法有效利用分配的内存。比如将100字节的数据缓存到128字节的chunk中，剩余的28字节就浪费了。 Page：分配给Slab的内存空间，默认是1MB。分配给Slab之后根据slab的大小切分成chunk。 Chunk：用于缓存记录的内存空间。 Slab Class：特定大小的chunk的组。 memcached根据收到的数据的大小，选择最适合数据大小的slab。 memcached中保存着slab内空闲chunk的列表，根据该列表选择chunk，然后将数据缓存于其中。 数据过期方式：Lazy Expiration + LRU Lazy Expiration memcached内部不会监视记录是否过期，而是在get时查看记录的时间戳，检查记录是否过 期。这种技术被称为lazy（惰性）expiration。因此，memcached不会在过期监视上耗费CPU时间。 LRU memcached会优先使用已超时的记录的空间，但即使如此，也会发生追加新记录时空间不 足的情况，此时就要使用名为 Least Recently Used（LRU）机制来分配空间。当memcached的内存空间不足时（无法从slab class 获取到新的空间时），就从最近未被使用的记录中搜索，并将其空间分配给新的记录。 二．memcached安装 1.安装libevent # wget http://www.monkey.org/~provos/libevent-2.0.11-stable.tar.gz # tar zxvpf libevent-2.0.11-stable.tar.gz -C ../software/ # ./configure --prefix=/usr/local # make # make install 2.安装memcached # wget http://memcached.googlecode.com/files/memcached-1.4.5.tar.gz # tar zxvf memcached-1.4.5.tar.gz -C ../software/ # ./configure --prefix=/usr/local/memcached-1.4.5 --with-libevent=/usr/local # make # make install 3.相关错误以及解决方案 错误信息： ./memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory 解决方法： # whereis libevent-2.0.so.5 libevent-2.0.so: /usr/local/lib/libevent-2.0.so.5 # ldd /usr/local/memcached-1.4.5/bin/memcached libevent-2.0.so.5 => not found libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032f2600000) libc.so.6 => /lib64/libc.so.6 (0x00000032f1a00000) /lib64/ld-linux-x86-64.so.2 (0x00000032f1600000) # LD_DEBUG=libs ./memcached -v 9873: find library=libevent-2.0.so.5 [0]; searching 9873: search cache=/etc/ld.so.cache 9873: search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64 (system search path) 9873: trying file=/lib64/tls/x86_64/libevent-2.0.so.5 9873: trying file=/lib64/tls/libevent-2.0.so.5 9873: trying file=/lib64/x86_64/libevent-2.0.so.5 9873: trying file=/lib64/libevent-2.0.so.5 9873: trying file=/usr/lib64/tls/x86_64/libevent-2.0.so.5 9873: trying file=/usr/lib64/tls/libevent-2.0.so.5 9873: trying file=/usr/lib64/x86_64/libevent-2.0.so.5 9873: trying file=/usr/lib64/libevent-2.0.so.5 9873: ./memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory 搜索路径为：/lib64 /usr/lib64 因此做个软连接到/usr/lib64目录下： # ln -s /usr/local/lib/libevent-2.0.so.5 /usr/lib64/libevent-2.0.so.5 # ldd /usr/local/memcached-1.4.5/bin/memcached libevent-2.0.so.5 => /usr/lib64/libevent-2.0.so.5 (0x00002adde8270000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00000032f2600000) libc.so.6 => /lib64/libc.so.6 (0x00000032f1a00000) librt.so.1 => /lib64/librt.so.1 (0x00000032f2a00000) /lib64/ld-linux-x86-64.so.2 (0x00000032f1600000) 可看出已经ok了。很多时候遇到找不到lib库的问题可以通过上面的方法来解决。 三．启动memcached 1.memcached启动选项： -p TCP监听端口 (default: 11211) -U UDP 监听端口 (default: 11211, 0 is off) -s UNIX socket监听路径，不支持网络 -a UNIX socket访问掩码, 八进制 (default: 0700) -l <ip_addr> 监听的服务器IP地址 (default: all addresses) -d 启动一个守护进程 -r 最大限度利用核心文件限制 -u 运行memcached用户 -m 最大的内存使用 (default: 64 MB) -M 内存耗尽返回错误 -c 最大并发连接 (default: 1024) -k 锁定所有分页内存 -v 输出警告和错误信息 -vv 同时打印客户端请求和返回信息 -vvv 打印内部状态转换信息 -i 打印memcached 和 libevent 版本信息 -P 设置保存pid文件, only used with -d option -f 块大小增长倍数 (default: 1.25) -n key+value+flags最小分配空间(default: 48) -L 如何有效，尝试使用大内存页。增加内存页大小可以减少失误的TLB数量，提高性能。 -D 指定key和IDs的分隔符 default is ":" (colon). 如果指定此选项，统计信息收集自动开启; -t 使用的线程数量 (default: 4) -R 每个事件的最大请求数 (default: 20) -C 禁止使用 CAS -b 设置积压队列数限制 (default: 1024) -B 绑定协议 - one of ascii, binary, or auto (default) -I 分配给每个slab页(default: 1mb, min: 1k, max: 128m) 2.启动 # ./memcached -d -m 500 -u root -l 192.168.50.10 -p 12000 -c 256 -P /tmp/memcached.pid -vvv slab class 1: chunk size 96 perslab 10922 slab class 2: chunk size 120 perslab 8738 slab class 3: chunk size 152 perslab 6898 slab class 4: chunk size 192 perslab 5461 slab class 5: chunk size 240 perslab 4369 slab class 6: chunk size 304 perslab 3449 slab class 7: chunk size 384 perslab 2730 slab class 8: chunk size 480 perslab 2184 slab class 9: chunk size 600 perslab 1747 slab class 10: chunk size 752 perslab 1394 slab class 11: chunk size 944 perslab 1110 slab class 12: chunk size 1184 perslab 885 slab class 13: chunk size 1480 perslab 708 slab class 14: chunk size 1856 perslab 564 slab class 15: chunk size 2320 perslab 451 slab class 16: chunk size 2904 perslab 361 slab class 17: chunk size 3632 perslab 288 slab class 18: chunk size 4544 perslab 230 slab class 19: chunk size 5680 perslab 184 slab class 20: chunk size 7104 perslab 147 slab class 21: chunk size 8880 perslab 118 slab class 22: chunk size 11104 perslab 94 slab class 23: chunk size 13880 perslab 75 slab class 24: chunk size 17352 perslab 60 slab class 25: chunk size 21696 perslab 48 slab class 26: chunk size 27120 perslab 38 slab class 27: chunk size 33904 perslab 30 slab class 28: chunk size 42384 perslab 24 slab class 29: chunk size 52984 perslab 19 slab class 30: chunk size 66232 perslab 15 slab class 31: chunk size 82792 perslab 12 slab class 32: chunk size 103496 perslab 10 slab class 33: chunk size 129376 perslab 8 slab class 34: chunk size 161720 perslab 6 slab class 35: chunk size 202152 perslab 5 slab class 36: chunk size 252696 perslab 4 slab class 37: chunk size 315872 perslab 3 slab class 38: chunk size 394840 perslab 2 slab class 39: chunk size 493552 perslab 2 slab class 40: chunk size 616944 perslab 1 slab class 41: chunk size 771184 perslab 1 slab class 42: chunk size 1048576 perslab 1 使用12000端口，root用户，最大使用500M内存，253个并发连接，输出详细信息，以守护进程方式运行。 从输出信息可看出memcached分配内存的过程。 四．安装php的memcache扩展 # wget http://pecl.php.net/get/memcache-2.2.6.tgz # tar zxvf memcache-2.2.6.tgz -C ../software/ # /usr/local/php5.2.13/bin/phpize # ./configure --enable-memcache --with-php-config=/usr/local/php5.2.13/bin/php-config --with-zlib-dir # make # make install 编辑php.ini # vim /usr/local/Zend/etc/php.ini extension=/usr/local/php5.2.13/lib/php/extensions/no-debug-zts-20060613/memcache.so extension=memcache.so 五．查看memcached状态信息 1.telnet 192.168.50.10 12000 2.memstat命令 需要安装libmemcached # wget http://launchpad.net/libmemcached/1.0/0.49/+download/libmemcached-0.49.tar.gz # tar zxvf libmemcached-0.49.tar.gz -C ../software/ # ./configure --with-memcached # make # make install 提供一下几个工具： memcat - Copy the value of a key to standard output memflush - Flush the contents of your servers. memrm - Remove a key(s) from the server. memstat - Dump the stats of your servers to standard output memslap - Load generation utility (benchmark!) # memstat --server=192.168.50.10:12000 Server: 192.168.50.10 (12000) pid: 26204 uptime: 8 time: 1306137602 version: 1.4.5 pointer_size: 64 rusage_user: 0.000999 rusage_system: 0.000999 curr_connections: 5 total_connections: 6 connection_structures: 6 cmd_get: 0 cmd_set: 0 cmd_flush: 0 get_hits: 0 get_misses: 0 delete_misses: 0 delete_hits: 0 incr_misses: 0 incr_hits: 0 decr_misses: 0 decr_hits: 0 cas_misses: 0 cas_hits: 0 cas_badval: 0 auth_cmds: 0 auth_errors: 0 bytes_read: 16 bytes_written: 15 limit_maxbytes: 524288000 accepting_conns: 1 listen_disabled_num: 0 threads: 4 conn_yields: 0 bytes: 0 curr_items: 0 total_items: 0 evictions: 0 reclaimed: 0 3. memcached-tool # ./memcached-tool 192.168.50.10:12000 # Item_Size Max_age 1MB_pages Count Full? 1 96 B 149761 s 1 2259 no 2 120 B 149692 s 1 318 no 3 152 B 147345 s 1 1 no 4 192 B 113194 s 1 105 no 5 240 B 82055 s 1 92 no 6 304 B 81829 s 1 195 no 7 384 B 87061 s 1 12 no 8 480 B 149783 s 1 10 no 9 600 B 127498 s 1 1 no 10 752 B 45682 s 34 39453 no 11 944 B 45684 s 17 17876 no 12 1.2 kB 53027 s 62 54796 no 13 1.4 kB 75019 s 1 246 no 14 1.8 kB 74749 s 1 406 no 15 2.3 kB 149917 s 1 21 no 16 2.8 kB 147832 s 1 24 no 17 3.5 kB 149802 s 1 7 no 18 4.4 kB 87529 s 1 6 no 19 5.5 kB 87529 s 1 27 no 20 6.9 kB 87531 s 1 6 no 21 8.7 kB 94683 s 1 5 no 22 10.8 kB 149594 s 1 7 no 23 13.6 kB 132613 s 1 5 no 24 16.9 kB 124454 s 1 1 no 25 21.2 kB 148060 s 1 3 no 26 26.5 kB 67672 s 1 3 no 27 33.1 kB 74285 s 1 3 no 28 41.4 kB 147603 s 1 1 no 29 51.7 kB 147903 s 1 1 no 含义如下： # slab class编号 Item_Size chunk大小 Max_age LRU内最旧的记录的生存时间 1MB_pages 分配给slab的页数 Count slab内的记录数 Full？ Slab内是否含有空闲chunk 六．memcached监控 Cacti监控： # wget ftp://ftp.tummy.com/pub/python-memcached/python-memcached-latest.tar.gz # tar zxvf python-memcached-latest.tar.gz -C ../software/ # cd ../software/python-memcached-1.47/ # yum install python-setuptools # python setup.py install # wget http://s1.dlnws.com/dealnews/developers/cacti-memcached-1.0.tar.gz # tar zxvf cacti-memcached-1.0.tar.gz -C ../software/ # cd ../software/cacti-memcached-1.0/ # cp memcached.py /usr/local/cacti/scripts/ 按照cacti添加节点添加新的监控主机。 Nagios监控： define command{ command_name check_memcached command_line $USER1$/check_tcp -H $ARG1$ -p $ARG2$ -t $ARG3$ -E -s 'stats\r\nquit\r\n' -e 'uptime' -M crit } define service{ use local-service host_name XXX.XXX.XXX.XXX service_description 192.168.1.1 check_command check_memcached!192.168.1.1!12000!5 }
转载于:https://my.oschina.net/766/blog/211237
扫一扫
73
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
