redis技术之旅九

最新推荐文章于 2024-10-23 22:05:01 发布
原创 最新推荐文章于 2024-10-23 22:05:01 发布 · 728 阅读 · 0 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#redis

redis技术之旅九 redis中间件 twemproxy介绍

twemproxy简介
redis在其3.0的发布版本中，开始支持自身的cluster。在这之前。redis本身并没有提供集群策略，而一些第三方开发的工具完成了这些需要。本文介绍一种第三方开源工具来完成redis的集群需要。
twemproxy：
twemproxy,也叫nutcraker。这是一个开源的redis和memcached代理服务器。除了twemproxy外还有许多其他的开源项目，在这里就不一一介绍了（RedBrige，Webdis，redis-proxy等），本文只着重介绍用twemproxy来实现redis集群和管理解决办法。通过twemproxy可以水平扩展redis服务器，有效避免单点故障的问题。因为增加了中间层，所以redis会有性能损失（在20%左右），这个需要在HA和性能之间做权衡。twemproxy可以把数据sharding到多个服务器，每台服务器是整个数据集的一部分。当有一台服务器宕机时，那么就失去了一部分数据，借助M-S，来保证整个数据集的完整性。

twemproxy编译安装
防止出现一些意外的情况将
automake
libtool
autoreconf
m4
都安装好。
下载nutcracker-0.2.1.tar.gz
tar -zxvf nutcracker-0.2.1.tar.gz
cd nutcracker-0.2.1
./configure
make
make install

nutcracker -h
This is nutcracker-0.2.1

Usage: nutcracker [-?hVdDt] [-v verbosity level] [-o output file]
[-c conf file] [-s stats port] [-a stats addr]
[-i stats interval] [-p pid file] [-m mbuf size]

Options:
-h, –help : this help
-V, –version : show version and exit
-t, –test-conf : test configuration for syntax errors and exit
-d, –daemonize : run as a daemon
-D, –describe-stats : print stats description and exit
-v, –verbosity=N : set logging level (default: 5, min: 0, max: 11)
-o, –output=S : set logging file (default: stderr)
-c, –conf-file=S : set configuration file (default: conf/nutcracker.yml)
-s, –stats-port=N : set stats monitoring port (default: 22222)
-a, –stats-addr=S : set stats monitoring ip (default: 0.0.0.0)
-i, –stats-interval=N : set stats aggregation interval in msec (default: 30000 msec)
-p, –pid-file=S : set pid file (default: off)
-m, –mbuf-size=N : set size of mbuf chunk in bytes (default: 16384 bytes)

看到这些输出，证明安装成功。
-d, –daemonize : run as a daemon
守护进程运行
-D, –describe-stats : print stats description and exit
描述
-v, –verbosity=N : set logging level (default: 5, min: 0, max: 11)
默认情况下debug日志是不可用的，不过在生产环境中打开debug日志并将日志级别设置为verbosity,LOG_INFO(-v 6 或者 –verbosity=6)
-c, –conf-file=S : set configuration file (default: conf/nutcracker.yml)
如果不指定此选项，默认为conf/nutcracker.yml，我们也可以设定自己的配置文件；

编辑配置文件：
cat nutcracker.yml
alpha:
listen: 172.21.24.20:55555
hash: fnv1a_64
distribution: ketama
auto_eject_hosts: true
redis: true
server_retry_timeout: 2000
server_failure_limit: 1
servers:
- 172.21.24.20:6679:1
- 172.21.24.20:6779:1

启动nutcracker ：
nutcracker -d -c /tmp/nutcracker-0.2.1/conf/nutcracker.yml -p /data01/redis/nutcracker/redisproxy.pid -o /data01/redis/nutcracker/redisproxy.log -v 6

配置文件参数：
listen: The listening address and port (name:port or ip:port) for this server pool.
nutcracker服务器池监听端口和地址;

hash: The name of the hash function
hash函数，支持md5,crc16,crc32,finv1a_32等十多种；

hash_tag: A two character string that specifies the part of the key used for hashing. Eg “{}” or “$$”.
Hash tagenable mapping different keys to the same server as long as the part of the key within the tag
is the same.
启用hash tags 意味着你将使用key的一部分来计算hash值，当hash tages存在的时候 ,我们使用在标签内的key的一部分
来构建一致性hash,其他情况，我们使用全部的key去构建。hash tags能够让你将不同的key(只要在标签内的部分相同)映
射到相同的服务器上。例如，服务池beta的配置如下，指定了两个hash_tag字符“{}”,这意味着keys”user:{user1}:ids”
和 “user:{user1}:tweets” 将会映射到同一台服务器上，因为我们使用的是’user1′来计算hash,对于key
“user:user1:ids”将会是用整个字符串来计算hash,所以可能会映射到其他服务器上。

timeout: The timeout value in msec that we wait for to establish a connection to the server or receive
a response from a server. By default, we wait indefinitely.
为nutcracker的每一个服务池配置timeout 要比仅仅依靠客户端超时要好的多。
比如 ：timeout: 400
仅仅依靠客户端超时设置并不能达到理想的超时效果，反而起到了相反的作用，因为客户端的超时设置在这里变成了
客户端对代理的超时，但代理对服务端的链接是一直保持的，客户端重试请求对于服务端是没有效果的。默认情况下，
任何发送给服务端的请求，nutcracker都会无限期的等待，当timeout被设置后，如果在timeout的时间过后还没有从
服务端得到回应，这时会将超时错误信息SERVER_ERROR Connection time out发送给客户端。

backlog: The TCP backlog argument. Defaults to 512.

preconnect: A boolean value that controls if nutcracker should preconnect to all the servers in this pool
on process start. Defaults to false.

redis: A boolean value that controls if a server pool speaks redis or memcached protocol.
Defaults to false.

server_connections: The maximum number of connections that can be opened to each server.
By default, we open at most 1 server connection.
twemproxy的设计意图是通过少量的服务端的链接来响应更多的客户端的链接，但是需要注意的是当twemproxy配置了
server_connections: > 1时，情况就不一定是这样的。为了说明这一点，假设在twemproxy 配置为
server_connections: 2的场景下，一个客户端发出了以set foo 0 0 3\r\nbar\r\n（写）为开始，
然后第二个命令是get foo\r\n (读)的piplined请求，期望的结果是读取foo的时候能够返回bar,然而，
由于配置了两个服务端链接，读写请求可能被发送到不同的链接上，也意味着他们的执行顺序要看哪一个先到达服务端，
总结一下，如果客户端期望得到的是我最后写的内容，需要将twemproxy配置为 server_connections:1或者客户端
只发起同步的请求。

auto_eject_hosts: A boolean value that controls if server should be ejected temporarily when it fails
consecutively server_failure_limit times. See liveness recommendations for information. Defaults to false.

server_retry_timeout: The timeout value in msec to wait for before retrying on a temporarily ejected
server, when auto_eject_host is set to true. Defaults to 30000 msec.

server_failure_limit: The number of consecutive failures on a server that would lead to it being
temporarily ejected when auto_eject_host is set to true. Defaults to 2.

servers: A list of server address, port and weight (name:port:weight or ip:port:weight) for this server pool.
twemproxy上代理的服务实例可以通过两种字符串格式指定‘host:port:weight’ 或者 ‘host:port:weight name’.

高容量redis集群部署方案：
面对大容量redis实例的需求，根据以往的经验和测试结论，redis单实例的性能控制在8G以内，而实际应用对redis的需求可能远远大于这个值。因此需要找到一个redis性能不下降，但可以扩充redis的容量的方案。twemproxy可以解决这个问题。
扩充twemproxy的并发能力，twemproxy集群能提供的并发能力是其后端所有redis服务器的综合数量（小于），而扩充并发量的方式是增加twemproxy的集群访问入口数量。

Redis Command Support

Keys Command

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|        DEL        |    Yes     | DEL key [key …]                                                                                                     |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       DUMP        |    Yes     | DUMP key                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      EXISTS       |    Yes     | EXISTS key                                                                                                          |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      EXPIRE       |    Yes     | EXPIRE key seconds                                                                                                  |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     EXPIREAT      |    Yes     | EXPIREAT key timestamp                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       KEYS        |    No      | KEYS pattern                                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      MIGRATE      |    No      | MIGRATE host port key destination-db timeout                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       MOVE        |    No      | MOVE key db                                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      OBJECT       |    No      | OBJECT subcommand [arguments [arguments …]]                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      PERSIST      |    Yes     | PERSIST key                                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      PEXPIRE      |    Yes     | PEXPIRE key milliseconds                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     PEXPIREAT     |    Yes     | PEXPIREAT key milliseconds-timestamp                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      PTTL         |    Yes     | PTTL key                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     RANDOMKEY     |    No      | RANDOMKEY                                                                                                           |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      RENAME       |    No      | RENAME key newkey                                                                                                   |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     RENAMENX      |    No      | RENAMENX key newkey                                                                                                 |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      RESTORE      |    Yes     | RESTORE key ttl serialized-value                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SORT         |    Yes     | SORT key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination]   |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       TTL         |    Yes     | TTL key                                                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      TYPE         |    Yes     | TYPE key                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SCAN         |    No      | SCAN cursor [MATCH pattern] [COUNT count]                                                                           |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

Strings Command

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       APPEND      |    Yes     | APPEND key value                                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      BITCOUNT     |    Yes     | BITCOUNT key [start] [end]                                                                                          |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      BITPOS       |    Yes     | BITPOS key bit [start] [end]                                                                                          |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       BITOP       |    No      | BITOP operation destkey key [key ...]                                                                               |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       DECR        |    Yes     | DECR key                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      DECRBY       |    Yes     | DECRBY key decrement                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       GET         |    Yes     | GET key                                                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      GETBIT       |    Yes     | GETBIT key offset                                                                                                   |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     GETRANGE      |    Yes     | GETRANGE key start end                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      GETSET       |    Yes     | GETSET key value                                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      INCR         |    Yes     | INCR key                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      INCRBY       |    Yes     | INCRBY key increment                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     INCRBYFLOAT   |    Yes     | INCRBYFLOAT key increment                                                                                           |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      MGET         |    Yes     | MGET key [key ...]                                                                                                  |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      MSET         |    Yes*    | MSET key value [key value ...]                                                                                      | 
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      MSETNX       |    No      | MSETNX key value [key value ...]                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      PSETEX       |    Yes     | PSETEX key milliseconds value                                                                                       |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SET          |    Yes     | SET key value [EX seconds] [PX milliseconds] [NX|XX]                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SETBIT       |    Yes     | SETBIT key offset value                                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SETEX        |    Yes     | SETEX key seconds value                                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SETNX        |    Yes     | SETNX key value                                                                                                     |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SETRANGE     |    Yes     | SETRANGE key offset value                                                                                           |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      STRLEN       |    Yes     | STRLEN key                                                                                                          |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
  • MSET support is not Atomic

Hashes

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       HDEL        |    Yes     | HDEL key field [field ...]                                                                                          |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HEXISTS      |    Yes     | HEXISTS key field                                                                                                   |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       HGET        |    Yes     | HGET key field                                                                                                      |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HGETALL      |    Yes     | HGETALL key                                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HINCRBY      |    Yes     | HINCRBY key field increment                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    HINCRBYFLOAT   |    Yes     | HINCRBYFLOAT key field increment                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HKEYS        |    Yes     | HKEYS key                                                                                                           |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HLEN         |    Yes     | HLEN key                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HMGET        |    Yes     | HMGET key field [field ...]                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HMSET        |    Yes     | HMSET key field value [field value ...]                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HSET         |    Yes     | HSET key field value                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HSETNX       |    Yes     | HSETNX key field value                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HVALS        |    Yes     | HVALS key                                                                                                           |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      HSCAN        |    Yes     | HSCAN key cursor [MATCH pattern] [COUNT count]                                                                      |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

Lists

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       BLPOP       |    No      | BLPOP key [key ...] timeout                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       BRPOP       |    No      | BRPOP key [key ...] timeout                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     BRPOPLPUSH    |    No      | BRPOPLPUSH source destination timeout                                                                               |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      LINDEX       |    Yes     | LINDEX key index                                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      LINSERT      |    Yes     | LINSERT key BEFORE|AFTER pivot value                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      LLEN         |    Yes     | LLEN key                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      LPOP         |    Yes     | LPOP key                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      LPUSH        |    Yes     | LPUSH key value [value ...]                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      LPUSHX       |    Yes     | LPUSHX key value                                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      LRANGE       |    Yes     | LRANGE key start stop                                                                                               |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      LREM         |    Yes     | LREM key count value                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      LSET         |    Yes     | LSET key index value                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      LTRIM        |    Yes     | LTRIM key start stop                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      RPOP         |    Yes     | RPOP key                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     RPOPLPUSH     |    Yes*    | RPOPLPUSH source destination                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      RPUSH        |    Yes     | RPUSH key value [value ...]                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      RPUSHX       |    Yes     | RPUSHX key value                                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
  • RPOPLPUSH support requires that source and destination keys hash to the same server. You can ensure this by using the same hashtag for source and destination key. Twemproxy does no checking on its end to verify that source and destination key hash to the same server, and the RPOPLPUSH command is forwarded to the server that the source key hashes to

Sets

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SADD         |    Yes     | SADD key member [member ...]                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SCARD        |    Yes     | SCARD key                                                                                                           |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SDIFF        |    Yes*    | SDIFF key [key ...]                                                                                                 |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     SDIFFSTORE    |    Yes*    | SDIFFSTORE destination key [key ...]                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SINTER       |    Yes*    | SINTER key [key ...]                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    SINTERSTORE    |    Yes*    | SINTERSTORE destination key [key ...]                                                                               |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     SISMEMBER     |    Yes     | SISMEMBER key member                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     SMEMBERS      |    Yes     | SMEMBERS key                                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SMOVE        |    Yes*    | SMOVE source destination member                                                                                     |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SPOP         |    Yes     | SPOP key                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    SRANDMEMBER    |    Yes     | SRANDMEMBER key [count]                                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SREM         |    Yes     | SREM key member [member ...]                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     SUNION        |    Yes*    | SUNION key [key ...]                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|   SUNIONSTORE     |    Yes*    | SUNIONSTORE destination key [key ...]                                                                               |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SSCAN        |    Yes     | SSCAN key cursor [MATCH pattern] [COUNT count]                                                                      |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
  • SIDFF, SDIFFSTORE, SINTER, SINTERSTORE, SMOVE, SUNION and SUNIONSTORE support requires that the supplied keys hash to the same server. You can ensure this by using the same hashtag for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.

Sorted Sets

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      ZADD         |    Yes     | ZADD key score member [score] [member]                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      ZCARD        |    Yes     | ZCARD key                                                                                                           |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      ZCOUNT       |    Yes     | ZCOUNT key min max                                                                                                  |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      ZINCRBY      |    Yes     | ZINCRBY key increment member                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     ZINTERSTORE   |    Yes*    | ZINTERSTORE destination numkeys key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX]                 |
+------------------------------------------------------------------------------------------------------------------------------------------------------+
|      ZLEXCOUNT    |    Yes     | ZLEXCOUNT key min max                                                                                               |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      ZRANGE       |    Yes     | ZRANGE key start stop [WITHSCORES]                                                                                  |
+------------------------------------------------------------------------------------------------------------------------------------------------------+
|    ZRANGEBYLEX    |    Yes     | ZRANGEBYLEX key min max [LIMIT offset count]                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    ZRANGEBYSCORE  |    Yes     | ZRANGEBYSCORE key min max [WITHSCORES] [LIMIT offset count]                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      ZRANK        |    Yes     | ZRANK key member                                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       ZREM        |    Yes     | ZREM key member [member ...]                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|   ZREMRANGEBYLEX  |    Yes     | ZREMRANGEBYLEX key min max                                                                                          |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|   ZREMRANGEBYRANK |    Yes     | ZREMRANGEBYRANK key start stop                                                                                      |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|  ZREMRANGEBYSCORE |    Yes     | ZREMRANGEBYSCORE key min max                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    ZREVRANGE      |    Yes     | ZREVRANGE key start stop [WITHSCORES]                                                                               |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|  ZREVRANGEBYSCORE |    Yes     | ZREVRANGEBYSCORE key max min [WITHSCORES] [LIMIT offset count]                                                      |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     ZREVRANK      |    Yes     | ZREVRANK key member                                                                                                 |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     ZSCORE        |    Yes     | ZSCORE key member                                                                                                   |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    ZUNIONSTORE    |    Yes*    | ZUNIONSTORE destination numkeys key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX]                 |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      ZSCAN        |    Yes     | ZSCAN key cursor [MATCH pattern] [COUNT count]                                                                      |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
  • ZINTERSTORE and ZUNIONSTORE support requires that the supplied keys hash to the same server. You can ensure this by using the same hashtag for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.

HyperLogLog

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       PFADD       |    Yes     | PFADD key element [element ...]                                                                                     |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      PFCOUNT      |    Yes     | PFCOUNT key [key ...]                                                                                               |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      PFMERGE      |    Yes*    | PFMERGE destkey sourcekey [sourcekey ...]                                                                           |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
  • PFMERGE support requires that the supplied keys hash to the same server. You can ensure this by using the same hashtag for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.

Pub/Sub

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     PSUBSCRIBE    |    No      | PSUBSCRIBE pattern [pattern ...]                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     PUBLISH       |    No      | PUBLISH channel message                                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    PUNSUBSCRIBE   |    No      | PUNSUBSCRIBE [pattern [pattern ...]]                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     SUBSCRIBE     |    No      | SUBSCRIBE channel [channel ...]                                                                                     |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     UNSUBSCRIBE   |    No      | UNSUBSCRIBE [channel [channel ...]]                                                                                 |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

Transactions

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      DISCARD      |    No      | DISCARD                                                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       EXEC        |    No      | EXEC                                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       MULTI       |    No      | MULTI                                                                                                               |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      UNWATCH      |    No      | UNWATCH                                                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       WATCH       |    No      | WATCH key [key ...]                                                                                                 |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

Scripting

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       EVAL        |    Yes*    | EVAL script numkeys key [key ...] arg [arg ...]                                                                     |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     EVALSHA       |    Yes*    | EVALSHA sha1 numkeys key [key ...] arg [arg ...]                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    SCRIPT EXISTS  |    No      | SCRIPT EXISTS script [script ...]                                                                                   |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    SCRIPT FLUSH   |    No      | SCRIPT FLUSH                                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    SCRIPT KILL    |    No      | SCRIPT KILL                                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    SCRIPT LOAD    |    No      | SCRIPT LOAD script                                                                                                  |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
  • EVAL and EVALSHA support is limited to scripts that take at least 1 key. If multiple keys are used, all keys must hash to the same server. You can ensure this by using the same hashtag for all keys. If you use more than 1 key, the proxy does no checking to verify that all keys hash to the same server, and the entire command is forwarded to the server that the first key hashes to

Connection

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       AUTH        |    No      | AUTH password                                                                                                       |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       ECHO        |    No      | ECHO message                                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       PING        |    No      | PING                                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|       QUIT        |    No      | QUIT                                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SELECT       |    No      | SELECT index                                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

Server

+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      Command      | Supported? | Format                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    BGREWRITEAOF   |    No      | BGREWRITEAOF                                                                                                        |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      BGSAVE       |    No      | BGSAVE                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    CLIENT KILL    |    No      | CLIENT KILL ip:port                                                                                                 |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    CLIENT LIST    |    No      | CLIENT LIST                                                                                                         |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    CONFIG GET     |    No      | CONFIG GET parameter                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    CONFIG SET     |    No      | CONFIG SET parameter value                                                                                          |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|  CONFIG RESETSTAT |    No      | CONFIG RESETSTAT                                                                                                    |
+-------------------+-------------+--------------------------------------------------------------------------------------------------------------------+
|     DBSIZE        |    No      | DBSIZE                                                                                                              |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    DEBUG OBJECT   |    No      | DEBUG OBJECT key                                                                                                    |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|    DEBUG SEGFAULT |    No      | DEBUG SEGFAULT                                                                                                      |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     FLUSHALL      |    No      | FLUSHALL                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     FLUSHDB       |    No      | FLUSHDB                                                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      INFO         |    No      | INFO                                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     LASTSAVE      |    No      | LASTSAVE                                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     MONITOR       |    No      | MONITOR                                                                                                             |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SAVE         |    No      | SAVE                                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     SHUTDOWN      |    No      | SHUTDOWN [NOSAVE] [SAVE]                                                                                            |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     SLAVEOF       |    No      | SLAVEOF host port                                                                                                   |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|     SLOWLOG       |    No      | SLOWLOG subcommand [argument]                                                                                       |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      SYNC         |    No      | SYNC                                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
|      TIME         |    No      | TIME                                                                                                                |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+

Note

  • redis commands are not case sensitive
  • only vectored commands ‘MGET key [key …]’, ‘MSET key value [key value …]’, ‘DEL key [key …]’ needs to be fragmented

Performance

Setup

  • redis-server running on machine A.
  • nutcracker running on machine A as a local proxy to redis-server.
  • redis-benchmark running on machine B.
  • machine A != machine B.
  • nutcracker built with –enable-debug=no
  • nutcracker running with mbuf-size of 512 (-m 512)
  • redis-server built from redis 2.6 branch

redis-benchmark against redis-server

$ redis-benchmark -h <machine-A> -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 6379
SET: 89285.71 requests per second
GET: 92592.59 requests per second
INCR: 89285.71 requests per second
LPUSH: 90090.09 requests per second
LPOP: 90090.09 requests per second
SADD: 90090.09 requests per second
SPOP: 93457.95 requests per second
LPUSH (needed to benchmark LRANGE): 89285.71 requests per second
LRANGE_100 (first 100 elements): 36496.35 requests per second
LRANGE_300 (first 300 elements): 15748.03 requests per second
LRANGE_500 (first 450 elements): 11135.86 requests per second
LRANGE_600 (first 600 elements): 8650.52 requests per second

redis-benchmark against nutcracker proxing redis-server

$ redis-benchmark -h <machine-A> -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 22121
SET: 85470.09 requests per second
GET: 86956.52 requests per second
INCR: 85470.09 requests per second
LPUSH: 84745.77 requests per second
LPOP: 86206.90 requests per second
SADD: 84745.77 requests per second
SPOP: 86956.52 requests per second
LPUSH (needed to benchmark LRANGE): 84745.77 requests per second
LRANGE_100 (first 100 elements): 29761.90 requests per second
LRANGE_300 (first 300 elements): 12376.24 requests per second
LRANGE_500 (first 450 elements): 8605.85 requests per second
LRANGE_600 (first 600 elements): 6587.62 requests per second

redis-auth feature

  • you can enable redis-auth for a pool with ‘redis_auth’:

    alpha:
  listen: 127.0.0.1:22121
  hash: fnv1a_64
  distribution: ketama
  redis: true
  redis_auth: testpass

  • notice:

    • MUST set all redis with a same passwd, and all twemproxy with the same passwd
    • Length of password should less than 256
Redis数据库安全之旅
Huangjiazhen711的博客
11-12 774
Redis相信大家都或多或少都听说过吧,作为内存数据库的代表, 但是近些年Redis被攻击的典范也是越来越多,我们将如何防护Redis安全呢?跟着我们的脚本,来看看这篇文章吧.对于Redis而言，我们设置的方法大概有以下几种通过redis.conf设置。在已经启动的Redis中使用CONFIG SET来设置。如之前所述，安全方面归根结底，总结一句话就是：最好的安全设置就是最小化权限。那我们来看下，Redis有哪些安全设置呢？安全一直是大家比较关心的话题,在Redis。
Redis学习之旅 管道/脚本/过期/事务
小萤哥的博客
06-02 252
Redis学习之旅 管道/脚本/过期/事务 本文尝试将 管道、脚本、过期和事务一起归纳一下 管道 我们都知道，一个命令通常都是由一个客户端通过网络连接发送到某个服务，由服务做出响应并给出回复，相对于Redis的快速，网络的传输速度往往是制约Redis发挥最大功效的因素，因此Redis设计了管道的概念 常规的使用，就是由客户端 1.通过网络发送请求-2.Redis处理计算-3.通过网络返回结果，所有的命令都是这样123-123-123式的进行处理，性能的消耗基本上都消耗在了网络传输上，在此基础
TwemProxy（nutcracker）参数选项及配置解析
q342733584的博客
08-21 4652
twemproxy,又名nutracker,是一个快速,轻量级的代理程序，支持memcache 和redis协议。它主要用来降低后端缓存服务器的连接数 目前的最新版本为0.3.0.git地址:https://github.com/yuyunliuhen/twemproxy。     编译：     $get clone https://github.com/yuyunliuhen/twe
twemproxy的参数解析和监控
06-08 406
之前写了twemproxy的安装配置，今天主要内容是参数解析和它提供监控： 参数解读： listen： 监听地址和端口（name:port 或者ip:port）,也可以用sock文件（/var/r...
基于twemproxy的redis集群方案实践
星空的风fly
01-09 664
Twemproxy又名nutcraker，是由Twtter 开源，是一种代理分片机制的中间件（类似于mycat），主要用于管理 Redis Memcached 集群，可接受来自多个客户端的请求，按照路由规则，转发给后端cache节点，减少了客户端与cache 服务器直接连接的数量。 一、twemproxy特性 1、优点 快速。 轻量级。 保持与服务器的长连接。 减少了直接与缓存服务器连...
Twemproxy源码走读（5）：事件处理
Chris.Xiao
12-02 1857
概述 Twemproxy中的IO复用考虑了跨平台的情况，针对不同平台采用不同的IO复用机制，比如Linux下使用epoll、FreeBSD使用kqueue等，在event目录下都有实现，所有的IO复用机制对外实现了统一的接口（event/nc_event.h）： struct event_base *event_base_create(int size, event_cb_t cb); voi
Redis集群下，不同的Key可以映射到同一个槽吗？
qq_36286910的博客
03-27 2146
Redis集群下，不同的Key可以映射到同一个槽吗？
《Spring Boot邂逅Redis：缓存加速的奇妙之旅》
最新发布
09-05
内容概要：本文详细介绍了Spring Boot与Redis整合实现缓存加速的技术实践。首先概述了Redis的核心特性，包括高性能内存存储、丰富的数据结构支持以及RDB和AOF两种持久化机制，并分别指导在Windows和Linux系统中搭建...
释放Redis的力量闪电般快速的数据管理之旅小教程
05-17
### 释放Redis的力量：闪电般快速的数据管理之旅 #### Redis：基础知识和技术特点 **1. 内存存储：** Redis的核心特性之一就是采用内存作为主要的存储介质，这意味着所有的数据都存储在RAM（随机访问存储器）中。...
揭秘2017云栖大会：云数据库Redis开源之旅与实战应用
在2017年的云栖大会上，...这次演讲不仅展示了阿里云在Redis技术上的创新和实践，也透露了未来的产品规划方向，对于想要了解阿里云Redis产品及其在实际业务中的应用和技术优势的用户来说，是一次非常有价值的学习资料。
Redis基础知识（学习笔记5--Redis Cluster）
hongjian006的博客
10-23 910
（1）.Redis Cluster 是Reids 自己本身提供的Redis 集群方案。【此图来源于 https://www.bilibili.com/video/BV1Gs4y1Q7Ls?（2）.Redis 是去中心化的，集群由多个redis节点组成，每个节点负责整个集群的一部分数据，每个节点负责的数据多少可能不一样。节点之间相互连接组成一个对能的集群，他们呢之间通过一个特殊的二进制协议交换集群信息。（3）.Redis Cluster 将所有的数据划分为16384个槽位，每个节点负责其中一部分槽位。
基于nutcracker 的redis分布式缓存的实现的注意
单向街的夏天
04-25 6167
TwemP What’s awesome about Twemproxy is that it can be configured both to disable nodes on failure, and retry after some time, or to stick to the specified keys -> servers map. This means that it i
twemproxy配置详解
IT老兵
07-10 936
twemproxy的配置信息填写在nutcracker.yml之中，默认的查找位置是在conf目录下，也可以通过-c参数指定。 nutcracker.yml的例子： [code="java"]redis1: listen: 10.18.129.48:11299 redis: true hash: fnv1a_64 distribution: ketama ...
Redis代理twemproxy安装,配置,使用
每天积累一点，一年后你会发现，自己变化很大
03-24 701
Twemproxy(nutcracker)是一个memcache、redis协议的轻量级代理。它的主要作用是减少后端缓存服务器的连接数；它支持作为多个后端服务器的代理，并对数据进行sharding分片存储到多个服务器；能够暂时移除后端不可用的服务器节点。 （一）安装 1、安装libtool yum -y install libtool (解决运行autoconf 缺少 acloc
基于Twemproxy的群集部署方案
zhaokuner的专栏
04-10 7180
概述   本文描述的twemproxy基于nutcracker-0.2.4版本。   twemproxy是memcached 和redis的协议层面的代理，其提供的features如下：   注： twemproxy不会增加redis的性能指标数据，据业界测算，使用twemproxy相比直接使用redis会带来~10%的性能下降。   但是单个redis进程的内存管理能力有限
Twemproxy调研总结
逍遥子曰：
08-03 9887
本文档主要针对Redis的代理程序TwemProxy的使用进行调研和总结，共分为以下几个部分： 1、TwemProxy简介；该部分主要介绍TwemProxy是什么，能干哪些事情，有什么主要特点； 2、TwemProxy的集群部署；该部分主要介绍TwemProxy的集群部署方式； 3、TwemProxy的功能及性能分析；该部分主要介绍TwemProxy集群架构时的功能和性能情况。 4、T
彻底解决twemproxy半连接问题
a13805667702的专栏
12-02 2526
LVS+TWEMPROXY+REDIS的这种方式会产生很多半连接，在我的http://blog.youkuaiyun.com/a13805667702/article/details/53411576这篇文章中已经讲过 生产事故回顾（血泪史）： 首先是应公司安全要求对核心交换机做升级，升级期间公司网络是不可用的！但网络异常会导致web应用和redis的proxy会产生大量的半连接，应用的半连接过多会把连接
twemproxy redis ttserver phpadmin memcache
汪迎春的专栏
12-19 1509
1.system update LANG=C yum -y install gcc gcc-c++ autoconf libjpeg libjpeg-devel libpng libpng-devel freetype freetype-devel libxml2 libxml2-devel zlib zlib-devel glibc glibc-devel glib2 glib2-devel
Twemproxy和Redis auth
qq_33374489的博客
08-23 1138
Twemproxy twemproxy自身不支持auth 但是支持redis的auth redis_14600:   listen: 192.168.3.39:14600   hash: fnv1a_64   distribution: ketama   auto_eject_hosts: true   redis: true   red...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值