Jedis和ShardedJedis设计分析

本文详细介绍了Jedis作为Java语言Redis客户端的内部结构及其实现方式,包括对象池设计、面向用户和服务器的操作封装等内容。此外,还探讨了ShardedJedis的一致性哈希算法实现细节。

Jedis分析

Jedis作为推荐的java语言redis客户端,其抽象封装为三部分:

1.对象池设计:Pool,JedisPool,JedisFactory.主要引用了common-pool里的GenericObjectPool,BasePoolableObjectFactory。

对象池技术主要使用了commons-pool
参考文章:开源项目剖析之apache-common-pool
对象池技术
Apache common-pool, common-dbcp源码解读与对象池原理剖析

2.面向用户的redis操作封装:JedisCommands, MultiKeyCommands, AdvancedJedisCommands, ScriptingCommands,BinaryJedis,Jedis

3.面向redis服务器的操作封装:Commands,Client,BinaryClient,Connection,Protocol

Pool 抽象Jedis对象池操作;并委托给操作给GenericObjectPool

JedisPool 实现Pool并提供JedisFactory工厂

JedisFactory 实现BasePoolableObjectFactory,提供创建,销毁Jedis方法

BinaryJedisCommands 抽象面向客户端操作的Redis命令;key,value都为序列化后的byte数组

JedisCommands 抽象面向客户端操作的Redis命令;提供String类型的key,value

BinaryJedis 实现BinaryJedisCommands接口,并将实际操作委托给Client

Jedis 实现JedisCommands接口,并将操作委托给Client

Commands 抽象Redis操作接口,提供String类型的key,value操作;被Jedis调用

Connection 抽象了Redis连接;包括host,port,pass,socket,inputstream,outputstream,protocol 完成与Redis服务器的通信

Protocol 抽象了Redis协议处理

BinaryClient 继承Connection类,封装了基于Byte[]的key,value操作

Client 继承BinaryClient同时实现了Commands,对上层提供基于String类型的操作

类图如下:
jedis


ShardedJedis分析

ShardedJedis是基于一致性哈希算法实现的分布式Redis集群客户端;ShardedJedis的设计分为以下几块:

1.对象池设计:Pool,ShardedJedisPool,JedisSentinelPool.

Sentinel是Redis2.6版开始加入的另一组独立运行的节点, 提供自动Failover的支持。Redis Sentinel具有的功能包括:

(1)监控:定时检查Redis的Master和Slave服务器是否正常;

(2)通知:在发生异常时通知系统管理员和其他程序;

(3)自动Failover:如果Master不可用,Sentinel会自动执行Failover,选择一个Slave提升为Master,其他的Salve会重新设置使用新的Master。当应用程序重新连接时会被通知连接新的Master。

Redis Sentinel 是一个分布式系统, 你可以在架构中运行多个 Sentinel 进程, 这些 Sentinel 进程通过相互通讯来判断一个主服务器是否断线, 以及是否应该执行故障转移。

虽然Redis Sentinel为一个单独的可执行文件 redis-sentinel,但实际上它只是一个运行在特殊模式下的 Redis 服务器, 你可以在启动一个普通 Redis 服务器时通过给定 –sentinel 选项来启动 Redis Sentinel

JedisSentinelPool是jedis对Redis Sentinel的客户端连接实现。

2.面向用户的操作封装:BinaryShardedJedis,ShardedJedis

3.一致性哈希实现:Sharded

类图如下:
shardedjedis

Sharded :抽象了基于一致性哈希算法的划分设计,设计思路。
(1)基于hash算法划分redis服务器
(2)保持每台Redis服务器的Jedis客户端
(3)提供基于Key的划分方法;提供了ShardKeyTag实现

BinaryShardedJedis: 同BinaryJedis类似,实现BinaryJedisCommands对外提供基于Byte[]的key,value操作

ShardedJedis: 同Jedis类似,实现JedisCommands对外提供基于String的key,value操作


Sharded一致性哈希实现

shared一致性哈希采用以下方案:

Redis服务器节点划分:将每台服务器节点采用hash算法划分为160个虚拟节点(可以配置划分权重)
(1)将划分虚拟节点采用TreeMap存储
(2)对每个Redis服务器的物理连接采用LinkedHashMap存储
(3)对Key or KeyTag 采用同样的hash算法,然后从TreeMap获取大于等于键hash值得节点,取最邻近节点存储;当key的hash值大于虚拟节点hash值得最大值时,存入第一个虚拟节点
sharded采用的hash算法:MD5 和 MurmurHash两种;默认采用64位的MurmurHash算法;MurmurHash是一种高效,低碰撞的hash算法.

参考地址:

http://blog.youkuaiyun.com/yfkiss/article/details/7337382

https://sites.google.com/site/murmurhash/

http://yychao.iteye.com/blog/1751583

java.lang.IllegalStateException: Failed to load ApplicationContext for [WebMergedContextConfiguration@6b3e7aa2 testClass = com.huawei.vmall.pms.interfaces.AlphaTest, locations = ["classpath*:META-INF/uuid/uuid_context.xml"], classes = [com.huawei.vmall.pms.front.server.PmsFrontApplication], contextInitializerClasses = [], activeProfiles = ["cse-enable", "r_druid", "w_druid", "REDIS"], propertySourceDescriptors = [], propertySourceProperties = ["org.springframework.boot.test.context.SpringBootTestContextBootstrapper=true"], contextCustomizers = [[ImportsContextCustomizer@54a6f06 key = [@org.springframework.test.context.ActiveProfiles(inheritProfiles=true, profiles={"cse-enable", "r_druid", "w_druid", "REDIS"}, resolver=org.springframework.test.context.ActiveProfilesResolver.class, value={"cse-enable", "r_druid", "w_druid", "REDIS"}), @org.apiguardian.api.API(consumers={"*"}, since="5.0", status=STABLE), @org.springframework.test.context.BootstrapWith(value=org.springframework.boot.test.context.SpringBootTestContextBootstrapper.class), @com.huawei.vmall.mock.annotations.ProxyScanner(includes={"com.huawei.vmall.fss.client.service"}, excludes={}), @org.springframework.context.annotation.Import(value={com.ctrip.framework.apollo.spring.annotation.ApolloConfigRegistrar.class}), @org.springframework.test.context.ContextConfiguration(classes={}, inheritInitializers=true, inheritLocations=true, initializers={}, loader=org.springframework.test.context.ContextLoader.class, locations={"classpath*:META-INF/uuid/uuid_context.xml"}, name="", value={"classpath*:META-INF/uuid/uuid_context.xml"}), @com.ctrip.framework.apollo.spring.annotation.EnableApolloConfig(value={"application", "global_rocketmq", "global_common"}, order=2147483647), @org.springframework.boot.test.context.SpringBootTest(args={}, classes={com.huawei.vmall.pms.front.server.PmsFrontApplication.class}, properties={}, useMainMethod=NEVER, value={}, webEnvironment=MOCK)]], org.springframework.boot.test.context.filter.ExcludeFilterContextCustomizer@607b2792, org.springframework.boot.test.json.DuplicateJsonObjectContextCustomizerFactory$DuplicateJsonObjectContextCustomizer@3e598df9, org.springframework.boot.test.mock.mockito.MockitoContextCustomizer@0, org.springframework.boot.test.web.client.TestRestTemplateContextCustomizer@38f2e97e, org.springframework.boot.test.web.reactor.netty.DisableReactorResourceFactoryGlobalResourcesContextCustomizerFactory$DisableReactorResourceFactoryGlobalResourcesContextCustomizerCustomizer@7fae4d4a, org.springframework.boot.test.autoconfigure.OnFailureConditionReportContextCustomizerFactory$OnFailureConditionReportContextCustomizer@46baf579, org.springframework.boot.test.autoconfigure.actuate.observability.ObservabilityContextCustomizerFactory$DisableObservabilityContextCustomizer@1f, org.springframework.boot.test.autoconfigure.properties.PropertyMappingContextCustomizer@0, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverContextCustomizer@39aa45a1, org.springframework.test.context.support.DynamicPropertiesContextCustomizer@0, org.springframework.boot.test.context.SpringBootTestAnnotation@b9e9c40d], resourceBasePath = "src/main/webapp", contextLoader = org.springframework.boot.test.context.SpringBootContextLoader, parent = null] at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:180) at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:130) at org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:200) at org.springframework.test.context.web.ServletTestExecutionListener.prepareTestInstance(ServletTestExecutionListener.java:139) at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:260) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:228) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:289) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:291) at com.huawei.vmall.mock.hotswap.SpringDelegateRunner.agentStart(SpringDelegateRunner.java:125) at com.huawei.vmall.mock.hotswap.SpringDelegateRunner.runTestInWin(SpringDelegateRunner.java:100) at com.huawei.vmall.mock.hotswap.SpringDelegateRunner.runChild(SpringDelegateRunner.java:90) at com.huawei.vmall.mock.hotswap.SpringDelegateRunner.runChild(SpringDelegateRunner.java:49) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191) at com.huawei.vmall.mock.hotswap.SpringDelegateRunner.run(SpringDelegateRunner.java:234) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69) at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38) at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55) Caused by: java.lang.IllegalStateException: Failed to introspect Class [org.springframework.data.redis.connection.jedis.JedisConnectionFactory] from ClassLoader [jdk.internal.loader.ClassLoaders$AppClassLoader@36baf30c] at org.springframework.util.ReflectionUtils.getDeclaredFields(ReflectionUtils.java:756) at org.springframework.util.ReflectionUtils.doWithFields(ReflectionUtils.java:722) at org.springframework.util.ReflectionUtils.doWithFields(ReflectionUtils.java:707) at org.apache.dubbo.config.spring.beans.factory.annotation.AbstractAnnotationBeanPostProcessor.findFieldAnnotationMetadata(AbstractAnnotationBeanPostProcessor.java:142) at org.apache.dubbo.config.spring.beans.factory.annotation.AbstractAnnotationBeanPostProcessor.buildAnnotatedMetadata(AbstractAnnotationBeanPostProcessor.java:218) at org.apache.dubbo.config.spring.beans.factory.annotation.AbstractAnnotationBeanPostProcessor.findInjectionMetadata(AbstractAnnotationBeanPostProcessor.java:239) at org.apache.dubbo.config.spring.beans.factory.annotation.ReferenceAnnotationBeanPostProcessor.postProcessBeanFactory(ReferenceAnnotationBeanPostProcessor.java:162) at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:363) at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:204) at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:791) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:609) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:752) at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:439) at org.springframework.boot.SpringApplication.run(SpringApplication.java:318) at org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:144) at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58) at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46) at org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1461) at org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:563) at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:144) at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:110) at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:225) at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:152) ... 34 more Caused by: java.lang.NoClassDefFoundError: redis/clients/jedis/JedisShardInfo at java.base/java.lang.Class.getDeclaredFields0(Native Method) at java.base/java.lang.Class.privateGetDeclaredFields(Class.java:3473) at java.base/java.lang.Class.getDeclaredFields(Class.java:2542) at org.springframework.util.ReflectionUtils.getDeclaredFields(ReflectionUtils.java:751) ... 56 more Caused by: java.lang.ClassNotFoundException: redis.clients.jedis.JedisShardInfo at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526) ... 60 more
最新发布
12-02
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值