在学习使用Java_Memcache操作memcache后,饶有兴趣的研究了一下Java_Memcache的源码。Java_Memcache在类AscIIClient中封装了数据操作方法set/add/delete/append/get等。
存储数据set
由Memcache命令详解,我们知道memcache原始的set命令格式为
set <key> <flag> <expiretime> <bytes> \r\n
<value> \r\n
而在Java_Memcache中set操作数据最始调用的是set(String, String, Object, Date, Integer, Long, flag)方法。
private boolean set(String s, String s1, Object obj, Date date, Integer integer, Long long1, boolean flag)
{
SchoonerSockIO schoonersockio;
int i;
String s2;
......
s1 = sanitizeKey(s1);
......
schoonersockio = pool.getSock(s1, integer);
if(schoonersockio == null)
{
if(errorHandler != null)
errorHandler.handleErrorOnSet(this, new IOException("no socket to server available"), s1);
return false;
}
if(date == null)
date = new Date(0L);
i = NativeHandler.getMarkerFlag(obj);
s2 = s + " " + s1 + " " + i + " " + date.getTime() / 1000L + " ";
boolean flag1;
schoonersockio.writeBuf.clear();
schoonersockio.writeBuf.put(s2.getBytes());
int j = schoonersockio.writeBuf.position();
schoonersockio.writeBuf.put(BLAND_DATA_SIZE);
if(long1.longValue() != 0L)
schoonersockio.writeBuf.put((new StringBuilder()).append(" ").append(long1.toString()).toString().getBytes());
schoonersockio.writeBuf.put(B_RETURN);
SockOutputStream sockoutputstream = new SockOutputStream(schoonersockio);
int k = 0;
if(i != 0)
{
byte abyte0[];
if(flag)
abyte0 = obj.toString().getBytes(defaultEncoding);
else
abyte0 = NativeHandler.encode(obj);
sockoutputstream.write(abyte0);
k = abyte0.length;
} else
{
k = transCoder.encode(sockoutputstream, obj);
}
schoonersockio.writeBuf.put(B_RETURN);
byte abyte1[] = (new Integer(k)).toString().getBytes();
int l = schoonersockio.writeBuf.position();
schoonersockio.writeBuf.position(j);
schoonersockio.writeBuf.put(abyte1);
schoonersockio.writeBuf.position(l);
schoonersockio.flush();
String s3 = (new SockInputStream(schoonersockio, 2147483647)).getLine();
if(!"STORED\r\n".equals(s3))
break MISSING_BLOCK_LABEL_538;
......
return false;
}
当我们使用MemCacheClient.set(“name”,”abcdef”),通过对比上面的code,首先i=NativeHandler.getMarkerFlag(“abcdef”)值为32, 然后拼接字符串s2=”set name 32 0 ”并将s2放入ByteBuffer中,然后调用schoonersockio.writeBuf.put(BLAND_DATA_SIZE),而BLADN_DATA_SIZE是" ".getBytes(),其实就是“set name 32 0 ”,最后schoonersockio.writeBuf.put(B_RETURN)即输入\r\n,最后再通过transCoder.encode()将value写入。
public void encode(OutputStream outputstream, Object obj)throws IOException
{
ObjectOutputStream objectoutputstream = new ObjectOutputStream(outputstream);
objectoutputstream.writeObject(obj);
objectoutputstream.close();
}
其实就是将对象序列化,因此在使用Java_Memcache操作的自定义对象必须都实现Serializable接口。
但是,如果我们直接运行”set name 32 0 \r\n“会提示错误信息。
我很困惑set命令最后的<byte>参数是如何处理的????
反编译后的代码不全 or 反编译有误?
数据提取get
同样,memcache原始的get命令很简单,其基本格式为:
get <key> \r\n
而在Java_Memcache中,AscIIClient中是按如下处理的:
public Object get(String s)
{
return get(s, null);
}
public Object get(String s, Integer integer)
{
return get("get", s, integer, false);
}
private Object get(String s, String s1, Integer integer, boolean flag)
{
SchoonerSockIO schoonersockio;
String s2;
....
s1 = sanitizeKey(s1);
schoonersockio = pool.getSock(s1, integer);
s2 = (new StringBuilder()).append(s).append(" ").append(s1).toString();
int i;
int j;
SockInputStream sockinputstream;
boolean flag1;
StringBuffer stringbuffer;
int l;
schoonersockio.writeBuf.clear();
schoonersockio.writeBuf.put(s2.getBytes());
schoonersockio.writeBuf.put(B_RETURN);
schoonersockio.flush();
i = 0;
j = 0;
sockinputstream = new SockInputStream(schoonersockio, 2147483647);
flag1 = false;
stringbuffer = new StringBuffer();
l = 0;
<span style="color:#006600;">//前面一部分就是构建字符串 “get name \r\n”,即向Memcache server发出get命令</span>
_L5:
int k;
if(flag1)
break MISSING_BLOCK_LABEL_365;
k = sockinputstream.read();
if(k != 32 && k != 13)
break MISSING_BLOCK_LABEL_353;
<span style="color:#006600;">//一直读取到回车或换行
</span> l;
JVM INSTR tableswitch 0 3: default 322
goto _L1 _L2 _L1 _L3 _L4
<span style="color:#006600;">//这里是一个while循环接收从memcache server返回的消息字符</span>
_L1:
break; /* Loop/switch isn't completed */
_L2:
Object obj;
if(!"END\r\n".startsWith(stringbuffer.toString()))
break; /* Loop/switch isn't completed */
<span style="color:#006600;">//如果不是END\r\r行,就继续处理,否则就直接返回空</span>
obj = null;
if(schoonersockio != null)
{
schoonersockio.close();
schoonersockio = null;
}
return obj;
<span style="color:#006600;">//这里处理在memcache中没有对应key值时取不到任何值,调用get命令后直接得到END/r/n,返回空</span>
_L3:
j = Integer.parseInt(stringbuffer.toString());
break; /* Loop/switch isn't completed */
_L4:
i = Integer.parseInt(stringbuffer.toString());
l++;
stringbuffer = new StringBuffer();
if(k == 13)
{
sockinputstream.read();
flag1 = true;
}
goto _L5
stringbuffer.append((char)k);
goto _L5
Object obj3;
Object obj1 = null;
sockinputstream.willRead(i);
if(i > 0)
if(NativeHandler.isHandled(j))
{
byte abyte0[] = sockinputstream.getBuffer();
if((j & 2) == 2)
{
GZIPInputStream gzipinputstream = new GZIPInputStream(new ByteArrayInputStream(abyte0));
ByteArrayOutputStream bytearrayoutputstream = new ByteArrayOutputStream(abyte0.length);
byte abyte1[] = new byte[2048];
int i1;
while((i1 = gzipinputstream.read(abyte1)) != -1)
bytearrayoutputstream.write(abyte1, 0, i1);
abyte0 = bytearrayoutputstream.toByteArray();
<span style="color:#006600;">//读取数据到字节数组中</span>
gzipinputstream.close();
}
if(primitiveAsString || flag)
obj1 = new String(abyte0, defaultEncoding);
else
obj1 = NativeHandler.decode(abyte0, j);
<span style="color:#006600;">//最后调用decodeString解码字符串</span>
}
else
if(transCoder != null)
{
Object obj2 = sockinputstream;
if((j & 2) == 2)
obj2 = new GZIPInputStream(((java.io.InputStream) (obj2)));
if(classLoader == null)
obj1 = transCoder.decode(((java.io.InputStream) (obj2)));
else
obj1 = ((ObjectTransCoder)transCoder).decode(((java.io.InputStream) (obj2)), classLoader);
}
sockinputstream.willRead(2147483647);
sockinputstream.getLine();
sockinputstream.getLine();
obj3 = obj1;
if(schoonersockio != null)
{
schoonersockio.close();
schoonersockio = null;
}
return obj3;
}
我们可以对比直接使用memcache命令操作的输入输出来理解,就是在输入完get命令后解析字符序列,编码成实际的data。
使用反编译后的代码可能理解起来比较困难,我们可以查看Java_Memcache client原始代码,学习起来更省心。
有些版本原代码是在MemcacheClient中直接get/set data的。
这份代码分析Memcache处理数据会更加简单。
总结
至此,我们介绍了如何在windows安装并操作Memcache,分析了Memcache分布式原理,探讨了一般Hash算法和一致性Hash算法,如何操作数据。我们从整体上对Java_Memcache Client有了深刻的认识,其主要思想就是维护一套分布式Server的信息,对每台Server维护其Socket连接,根据操作的数据进行Hash映射到某台Server上,最后构建memcache命令操作和解析数据。
相信在掌握Java_Memcache_Client的思想和实现过程后,您会使用得更加得心应手。
Windows环境下的Memcache实战 http://blog.youkuaiyun.com/musa875643dn/article/details/45439417
Memcache命令详解 http://blog.youkuaiyun.com/musa875643dn/article/details/45935895
Memcache分布式原理解析 http://blog.youkuaiyun.com/musa875643dn/article/details/45675711
Memcache Hash算法揭秘 http://blog.youkuaiyun.com/musa875643dn/article/details/45796887