先出个题目:
有server1,
其处理能力为 qps = 10000,
当前运行中的qps=1000,
server1中有一个hashtable 为 h1,
目前h1的hash冲突情况很严重,需要进行hash重整,
此时如何处理才能在不影响server服务的情况下完成优化重整任务?
答案就在下面:
今天看redis源码,看到一段代码,对写server的同学有借鉴简直。
我们知道,Server里面一般是一个死循环,然后就是get_msg, handle_msg, send_response, 而如果没有获取到消息,则一般是在进行usleep(x)。 而redis里面则把这个本来要进行usleep的时间利用起来,他把一个例行工作按时间片进行严格拆分,比如拆分为多个1ms的执行段。
如下面代码,rehash是一个hashtable重整的操作,一般比较耗时,这里便用dictRehashMilliseconds将这个操作拆成多个时间片区完成,
从而在完成重整任务的同时也保证了server的及时响应。
/* Rehash for an amount of time between ms milliseconds and ms+1 milliseconds */
int dictRehashMilliseconds(dict *d, int ms) {
long long start = timeInMilliseconds();
int rehashes = 0;
while(dictRehash(d,100)) {
rehashes += 100;
if (timeInMilliseconds()-start > ms) break;
}
return rehashes;
}
rehash支持将整个hashtable重整分为每次N步的片段执行。
/* Performs N steps of incremental rehashing. Returns 1 if there are still
* keys to move from the old to the new hash table, otherwise 0 is returned.
* Note that a rehashing step consists in moving a bucket (that may have more
* thank one key as we use chaining) from the old to the new hash table. */
int dictRehash(dict *d, int n) {
if (!dictIsRehashing(d)) return 0;
while(n--) {
dictEntry *de, *nextde;
/* Check if we already rehashed the whole table... */
if (d->ht[0].used == 0) {
_dictFree(d->ht[0].table);
d->ht[0] = d->ht[1];
_dictReset(&d->ht[1]);
d->rehashidx = -1;
return 0;
}
/* Note that rehashidx can't overflow as we are sure there are more
* elements because ht[0].used != 0 */
while(d->ht[0].table[d->rehashidx] == NULL) d->rehashidx++;
de = d->ht[0].table[d->rehashidx];
/* Move all the keys in this bucket from the old to the new hash HT */
while(de) {
unsigned int h;
nextde = de->next;
/* Get the index in the new hash table */
h = dictHashKey(d, de->key) & d->ht[1].sizemask;
de->next = d->ht[1].table[h];
d->ht[1].table[h] = de;
d->ht[0].used--;
d->ht[1].used++;
de = nextde;
}
d->ht[0].table[d->rehashidx] = NULL;
d->rehashidx++;
}
return 1;
}