转载两篇不错的文章:
第一篇:
前言
HashMap是我们平时开发过程中用的比较多的集合,但它是非线程安全的,在涉及到多线程并发的情况,进行put操作有可能会引起死循环,导致CPU利用率接近100%。
- final HashMap<String, String> map = new HashMap<String, String>(2);
- for (int i = 0; i < 10000; i++) {
- new Thread(new Runnable() {
- @Override
- public void run() {
- map.put(UUID.randomUUID().toString(), "");
- }
- }).start();
- }
解决方案有Hashtable和Collections.synchronizedMap(hashMap),不过这两个方案基本上是对读写进行加锁操作,一个线程在读写元素,其余线程必须等待,性能可想而知。
所以,Doug Lea给我们带来了并发安全的ConcurrentHashMap,它的实现是依赖于 Java 内存模型,所以我们在了解 ConcurrentHashMap 的之前必须了解一些底层的知识:
- java内存模型
- java中的Unsafe
- java中的CAS
- 深入浅出java同步器
- 深入浅出ReentrantLock
本文源码是JDK8的版本,与之前的版本有较大差异。
JDK1.6分析
ConcurrentHashMap采用 分段锁的机制,实现并发的更新操作,底层采用数组+链表+红黑树的存储结构。
其包含两个核心静态内部类 Segment和HashEntry。
- Segment继承ReentrantLock用来充当锁的角色,每个 Segment 对象守护每个散列映射表的若干个桶。
- HashEntry 用来封装映射表的键 / 值对;
- 每个桶是由若干个 HashEntry 对象链接起来的链表。
一个 ConcurrentHashMap 实例中包含由若干个 Segment 对象组成的数组,下面我们通过一个图来演示一下 ConcurrentHashMap 的结构:
ConcurrentHashMap存储结构.png
JDK1.8分析
1.8的实现已经抛弃了Segment分段锁机制,利用CAS+Synchronized来保证并发更新的安全,底层依然采用数组+链表+红黑树的存储结构。
Paste_Image.png
重要概念
在开始之前,有些重要的概念需要介绍一下:
- table:默认为null,初始化发生在第一次插入操作,默认大小为16的数组,用来存储Node节点数据,扩容时大小总是2的幂次方。
- nextTable:默认为null,扩容时新生成的数组,其大小为原数组的两倍。
- sizeCtl :默认为0,用来控制table的初始化和扩容操作,具体应用在后续会体现出来。
- -1 代表table正在初始化
- -N 表示有N-1个线程正在进行扩容操作
- 其余情况:
1、如果table未初始化,表示table需要初始化的大小。
2、如果table初始化完成,表示table的容量,默认是table大小的0.75倍,居然用这个公式算0.75(n - (n >>> 2))。
- Node:保存key,value及key的hash值的数据结构。
- class Node<K,V> implements Map.Entry<K,V> {
- final int hash;
- final K key;
- volatile V val;
- volatile Node<K,V> next;
- ... 省略部分代码
- }
其中value和next都用volatile修饰,保证并发的可见性。 - ForwardingNode:一个特殊的Node节点,hash值为-1,其中存储nextTable的引用。
- final class ForwardingNode<K,V> extends Node<K,V> {
- final Node<K,V>[] nextTable;
- ForwardingNode(Node<K,V>[] tab) {
- super(MOVED, null, null, null);
- this.nextTable = tab;
- }
- }
- 只有table发生扩容的时候,ForwardingNode才会发挥作用,作为一个占位符放在table中表示当前节点为null或则已经被移动。
实例初始化
实例化ConcurrentHashMap时带参数时,会根据参数调整table的大小,假设参数为100,最终会调整成256,确保table的大小总是2的幂次方,算法如下:
- ConcurrentHashMap<String, String> hashMap = new ConcurrentHashMap<>(100);
- private static final int tableSizeFor(int c) {
- int n = c - 1;
- n |= n >>> 1;
- n |= n >>> 2;
- n |= n >>> 4;
- n |= n >>> 8;
- n |= n >>> 16;
- return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? MAXIMUM_CAPACITY : n + 1;
- }
注意,ConcurrentHashMap在构造函数中只会初始化sizeCtl值,并不会直接初始化table,而是延缓到第一次put操作。
table初始化
前面已经提到过,table初始化操作会延缓到第一次put行为。但是put是可以并发执行的,Doug Lea是如何实现table只初始化一次的?让我们来看看源码的实现。
- private final Node<K,V>[] initTable() {
- Node<K,V>[] tab; int sc;
- while ((tab = table) == null || tab.length == 0) {
-
- if ((sc = sizeCtl) < 0)
- Thread.yield();
- else if (U.compareAndSwapInt(this, SIZECTL, sc, -1)) {
- try {
- if ((tab = table) == null || tab.length == 0) {
- int n = (sc > 0) ? sc : DEFAULT_CAPACITY;
- @SuppressWarnings("unchecked")
- Node<K,V>[] nt = (Node<K,V>[])new Node<?,?>[n];
- table = tab = nt;
- sc = n - (n >>> 2);
- }
- } finally {
- sizeCtl = sc;
- }
- break;
- }
- }
- return tab;
- }
sizeCtl默认为0,如果ConcurrentHashMap实例化时有传参数,sizeCtl会是一个2的幂次方的值。所以执行第一次put操作的线程会执行Unsafe.compareAndSwapInt方法修改sizeCtl为-1,有且只有一个线程能够修改成功,其它线程通过Thread.yield()让出CPU时间片等待table初始化完成。
put操作
假设table已经初始化完成,put操作采用CAS+synchronized实现并发插入或更新操作,具体实现如下。
- final V putVal(K key, V value, boolean onlyIfAbsent) {
- if (key == null || value == null) throw new NullPointerException();
- int hash = spread(key.hashCode());
- int binCount = 0;
- for (Node<K,V>[] tab = table;;) {
- Node<K,V> f; int n, i, fh;
- if (tab == null || (n = tab.length) == 0)
- tab = initTable();
- else if ((f = tabAt(tab, i = (n - 1) & hash)) == null) {
- if (casTabAt(tab, i, null, new Node<K,V>(hash, key, value, null)))
- break;
- }
- else if ((fh = f.hash) == MOVED)
- tab = helpTransfer(tab, f);
- ...省略部分代码
- }
- addCount(1L, binCount);
- return null;
- }
- hash算法
static final int spread(int h) {return (h ^ (h >>> 16)) & HASH_BITS;}
- table中定位索引位置,n是table的大小
int index = (n - 1) & hash
- 获取table中对应索引的元素f。
Doug Lea采用Unsafe.getObjectVolatile来获取,也许有人质疑,直接table[index]不可以么,为什么要这么复杂?
在java内存模型中,我们已经知道每个线程都有一个工作内存,里面存储着table的副本,虽然table是volatile修饰的,但不能保证线程每次都拿到table中的最新元素,Unsafe.getObjectVolatile可以直接获取指定内存的数据,保证了每次拿到数据都是最新的。 - 如果f为null,说明table中这个位置第一次插入元素,利用Unsafe.compareAndSwapObject方法插入Node节点。
- 如果CAS成功,说明Node节点已经插入,随后addCount(1L, binCount)方法会检查当前容量是否需要进行扩容。
- 如果CAS失败,说明有其它线程提前插入了节点,自旋重新尝试在这个位置插入节点。
- 如果f的hash值为-1,说明当前f是ForwardingNode节点,意味有其它线程正在扩容,则一起进行扩容操作。
- 其余情况把新的Node节点按链表或红黑树的方式插入到合适的位置,这个过程采用同步内置锁实现并发,代码如下:
- synchronized (f) {
- if (tabAt(tab, i) == f) {
- if (fh >= 0) {
- binCount = 1;
- for (Node<K,V> e = f;; ++binCount) {
- K ek;
- if (e.hash == hash &&
- ((ek = e.key) == key ||
- (ek != null && key.equals(ek)))) {
- oldVal = e.val;
- if (!onlyIfAbsent)
- e.val = value;
- break;
- }
- Node<K,V> pred = e;
- if ((e = e.next) == null) {
- pred.next = new Node<K,V>(hash, key,
- value, null);
- break;
- }
- }
- }
- else if (f instanceof TreeBin) {
- Node<K,V> p;
- binCount = 2;
- if ((p = ((TreeBin<K,V>)f).putTreeVal(hash, key,
- value)) != null) {
- oldVal = p.val;
- if (!onlyIfAbsent)
- p.val = value;
- }
- }
- }
- }
在节点f上进行同步,节点插入之前,再次利用tabAt(tab, i) == f判断,防止被其它线程修改。
- 如果f.hash >= 0,说明f是链表结构的头结点,遍历链表,如果找到对应的node节点,则修改value,否则在链表尾部加入节点。
- 如果f是TreeBin类型节点,说明f是红黑树根节点,则在树结构上遍历元素,更新或增加节点。
- 如果链表中节点数binCount >= TREEIFY_THRESHOLD(默认是8),则把链表转化为红黑树结构。
table扩容
当table容量不足的时候,即table的元素数量达到容量阈值sizeCtl,需要对table进行扩容。
整个扩容分为两部分:
- 构建一个nextTable,大小为table的两倍。
- 把table的数据复制到nextTable中。
这两个过程在单线程下实现很简单,但是ConcurrentHashMap是支持并发插入的,扩容操作自然也会有并发的出现,这种情况下,第二步可以支持节点的并发复制,这样性能自然提升不少,但实现的复杂度也上升了一个台阶。
先看第一步,构建nextTable,毫无疑问,这个过程只能只有单个线程进行nextTable的初始化,具体实现如下:
- private final void addCount(long x, int check) {
- ... 省略部分代码
- if (check >= 0) {
- Node<K,V>[] tab, nt; int n, sc;
- while (s >= (long)(sc = sizeCtl) && (tab = table) != null &&
- (n = tab.length) < MAXIMUM_CAPACITY) {
- int rs = resizeStamp(n);
- if (sc < 0) {
- if ((sc >>> RESIZE_STAMP_SHIFT) != rs || sc == rs + 1 ||
- sc == rs + MAX_RESIZERS || (nt = nextTable) == null ||
- transferIndex <= 0)
- break;
- if (U.compareAndSwapInt(this, SIZECTL, sc, sc + 1))
- transfer(tab, nt);
- }
- else if (U.compareAndSwapInt(this, SIZECTL, sc,
- (rs << RESIZE_STAMP_SHIFT) + 2))
- transfer(tab, null);
- s = sumCount();
- }
- }
- }
通过Unsafe.compareAndSwapInt修改sizeCtl值,保证只有一个线程能够初始化nextTable,扩容后的数组长度为原来的两倍,但是容量是原来的1.5。
节点从table移动到nextTable,大体思想是遍历、复制的过程。
- 首先根据运算得到需要遍历的次数i,然后利用tabAt方法获得i位置的元素f,初始化一个forwardNode实例fwd。
- 如果f == null,则在table中的i位置放入fwd,这个过程是采用Unsafe.compareAndSwapObjectf方法实现的,很巧妙的实现了节点的并发移动。
- 如果f是链表的头节点,就构造一个反序链表,把他们分别放在nextTable的i和i+n的位置上,移动完成,采用Unsafe.putObjectVolatile方法给table原位置赋值fwd。
- 如果f是TreeBin节点,也做一个反序处理,并判断是否需要untreeify,把处理的结果分别放在nextTable的i和i+n的位置上,移动完成,同样采用Unsafe.putObjectVolatile方法给table原位置赋值fwd。
遍历过所有的节点以后就完成了复制工作,把table指向nextTable,并更新sizeCtl为新数组大小的0.75倍 ,扩容完成。
红黑树构造
注意:如果链表结构中元素超过TREEIFY_THRESHOLD阈值,默认为8个,则把链表转化为红黑树,提高遍历查询效率。
if (binCount != 0) {
if (binCount >= TREEIFY_THRESHOLD)
treeifyBin(tab, i);
if (oldVal != null)
return oldVal;
break;
}
接下来我们看看如何构造树结构,代码如下:
- private final void treeifyBin(Node<K,V>[] tab, int index) {
- Node<K,V> b; int n, sc;
- if (tab != null) {
- if ((n = tab.length) < MIN_TREEIFY_CAPACITY)
- tryPresize(n << 1);
- else if ((b = tabAt(tab, index)) != null && b.hash >= 0) {
- synchronized (b) {
- if (tabAt(tab, index) == b) {
- TreeNode<K,V> hd = null, tl = null;
- for (Node<K,V> e = b; e != null; e = e.next) {
- TreeNode<K,V> p =
- new TreeNode<K,V>(e.hash, e.key, e.val,
- null, null);
- if ((p.prev = tl) == null)
- hd = p;
- else
- tl.next = p;
- tl = p;
- }
- setTabAt(tab, index, new TreeBin<K,V>(hd));
- }
- }
- }
- }
- }
可以看出,生成树节点的代码块是同步的,进入同步代码块之后,再次验证table中index位置元素是否被修改过。
1、根据table中index位置Node链表,重新生成一个hd为头结点的TreeNode链表。
2、根据hd头结点,生成TreeBin树结构,并把树结构的root节点写到table的index位置的内存中,具体实现如下:
- TreeBin(TreeNode<K,V> b) {
- super(TREEBIN, null, null, null);
- this.first = b;
- TreeNode<K,V> r = null;
- for (TreeNode<K,V> x = b, next; x != null; x = next) {
- next = (TreeNode<K,V>)x.next;
- x.left = x.right = null;
- if (r == null) {
- x.parent = null;
- x.red = false;
- r = x;
- }
- else {
- K k = x.key;
- int h = x.hash;
- Class<?> kc = null;
- for (TreeNode<K,V> p = r;;) {
- int dir, ph;
- K pk = p.key;
- if ((ph = p.hash) > h)
- dir = -1;
- else if (ph < h)
- dir = 1;
- else if ((kc == null &&
- (kc = comparableClassFor(k)) == null) ||
- (dir = compareComparables(kc, k, pk)) == 0)
- dir = tieBreakOrder(k, pk);
- TreeNode<K,V> xp = p;
- if ((p = (dir <= 0) ? p.left : p.right) == null) {
- x.parent = xp;
- if (dir <= 0)
- xp.left = x;
- else
- xp.right = x;
- r = balanceInsertion(r, x);
- break;
- }
- }
- }
- }
- this.root = r;
- assert checkInvariants(root);
- }
主要根据Node节点的hash值大小构建二叉树。这个红黑树的构造过程实在有点复杂,感兴趣的同学可以看看源码。
get操作
get操作和put操作相比,显得简单了许多。
- public V get(Object key) {
- Node<K,V>[] tab; Node<K,V> e, p; int n, eh; K ek;
- int h = spread(key.hashCode());
- if ((tab = table) != null && (n = tab.length) > 0 &&
- (e = tabAt(tab, (n - 1) & h)) != null) {
- if ((eh = e.hash) == h) {
- if ((ek = e.key) == key || (ek != null && key.equals(ek)))
- return e.val;
- }
- else if (eh < 0)
- return (p = e.find(h, key)) != null ? p.val : null;
- while ((e = e.next) != null) {
- if (e.hash == h &&
- ((ek = e.key) == key || (ek != null && key.equals(ek))))
- return e.val;
- }
- }
- return null;
- }
总结
ConcurrentHashMap 是一个并发散列映射表的实现,它允许完全并发的读取,并且支持给定数量的并发更新。相比于 HashTable 和同步包装器包装的 HashMap,使用一个全局的锁来同步不同线程间的并发访问,同一时间点,只能有一个线程持有锁,也就是说在同一时间点,只能有一个线程能访问容器,这虽然保证多线程间的安全并发访问,但同时也导致对容器的访问变成串行化的了。
1.6中采用ReentrantLock 分段锁的方式,使多个线程在不同的segment上进行写操作不会发现阻塞行为;1.8中直接采用了内置锁synchronized,难道是因为1.8的虚拟机对内置锁已经优化的足够快了?
第二篇
本文首写于有道云笔记,并在小组分享会分享,先整理发布,希望和大家交流探讨。云笔记地址
概述:
1、设计首要目的:维护并发可读性(get、迭代相关);次要目的:使空间消耗比HashMap相同或更好,且支持多线程高效率的初始插入(empty table)。
2、HashTable
线程安全,但采用synchronized,多线程下效率低下。线程1put时,线程2无法put或get。
实现原理:
锁分离:
在HashMap的基础上,将数据分段存储,
ConcurrentHashMap由多个Segment组成,每个Segment都有把锁。Segment下包含很多Node,也就是我们的键值对了。
如果还停留在锁分离、Segment,那已经out了。
Segment虽保留,但已经简化属性,仅仅是为了兼容旧版本。
- CAS算法;unsafe.compareAndSwapInt(this, valueOffset, expect, update); CAS(Compare And Swap),意思是如果valueOffset位置包含的值与expect值相同,则更新valueOffset位置的值为update,并返回true,否则不更新,返回false。
- 与Java8的HashMap有相通之处,底层依然由“数组”+链表+红黑树;
- 底层结构存放的是TreeBin对象,而不是TreeNode对象;
- CAS作为知名无锁算法,那ConcurrentHashMap就没用锁了么?当然不是,hash值相同的链表的头结点还是会synchronized上锁。
- private static final int MAXIMUM_CAPACITY = 1 << 30;
-
- private static final intDEFAULT_CAPACITY = 16;
-
- static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8;
-
- private static finalint DEFAULT_CONCURRENCY_LEVEL = 16;
-
- private static final float LOAD_FACTOR = 0.75f;
-
- static final int TREEIFY_THRESHOLD = 8;
-
- static final int UNTREEIFY_THRESHOLD = 6;
-
- static final int MIN_TREEIFY_CAPACITY = 64;
-
- private static final int MIN_TRANSFER_STRIDE = 16;
-
- private static int RESIZE_STAMP_BITS = 16;
-
- private static final int MAX_RESIZERS = (1 << (32 - RESIZE_STAMP_BITS)) - 1;
-
- private static final int RESIZE_STAMP_SHIFT = 32 - RESIZE_STAMP_BITS;
-
- static final int MOVED = -1;
-
- static final int TREEBIN = -2;
-
- static final int RESERVED = -3;
-
- static final int HASH_BITS = 0x7fffffff;
-
- static final int NCPU = Runtime.getRuntime().availableProcessors();
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- private transient volatile int sizeCtl;
-
- sizeCtl是控制标识符,不同的值表示不同的意义。
-
- 负数代表正在进行初始化或扩容操作
- -1代表正在初始化
- -N 表示有N-1个线程正在进行扩容操作
- 正数或0代表hash表还没有被初始化,这个数值表示初始化或下一次进行扩容的大小,类似于扩容阈值。它的值始终是当前ConcurrentHashMap容量的0.75倍,这与loadfactor是对应的。实际容量>=sizeCtl,则扩容。
部分构造函数:
- public ConcurrentHashMap(int initialCapacity,
- float loadFactor, int concurrencyLevel) {
- if (!(loadFactor > 0.0f) || initialCapacity < 0 || concurrencyLevel <= 0)
- thrownew IllegalArgumentException();
- if (initialCapacity < concurrencyLevel)
- initialCapacity = concurrencyLevel;
- long size = (long)(1.0 + (long)initialCapacity / loadFactor);
- int cap = (size >= (long)MAXIMUM_CAPACITY) ?
- MAXIMUM_CAPACITY : tableSizeFor((int)size);
- this.sizeCtl = cap;
- }
concurrencyLevel:
concurrencyLevel,能够同时更新ConccurentHashMap且不产生锁竞争的最大线程数,在Java8之前实际上就是ConcurrentHashMap中的分段锁个数,即Segment[]的数组长度
。
正确地估计很重要,当低估,数据结构将根据额外的竞争,从而导致线程试图写入当前锁定的段时阻塞;
相反,如果高估了并发级别,你遇到过大的膨胀,由于段的不必要的数量;
这种膨胀可能会导致性能下降,由于高数缓存未命中。
在Java8里,仅仅是为了兼容旧版本而保留。唯一的作用就是保证构造map时初始容量不小于concurrencyLevel。
源码122行:
Also, for compatibility with previous
versions of this class, constructors may optionally specify an
expected {@code concurrencyLevel} as an additional hint for
internal sizing.
源码482行:
Mainly: We
leave untouched but unused constructor arguments refering to
concurrencyLevel .……
……
1、重要属性:
1.1 Node:
- static class Node<K,V> implements Map.Entry<K,V> {
- final int hash;
- final K key;
- volatile V val;
- volatile Node<K,V> next;
-
- Node(inthash, K key, V val, Node<K,V> next) {
- this.hash = hash;
- this.key = key;
- this.val = val;
- this.next = next;
- }
-
- public final K getKey() { return key; }
- public final V getValue() { return val; }
-
- public final int hashCode() { returnkey.hashCode() ^ val.hashCode(); }
- public final String toString(){ returnkey + "=" + val; }
- public final V setValue(V value) {
- throw new UnsupportedOperationException();
- }
-
- public final boolean equals(Object o) {
- Object k, v, u; Map.Entry<?,?> e;
- return ((oinstanceof Map.Entry) &&
- (k = (e = (Map.Entry<?,?>)o).getKey()) != null &&
- (v = e.getValue()) != null &&
- (k == key || k.equals(key)) &&
- (v == (u = val) || v.equals(u)));
- }
-
-
-
-
- Node<K,V> find(inth, Object k) {
- Node<K,V> e = this;
- if (k != null) {
- do {
- K ek;
- if (e.hash == h &&
- ((ek = e.key) == k || (ek != null && k.equals(ek))))
- returne;
- } while ((e = e.next) != null);
- }
- returnnull;
- }
- }
1.2 TreeNode
-
-
- static final class TreeNode<K,V> extends Node<K,V> {
- TreeNode<K,V> parent;
- TreeNode<K,V> left;
- TreeNode<K,V> right;
- TreeNode<K,V> prev;
- boolean red;
-
- TreeNode(inthash, K key, V val, Node<K,V> next,
- TreeNode<K,V> parent) {
- super(hash, key, val, next);
- this.parent = parent;
- }
-
- Node<K,V> find(inth, Object k) {
- return findTreeNode(h, k, null);
- }
-
-
-
-
-
- final TreeNode<K,V> findTreeNode(int h, Object k, Class<?> kc) {
- if (k != null) {
- TreeNode<K,V> p = this;
- do {
- intph, dir; K pk; TreeNode<K,V> q;
- TreeNode<K,V> pl = p.left, pr = p.right;
- if ((ph = p.hash) > h)
- p = pl;
- elseif (ph < h)
- p = pr;
- elseif ((pk = p.key) == k || (pk != null && k.equals(pk)))
- returnp;
- elseif (pl == null)
- p = pr;
- elseif (pr == null)
- p = pl;
- elseif ((kc != null ||
- (kc = comparableClassFor(k)) != null) &&
- (dir = compareComparables(kc, k, pk)) != 0)
- p = (dir < 0) ? pl : pr;
- elseif ((q = pr.findTreeNode(h, k, kc)) != null)
- returnq;
- else
- p = pl;
- } while (p != null);
- }
- return null;
- }
- }
-
1.3 TreeBin
-
-
- TreeBin(TreeNode<K,V> b) {
- super(TREEBIN, null, null, null);
- this.first = b;
- TreeNode<K,V> r = null;
- for (TreeNode<K,V> x = b, next; x != null; x = next) {
- next = (TreeNode<K,V>)x.next;
- x.left = x.right = null;
- if (r == null) {
- x.parent = null;
- x.red = false;
- r = x;
- }
- else {
- K k = x.key;
- inth = x.hash;
- Class<?> kc = null;
- for (TreeNode<K,V> p = r;;) {
- intdir, ph;
- K pk = p.key;
- if ((ph = p.hash) > h)
- dir = -1;
- elseif (ph < h)
- dir = 1;
- elseif ((kc == null &&
- (kc = comparableClassFor(k)) == null) ||
- (dir = compareComparables(kc, k, pk)) == 0)
- dir = tieBreakOrder(k, pk);
- TreeNode<K,V> xp = p;
- if ((p = (dir <= 0) ? p.left : p.right) == null) {
- x.parent = xp;
- if (dir <= 0)
- xp.left = x;
- else
- xp.right = x;
- r = balanceInsertion(r, x);
- break;
- }
- }
- }
- }
- this.root = r;
- assert checkInvariants(root);
- }
1.4 treeifyBin
-
-
-
-
- private final void treeifyBin(Node<K,V>[] tab, int index) {
- Node<K,V> b; intn, sc;
- if (tab != null) {
- if ((n = tab.length) < MIN_TREEIFY_CAPACITY)
- tryPresize(n << 1);
- else if ((b = tabAt(tab, index)) != null && b.hash >= 0) {
- synchronized (b) {
- if (tabAt(tab, index) == b) {
- TreeNode<K,V> hd = null, tl = null;
- for (Node<K,V> e = b; e != null; e = e.next) {
- TreeNode<K,V> p =
- new TreeNode<K,V>(e.hash, e.key, e.val,
- null, null);
- if ((p.prev = tl) == null)
- hd = p;
- else
- tl.next = p;
- tl = p;
- }
- setTabAt(tab, index, new TreeBin<K,V>(hd));
- }
- }
- }
- }
- }
1.5 ForwardingNode
-
-
- static final class ForwardingNode<K,V> extends Node<K,V> {
- final Node<K,V>[] nextTable;
- ForwardingNode(Node<K,V>[] tab) {
- super(MOVED, null, null, null);
- this.nextTable = tab;
- }
-
- Node<K,V> find(int h, Object k) {
-
- outer: for (Node<K,V>[] tab = nextTable;;) {
- Node<K,V> e; intn;
- if (k == null || tab == null || (n = tab.length) == 0 ||
- (e = tabAt(tab, (n - 1) & h)) == null)
- returnnull;
- for (;;) {
- int eh; K ek;
- if ((eh = e.hash) == h &&
- ((ek = e.key) == k || (ek != null && k.equals(ek))))
- returne;
- if (eh < 0) {
- if (e instanceof ForwardingNode) {
- tab = ((ForwardingNode<K,V>)e).nextTable;
- continue outer;
- }
- else
- return e.find(h, k);
- }
- if ((e = e.next) == null)
- return null;
- }
- }
- }
- }
1.6 3个原子操作(调用频率很高)
- @SuppressWarnings("unchecked")
- static final <K,V> Node<K,V> tabAt(Node<K,V>[] tab, int i) {
- return (Node<K,V>)U.getObjectVolatile(tab, ((long)i << ASHIFT) + ABASE);
- }
-
- static final <K,V> boolean casTabAt(Node<K,V>[] tab, int i,
- Node<K,V> c, Node<K,V> v) {
- return U.compareAndSwapObject(tab, ((long)i << ASHIFT) + ABASE, c, v);
- }
-
- static final <K,V> void setTabAt(Node<K,V>[] tab, int i, Node<K,V> v) {
- U.putObjectVolatile(tab, ((long)i << ASHIFT) + ABASE, v);
- }
1.7 Unsafe
-
-
- private static final sun.misc.Unsafe U;
- private static final long SIZECTL;
- private static final long TRANSFERINDEX;
- private static final long BASECOUNT;
- private static final long CELLSBUSY;
- private static final long CELLVALUE;
- private static final long ABASE;
- private static final int ASHIFT;
-
- static {
- try {
- U = sun.misc.Unsafe.getUnsafe();
- Class<?> k = ConcurrentHashMap.class;
- SIZECTL = U.objectFieldOffset (k.getDeclaredField("sizeCtl"));
- TRANSFERINDEX=U.objectFieldOffset(k.getDeclaredField("transferIndex"));
- BASECOUNT = U.objectFieldOffset (k.getDeclaredField("baseCount"));
- CELLSBUSY = U.objectFieldOffset (k.getDeclaredField("cellsBusy"));
- Class<?> ck = CounterCell.class;
- CELLVALUE = U.objectFieldOffset (ck.getDeclaredField("value"));
- Class<?> ak = Node[].class;
- ABASE = U.arrayBaseOffset(ak);
- intscale = U.arrayIndexScale(ak);
- if ((scale & (scale - 1)) != 0)
- thrownew Error("data type scale not a power of two");
- ASHIFT = 31 - Integer.numberOfLeadingZeros(scale);
- } catch (Exception e) {
- thrownew Error(e);
- }
- }
1.8 扩容相关
tryPresize
在putAll以及treeifyBin中调用
- private final void tryPresize(int size) {
-
- int c = (size >= (MAXIMUM_CAPACITY >>> 1)) ? MAXIMUM_CAPACITY :
- tableSizeFor(size + (size >>> 1) + 1);
- int sc;
- while ((sc = sizeCtl) >= 0) {
- Node<K,V>[] tab = table; int n;
- if(tab == null || (n = tab.length) == 0) {
- n = (sc > c) ? sc : c;
-
- if (U.compareAndSwapInt(this, SIZECTL, sc, -1)) {
- try {
- if (table == tab) {
- @SuppressWarnings("unchecked")
- Node<K,V>[] nt = (Node<K,V>[])new Node<?,?>[n];
- table = nt;
- sc = n - (n >>> 2);
- }
- } finally {
- sizeCtl = sc;
- }
- }
- }
- else if (c <= sc || n >= MAXIMUM_CAPACITY)
- break;
- else if (tab == table) {
- int rs = resizeStamp(n);
- if (sc < 0) {
- Node<K,V>[] nt;
- if ((sc >>> RESIZE_STAMP_SHIFT) != rs || sc == rs + 1 ||
- sc == rs + MAX_RESIZERS || (nt = nextTable) == null ||
- transferIndex <= 0)
- break;
- if (U.compareAndSwapInt(this, SIZECTL, sc, sc + 1))
- transfer(tab, nt);
- }
- else if (U.compareAndSwapInt(this, SIZECTL, sc,
- (rs << RESIZE_STAMP_SHIFT) + 2))
- transfer(tab, null);
- }
- }
- }
- private static final int tableSizeFor(int c){
- int n = c - 1;
- n |= n >>> 1;
- n |= n >>> 2;
- n |= n >>> 4;
- n |= n >>> 8;
- n |= n >>> 16;
- return (n < 0) ? 1 : (n >= MAXIMUM_CAPACITY) ? MAXIMUM_CAPACITY : n + 1;
- }
-
-
-
-
- static final int resizeStamp(int n) {
- return Integer.numberOfLeadingZeros(n) | (1 << (RESIZE_STAMP_BITS - 1));
- }
-
ConcurrentHashMap无锁多线程扩容,减少扩容时的时间消耗。
transfer扩容操作
:单线程构建两倍容量的nextTable;允许多线程复制原table元素到nextTable。
- 为每个内核均分任务,并保证其不小于16;
- 若nextTab为null,则初始化其为原table的2倍;
- 死循环遍历,直到finishing。
- 节点为空,则插入ForwardingNode;
- 链表节点(fh>=0),分别插入nextTable的i和i+n的位置;【逆序链表??】
- TreeBin节点(fh<0),判断是否需要untreefi,分别插入nextTable的i和i+n的位置;【逆序树??】
- finishing时,nextTab赋给table,更新sizeCtl为新容量的0.75倍 ,完成扩容。
以上说的都是单线程,多线程又是如何实现的呢?
遍历到ForwardingNode节点((fh = f.hash) == MOVED),说明此节点被处理过了,直接跳过。这是控制并发扩容的核心
。
由于给节点上了锁,只允许当前线程完成此节点的操作,处理完毕后,将对应值设为ForwardingNode(fwd),其他线程看到forward,直接向后遍历。如此便完成了多线程的复制工作,也解决了线程安全问题。
private transient volatile Node<K,V>[] nextTable; //仅仅在扩容使用,并且此时非空
-
-
- private final void transfer(Node<K,V>[] tab, Node<K,V>[] nextTab) {
- int n = tab.length, stride;
-
- if ((stride = (NCPU > 1) ? (n >>> 3) / NCPU : n) < MIN_TRANSFER_STRIDE)
- stride = MIN_TRANSFER_STRIDE;
- if (nextTab == null) {
- try {
- @SuppressWarnings("unchecked")
- Node<K,V>[] nt = (Node<K,V>[])new Node<?,?>[n << 1];
- nextTab = nt;
- } catch (Throwable ex) {
- sizeCtl = Integer.MAX_VALUE;
- return;
- }
- nextTable = nextTab;
- transferIndex = n;
- }
- int nextn = nextTab.length;
-
- ForwardingNode<K,V> fwd= new ForwardingNode<K,V>(nextTab);
- boolean advance= true;
- boolean finishing = false;
- for (int i = 0, bound = 0;;) {
- Node<K,V> f; int fh;
- while (advance) {
- int nextIndex, nextBound;
- if (--i >= bound || finishing)
- advance = false;
- else if ((nextIndex = transferIndex) <= 0) {
- i = -1;
- advance = false;
- }
- else if (U.compareAndSwapInt
- (this, TRANSFERINDEX, nextIndex,
- nextBound = (nextIndex > stride ?
- nextIndex - stride : 0))) {
- bound = nextBound;
- i = nextIndex - 1;
- advance = false;
- }
- }
- if (i < 0 || i >= n || i + n >= nextn) {
- int sc;
- if (finishing) {
- nextTable = null;
- table = nextTab;
- sizeCtl = (n << 1) - (n >>> 1);
- return;
- }
- if (U.compareAndSwapInt(this, SIZECTL, sc = sizeCtl, sc - 1)) {
- if ((sc - 2) != resizeStamp(n) << RESIZE_STAMP_SHIFT)
- return;
- finishing = advance = true;
- i = n;
- }
- }
- else if ((f = tabAt(tab, i)) == null)
- advance = casTabAt(tab, i, null, fwd);
-
- else if ((fh = f.hash) == MOVED)
- advance = true;
- else {
- synchronized (f) {
- if (tabAt(tab, i) == f) {
- Node<K,V> ln, hn;
- if (fh >= 0) {
- int runBit = fh & n;
- Node<K,V> lastRun = f;
- for (Node<K,V> p = f.next; p != null; p = p.next) {
- int b = p.hash & n;
- if (b != runBit) {
- runBit = b;
- lastRun = p;
- }
- }
- if (runBit == 0) {
- ln = lastRun;
- hn = null;
- }
- else {
- hn = lastRun;
- ln = null;
- }
- for (Node<K,V> p = f; p != lastRun; p = p.next) {
- int ph = p.hash; K pk = p.key; V pv = p.val;
- if ((ph & n) == 0)
- ln = new Node<K,V>(ph, pk, pv, ln);
- else
- hn = new Node<K,V>(ph, pk, pv, hn);
- }
- setTabAt(nextTab, i, ln);
- setTabAt(nextTab, i + n, hn);
-
- setTabAt(tab, i, fwd);
-
- advance = true;
- }
- else if (f instanceof TreeBin) {
- TreeBin<K,V> t = (TreeBin<K,V>)f;
- TreeNode<K,V> lo = null, loTail = null;
- TreeNode<K,V> hi = null, hiTail = null;
-
- int lc = 0, hc = 0;
- for (Node<K,V> e = t.first; e != null; e = e.next) {
- int h = e.hash;
- TreeNode<K,V> p = new TreeNode<K,V>
- (h, e.key, e.val, null, null);
- if ((h & n) == 0) {
- if ((p.prev = loTail) == null)
- lo = p;
- else
- loTail.next = p;
- loTail = p;
- ++lc;
- }
- else {
- if ((p.prev = hiTail) == null)
- hi = p;
- else
- hiTail.next = p;
- hiTail = p;
- ++hc;
- }
- }
- ln = (lc <= UNTREEIFY_THRESHOLD) ? untreeify(lo) :
- (hc != 0) ? new TreeBin<K,V>(lo) : t;
- hn = (hc <= UNTREEIFY_THRESHOLD) ? untreeify(hi) :
- (lc != 0) ? new TreeBin<K,V>(hi) : t;
- setTabAt(nextTab, i, ln);
- setTabAt(nextTab, i + n, hn);
- setTabAt(tab, i, fwd);
- advance = true;
- }
- }
- }
- }
- }
- }
-
-
- final Node<K,V>[] helpTransfer(Node<K,V>[] tab, Node<K,V> f) {
- Node<K,V>[] nextTab; intsc;
- if (tab != null && (finstanceof ForwardingNode) &&
- (nextTab = ((ForwardingNode<K,V>)f).nextTable) != null) {
- intrs = resizeStamp(tab.length);
- while (nextTab == nextTable && table == tab &&
- (sc = sizeCtl) < 0) {
- if ((sc >>> RESIZE_STAMP_SHIFT) != rs || sc == rs + 1 ||
- sc == rs + MAX_RESIZERS || transferIndex <= 0)
- break;
- if (U.compareAndSwapInt(this, SIZECTL, sc, sc + 1)) {
- transfer(tab, nextTab);
- break;
- }
- }
- return nextTab;
- }
- return table;
- }
2、 put相关:
理一下put的流程:
①
判空:null直接抛空指针异常;
②
hash:计算h=key.hashcode;调用spread计算hash=
(
h
^(
h
>>>
16
))&
HASH_BITS;
③
遍历table
- 若table为空,则初始化,仅设置相关参数;
- @@@计算当前key存放位置,即table的下标i=(n - 1) & hash;
- 若待存放位置为null,casTabAt无锁插入;
- 若是forwarding nodes(检测到正在扩容),则helpTransfer(帮助其扩容);
- else(待插入位置非空且不是forward节点,即碰撞了),将头节点上锁(保证了线程安全):区分链表节点和树节点,分别插入(遇到hash值与key值都与新节点一致的情况,只需要更新value值即可。否则依次向后遍历,直到链表尾插入这个结点);
- 若链表长度>8,则treeifyBin转树(Note:若length<64,直接tryPresize,两倍table.length;不转树)。
④
addCount(1L, binCount)。
Note:
1、put操作共计两次hash操作,再利用“与&”操作计算Node的存放位置。
2、ConcurrentHashMap不允许key或value为null。
3、
addCount(longx,intcheck)方法:
①利用CAS快速更新baseCount的值;
②check>=0.则检验是否需要扩容;
if sizeCtl<0(正在进行初始化或扩容操作)【nexttable null等情况break;如果有线程正在扩容,则协助扩容】;
else if
仅当前线程在扩容,调用协助扩容函数,注其参数nextTable为null。
public V put(K key, V value) {
return putVal(key, value, false);
}
- final V <span style="background-color: rgb(255, 255, 51);">putVal</span>(K key, V value, boolean onlyIfAbsent) {
-
- if (key == null || value == null) throw new NullPointerException();
- int hash = spread(key.hashCode());
- int binCount = 0;
- for (Node<K,V>[] tab = table;;) {
- Node<K,V> f; int n, i, fh;
- if (tab == null || (n = tab.length) == 0)
- tab = initTable();
- else if ((f = tabAt(tab, i = (n - 1) & hash)) == null) {
- if (casTabAt(tab, i, null,
- new Node<K,V>(hash, key, value, null)))
- break;
- }
- else if ((fh = f.hash) == MOVED)
- tab = helpTransfer(tab, f);
- else {
- V oldVal = null;
- synchronized (f) {
- if (tabAt(tab, i) == f) {
- if (fh >= 0) {
- binCount = 1;
- for (Node<K,V> e = f;; ++binCount) {
- K ek;
- if (e.hash == hash &&
- ((ek = e.key) == key ||(ek != null && key.equals(ek)))) {
- oldVal = e.val;
- if (!onlyIfAbsent)
- e.val = value;
- break;
- }
- Node<K,V> pred = e;
- if ((e = e.next) == null) {
- pred.next = new Node<K,V>(hash, key, value, null);
- break;
- }
- }
- }
- else if (f instanceof TreeBin) {
- Node<K,V> p;
- binCount = 2;
- if ((p = ((TreeBin<K,V>)f).putTreeVal(hash, key,value)) != null) {
- oldVal = p.val;
- if (!onlyIfAbsent)
- p.val = value;
- }
- }
- }
- }
- if (binCount != 0) {
- if (binCount >= TREEIFY_THRESHOLD)
- treeifyBin(tab, i);
- if (oldVal != null)
- return oldVal;
- break;
- }
- }
- }
- addCount(1L, binCount);
- return null;
- }
-
- private final Node<K,V>[] <span style="background-color: rgb(255, 255, 51);">initTable</span>() {
- Node<K,V>[] tab; intsc;
- while ((tab = table) == null || tab.length == 0) {
- if ((sc = sizeCtl) < 0)
- Thread.yield();
-
- elseif (U.compareAndSwapInt(this, SIZECTL, sc, -1)) {
- try {
- if ((tab = table) == null || tab.length == 0) {
- intn = (sc > 0) ? sc : DEFAULT_CAPACITY;
- @SuppressWarnings("unchecked")
- Node<K,V>[] nt = (Node<K,V>[])new Node<?,?>[n];
- table = tab = nt;
- sc = n - (n >>> 2);
- }
- } finally {
- sizeCtl = sc;
- }
- break;
- }
- }
- return tab;
- }
3、 get、contains相关
- public V <span style="background-color: rgb(255, 255, 51);">get</span>(Object key) {
- Node<K,V>[] tab; Node<K,V> e, p; intn, eh; K ek;
- inth = spread(key.hashCode());
- if ((tab = table) != null && (n = tab.length) > 0 &&
- (e = tabAt(tab, (n - 1) & h)) != null) {
- if ((eh = e.hash) == h) {
- if ((ek = e.key) == key || (ek != null && key.equals(ek)))
- returne.val;
- }
- elseif (eh < 0)
- return (p = e.find(h, key)) != null ? p.val : null;
- while ((e = e.next) != null) {
- if (e.hash == h &&
- ((ek = e.key) == key || (ek != null && key.equals(ek))))
- returne.val;
- }
- }
- return null;
- }
- public boolean containsKey(Object key) {return get(key) != null;}
- public boolean containsValue(Object value) {}
理一下get的流程:
①spread计算hash值;
②table不为空;
③tabAt(i)处桶位不为空;
④check first,是则返回当前Node的value;否则分别根据树、链表查询。
4、 Size相关:
由于ConcurrentHashMap在统计size时
可能正被
多个线程操作,而我们又不可能让他停下来让我们计算,所以只能计量一个估计值。
计数辅助:
// Table of counter cells. When non-null, size is a power of 2
private transient volatile CounterCell[] counterCells;
|
@sun.misc.Contended static final class CounterCell {
volatile long value;
CounterCell(long x) { value = x; }
}
|
final long sumCount(){
CounterCell as[] = counterCells;
long sum = baseCount;
if(as != null){
for(int i = 0; i < as.length; i++){
CounterCell a;
if((a = as[i]) != null)
sum += a.value;
}
}
return sum;
}
|
private final void fullAddCount(long x, boolean wasUncontended) {}
|
public int size() { // 旧版本方法,和推荐的mappingCount返回的值基本无区别
longn = sumCount();
return ((n < 0L) ? 0 :
(n > (long)Integer.MAX_VALUE) ? Integer.MAX_VALUE :
(int)n);
}
|
// 返回Mappings中的元素个数,官方建议用来替代size。此方法返回的是一个估计值;如果sumCount时有线程插入或删除,实际数量是和mappingCount不同的。since 1.8
public long mappingCount() {
longn = sumCount();
return (n < 0L) ? 0L : n; // ignore transient negative values
}
|
private
transient
volatile
long
baseCount
;
//ConcurrentHashMap中元素个数,基于CAS无锁更新,但返回的不一定是当前Map的真实元素个数。
|
5、remove、clear相关:
- public void clear() {
- long delta = 0L;
- inti = 0;
- Node<K,V>[] tab = table;
- while (tab != null && i < tab.length) {
- intfh;
- Node<K,V> f = tabAt(tab, i);
- if (f == null)
- ++i;
- else if ((fh = f.hash) == MOVED) {
-
- tab = helpTransfer(tab, f);
- i = 0;
- }
- else{
- synchronized (f) {
- if (tabAt(tab, i) == f) {
- Node<K,V> p = (fh >= 0 ? f :
- (finstanceof TreeBin) ?
- ((TreeBin<K,V>)f).first : null);
- while (p != null) {
- --delta;
- p = p.next;
- }
- setTabAt(tab, i++, null);
- }
- }
- }
- }
- if (delta != 0L)
- addCount(delta, -1);
- }
- public V remove(Object key) {
- return replaceNode(key, null, null);
- }
- public boolean remove(Object key, Object value) {
- if (key == null)
- thrownew NullPointerException();
- returnvalue != null && replaceNode(key, null, value) != null;
- }
-
- final V replaceNode(Object key, V value, Object cv) {
- inthash = spread(key.hashCode());
- for (Node<K,V>[] tab = table;;) {
- Node<K,V> f; intn, i, fh;
- if (tab == null || (n = tab.length) == 0 ||
- (f = tabAt(tab, i = (n - 1) & hash)) == null)
- break;
- elseif ((fh = f.hash) == MOVED)
- tab = helpTransfer(tab, f);
- else {
- V oldVal = null;
- booleanvalidated = false;
- synchronized (f) {
- if (tabAt(tab, i) == f) {
- if (fh >= 0) {
- validated = true;
-
- for (Node<K,V> e = f, pred = null;;) {
- K ek;
- if (e.hash == hash &&((ek = e.key) == key ||
- (ek != null && key.equals(ek)))){
- V ev = e.val;
-
- if (cv == null || cv == ev ||
- (ev != null && cv.equals(ev))) {
- oldVal = ev;
- if (value != null)
- e.val = value;
- elseif (pred != null)
- pred.next = e.next;
- else
- setTabAt(tab, i, e.next);
- }
- break;
- }
- pred = e;
- if ((e = e.next) == null)
- break;
- }
- }
- elseif (finstanceof TreeBin) {
- validated = true;
- TreeBin<K,V> t = (TreeBin<K,V>)f;
- TreeNode<K,V> r, p;
- if ((r = t.root) != null &&
- (p = r.findTreeNode(hash, key, null)) != null) {
- V pv = p.val;
- if (cv == null || cv == pv ||
- (pv != null && cv.equals(pv))) {
- oldVal = pv;
- if (value != null)
- p.val = value;
- elseif (t.removeTreeNode(p))
- setTabAt(tab, i, untreeify(t.first));
- }
- }
- }
- }
- }
- if (validated) {
- if (oldVal != null) {
- if (value == null)
- addCount(-1L, -1);
- returnoldVal;
- }
- break;
- }
- }
- }
- return null;
- }
- public boolean replace(K key, V oldValue, V newValue) {}
6、其他函数:
public boolean isEmpty() {
return sumCount() <= 0L; // ignore transient negative values
}
参考资料:
、、、、、、、、、、
1
2
3
4
5
6
7
8
9
10
11
12 |
ArrayList源码分析(jdk1.8):http://blog.youkuaiyun.com/u010887744/article/details/49496093
HashMap源码分析(jdk1.8):http://write.blog.youkuaiyun.com/postedit/50346257
ConcurrentHashMap源码分析--Java8:http://blog.youkuaiyun.com/u010887744/article/details/50637030
ThreadLocal源码分析(JDK8) :http://blog.youkuaiyun.com/u010887744/article/details/54730556
每篇文章都包含 有道云笔记地址,可直接保存。
在线查阅JDK源码:
JDK8:https://github.com/zxiaofan/JDK1.8-Src
JDK7:https://github.com/zxiaofan/JDK_Src_1.7
史上最全Java集合关系图:http://blog.youkuaiyun.com/u010887744/article/details/50575735
|