"IPC Server handler 26 on 60020" tid=72 daemon=true priority=5 state=BLOCKED
at org.apache.hadoop.hbase.regionserver.HLog.append
at org.apache.hadoop.hbase.regionserver.HRegion.put
at org.apache.hadoop.hbase.regionserver.HRegion.put
at org.apache.hadoop.hbase.regionserver.HRegion.put
at org.apache.hadoop.hbase.regionserver.HRegionServer.put
at sun.reflect.GeneratedMethodAccessor3.invoke
at sun.reflect.DelegatingMethodAccessorImpl.invoke
at java.lang.reflect.Method.invoke
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run
"IPC Server handler 64 on 60020" tid=110 daemon=true priority=5 state=BLOCKED
at org.apache.hadoop.hbase.regionserver.HRegion.obtainRowLock
at org.apache.hadoop.hbase.regionserver.HRegion.getLock
at org.apache.hadoop.hbase.regionserver.HRegion.put
at org.apache.hadoop.hbase.regionserver.HRegion.put
at org.apache.hadoop.hbase.regionserver.HRegionServer.put
at sun.reflect.GeneratedMethodAccessor3.invoke
at sun.reflect.DelegatingMethodAccessorImpl.invoke
at java.lang.reflect.Method.invoke
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run
"IPC Server handler 33 on 60020" tid=79 daemon=true priority=5 state=BLOCKED
at org.apache.hadoop.hbase.regionserver.HRegion.releaseRowLock
at org.apache.hadoop.hbase.regionserver.HRegion.put
at org.apache.hadoop.hbase.regionserver.HRegion.put
at org.apache.hadoop.hbase.regionserver.HRegionServer.put
at sun.reflect.GeneratedMethodAccessor3.invoke
at sun.reflect.DelegatingMethodAccessorImpl.invoke
at java.lang.reflect.Method.invoke
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run
这是HBase的client在做大量插入record的时候的log,可以发现很多时候线程阻塞在行锁上以及HLog获得写锁上。因为一个regionserver上面的所有region拥有同一个HLog,所以不可避免会有竞争。