hbase
关注
分享
扫一扫
文章平均质量分 63
iteye_15479
这个作者很懒，什么都没留下…
展开
专栏收录文章
- 默认排序
- 最新发布
- 最早发布
- 最多阅读
- 最少阅读
-
当hbase的master无法停掉的时候……
some times you will find the master in a hbase cluster cannot be stopped, so check the following two items suggested by some guy in hbase user mail list. But I'm not sure it helps. > If zookee...原创 2010-09-28 11:19:48 · 138 阅读 · 0 评论 -
Question on HBase source code
I'm reading source code of hbase. When come to class org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher I find this class has private member *unassignedNodes* but I cannot find where nodes are added i...2013-05-22 15:05:16 · 122 阅读 · 0 评论 -
Using the libjars option with Hadoop
As I have said in my last post, I was developing a hbase based mapreduce application. But one damn thing is the hadoop cluster managed by our system admin has no hbase jars in its classpath... So I...原创 2013-05-20 15:03:23 · 151 阅读 · 0 评论 -
Use HBase to Solve Page Access Problem
Currrently I'm working on sth like calculating how many hits on a page. The problem is the raw data can be huge, so it may not scale if you use RDBMS. The raw input is as follows. Date Page Use...2013-05-17 14:48:31 · 137 阅读 · 0 评论 -
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/thi
If you meet this exception, make sure you include the guava-r09-jarjar.jar JAR in your build path. This is usually located in /usr/lib/hadoop-0.20/lib.原创 2013-05-16 15:27:09 · 381 阅读 · 0 评论 -
[youtube] Scaling the Web: Databases & NoSQL
Very good talk on this subject. http://www.youtube.com/watch?NR=1&feature=endscreen&v=oL-A4JYwgH4原创 2012-03-23 13:11:27 · 94 阅读 · 0 评论 -
【读书笔记】Data warehousing and analytics infrastructure at facebook
这好像是sigmod2010上的paper。 读了之后做了以下几点记录： 1. facebook的hadoop cluster分成 scribe hadoop cluster: scribe servers将web service的log汇总然后存到HDFS上。通常会有带宽成为bottleneck的问题，这时候可以考虑压缩，但是一个副作用就是在buffer待压缩的数据的同时导致latency...原创 2011-03-18 22:03:27 · 193 阅读 · 0 评论 -
hbase中region的flush
都知道memstore大小到一定程度后会flush到disk上去，这个大小是由hbase.hregion.memstore.flush.size定义的。flush的时候也不是说马上就flush出去，会有个检查，就是下面这个方法了： code：hbase 0.20.6， MemStoreFlusher.java /* * A flushRegion that checks ...2011-01-20 15:52:20 · 318 阅读 · 0 评论 -
impact of total region numbers?
这几天tune了hbase的几个参数，有些有意思的结果。具体看我下面的邮件吧。 For example, I have total some data and I can tune hbase.hregion.max.filesize to increase/decrease total region number, rite? I want ...原创 2011-01-19 16:31:09 · 151 阅读 · 0 评论 -
Will all HFiles managed by a regionserver kept open
code 没看仔细，所以在hbase 的mail list上面问了这么个问题。其实再仔细看一下big table的paper就知道肯定是open的。现在分析的结果是hbase random read的performance决定在几个方面： 1）HDFS的seek操作，平均每次random get导致几次seek？ 2）memory copy; 这个问题尤其在data locality差的时候，...原创 2011-01-19 10:29:25 · 237 阅读 · 0 评论 -
细说HBase怎么完成一个Get操作 (server side)
上面有一篇记录了client边的过程，现在看看RegionSever这边怎么进行下去。 经过HBaseRPC后，调用传递到HRegionServer.get(byte[] regionName, Get get). HRegion region = getRegion(regionName); return region.get(get, getLockFromId(get.ge...原创 2011-01-14 16:33:50 · 163 阅读 · 0 评论 -
HBase importing
slides by hbase developer Ryan Rawson.原创 2010-11-30 12:54:03 · 106 阅读 · 0 评论 -
hadoop-0.20.2+737 and hbase-0.20.6 not compatible?
master log里面发现 0 region servers, 0 dead, average load NaN regionserver hang住。 原因不明。原创 2010-11-11 13:28:29 · 132 阅读 · 0 评论 -
hbase master cannot start up
2010-10-06 15:21:44,704 INFO org.apache.hadoop.hbase.master.HMaster: vmName=Java HotSpot(TM) Client VM, vmVendor=Sun Microsystems Inc., vmVersion=16.3-b01 2010-10-06 15:21:44,704 INFO org.apache.h...原创 2010-11-11 13:24:31 · 166 阅读 · 0 评论 -
细说HBase怎么完成一个Get操作 (client side)
源码解析基于HBase-0.20.6。 先看HTable类get()方法的code： HTable.java /** * Extracts certain cells from a given row. * @param get The object that specifies what data to fetch and from which ...原创 2010-10-14 14:37:33 · 262 阅读 · 0 评论 -
Hbase read performance with increasing number of client threads
在跑ycsb的时候，发现在做heavy的get操作的时候，ycsb统计的latency很大，100个thread的时候就接近100ms，而从ganglia上看hbase的“get_avg_time"这个metric发现只有20~30左右。最终查看code，发现100个thread共享同一个连接，所有Call的请求数据都走这一个连接，所以在大量请求并发时会造成拥堵，latency变大。具体看下...原创 2010-10-12 23:19:32 · 151 阅读 · 0 评论 -
What are some tips for configuring HBase?
Jeff Hammerbacher 8 endorsements 2 votes by Oleksiy Kovyrin and Alex Kamil Much of this content is taken from the HBase Overview [1] and the HBase Default Configur...原创 2010-10-11 15:42:59 · 141 阅读 · 0 评论 -
HBase Schema Design
As someone has said [url=http://www.stephenonsoftware.com/2012/06/hbase-schema-design.html]here[/url] [quote]You need to look more at the types of queries that you will be needing to access your da...原创 2013-05-24 11:41:20 · 151 阅读 · 0 评论