You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by on 2016/10/24 03:01:09 UTC
[In Memory] Questions about Make HBase Table In Memory
Hi, HBase Experts
I have several questions related to HBase BlockCache and IN_MEMORY setting
1. Does BlockCache cache the data when do PUT request, so next time
GET/SCAN request against the same data will be served from Memory only?
2. If HBase Table ColumnFamily have *IN_MEMORY=true*, and every time I do
PUT followed immediately with a GET/SCAN against the same data, assume I
have memory larger than the specific table, is that this way I can make
sure the whole table is in memory and all read requests against this table
will be served from memory only?
3. How can I determine whether a GET served from Disk Reading or Memory?
4. Compared to Item 2 above against "Prefetch Option for Blockcache", which
one is best for read performance?
The purpose I try to achieve is best and stable read performance even when
have high HDFS IO like re-balance when new servers added. we want the
performance could be comparable against memory solution like Redis.
Thanks
Proust
--
地址: 上海市浦东新区金科路2889弄长泰广场C座7楼
Address: 7th floor, No#3, Jinke road 2889, Pudong district, Shanghai, China.
Mobile: +86 13621672634
Re: [In Memory] Questions about Make HBase Table In Memory
Posted by spats <sp...@gmail.com>.
Hi @Proust , Could you please share your findings? is in_memory table
performance is comparable to redis etc?
--
Sent from: http://apache-hbase.679495.n3.nabble.com/HBase-User-f4020416.html
Re: [In Memory] Questions about Make HBase Table In Memory
Posted by .
Thanks Ted for your reply, the idea actually based on the HBase doc about
Block Cache Design, so with IN_MEMORY=true and proactively read(GET/SCAN)
the table, we can get the table served from memory for best and stable read
performance.
On 24 October 2016 at 11:19, Ted Yu <yu...@gmail.com> wrote:
> Puts are stored in memstore. Get / Scan would include such data in memstore
> when row keys match.
>
> For #2, see http://hbase.apache.org/book.html#block.cache.design
>
> For #3, one indication is latency of the result. Latency would be longer
> when disk is involved.
>
>
> On Sun, Oct 23, 2016 at 8:01 PM, Proust(Proust/Feng Guizhou) [Saipan] <
> pfeng@coupang.com> wrote:
>
> > Hi, HBase Experts
> >
> > I have several questions related to HBase BlockCache and IN_MEMORY
> setting
> > 1. Does BlockCache cache the data when do PUT request, so next time
> > GET/SCAN request against the same data will be served from Memory only?
> > 2. If HBase Table ColumnFamily have *IN_MEMORY=true*, and every time I do
> > PUT followed immediately with a GET/SCAN against the same data, assume I
> > have memory larger than the specific table, is that this way I can make
> > sure the whole table is in memory and all read requests against this
> table
> > will be served from memory only?
> > 3. How can I determine whether a GET served from Disk Reading or Memory?
> > 4. Compared to Item 2 above against "Prefetch Option for Blockcache",
> which
> > one is best for read performance?
> >
> > The purpose I try to achieve is best and stable read performance even
> when
> > have high HDFS IO like re-balance when new servers added. we want the
> > performance could be comparable against memory solution like Redis.
> >
> > Thanks
> > Proust
> >
> > --
> >
> > 地址: 上海市浦东新区金科路2889弄长泰广场C座7楼
> > Address: 7th floor, No#3, Jinke road 2889, Pudong district, Shanghai,
> > China.
> > Mobile: +86 13621672634
> >
>
--
地址: 上海市浦东新区金科路2889弄长泰广场C座7楼
Address: 7th floor, No#3, Jinke road 2889, Pudong district, Shanghai, China.
Mobile: +86 13621672634
Re: [In Memory] Questions about Make HBase Table In Memory
Posted by Ted Yu <yu...@gmail.com>.
Puts are stored in memstore. Get / Scan would include such data in memstore
when row keys match.
For #2, see http://hbase.apache.org/book.html#block.cache.design
For #3, one indication is latency of the result. Latency would be longer
when disk is involved.
On Sun, Oct 23, 2016 at 8:01 PM, Proust(Proust/Feng Guizhou) [Saipan] <
pfeng@coupang.com> wrote:
> Hi, HBase Experts
>
> I have several questions related to HBase BlockCache and IN_MEMORY setting
> 1. Does BlockCache cache the data when do PUT request, so next time
> GET/SCAN request against the same data will be served from Memory only?
> 2. If HBase Table ColumnFamily have *IN_MEMORY=true*, and every time I do
> PUT followed immediately with a GET/SCAN against the same data, assume I
> have memory larger than the specific table, is that this way I can make
> sure the whole table is in memory and all read requests against this table
> will be served from memory only?
> 3. How can I determine whether a GET served from Disk Reading or Memory?
> 4. Compared to Item 2 above against "Prefetch Option for Blockcache", which
> one is best for read performance?
>
> The purpose I try to achieve is best and stable read performance even when
> have high HDFS IO like re-balance when new servers added. we want the
> performance could be comparable against memory solution like Redis.
>
> Thanks
> Proust
>
> --
>
> 地址: 上海市浦东新区金科路2889弄长泰广场C座7楼
> Address: 7th floor, No#3, Jinke road 2889, Pudong district, Shanghai,
> China.
> Mobile: +86 13621672634
>