You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Mikhail Bautin (Created) (JIRA)" <ji...@apache.org> on 2011/12/09 22:39:40 UTC
[jira] [Created] (HBASE-5000) Speed up simultaneous reads of a
block when block caching is turned off
Speed up simultaneous reads of a block when block caching is turned off
-----------------------------------------------------------------------
Key: HBASE-5000
URL: https://issues.apache.org/jira/browse/HBASE-5000
Project: HBase
Issue Type: Bug
Reporter: Mikhail Bautin
Priority: Minor
With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a "sparse lock" primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it.
Setting priority as minor, as it is very unusual to turn off block caching.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5000) Speed up simultaneous reads of a
block when block caching is turned off
Posted by "Mikhail Bautin (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mikhail Bautin reassigned HBASE-5000:
-------------------------------------
Assignee: Mikhail Bautin
> Speed up simultaneous reads of a block when block caching is turned off
> -----------------------------------------------------------------------
>
> Key: HBASE-5000
> URL: https://issues.apache.org/jira/browse/HBASE-5000
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
>
> With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a "sparse lock" primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it.
> Setting priority as minor, as it is very unusual to turn off block caching.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5000) Speed up simultaneous
reads of a block when block caching is turned off
Posted by "Lars Hofhansl (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167360#comment-13167360 ]
Lars Hofhansl edited comment on HBASE-5000 at 12/12/11 5:39 AM:
----------------------------------------------------------------
+1 on documenting. Could state there that block caching can be disabled on a per scan bases, and in fact should be disabled if the data scanned is known (or likely to) not to fit in the cache.
Generally shouldn't aim for loading the same block only once per JVM if requested by multiple threads at the same time? Could do some kind of reference counting to keep track of other requesters, or register a call back when the block is loaded...
was (Author: lhofhansl):
+1 on documenting. Could state there, that block caching can be disabled on a per scan bases, and in fact should be disable if the data scanned is known (or likely to) not to fit in the cache.
Generally shouldn't aim for loading the same block only once per JVM if requested by multiple threads at the same time? Could do some kind of reference counting to keep track of other requesters, or register a call back when the block is loaded...
> Speed up simultaneous reads of a block when block caching is turned off
> -----------------------------------------------------------------------
>
> Key: HBASE-5000
> URL: https://issues.apache.org/jira/browse/HBASE-5000
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
>
> With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a "sparse lock" primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it.
> Setting priority as minor, as it is very unusual to turn off block caching.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5000) Speed up simultaneous reads of a
block when block caching is turned off
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167761#comment-13167761 ]
Hudson commented on HBASE-5000:
-------------------------------
Integrated in HBase-TRUNK-security #29 (See [https://builds.apache.org/job/HBase-TRUNK-security/29/])
HBASE-5000 Speed up simultaneous reads of a block when block caching is turned off; bit of doc and why its a bad idea disabling block cache completely
stack :
Files :
* /hbase/trunk/src/docbkx/configuration.xml
> Speed up simultaneous reads of a block when block caching is turned off
> -----------------------------------------------------------------------
>
> Key: HBASE-5000
> URL: https://issues.apache.org/jira/browse/HBASE-5000
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
>
> With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a "sparse lock" primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it.
> Setting priority as minor, as it is very unusual to turn off block caching.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5000) Speed up simultaneous reads of a
block when block caching is turned off
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167655#comment-13167655 ]
stack commented on HBASE-5000:
------------------------------
I added a bit of rough doc suggesting users not disable blockcache completely.
{code}
<section xml:id="disabling.blockcache"><title>Disabling Blockcache</title>
<para>Do not turn off block cache (You'd do it by setting <varname>hbase.block.cache.size</varname> to zero).
Currently we do not do well if you do this because the regionserver will spend all its time loading hfile
indices over and over again. If your working set it such that block cache does you no good, at least
size the block cache such that hfile indices will stay up in the cache (you can get a rough idea
on the size you need by surveying regionserver UIs; you'll see index block size accounted near the
top of the webpage).</para>
</section>
{code}
> Speed up simultaneous reads of a block when block caching is turned off
> -----------------------------------------------------------------------
>
> Key: HBASE-5000
> URL: https://issues.apache.org/jira/browse/HBASE-5000
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
>
> With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a "sparse lock" primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it.
> Setting priority as minor, as it is very unusual to turn off block caching.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5000) Speed up simultaneous reads of a
block when block caching is turned off
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167360#comment-13167360 ]
Lars Hofhansl commented on HBASE-5000:
--------------------------------------
+1 on documenting. Could state there, that block caching can be disabled on a per scan bases, and in fact should be disable if the data scanned is known (or likely to) not to fit in the cache.
Generally shouldn't aim for loading the same block only once per JVM if requested by multiple threads at the same time? Could do some kind of reference counting to keep track of other requesters, or register a call back when the block is loaded...
> Speed up simultaneous reads of a block when block caching is turned off
> -----------------------------------------------------------------------
>
> Key: HBASE-5000
> URL: https://issues.apache.org/jira/browse/HBASE-5000
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
>
> With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a "sparse lock" primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it.
> Setting priority as minor, as it is very unusual to turn off block caching.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5000) Speed up simultaneous
reads of a block when block caching is turned off
Posted by "Lars Hofhansl (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167360#comment-13167360 ]
Lars Hofhansl edited comment on HBASE-5000 at 12/12/11 5:40 AM:
----------------------------------------------------------------
+1 on documenting. Could state there that block caching can be disabled on a per scan bases, and in fact should be disabled if the data scanned is known (or likely to) not to fit in the cache.
Generally shouldn't we aim for loading the same block only once per JVM if requested by multiple threads at the same time? Could do some kind of reference counting to keep track of other requesters, or register a call back when the block is loaded...
was (Author: lhofhansl):
+1 on documenting. Could state there that block caching can be disabled on a per scan bases, and in fact should be disabled if the data scanned is known (or likely to) not to fit in the cache.
Generally shouldn't aim for loading the same block only once per JVM if requested by multiple threads at the same time? Could do some kind of reference counting to keep track of other requesters, or register a call back when the block is loaded...
> Speed up simultaneous reads of a block when block caching is turned off
> -----------------------------------------------------------------------
>
> Key: HBASE-5000
> URL: https://issues.apache.org/jira/browse/HBASE-5000
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
>
> With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a "sparse lock" primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it.
> Setting priority as minor, as it is very unusual to turn off block caching.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5000) Speed up simultaneous reads of a
block when block caching is turned off
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166608#comment-13166608 ]
stack commented on HBASE-5000:
------------------------------
I agree this priority minor. We should probably add to the book that currently disabling block cache completed is not recommended.
> Speed up simultaneous reads of a block when block caching is turned off
> -----------------------------------------------------------------------
>
> Key: HBASE-5000
> URL: https://issues.apache.org/jira/browse/HBASE-5000
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Priority: Minor
>
> With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a "sparse lock" primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it.
> Setting priority as minor, as it is very unusual to turn off block caching.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5000) Speed up simultaneous reads of a
block when block caching is turned off
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167900#comment-13167900 ]
Hudson commented on HBASE-5000:
-------------------------------
Integrated in HBase-TRUNK #2540 (See [https://builds.apache.org/job/HBase-TRUNK/2540/])
HBASE-5000 Speed up simultaneous reads of a block when block caching is turned off; bit of doc and why its a bad idea disabling block cache completely
stack :
Files :
* /hbase/trunk/src/docbkx/configuration.xml
> Speed up simultaneous reads of a block when block caching is turned off
> -----------------------------------------------------------------------
>
> Key: HBASE-5000
> URL: https://issues.apache.org/jira/browse/HBASE-5000
> Project: HBase
> Issue Type: Bug
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Minor
>
> With block caching, when one client starts reading a block and another one comes around asking for the same block, the second client waits for the first one to finish reading and returns the block from cache. This is achieved by locking on the block offset using IdLock, a "sparse lock" primitive allowing to lock on arbitrary long numbers. However, in case there is no block caching, there is no reason to wait for other clients that are reading the same block. One challenge optimizing this that we don't necessary have accurate information about whether other HFile API clients interested in the block would cache it.
> Setting priority as minor, as it is very unusual to turn off block caching.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira