You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2009/06/05 21:40:07 UTC
[jira] Created: (HBASE-1486) BLOCKCACHE always on even when
disabled
BLOCKCACHE always on even when disabled
---------------------------------------
Key: HBASE-1486
URL: https://issues.apache.org/jira/browse/HBASE-1486
Project: Hadoop HBase
Issue Type: Bug
Reporter: stack
Fix For: 0.20.0
Comes from Billy Pearson up on list.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Updated: (HBASE-1486) BLOCKCACHE always on even when
disabled
Posted by Ryan Rawson <ry...@gmail.com>.
Lars, can you make sure this bit defaults to true?
On Jun 6, 2009 4:28 PM, "Lars George (JIRA)" <ji...@apache.org> wrote:
[
https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issue.
..
Attachment: 1486-2.patch
Oops, sorry, my IDE missed a few spots. The ant build found them though.
This second patch fixes all compile issues.
But please note that now the TestStoreFile.java has the option to test with
and without the block cache. I have set it to "false" but maybe the tests
need to be run twice? Once with and once without block cache?
> BLOCKCACHE always on even when disabled >
--------------------------------------- > > ...
> Attachments: 1486-2.patch, 1486.patch
> > > Comes from Billy Pearson up on list. -- This message is automatically
generated by JIRA. - Y...
[jira] Updated: (HBASE-1486) BLOCKCACHE always on even when
disabled
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars George updated HBASE-1486:
-------------------------------
Attachment: 1486.patch
I am taking a stab at this. Maybe Billy can verify as well if this fixes the issue.
The issue was that the isBlockCacheEnabled from the HColumnDefinition was *not* handed to the Store and StoreFiles eventually. I added this to these two classes and when a block cache is asked for during the creation of a Reader it skips it when blockcache is set to false.
> BLOCKCACHE always on even when disabled
> ---------------------------------------
>
> Key: HBASE-1486
> URL: https://issues.apache.org/jira/browse/HBASE-1486
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Lars George
> Fix For: 0.20.0
>
> Attachments: 1486.patch
>
>
> Comes from Billy Pearson up on list.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1486) BLOCKCACHE always on even when
disabled
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-1486.
--------------------------
Resolution: Fixed
Committed. Thanks for the patch Lars.
In 0.19.x, we had block cache on for catalog tables as default (off for historian). We wanted to have block cache on for all tables by default for same reasons as I see being put forward up on #hbase but in 0.19, block cache could cause us to OOME so we were selective -- the catalog tables only.
I added note that migration needs to enable block cache on tables or at least recommend that old tables have it manually enabled.
> BLOCKCACHE always on even when disabled
> ---------------------------------------
>
> Key: HBASE-1486
> URL: https://issues.apache.org/jira/browse/HBASE-1486
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Lars George
> Fix For: 0.20.0
>
> Attachments: 1486-2.patch, 1486-3.patch, 1486.patch
>
>
> Comes from Billy Pearson up on list.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1486) BLOCKCACHE always on even when
disabled
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717001#action_12717001 ]
Billy Pearson commented on HBASE-1486:
--------------------------------------
BLOCKCACHE => 'true' for info in meta
and
BLOCKCACHE => 'false' for historian
> BLOCKCACHE always on even when disabled
> ---------------------------------------
>
> Key: HBASE-1486
> URL: https://issues.apache.org/jira/browse/HBASE-1486
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Lars George
> Fix For: 0.20.0
>
> Attachments: 1486-2.patch, 1486.patch
>
>
> Comes from Billy Pearson up on list.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1486) BLOCKCACHE always on even when
disabled
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars George reassigned HBASE-1486:
----------------------------------
Assignee: Lars George
> BLOCKCACHE always on even when disabled
> ---------------------------------------
>
> Key: HBASE-1486
> URL: https://issues.apache.org/jira/browse/HBASE-1486
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Lars George
> Fix For: 0.20.0
>
>
> Comes from Billy Pearson up on list.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1486) BLOCKCACHE always on even when
disabled
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars George updated HBASE-1486:
-------------------------------
Attachment: 1486-3.patch
OK, as per the discussion, I have
- changed HColumnDefinition to have the block cache set to *true* by default
- changed the TestStoreFile to test *with* block cache on as it was the case originally (since it was always on)
Also, I checked the HTableDescriptor and found that for
- ROOT block cache is *true*
- .META. block cache is *true*
- Historian block cache is *false*
Are these still OK. Sounds to me, not sure about the Historian, but is seems to be set to false on purpose?
> BLOCKCACHE always on even when disabled
> ---------------------------------------
>
> Key: HBASE-1486
> URL: https://issues.apache.org/jira/browse/HBASE-1486
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Lars George
> Fix For: 0.20.0
>
> Attachments: 1486-2.patch, 1486-3.patch, 1486.patch
>
>
> Comes from Billy Pearson up on list.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1486) BLOCKCACHE always on even when
disabled
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717002#action_12717002 ]
Billy Pearson commented on HBASE-1486:
--------------------------------------
Seams to be working HBASE-1304 killed my ability to import until its side effects get resolved I put some data and flushed via shell then confirmed that the region server was using 0 blocks of block cache where the region was stored.
so +1
> BLOCKCACHE always on even when disabled
> ---------------------------------------
>
> Key: HBASE-1486
> URL: https://issues.apache.org/jira/browse/HBASE-1486
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Lars George
> Fix For: 0.20.0
>
> Attachments: 1486-2.patch, 1486.patch
>
>
> Comes from Billy Pearson up on list.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1486) BLOCKCACHE always on even when
disabled
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716999#action_12716999 ]
ryan rawson commented on HBASE-1486:
------------------------------------
the block cache family value should default to 'true' or 'on'. Can you please make sure this is the case.
> BLOCKCACHE always on even when disabled
> ---------------------------------------
>
> Key: HBASE-1486
> URL: https://issues.apache.org/jira/browse/HBASE-1486
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Lars George
> Fix For: 0.20.0
>
> Attachments: 1486-2.patch, 1486.patch
>
>
> Comes from Billy Pearson up on list.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1486) BLOCKCACHE always on even when
disabled
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717000#action_12717000 ]
Billy Pearson commented on HBASE-1486:
--------------------------------------
when I create a new table and do not define block cache in the column it was getting set to true
also same in meta
{code}
hbase(main):002:0> describe '.META.'
09/06/06 19:37:36 INFO zookeeper.ZooKeeperWrapper: Quorum servers: server-1:2181
09/06/06 19:37:37 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state: SyncConnected, type: None, path: null
09/06/06 19:37:37 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/master got 10.0.1.1:60000
DESCRIPTION ENABLED
09/06/06 19:37:37 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state: SyncConnected, type: None, path: null
09/06/06 19:37:37 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.0.1.5:60020
09/06/06 19:37:37 DEBUG client.HConnectionManager$TableServers: Found ROOT at 10.0.1.5:60020
09/06/06 19:37:37 DEBUG client.HConnectionManager$TableServers: address: 10.0.1.8:60020, regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192, TABLE => {{NAME => '.META.', MEMCACHE_FLUSHSIZE => '16384', IS_META => 'true', FAMILIES => [{NAME => 'historian', VERSIONS => '2147483647', COMPRESSION => 'NONE', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '10', COMPRESSION => 'NONE', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}
09/06/06 19:37:37 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got 10.0.1.5:60020
09/06/06 19:37:37 DEBUG client.HConnectionManager$TableServers: Found ROOT at 10.0.1.5:60020
{NAME => '.META.', MEMCACHE_FLUSHSIZE => '16384', IS_META => 'true', F true
AMILIES => [{NAME => 'historian', COMPRESSION => 'NONE', VERSIONS => '
2147483647', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false
', BLOCKCACHE => 'false'}, {NAME => 'info', COMPRESSION => 'NONE', VER
SIONS => '10', TTL => '2147483647', BLOCKSIZE => '8192', IN_MEMORY =>
'false', BLOCKCACHE => 'true'}]}
1 row(s) in 0.2430 seconds
{code}
> BLOCKCACHE always on even when disabled
> ---------------------------------------
>
> Key: HBASE-1486
> URL: https://issues.apache.org/jira/browse/HBASE-1486
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Lars George
> Fix For: 0.20.0
>
> Attachments: 1486-2.patch, 1486.patch
>
>
> Comes from Billy Pearson up on list.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1486) BLOCKCACHE always on even when
disabled
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars George updated HBASE-1486:
-------------------------------
Attachment: 1486-2.patch
Oops, sorry, my IDE missed a few spots. The ant build found them though. This second patch fixes all compile issues.
But please note that now the TestStoreFile.java has the option to test with and without the block cache. I have set it to "false" but maybe the tests need to be run twice? Once with and once without block cache?
> BLOCKCACHE always on even when disabled
> ---------------------------------------
>
> Key: HBASE-1486
> URL: https://issues.apache.org/jira/browse/HBASE-1486
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Lars George
> Fix For: 0.20.0
>
> Attachments: 1486-2.patch, 1486.patch
>
>
> Comes from Billy Pearson up on list.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.