You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2008/11/24 23:57:44 UTC
[jira] Created: (HBASE-1027) Make global flusher check work with
percentages rather than hard code memory sizes.
Make global flusher check work with percentages rather than hard code memory sizes.
-----------------------------------------------------------------------------------
Key: HBASE-1027
URL: https://issues.apache.org/jira/browse/HBASE-1027
Project: Hadoop HBase
Issue Type: Improvement
Reporter: stack
Currently defaults are 512M for upperbound and 256 for the lowerbound. Comes of HBASE-1023.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1027) Make global flusher check work with
percentages rather than hard code memory sizes.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack reassigned HBASE-1027:
----------------------------
Assignee: stack
> Make global flusher check work with percentages rather than hard code memory sizes.
> -----------------------------------------------------------------------------------
>
> Key: HBASE-1027
> URL: https://issues.apache.org/jira/browse/HBASE-1027
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Assignee: stack
> Fix For: 0.19.0
>
> Attachments: 1027-v2.patch, 1027-v3.patch, 1027.patch
>
>
> Currently defaults are 512M for upperbound and 256 for the lowerbound. Comes of HBASE-1023.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1027) Make global flusher check work with
percentages rather than hard code memory sizes.
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653110#action_12653110 ]
Andrew Purtell commented on HBASE-1027:
---------------------------------------
Testing this patch now.
> Make global flusher check work with percentages rather than hard code memory sizes.
> -----------------------------------------------------------------------------------
>
> Key: HBASE-1027
> URL: https://issues.apache.org/jira/browse/HBASE-1027
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.19.0
>
> Attachments: 1027.patch
>
>
> Currently defaults are 512M for upperbound and 256 for the lowerbound. Comes of HBASE-1023.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1027) Make global flusher check work with
percentages rather than hard code memory sizes.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-1027.
--------------------------
Resolution: Fixed
Hadoop Flags: [Incompatible change]
Thanks Andrew for feedback. I tried it here and it works too so going ahead an committing.
> Make global flusher check work with percentages rather than hard code memory sizes.
> -----------------------------------------------------------------------------------
>
> Key: HBASE-1027
> URL: https://issues.apache.org/jira/browse/HBASE-1027
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.19.0
>
> Attachments: 1027-v2.patch, 1027-v3.patch, 1027.patch
>
>
> Currently defaults are 512M for upperbound and 256 for the lowerbound. Comes of HBASE-1023.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1027) Make global flusher check work with
percentages rather than hard code memory sizes.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1027:
-------------------------
Fix Version/s: 0.19.0
Bring into 0.19.
In streamy cluster, I see that the global is not running soon enough. Saw same testing locally. Making bounds for global percentages will help when heap is varied. Also, will set defaults down from 50% to 40% of default heap.
> Make global flusher check work with percentages rather than hard code memory sizes.
> -----------------------------------------------------------------------------------
>
> Key: HBASE-1027
> URL: https://issues.apache.org/jira/browse/HBASE-1027
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.19.0
>
>
> Currently defaults are 512M for upperbound and 256 for the lowerbound. Comes of HBASE-1023.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1027) Make global flusher check work with
percentages rather than hard code memory sizes.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1027:
-------------------------
Attachment: 1027.patch
Patch that makes limits into percentages. Changes hbase-default values. Old limits are no longer read. Need to test.
> Make global flusher check work with percentages rather than hard code memory sizes.
> -----------------------------------------------------------------------------------
>
> Key: HBASE-1027
> URL: https://issues.apache.org/jira/browse/HBASE-1027
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.19.0
>
> Attachments: 1027.patch
>
>
> Currently defaults are 512M for upperbound and 256 for the lowerbound. Comes of HBASE-1023.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1027) Make global flusher check work with
percentages rather than hard code memory sizes.
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653435#action_12653435 ]
Andrew Purtell commented on HBASE-1027:
---------------------------------------
I tried the first patch. Memcache use is higher now because of the use of percentages instead of hard limits, of course. I see a substantial amount of flushing in my logs, and looks like flushing happens until it should stop. It looks like it does work.
I'll try the second patch now. I'm in the middle of a cluster restart so can rebuild and push out new jars.
> Make global flusher check work with percentages rather than hard code memory sizes.
> -----------------------------------------------------------------------------------
>
> Key: HBASE-1027
> URL: https://issues.apache.org/jira/browse/HBASE-1027
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.19.0
>
> Attachments: 1027-v2.patch, 1027.patch
>
>
> Currently defaults are 512M for upperbound and 256 for the lowerbound. Comes of HBASE-1023.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1027) Make global flusher check work with
percentages rather than hard code memory sizes.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1027:
-------------------------
Attachment: 1027-v2.patch
v2 removes failing testcase built on presumptions that no longer hold in hbase -- optional flushing, setting global flush max and min using hard datasizes rather than percentages. Also tidies up the messages out of memflusher some.
Andrew, you tried previous version of patch? It didn't break anything? Did you see more flushing because mem in excess of global memcache limit of 40% of heap?
> Make global flusher check work with percentages rather than hard code memory sizes.
> -----------------------------------------------------------------------------------
>
> Key: HBASE-1027
> URL: https://issues.apache.org/jira/browse/HBASE-1027
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.19.0
>
> Attachments: 1027-v2.patch, 1027.patch
>
>
> Currently defaults are 512M for upperbound and 256 for the lowerbound. Comes of HBASE-1023.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1027) Make global flusher check work with
percentages rather than hard code memory sizes.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1027:
-------------------------
Attachment: 1027-v3.patch
Forgot to add hbase-default.xml changes.
> Make global flusher check work with percentages rather than hard code memory sizes.
> -----------------------------------------------------------------------------------
>
> Key: HBASE-1027
> URL: https://issues.apache.org/jira/browse/HBASE-1027
> Project: Hadoop HBase
> Issue Type: Improvement
> Reporter: stack
> Fix For: 0.19.0
>
> Attachments: 1027-v2.patch, 1027-v3.patch, 1027.patch
>
>
> Currently defaults are 512M for upperbound and 256 for the lowerbound. Comes of HBASE-1023.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.