You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Jonathan Gray (JIRA)" <ji...@apache.org> on 2009/07/07 07:50:15 UTC
[jira] Created: (HBASE-1618) Investigate further into the
MemStoreFlusher StoreFile limit
Investigate further into the MemStoreFlusher StoreFile limit
------------------------------------------------------------
Key: HBASE-1618
URL: https://issues.apache.org/jira/browse/HBASE-1618
Project: Hadoop HBase
Issue Type: Bug
Components: regionserver
Reporter: Jonathan Gray
Assignee: Jonathan Gray
Fix For: 0.20.0
This seems to cause some weird behavior and not accomplish it's original intent (prevent a region growing to hundreds of storefiles).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1618) Investigate further into the
MemStoreFlusher StoreFile limit
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1618:
---------------------------------
Attachment: HBASE-1618-v2-19branch.patch
Patch that applies to 0.19 branch.
> Investigate further into the MemStoreFlusher StoreFile limit
> ------------------------------------------------------------
>
> Key: HBASE-1618
> URL: https://issues.apache.org/jira/browse/HBASE-1618
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1618-v1.patch, HBASE-1618-v2-19branch.patch, HBASE-1618-v2.patch
>
>
> This seems to cause some weird behavior and not accomplish it's original intent (prevent a region growing to hundreds of storefiles).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1618) Investigate further into the
MemStoreFlusher StoreFile limit
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1618:
---------------------------------
Status: Patch Available (was: Open)
Ready for review + commit.
> Investigate further into the MemStoreFlusher StoreFile limit
> ------------------------------------------------------------
>
> Key: HBASE-1618
> URL: https://issues.apache.org/jira/browse/HBASE-1618
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1618-v1.patch, HBASE-1618-v2-19branch.patch, HBASE-1618-v2.patch
>
>
> This seems to cause some weird behavior and not accomplish it's original intent (prevent a region growing to hundreds of storefiles).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1618) Investigate further into the
MemStoreFlusher StoreFile limit
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1618:
---------------------------------
Attachment: HBASE-1618-v2.patch
Cleans up the patch and removes most of the added nosie. Will make 0.19 patch later.
Please test this w/ HBASE-1620 at high load... should help significantly.
> Investigate further into the MemStoreFlusher StoreFile limit
> ------------------------------------------------------------
>
> Key: HBASE-1618
> URL: https://issues.apache.org/jira/browse/HBASE-1618
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1618-v1.patch, HBASE-1618-v2.patch
>
>
> This seems to cause some weird behavior and not accomplish it's original intent (prevent a region growing to hundreds of storefiles).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1618) Investigate further into the
MemStoreFlusher StoreFile limit
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728128#action_12728128 ]
Jean-Daniel Cryans commented on HBASE-1618:
-------------------------------------------
I saw that bug too and I remember that before either 0.19.2 or 0.19.3 something was reverted in that code. I guess that if your fix really fixes it, it should be going into 0.19.4.
> Investigate further into the MemStoreFlusher StoreFile limit
> ------------------------------------------------------------
>
> Key: HBASE-1618
> URL: https://issues.apache.org/jira/browse/HBASE-1618
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1618-v1.patch
>
>
> This seems to cause some weird behavior and not accomplish it's original intent (prevent a region growing to hundreds of storefiles).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1618) Investigate further into the
MemStoreFlusher StoreFile limit
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Gray updated HBASE-1618:
---------------------------------
Attachment: HBASE-1618-v1.patch
Patch fixes it and adds a bunch of debug. Quick fix needs to be cleaned up.
Want to spend more time with this issue, patch not for commit. But test with it, definitely seems to help and can watch how it behaves (very verbose).
Once we reach the StoreFile limit during a flush, we prevent the flushing and wait for compactions. Once we drop below the limit, or we wait past the max time limit, we proceed with the flush.
During the wait period, MemStores will start to back up. Once they reach the multiplier blocking size (i have it set to 4 not default 2), the blocking updates wall comes up.
My upload is behaving much better now, sustaining about 5k rows per second (rows have 12 columns total about 2K of data). This is on 3 slow nodes (2 core, 2gb, single hdd).
> Investigate further into the MemStoreFlusher StoreFile limit
> ------------------------------------------------------------
>
> Key: HBASE-1618
> URL: https://issues.apache.org/jira/browse/HBASE-1618
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1618-v1.patch
>
>
> This seems to cause some weird behavior and not accomplish it's original intent (prevent a region growing to hundreds of storefiles).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1618) Investigate further into the
MemStoreFlusher StoreFile limit
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1618:
-------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Ran in test. Doesn't break anything and has nicer log messages. Lets see if it makes a difference. Thanks for patch Jon.
> Investigate further into the MemStoreFlusher StoreFile limit
> ------------------------------------------------------------
>
> Key: HBASE-1618
> URL: https://issues.apache.org/jira/browse/HBASE-1618
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1618-v1.patch, HBASE-1618-v2-19branch.patch, HBASE-1618-v2.patch
>
>
> This seems to cause some weird behavior and not accomplish it's original intent (prevent a region growing to hundreds of storefiles).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1618) Investigate further into the
MemStoreFlusher StoreFile limit
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728367#action_12728367 ]
stack commented on HBASE-1618:
------------------------------
Committed branch and trunk.
> Investigate further into the MemStoreFlusher StoreFile limit
> ------------------------------------------------------------
>
> Key: HBASE-1618
> URL: https://issues.apache.org/jira/browse/HBASE-1618
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1618-v1.patch, HBASE-1618-v2-19branch.patch, HBASE-1618-v2.patch
>
>
> This seems to cause some weird behavior and not accomplish it's original intent (prevent a region growing to hundreds of storefiles).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1618) Investigate further into the
MemStoreFlusher StoreFile limit
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12727940#action_12727940 ]
Jonathan Gray commented on HBASE-1618:
--------------------------------------
There is definitely a bug in MemStoreFlusher.flushRegion()
Have a fix but still doing some investigation...
> Investigate further into the MemStoreFlusher StoreFile limit
> ------------------------------------------------------------
>
> Key: HBASE-1618
> URL: https://issues.apache.org/jira/browse/HBASE-1618
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
>
> This seems to cause some weird behavior and not accomplish it's original intent (prevent a region growing to hundreds of storefiles).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1618) Investigate further into the
MemStoreFlusher StoreFile limit
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728168#action_12728168 ]
Jonathan Gray commented on HBASE-1618:
--------------------------------------
Agreed, this needs to be fixed. Will make a path for 0.19 branch.
> Investigate further into the MemStoreFlusher StoreFile limit
> ------------------------------------------------------------
>
> Key: HBASE-1618
> URL: https://issues.apache.org/jira/browse/HBASE-1618
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Jonathan Gray
> Assignee: Jonathan Gray
> Fix For: 0.20.0
>
> Attachments: HBASE-1618-v1.patch
>
>
> This seems to cause some weird behavior and not accomplish it's original intent (prevent a region growing to hundreds of storefiles).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.