You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Billy Pearson (JIRA)" <ji...@apache.org> on 2009/06/12 21:43:07 UTC
[jira] Created: (HBASE-1513) Compactions to slow
Compactions to slow
-------------------
Key: HBASE-1513
URL: https://issues.apache.org/jira/browse/HBASE-1513
Project: Hadoop HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.20.0
Reporter: Billy Pearson
Priority: Blocker
Fix For: 0.20.0
know there was some changes to compaction a few days back I just got back
to where I could run imports with the new client api
and noticing that compactions are vary slow I meand 100x+ slower then they
used to be I thought one of the servers compaction thread had died
but after retesting and ruleing that out.
I tested with and without compression and with and with out
blockcache and all are running about the same
I got one small compaction to complete it took 1977 secs to compact 48.56MB
thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1513) Compactions too slow
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-1513:
-------------------------------
Attachment: HBASE-1513-v3.patch
this one should apply correctly
> Compactions too slow
> --------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513-v2.patch, HBASE-1513-v3.patch, HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-1513) Compactions to slow
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson reassigned HBASE-1513:
----------------------------------
Assignee: ryan rawson
> Compactions to slow
> -------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1513) Compactions to slow
Posted by "Erik Holstad (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719155#action_12719155 ]
Erik Holstad commented on HBASE-1513:
-------------------------------------
@Ryan
What does the new minor compaction do and what does it not do? I.e what did you remove?
> Compactions to slow
> -------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1513) Compactions to slow
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719206#action_12719206 ]
Billy Pearson commented on HBASE-1513:
--------------------------------------
second patch will not apply to trunk
{code}
patching file src/java/org/apache/hadoop/hbase/regionserver/Store.java
Hunk #1 FAILED at 829.
1 out of 1 hunk FAILED -- saving rejects to file src/java/org/apache/hadoop/hbase/regionserver/Store.java.rej
{code}
My compaction was with compression so might be much faster without.
> Compactions to slow
> -------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513-v2.patch, HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1513) Compactions too slow
Posted by "Justin Lynn (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Justin Lynn updated HBASE-1513:
-------------------------------
Summary: Compactions too slow (was: Compactions to slow)
> Compactions too slow
> --------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513-v2.patch, HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1513) Compactions to slow
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-1513:
-------------------------------
Attachment: HBASE-1513-v2.patch
the previous patch removed delete tracking, which was bad, and this one adds it back in.
On my cluster I can compact 70mb in 7s.
There may be a follow up bug 'major compaction too slow'.
> Compactions to slow
> -------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513-v2.patch, HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1513) Compactions to slow
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719124#action_12719124 ]
Billy Pearson commented on HBASE-1513:
--------------------------------------
+1 on minor compaction
1mins, 17sec for 103.3MB compaction = 1.34mb/s
What has changed to slow down the old way of compaction all the logic test should be the same or similar?
> Compactions to slow
> -------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1513) Compactions too slow
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719288#action_12719288 ]
Jonathan Gray commented on HBASE-1513:
--------------------------------------
Billy, the codepath is very new for compactions in 0.20, not the same at all. However, it should be able to be faster with the new stuff rather than slower we just need to spend more time optimizing.
Opened HBASE-1521 for further optimizations (marked for 0.20.1 for now), this issue really about "fixing" slowness introduced.
> Compactions too slow
> --------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513-v2.patch, HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1513) Compactions too slow
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719370#action_12719370 ]
stack commented on HBASE-1513:
------------------------------
+1 on v4 (after some back and forth with Ryan on IRC)
> Compactions too slow
> --------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513-v2.patch, HBASE-1513-v3.patch, HBASE-1513-v4.patch, HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1513) Compactions too slow
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-1513:
-------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Compactions too slow
> --------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513-v2.patch, HBASE-1513-v3.patch, HBASE-1513-v4.patch, HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1513) Compactions to slow
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-1513:
-------------------------------
Attachment: HBASE-1513.patch
looks like the logic to do the minor compaction is too expensive during minor compactions.
So I changed the minor compaction to offer an ability to just quickly iterate through a set of files. It's REALLY fast now!
> Compactions to slow
> -------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1513) Compactions too slow
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720523#action_12720523 ]
ryan rawson commented on HBASE-1513:
------------------------------------
im seeing major compact speeds at about 10+ minutes per region. the situation is not as dire as indicated, but we could probably be faster.
> Compactions too slow
> --------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513-v2.patch, HBASE-1513-v3.patch, HBASE-1513-v4.patch, HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1513) Compactions to slow
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719122#action_12719122 ]
Billy Pearson commented on HBASE-1513:
--------------------------------------
testing minor compaction
is major compaction going to be at the speed I seen above?
if so we still going to have major problems when you start talking about 256mb regions
256*1024/25kb/sec = 2.9 hours per region
that means running only major compactions we will only be able to major compact 8 regions a day not good for someone with 100 or so regions per server.
Billy
> Compactions to slow
> -------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1513) Compactions too slow
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-1513:
-------------------------------
Attachment: HBASE-1513-v4.patch
this one has the try/finally
> Compactions too slow
> --------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513-v2.patch, HBASE-1513-v3.patch, HBASE-1513-v4.patch, HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1513) Compactions to slow
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ryan rawson updated HBASE-1513:
-------------------------------
Status: Patch Available (was: Open)
minor compactor previously was too complex and did too much logic. new one kicks ass.
major compactions will be slow however (of course).
> Compactions to slow
> -------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Assignee: ryan rawson
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: HBASE-1513.patch
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1513) Compactions to slow
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718956#action_12718956 ]
stack commented on HBASE-1513:
------------------------------
Yes... something is going on. Thanks for filing issue Billy.
> Compactions to slow
> -------------------
>
> Key: HBASE-1513
> URL: https://issues.apache.org/jira/browse/HBASE-1513
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Reporter: Billy Pearson
> Priority: Blocker
> Fix For: 0.20.0
>
>
> know there was some changes to compaction a few days back I just got back
> to where I could run imports with the new client api
> and noticing that compactions are vary slow I meand 100x+ slower then they
> used to be I thought one of the servers compaction thread had died
> but after retesting and ruleing that out.
> I tested with and without compression and with and with out
> blockcache and all are running about the same
> I got one small compaction to complete it took 1977 secs to compact 48.56MB
> thats about 25kb sec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.