You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Kannan Muthukkaruppan (JIRA)" <ji...@apache.org> on 2012/06/15 19:28:42 UTC
[jira] [Created] (HBASE-6217) reduce overhead of maintaing get/next
size metric
Kannan Muthukkaruppan created HBASE-6217:
--------------------------------------------
Summary: reduce overhead of maintaing get/next size metric
Key: HBASE-6217
URL: https://issues.apache.org/jira/browse/HBASE-6217
Project: HBase
Issue Type: Improvement
Reporter: Kannan Muthukkaruppan
Assignee: Kannan Muthukkaruppan
[Forked off this specific issue as a separate JIRA from HBASE-6066].
Reduce overhead of "size metric" maintained in StoreScanner.next().
{code}
if (metric != null) {
HRegion.incrNumericMetric(this.metricNamePrefix + metric,
copyKv.getLength());
}
results.add(copyKv);
{code}
A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6217) reduce overhead of maintaing get/next
size metric
Posted by "M. Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
M. Chen updated HBASE-6217:
---------------------------
Labels: patch (was: )
Status: Patch Available (was: Open)
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Labels: patch
> Attachments: jira-6217.patch
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6217) reduce overhead of maintaing
get/next size metric
Posted by "Kannan Muthukkaruppan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kannan Muthukkaruppan reassigned HBASE-6217:
--------------------------------------------
Assignee: M. Chen (was: Kannan Muthukkaruppan)
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Attachments: StoreScanner.java
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6217) reduce overhead of maintaing
get/next size metric
Posted by "Elliott Clark (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295981#comment-13295981 ]
Elliott Clark commented on HBASE-6217:
--------------------------------------
This should be great, however can you upload a patch for trunk (see http://wiki.apache.org/hadoop/Hbase/HowToCommit on how to create the .patch file) so that we can release jenkins on it ?
Additionally I don't think that you need a null check before adding to cummulativeMetric. The most used case will be that metric is not null so optimizing for that seems reasonable.
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Attachments: StoreScanner.java
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6217) reduce overhead of maintaing get/next
size metric
Posted by "M. Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
M. Chen updated HBASE-6217:
---------------------------
Release Note: Reduce overhead of "size metric" maintained in StoreScanner.next().
Status: Patch Available (was: Open)
Reduce overhead of "size metric" maintained in StoreScanner.next() by accumulating the increments and applying them all at once at the end.
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Labels: patch
> Attachments: jira-6217.patch
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6217) reduce overhead of maintaing
get/next size metric
Posted by "M. Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295908#comment-13295908 ]
M. Chen commented on HBASE-6217:
--------------------------------
Diff uploaded at https://reviews.facebook.net/differential/diff/11895
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Attachments: StoreScanner.java
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6217) reduce overhead of maintaing
get/next size metric
Posted by "M. Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13295912#comment-13295912 ]
M. Chen commented on HBASE-6217:
--------------------------------
latest: https://reviews.facebook.net/differential/diff/11907
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Attachments: StoreScanner.java
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6217) reduce overhead of maintaing
get/next size metric
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296071#comment-13296071 ]
Hadoop QA commented on HBASE-6217:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12532287/jira-6217.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2174//console
This message is automatically generated.
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Labels: patch
> Attachments: jira-6217.patch
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6217) reduce overhead of maintaing get/next
size metric
Posted by "M. Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
M. Chen updated HBASE-6217:
---------------------------
Attachment: jira-6217.patch
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Labels: patch
> Attachments: jira-6217.patch
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6217) reduce overhead of maintaing get/next
size metric
Posted by "M. Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
M. Chen updated HBASE-6217:
---------------------------
Attachment: StoreScanner.java
Use a local variable to keep track of changes to the metrics and apply the changes all at once.
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: Kannan Muthukkaruppan
> Attachments: StoreScanner.java
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6217) reduce overhead of maintaing
get/next size metric
Posted by "Kannan Muthukkaruppan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296105#comment-13296105 ]
Kannan Muthukkaruppan commented on HBASE-6217:
----------------------------------------------
@ Ted: The patch is wrt to 89-fb.
@ Michael: Can you create a revision for your diff? Go to the https://reviews.facebook.net/differential/diff/11907 URL, and then hit "Continue" to create a revision, and fill in the details like reviewers. To the description, add [89-fb] [HBASE-6217] tags, and add JIRA to the CC on the review.
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Labels: patch
> Attachments: jira-6217.patch
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6217) reduce overhead of maintaing get/next
size metric
Posted by "M. Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
M. Chen updated HBASE-6217:
---------------------------
Attachment: (was: StoreScanner.java)
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Labels: patch
> Attachments: jira-6217.patch
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6217) reduce overhead of maintaing get/next
size metric
Posted by "M. Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
M. Chen updated HBASE-6217:
---------------------------
Status: Open (was: Patch Available)
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Labels: patch
> Attachments: jira-6217.patch
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6217) reduce overhead of maintaing
get/next size metric
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13296085#comment-13296085 ]
Zhihong Ted Yu commented on HBASE-6217:
---------------------------------------
@M:
The source file is now under hbase-server/
Please resubmit patch.
> reduce overhead of maintaing get/next size metric
> -------------------------------------------------
>
> Key: HBASE-6217
> URL: https://issues.apache.org/jira/browse/HBASE-6217
> Project: HBase
> Issue Type: Improvement
> Reporter: Kannan Muthukkaruppan
> Assignee: M. Chen
> Labels: patch
> Attachments: jira-6217.patch
>
>
> [Forked off this specific issue as a separate JIRA from HBASE-6066].
> Reduce overhead of "size metric" maintained in StoreScanner.next().
> {code}
> if (metric != null) {
> HRegion.incrNumericMetric(this.metricNamePrefix + metric,
> copyKv.getLength());
> }
> results.add(copyKv);
> {code}
> A single call to next() might fetch a lot of KVs. We can first add up the size of those KVs in a local variable and then in a finally clause increment the metric one shot, rather than updating AtomicLongs for each KV.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira