You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Benoit Sigoure (Created) (JIRA)" <ji...@apache.org> on 2012/03/06 04:41:01 UTC
[jira] [Created] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
PerformanceEvaluation: Report aggregate timings on a single line
----------------------------------------------------------------
Key: HBASE-5527
URL: https://issues.apache.org/jira/browse/HBASE-5527
Project: HBase
Issue Type: Improvement
Components: performance
Affects Versions: 0.92.0
Reporter: Benoit Sigoure
Assignee: Benoit Sigoure
Priority: Minor
Attachments: 0001-PerformanceEvaluation-fixes.patch
When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
{code}
12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
{code}
Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223028#comment-13223028 ]
Lars Hofhansl commented on HBASE-5527:
--------------------------------------
Looks good.
Why use nanoTime here, when it is scaled back milliseconds in the end anyway?
{code}
+ final long startTime = System.nanoTime();
...
+ return (System.nanoTime() - startTime) / 1000000;
{code}
> PerformanceEvaluation: Report aggregate timings on a single line
> ----------------------------------------------------------------
>
> Key: HBASE-5527
> URL: https://issues.apache.org/jira/browse/HBASE-5527
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.92.0
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-PerformanceEvaluation-fixes.patch
>
>
> When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
> {code}
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
> {code}
> Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222976#comment-13222976 ]
Hadoop QA commented on HBASE-5527:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12517190/0001-PerformanceEvaluation-fixes.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1110//console
This message is automatically generated.
> PerformanceEvaluation: Report aggregate timings on a single line
> ----------------------------------------------------------------
>
> Key: HBASE-5527
> URL: https://issues.apache.org/jira/browse/HBASE-5527
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.92.0
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-PerformanceEvaluation-fixes.patch
>
>
> When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
> {code}
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
> {code}
> Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
Posted by "Benoit Sigoure (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Sigoure updated HBASE-5527:
----------------------------------
Attachment: 0001-PerformanceEvaluation-fixes.patch
More code smell fixes and added {{numClientThreads}} to {{TestOptions}} as I'm going to need this for my next patch, which is integrating asynchbase into PE.
> PerformanceEvaluation: Report aggregate timings on a single line
> ----------------------------------------------------------------
>
> Key: HBASE-5527
> URL: https://issues.apache.org/jira/browse/HBASE-5527
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.92.0
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-PerformanceEvaluation-fixes.patch, 0001-PerformanceEvaluation-fixes.patch
>
>
> When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
> {code}
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
> {code}
> Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
Posted by "Benoit Sigoure (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Sigoure updated HBASE-5527:
----------------------------------
Status: Patch Available (was: Open)
> PerformanceEvaluation: Report aggregate timings on a single line
> ----------------------------------------------------------------
>
> Key: HBASE-5527
> URL: https://issues.apache.org/jira/browse/HBASE-5527
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.92.0
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-PerformanceEvaluation-fixes.patch
>
>
> When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
> {code}
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
> {code}
> Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399086#comment-13399086 ]
Zhihong Ted Yu commented on HBASE-5527:
---------------------------------------
+1 on patch.
Using nanoTime to measure duration is nice.
> PerformanceEvaluation: Report aggregate timings on a single line
> ----------------------------------------------------------------
>
> Key: HBASE-5527
> URL: https://issues.apache.org/jira/browse/HBASE-5527
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.92.0
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-PerformanceEvaluation-fixes.patch, 0001-PerformanceEvaluation-fixes.patch
>
>
> When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
> {code}
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
> {code}
> Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223042#comment-13223042 ]
Hadoop QA commented on HBASE-5527:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12517199/0001-PerformanceEvaluation-fixes.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1111//console
This message is automatically generated.
> PerformanceEvaluation: Report aggregate timings on a single line
> ----------------------------------------------------------------
>
> Key: HBASE-5527
> URL: https://issues.apache.org/jira/browse/HBASE-5527
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.92.0
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-PerformanceEvaluation-fixes.patch, 0001-PerformanceEvaluation-fixes.patch
>
>
> When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
> {code}
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
> {code}
> Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
Posted by "Benoit Sigoure (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223040#comment-13223040 ]
Benoit Sigoure commented on HBASE-5527:
---------------------------------------
Lars, I'm using {{nanoTime}} because it's the recommended way to measure time in benchmarks. It uses the most accurate monotonically increasing counter the CPU has. System time can change due to NTP adjusting the clock, leap seconds etc.
> PerformanceEvaluation: Report aggregate timings on a single line
> ----------------------------------------------------------------
>
> Key: HBASE-5527
> URL: https://issues.apache.org/jira/browse/HBASE-5527
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.92.0
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-PerformanceEvaluation-fixes.patch, 0001-PerformanceEvaluation-fixes.patch
>
>
> When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
> {code}
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
> {code}
> Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
Posted by "Benoit Sigoure (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399113#comment-13399113 ]
Benoit Sigoure commented on HBASE-5527:
---------------------------------------
Using {{nanoTime}} isn't only nice, it's also more correct :)
{{currentTimeMillis}} depends on system time and is not monotonic, whereas {{nanoTime}} is almost always implemented with a proper monotic clock (although I think technically this isn't _guaranteed_, but in practice on all reasonable platforms, it's the case).
> PerformanceEvaluation: Report aggregate timings on a single line
> ----------------------------------------------------------------
>
> Key: HBASE-5527
> URL: https://issues.apache.org/jira/browse/HBASE-5527
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.92.0
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-PerformanceEvaluation-fixes.patch, 0001-PerformanceEvaluation-fixes.patch
>
>
> When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
> {code}
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
> {code}
> Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-5527:
----------------------------------
Resolution: Fixed
Fix Version/s: 0.96.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> PerformanceEvaluation: Report aggregate timings on a single line
> ----------------------------------------------------------------
>
> Key: HBASE-5527
> URL: https://issues.apache.org/jira/browse/HBASE-5527
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.92.0
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Fix For: 0.96.0
>
> Attachments: 0001-PerformanceEvaluation-fixes.patch, 0001-PerformanceEvaluation-fixes.patch
>
>
> When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
> {code}
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
> {code}
> Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5527) PerformanceEvaluation: Report
aggregate timings on a single line
Posted by "Benoit Sigoure (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Sigoure updated HBASE-5527:
----------------------------------
Attachment: 0001-PerformanceEvaluation-fixes.patch
> PerformanceEvaluation: Report aggregate timings on a single line
> ----------------------------------------------------------------
>
> Key: HBASE-5527
> URL: https://issues.apache.org/jira/browse/HBASE-5527
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Affects Versions: 0.92.0
> Reporter: Benoit Sigoure
> Assignee: Benoit Sigoure
> Priority: Minor
> Attachments: 0001-PerformanceEvaluation-fixes.patch
>
>
> When running {{PerformanceEvaluation}} with {{--nomapred}} it's hard to locate all the lines saying {{Finished 14 in 292979ms writing 2000000 rows}} in the output. This change adds a couples line to summarize the run at the end, which makes parsing and scripting the output easier:
> {code}
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Summary of timings (ms): [15940, 15776, 15866, 15973, 15682, 15740, 15764, 15830, 15768, 15968, 15921, 15755, 15963, 15818, 15903, 15662]
> 12/03/06 00:43:58 INFO hbase.PerformanceEvaluation: [RandomWriteTest] Min: 15662ms Max: 15973ms Avg: 15833ms
> {code}
> Patch also removes a couple minor code smells.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira