You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "John Vines (Created) (JIRA)" <ji...@apache.org> on 2012/02/01 15:36:59 UTC

[jira] [Created] (ACCUMULO-359) Need to create automatic performance tests

Need to create automatic performance tests
------------------------------------------

                 Key: ACCUMULO-359
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-359
             Project: Accumulo
          Issue Type: New Feature
          Components: test
            Reporter: John Vines
            Assignee: John Vines


We have many tests for validating functionality of the system, but we lack any standard tests for any form of benchmarking. We should create appropriate performance tests in order to test performance differences between versions, impacts from code changes, and for just generally charting our growth as a project.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (ACCUMULO-359) Need to create automatic performance tests

Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/ACCUMULO-359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197905#comment-13197905 ] 

Keith Turner commented on ACCUMULO-359:
---------------------------------------

The continuous ingest test track batch write performance and the performance of little lookups over long periods of time.  The test suite also has a report generation tool that plots this info.  

The test suite needs to also track the performance of the verification.  This is a full scan of the table done via map reduce.  It would be nice to plot te read rates from when this is run in the report.

The primary purpose of the test is to detect data loss, performance is secondary.  So the test has not been stable w/ respect to measuring performance.  For example in 1.4 CRCs were added to the test.  It computes these on ingest and verifies them during the M/R verification.  So changes like this make comparisons of 1.3 CI and 1.4 CI for performance useful but not exact.  Would need to run the same version of continuous ingest against both 1.3 and 1.4, not the one that ships with it.   
                
> Need to create automatic performance tests
> ------------------------------------------
>
>                 Key: ACCUMULO-359
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-359
>             Project: Accumulo
>          Issue Type: New Feature
>          Components: test
>            Reporter: John Vines
>
> We have many tests for validating functionality of the system, but we lack any standard tests for any form of benchmarking. We should create appropriate performance tests in order to test performance differences between versions, impacts from code changes, and for just generally charting our growth as a project.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (ACCUMULO-359) Need to create automatic performance tests

Posted by "David Medinets (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/ACCUMULO-359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198494#comment-13198494 ] 

David Medinets commented on ACCUMULO-359:
-----------------------------------------

How would one track the cluster size, memory availability, machine power, and other tasks running during the testing? It seems like any given job has a lot of factors that affect performance.
                
> Need to create automatic performance tests
> ------------------------------------------
>
>                 Key: ACCUMULO-359
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-359
>             Project: Accumulo
>          Issue Type: New Feature
>          Components: test
>            Reporter: John Vines
>
> We have many tests for validating functionality of the system, but we lack any standard tests for any form of benchmarking. We should create appropriate performance tests in order to test performance differences between versions, impacts from code changes, and for just generally charting our growth as a project.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (ACCUMULO-359) Need to create automatic performance tests

Posted by "John Vines (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/ACCUMULO-359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

John Vines updated ACCUMULO-359:
--------------------------------

    Assignee:     (was: John Vines)
    
> Need to create automatic performance tests
> ------------------------------------------
>
>                 Key: ACCUMULO-359
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-359
>             Project: Accumulo
>          Issue Type: New Feature
>          Components: test
>            Reporter: John Vines
>
> We have many tests for validating functionality of the system, but we lack any standard tests for any form of benchmarking. We should create appropriate performance tests in order to test performance differences between versions, impacts from code changes, and for just generally charting our growth as a project.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira