You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Joel Koshy (JIRA)" <ji...@apache.org> on 2012/07/03 23:52:36 UTC
[jira] [Created] (KAFKA-392) Refactor and optimize system tests
Joel Koshy created KAFKA-392:
--------------------------------
Summary: Refactor and optimize system tests
Key: KAFKA-392
URL: https://issues.apache.org/jira/browse/KAFKA-392
Project: Kafka
Issue Type: Improvement
Components: core
Reporter: Joel Koshy
Priority: Minor
Fix For: 0.8
It would be helpful to refactor the system tests to make them as concise and
clear as possible, and get them to complete faster. Obviously this is
non-urgent but should ideally be done prior to adding new system tests if
any.
1 - Several functions (e.g., wait_for_zero_consumer_lag, start_cluster,
etc.) are all util methods that are relevant across system tests. Also,
these util functions can be switched to getopt (preferrably with long
options) which will make the scripts more readable. Anything that is not
super-specific to any test should be moved to the global util script.
2 - Might help to have all test output and logs sent to a directory in tmp
(use mktemp) so everything is contained there - right now some files are
local and some are in tmp. Also, clean up would simply be rm -rf the temp
directory instead of handling multiple locations.
3 - We should get rid of as many sleeps as possible. E.g., with producer
acks I don't think we need to sleep after producer-performance. Likewise
for all other sleeps, let us see if they are really needed and provide
tooling (if necessary) to eliminate/reduce them.
4 - User-variable test parameters should be command-line options (getopt) to
the script.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-392) Refactor and optimize system tests
Posted by "Neha Narkhede (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Neha Narkhede updated KAFKA-392:
--------------------------------
Issue Type: Sub-task (was: Improvement)
Parent: KAFKA-440
> Refactor and optimize system tests
> ----------------------------------
>
> Key: KAFKA-392
> URL: https://issues.apache.org/jira/browse/KAFKA-392
> Project: Kafka
> Issue Type: Sub-task
> Components: core
> Reporter: Joel Koshy
> Priority: Minor
> Labels: replication-testing
> Fix For: 0.8
>
>
> It would be helpful to refactor the system tests to make them as concise and
> clear as possible, and get them to complete faster. Obviously this is
> non-urgent but should ideally be done prior to adding new system tests if
> any.
> 1 - Several functions (e.g., wait_for_zero_consumer_lag, start_cluster,
> etc.) are all util methods that are relevant across system tests. Also,
> these util functions can be switched to getopt (preferrably with long
> options) which will make the scripts more readable. Anything that is not
> super-specific to any test should be moved to the global util script.
> 2 - Might help to have all test output and logs sent to a directory in tmp
> (use mktemp) so everything is contained there - right now some files are
> local and some are in tmp. Also, clean up would simply be rm -rf the temp
> directory instead of handling multiple locations.
> 3 - We should get rid of as many sleeps as possible. E.g., with producer
> acks I don't think we need to sleep after producer-performance. Likewise
> for all other sleeps, let us see if they are really needed and provide
> tooling (if necessary) to eliminate/reduce them.
> 4 - User-variable test parameters should be command-line options (getopt) to
> the script.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (KAFKA-392) Refactor and optimize system tests
Posted by "Joel Koshy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/KAFKA-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joel Koshy updated KAFKA-392:
-----------------------------
Labels: replication-testing (was: )
Probably handled by other jiras in 0.8 but will keep open for now.
> Refactor and optimize system tests
> ----------------------------------
>
> Key: KAFKA-392
> URL: https://issues.apache.org/jira/browse/KAFKA-392
> Project: Kafka
> Issue Type: Improvement
> Components: core
> Reporter: Joel Koshy
> Priority: Minor
> Labels: replication-testing
> Fix For: 0.8
>
>
> It would be helpful to refactor the system tests to make them as concise and
> clear as possible, and get them to complete faster. Obviously this is
> non-urgent but should ideally be done prior to adding new system tests if
> any.
> 1 - Several functions (e.g., wait_for_zero_consumer_lag, start_cluster,
> etc.) are all util methods that are relevant across system tests. Also,
> these util functions can be switched to getopt (preferrably with long
> options) which will make the scripts more readable. Anything that is not
> super-specific to any test should be moved to the global util script.
> 2 - Might help to have all test output and logs sent to a directory in tmp
> (use mktemp) so everything is contained there - right now some files are
> local and some are in tmp. Also, clean up would simply be rm -rf the temp
> directory instead of handling multiple locations.
> 3 - We should get rid of as many sleeps as possible. E.g., with producer
> acks I don't think we need to sleep after producer-performance. Likewise
> for all other sleeps, let us see if they are really needed and provide
> tooling (if necessary) to eliminate/reduce them.
> 4 - User-variable test parameters should be command-line options (getopt) to
> the script.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira