You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@cloudstack.apache.org by GitBox <gi...@apache.org> on 2022/08/17 15:51:46 UTC

[GitHub] [cloudstack] DaanHoogland opened a new issue, #4370: more fine grained categorisation for tests

DaanHoogland opened a new issue, #4370:
URL: https://github.com/apache/cloudstack/issues/4370

   <!--
   Verify first that your issue/request is not already reported on GitHub.
   Also test if the latest release and master branch are affected too.
   Always add information AFTER of these HTML comments, but no need to delete the comments.
   -->
   as a tester, one would want a more fine grained categorisation than "smoke", "component", "integration" to be able to quickly test a PR against features it touches on.
   
   ##### ISSUE TYPE
   <!-- Pick one below and delete the rest -->
    * Improvement Request
    * Enhancement Request
    * Feature Idea
    * Other : testing aid
   
   ##### COMPONENT NAME
   <!--
   Categorize the issue, e.g. API, VR, VPN, UI, etc.
   -->
   ~~~
   Marvin
   ~~~
   
   ##### CLOUDSTACK VERSION
   <!--
   New line separated list of affected versions, commit ID for issues on master branch.
   -->
   
   ~~~
   
   ~~~
   
   ##### CONFIGURATION
   <!--
   Information about the configuration if relevant, e.g. basic network, advanced networking, etc.  N/A otherwise
   -->
   
   
   ##### OS / ENVIRONMENT
   <!--
   Information about the environment if relevant, N/A otherwise
   -->
   
   
   ##### SUMMARY
   <!-- Explain the problem/feature briefly -->
   when a PR is tested it is usually tested against a set of smoke tests in travis or a private cloud with s.moke tests and than maybe avery feature specific (often manual) test. A better categorisation would help in testing more related code and would allow for bypassing any tests that have no baring on the subject of the change.
   A blueorangutan smoke test run takes anywhere between 8 and 15 hours and fails only once after the run, albeit with multiple reports. It would be helpful to run a smaller set of basic tests. if this set passes than a more high level batch could be choosen for a next step.
   
   ##### STEPS TO REPRODUCE
   <!--
   For bugs, show exactly how to reproduce the problem, using a minimal test-case. Use Screenshots if accurate.
   
   For new features, show how the feature would be used.
   -->
   
   <!-- Paste example playbooks or commands between quotes below -->
   ~~~
   go-test smoke
   go-test accounts-and-roles
   go-test instance-groups
   go-test ...
   ~~~
   
   `go-test` could be any test infra structure like blueorangutan. It is used to illustrate the categorisation, not the test-infrastructure.
   
   <!-- You can also paste gist.github.com links for larger files -->
   
   ##### EXPECTED RESULTS
   <!-- What did you expect to happen when running the steps above? -->
   
   ~~~
   
   ~~~
   
   ##### ACTUAL RESULTS
   <!-- What actually happened? -->
   
   <!-- Paste verbatim command output between quotes below -->
   ~~~
   
   ~~~
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] DaanHoogland commented on issue #4370: more fine grained categorisation for tests

Posted by GitBox <gi...@apache.org>.

DaanHoogland commented on issue #4370:
URL: https://github.com/apache/cloudstack/issues/4370#issuecomment-1218204477

   this is valid as long as we use integration tests and really not yet implemented to satisfaction


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rohityadavcloud closed issue #4370: more fine grained categorisation for tests

Posted by GitBox <gi...@apache.org>.

rohityadavcloud closed issue #4370: more fine grained categorisation for tests
URL: https://github.com/apache/cloudstack/issues/4370


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [cloudstack] rohityadavcloud commented on issue #4370: more fine grained categorisation for tests

Posted by GitBox <gi...@apache.org>.

rohityadavcloud commented on issue #4370:
URL: https://github.com/apache/cloudstack/issues/4370#issuecomment-1217942563

   Closing this - this has been open for 2+yrs; any categorisation effort won't speed up the CI/CD, will create new PRs/issues to track work towards improving CI/CD timings; one idea I want to explore is if it's possible to even run tests in parallel (or certain groups that can run in parallel).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org