You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by cardil <gi...@git.apache.org> on 2016/04/14 22:19:12 UTC
[GitHub] maven-surefire pull request: Adding support for externally passed ...
GitHub user cardil opened a pull request:
https://github.com/apache/maven-surefire/pull/112
Adding support for externally passed random seed and printing used seed on console
It's good practice to execute tests in random order. That ensures tests are atomic and not depend on each other or some stateful entity. But, if such error occurs, right now, there is no way to reproduce this exact, erroneous execution.
This PR adds the ability to reproduce those errors by adding support for externally passed random seed. If random seed is not passed, it will be generated automatically. Also, random seed will be printed on console. That's enable it to be achieved by CI server for later.
This operation is strongly influenced by Ruby's rspec order command line option: https://www.relishapp.com/rspec/rspec-core/v/2-13/docs/command-line/order-new-in-rspec-core-2-8
Showcase of execution:
[](https://asciinema.org/a/42342)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wavesoftware/maven-surefire feature/random-seed-usage
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/maven-surefire/pull/112.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #112
----
commit 6caf9b2819d4d7ded9944240af440a5833de579c
Author: Krzysztof Suszyński <kr...@wavesoftware.pl>
Date: 2016-04-02T17:59:55Z
Adding Randomizer to RunOrder and DefaultRunOrderCalculator, seed is printed on console output
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire issue #112: Adding support for externally passed random seed ...
Posted by Tibor17 <gi...@git.apache.org>.
Github user Tibor17 commented on the issue:
https://github.com/apache/maven-surefire/pull/112
Sorry @cardil . I have logged in just now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire issue #112: Adding support for externally passed random seed ...
Posted by Tibor17 <gi...@git.apache.org>.
Github user Tibor17 commented on the issue:
https://github.com/apache/maven-surefire/pull/112
@cardil
`-DrunOrder=random:123456` I meant as well.
The enums are only some model representation of real world but it does not mean that we cannot change it.
What we should not do is adding a new parameter, especially if we could reuse existing parameter and altering it's behavior and extendig the format of param value.
We have ambition in Surefire 3 to add parameters but in different way than this approach.
Maybe you was too fast and spent time on this code instead of asking for visions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire issue #112: Adding support for externally passed random seed ...
Posted by cardil <gi...@git.apache.org>.
Github user cardil commented on the issue:
https://github.com/apache/maven-surefire/pull/112
:palm_tree: anyone there?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire issue #112: Adding support for externally passed random seed ...
Posted by cardil <gi...@git.apache.org>.
Github user cardil commented on the issue:
https://github.com/apache/maven-surefire/pull/112
Is there something to fix with this PR or everything is okey now?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Adding support for externally passed ...
Posted by Tibor17 <gi...@git.apache.org>.
Github user Tibor17 commented on the pull request:
https://github.com/apache/maven-surefire/pull/112#issuecomment-210156799
@cardil Would it be possible without adding a new config parameter random-seed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
[GitHub] maven-surefire pull request: Adding support for externally passed ...
Posted by cardil <gi...@git.apache.org>.
Github user cardil commented on the pull request:
https://github.com/apache/maven-surefire/pull/112#issuecomment-210342842
@Tibor17 Yes. Actually, I started development in that way. For example: `-DrunOrder=random:123456`. But, that lead to some dirty code because `RunOrder` is a enum like class, and it's hard to store additional data there. I came up with specific classes to hold randomization and I figure out that it is pointless to make `RunOrder` responsible for two tasks (mode and seed). This is way I decided that additional parameter will be better solution. It's also slightly easier to pass your seed in that way.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [GitHub] maven-surefire pull request: Adding support for
externally passed ...
Posted by Rosa Marozzi <ro...@gmail.com>.
Unregister from the list please thanks :)
El 14/4/2016 10:19 p. m., "cardil" <gi...@git.apache.org> escribió:
GitHub user cardil opened a pull request:
https://github.com/apache/maven-surefire/pull/112
Adding support for externally passed random seed and printing used seed
on console
It's good practice to execute tests in random order. That ensures tests
are atomic and not depend on each other or some stateful entity. But, if
such error occurs, right now, there is no way to reproduce this exact,
erroneous execution.
This PR adds the ability to reproduce those errors by adding support
for externally passed random seed. If random seed is not passed, it will be
generated automatically. Also, random seed will be printed on console.
That's enable it to be achieved by CI server for later.
This operation is strongly influenced by Ruby's rspec order command
line option:
https://www.relishapp.com/rspec/rspec-core/v/2-13/docs/command-line/order-new-in-rspec-core-2-8
Showcase of execution:
[](https://asciinema.org/a/42342)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wavesoftware/maven-surefire
feature/random-seed-usage
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/maven-surefire/pull/112.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #112
----
commit 6caf9b2819d4d7ded9944240af440a5833de579c
Author: Krzysztof Suszyński <kr...@wavesoftware.pl>
Date: 2016-04-02T17:59:55Z
Adding Randomizer to RunOrder and DefaultRunOrderCalculator, seed is
printed on console output
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org