You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by GitBox <gi...@apache.org> on 2022/03/20 12:44:07 UTC

[GitHub] [maven-surefire] SammyVimes opened a new pull request #493: SUREFIRE-2039 Fix skipped test reporting

SammyVimes opened a new pull request #493:
URL: https://github.com/apache/maven-surefire/pull/493


   Following this checklist to help us incorporate your 
   contribution quickly and easily:
   
    - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/browse/SUREFIRE) filed 
          for the change (usually before you start working on it).  Trivial changes like typos do not 
          require a JIRA issue.  Your pull request should address just this issue, without 
          pulling in other changes.
    - [ ] Each commit in the pull request should have a meaningful subject line and body.
    - [ ] Format the pull request title like `[SUREFIRE-XXX] - Fixes bug in ApproximateQuantiles`,
          where you replace `SUREFIRE-XXX` with the appropriate JIRA issue. Best practice
          is to use the JIRA issue title in the pull request title and in the first line of the 
          commit message.
    - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
    - [ ] Run `mvn clean install` to make sure basic checks pass. A more thorough check will 
          be performed on your pull request automatically.
    - [ ] You have run the integration tests successfully (`mvn -Prun-its clean install`).
   
   If your pull request is about ~20 lines of code you don't need to sign an
   [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf) if you are unsure
   please ask on the developers list.
   
   To make clear that you license your contribution under 
   the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
   you have to acknowledge this by using the following check-box.
   
    - [ ] I hereby declare this contribution to be licenced under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0)
   
    - [ ] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [maven-surefire] Tibor17 commented on pull request #493: [SUREFIRE-2039] Fix skipped test classes reporting

Posted by GitBox <gi...@apache.org>.
Tibor17 commented on pull request #493:
URL: https://github.com/apache/maven-surefire/pull/493#issuecomment-1075633525


   Hey @SammyVimes ,
   
   I would rather see your help in the reimplementation of `TestSetRunListener` and `StatelessXmlReporter`.
   The patches don't make sense because they would have short life anyway and it would very consufe the development. These implementations are not in perfect state and adding workarounds would mean that we would not be sure during the refactoring.
   Anyway, I appreciate your efforts and I would like to invite you into the refactoring. Meanwhile pls close this ticket and write integration test which fails in current master and open a new PR with the test. This would really help! Pls see my notices from JIRA:
   
   > It really does not make sense to report any pull requests against these two classes with small workarounds or patches or fixes for three reasons:
   1. they are problematic and they exist for many years
   2. we have changes or prerequisite for (3)
   3. we have to refactor both classes but but...
   but as you have noticed the implementation relies on ordering of classes and methods, and the report is finished in one point of when test set has finished and not when the Provider has finished and thus we have problem with re-run feature, this issue, natural ordering.
   We have this in our roadmap and we have to do one thing, rework these two classes and employ the testRunId and RunMode and not to accept patches because these would kill us definitely. We have to smesh down the current implementation and then verify your requirements are met by the new implementation.
   So this effort you have made is pushing us but unfortunately it won't help, even if you want to help us, but the turn is on our side. So nowadays we should release M5, and then we should rework the implementation and we can do it together with your ITs in a new PR.
   What would definitely help us is to provide us with integration tests which fail now in master, and you can do it in a PR even if it would have a broken CI, that's not a problem, and after we have got these classes reimplemented, we would make an agreement to adopt these ITs and push the entire work to the master as one complete solution.
   Currently accepting the patches into an archaic reporter would be problematic and cannot be accepted, otherwise we as maintainers would blow up the problem to bigger and bigger one, and of course this has to be stopped.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [maven-surefire] Tibor17 edited a comment on pull request #493: [SUREFIRE-2039] Fix skipped test classes reporting

Posted by GitBox <gi...@apache.org>.
Tibor17 edited a comment on pull request #493:
URL: https://github.com/apache/maven-surefire/pull/493#issuecomment-1073361298


   @SammyVimes 
   We fixed some issue regarding skipped tests in SUREFIRE-1556 but I am not sure it is the same.
   Sorry, but I do not trust this JIRA report because i do not know what test framework you use, how you "skip" the test and I am not able to reproduce this problem upon the JIRA description. How can I prove that this is a general problem and not individual problem and then why should I accept a fix which is not the right one. If we accepted all patches like this without a deep understanding and debugging the problem, we would have a serious problem which would be bigger than the reported one in JIRA.
   
   So sometimes, it is better to make a small step and precisely write JIRA ticket.
   Of course if you have debugged the code and helped us with the findings around the root cause, it would help and maybe we would make the fix.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [maven-surefire] SammyVimes edited a comment on pull request #493: [SUREFIRE-2039] Fix skipped test classes reporting

Posted by GitBox <gi...@apache.org>.
SammyVimes edited a comment on pull request #493:
URL: https://github.com/apache/maven-surefire/pull/493#issuecomment-1073576107


   @Tibor17 JIRA report has a reproducer attached to it, it's really small (https://github.com/SammyVimesFiledIssues/SUREFIRE-2039). I fully understand that this patch at this moment is not the right one (it doesn't even have tests) and, of course, I do not expect it to be merged right away. On the contrary, as a fellow Apache Committer, I'm willing to work on this issue iff you deem it worthy


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [maven-surefire] Tibor17 edited a comment on pull request #493: [SUREFIRE-2039] Fix skipped test classes reporting

Posted by GitBox <gi...@apache.org>.
Tibor17 edited a comment on pull request #493:
URL: https://github.com/apache/maven-surefire/pull/493#issuecomment-1075633525


   Hey @SammyVimes ,
   
   I would rather see your help in the reimplementation of `TestSetRunListener` and `StatelessXmlReporter`.
   The patches don't make sense because they would have short life anyway and it would very consufe the development. These implementations are not in perfect state and adding workarounds would mean that we would not be sure during the refactoring.
   Anyway, I appreciate your efforts and I would like to invite you into the refactoring. Meanwhile pls close this ticket and write integration test which fails in current master and open a new PR with the test. This would really help! Pls see my notices from JIRA:
   
   > It really does not make sense to report any pull requests against these two classes with small workarounds or patches or fixes for three reasons:
   > 1. they are problematic and they exist for many years
   > 2. we have changes or prerequisite for (3)
   > 3. we have to refactor both classes but but...
   but as you have noticed the implementation relies on ordering of classes and methods, and the report is finished in one point of when test set has finished and not when the Provider has finished and thus we have problem with re-run feature, this issue, natural ordering.
   We have this in our roadmap and we have to do one thing, rework these two classes and employ the testRunId and RunMode and not to accept patches because these would kill us definitely. We have to smesh down the current implementation and then verify your requirements are met by the new implementation.
   So this effort you have made is pushing us but unfortunately it won't help, even if you want to help us, but the turn is on our side. So nowadays we should release M5, and then we should rework the implementation and we can do it together with your ITs in a new PR.
   What would definitely help us is to provide us with integration tests which fail now in master, and you can do it in a PR even if it would have a broken CI, that's not a problem, and after we have got these classes reimplemented, we would make an agreement to adopt these ITs and push the entire work to the master as one complete solution.
   Currently accepting the patches into an archaic reporter would be problematic and cannot be accepted, otherwise we as maintainers would blow up the problem to bigger and bigger one, and of course this has to be stopped.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [maven-surefire] Tibor17 commented on pull request #493: [SUREFIRE-2039] Fix skipped test classes reporting

Posted by GitBox <gi...@apache.org>.
Tibor17 commented on pull request #493:
URL: https://github.com/apache/maven-surefire/pull/493#issuecomment-1073361298


   @SammyVimes 
   We fixed some issue regarding skipped tests in SUREFIRE-1556 but I am not sure it is the same.
   Sorry, but I do not trust this JIRA report because i do not know what test framework you use, how you "skip" the test and I am not able to reproduce this problem upon the JIRA description. How can I prove that this is a general problem and not individual problem and then why should I accept a fix which is not the right one. If we accepted all patches like this without a deep understanding and debugging the problem, we may have a serious problem which is a bigger than the reported one in the JIRA.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [maven-surefire] SammyVimes commented on pull request #493: [SUREFIRE-2039] Fix skipped test classes reporting

Posted by GitBox <gi...@apache.org>.
SammyVimes commented on pull request #493:
URL: https://github.com/apache/maven-surefire/pull/493#issuecomment-1073576107


   @Tibor17 JIRA report has a reproducer attached to it, it's really small (https://github.com/SammyVimesFiledIssues/SUREFIRE-2039). I fully understand that this patch at this moment it's not the right one (it doesn't even have tests) and, of course, I do not expect it to be merged right away. On the contrary, as a fellow Apache Committer, I'm willing to work on this issue iff you deem it worthy


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [maven-surefire] Tibor17 edited a comment on pull request #493: [SUREFIRE-2039] Fix skipped test classes reporting

Posted by GitBox <gi...@apache.org>.
Tibor17 edited a comment on pull request #493:
URL: https://github.com/apache/maven-surefire/pull/493#issuecomment-1075633525


   Hey @SammyVimes ,
   
   I would rather see your help in the reimplementation of `TestSetRunListener` and `StatelessXmlReporter`.
   The patches don't make sense because they would have short life anyway and it would very consufe the development. These implementations are not in perfect state and adding workarounds would mean that we would not be sure during the refactoring.
   Anyway, I appreciate your efforts and I would like to invite you into the refactoring. Meanwhile pls close this ticket and write integration test which fails in current master and open a new PR with the test. This would really help! Pls see my notices from JIRA:
   
   > It really does not make sense to report any pull requests against these two classes with small workarounds or patches or fixes for three reasons:
   > 1. they are problematic and they exist for many years
   > 2. we have changes or prerequisite for (3)
   > 3. we have to refactor both classes
   but as you may have noticed the implementation relies on ordering of classes and methods, and the report is finished in one point of time when the test set has finished and not when the Provider has finished. Now we have problem with re-run feature, this issue, natural ordering, parallel tests executions.
   We have this refactoring in our roadmap and we have to do one thing, rework these two classes and employ the {{testRunId}} and {{RunMode}} and not to accept patches because these would definitely overload us. We have to smesh down the current implementation and then verify your requirements are met by the new implementation.
   So this effort you have made is pushing us but unfortunately it won't help, even if you want to help us, but the turn is on our side. So nowadays we should release M5, and then we should rework the implementation and we can do it together with your ITs in a new PR.
   What would definitely help us is to provide us with integration tests which fail now in master, and you can do it in a PR even if it would have a broken CI, that's not a problem, and after we have got these classes reimplemented, we would make an agreement to adopt these ITs and push the entire work to the master as one complete solution.
   Currently accepting the patches into an archaic reporter would be problematic and cannot be accepted, otherwise we as maintainers would blow up the problem to bigger and bigger one, and of course this has to be stopped.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@maven.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org