You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2023/01/17 08:02:57 UTC

[GitHub] [pulsar] lhotari opened a new pull request, #19252: [improve][ci] Improve test fail fast: disable for non-PR builds, fail parallel forked test procs

lhotari opened a new pull request, #19252:
URL: https://github.com/apache/pulsar/pull/19252

   ### Motivation
   
   Currently, there's a custom fail fast mode implemented as a TestNG listener in https://github.com/apache/pulsar/blob/master/buildtools/src/main/java/org/apache/pulsar/tests/FailFastNotifier.java. 
   The purpose of this is to speed up pull request builds by failing the build at the first error.
   Surefire's fail fast feature (`-Dsurefire.skipAfterFailureCount=1`) was unusable because of bug https://issues.apache.org/jira/browse/SUREFIRE-1762, when the solution was implemented.
   
   The FailFastNotifier solution doesn't currently work properly since it doesn't stop parallel test processes.
   This PR implements a solution for that gap.
   
   This PR also disables the fail fast mode for all non-PR builds in GitHub Actions so that there will be better coverage for Gradle Enterprise build scans that run for apache/pulsar branches.
   For example, the builds for master branch can be found at https://github.com/apache/pulsar/actions/workflows/pulsar-ci.yaml?query=branch%3Amaster . The summary view of each build contains the links to ge.apache.org Gradle Enterprise instance.
   
   ### Documentation
   
   <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
   
   - [ ] `doc` <!-- Your PR contains doc changes. Please attach the local preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR description, or else your PR might not get merged. -->
   - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
   - [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
   - [ ] `doc-complete` <!-- Docs have been already added -->
   
   ### Matching PR in forked repository
   
   PR in forked repository: <!-- ENTER URL HERE -->
   
   <!--
   After opening this PR, the build in apache/pulsar will fail and instructions will
   be provided for opening a PR in the PR author's forked repository.
   
   apache/pulsar pull requests should be first tested in your own fork since the 
   apache/pulsar CI based on GitHub Actions has constrained resources and quota.
   GitHub Actions provides separate quota for pull requests that are executed in 
   a forked repository.
   
   The tests will be run in the forked repository until all PR review comments have
   been handled, the tests pass and the PR is approved by a reviewer.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari merged pull request #19252: [improve][ci] Improve test fail fast: disable for non-PR builds, fail parallel forked test procs

Posted by GitBox <gi...@apache.org>.
lhotari merged PR #19252:
URL: https://github.com/apache/pulsar/pull/19252


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] codecov-commenter commented on pull request #19252: [improve][ci] Improve test fail fast: disable for non-PR builds, fail parallel forked test procs

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #19252:
URL: https://github.com/apache/pulsar/pull/19252#issuecomment-1385469935

   # [Codecov](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#19252](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cbcfc6e) into [master](https://codecov.io/gh/apache/pulsar/commit/299bd70fdfa023768e94a8ee4347d39337b6cbd4?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (299bd70) will **increase** coverage by `0.81%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pulsar/pull/19252/graphs/tree.svg?width=650&height=150&src=pr&token=acYqCpsK9J&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #19252      +/-   ##
   ============================================
   + Coverage     46.37%   47.19%   +0.81%     
   - Complexity    18221    18824     +603     
   ============================================
     Files          1599     1625      +26     
     Lines        130054   132308    +2254     
     Branches      14317    14562     +245     
   ============================================
   + Hits          60310    62439    +2129     
   + Misses        63630    63557      -73     
   - Partials       6114     6312     +198     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `47.19% <ø> (+0.81%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...etadata/bookkeeper/PulsarLedgerManagerFactory.java](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLW1ldGFkYXRhL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9wdWxzYXIvbWV0YWRhdGEvYm9va2tlZXBlci9QdWxzYXJMZWRnZXJNYW5hZ2VyRmFjdG9yeS5qYXZh) | `27.77% <0.00%> (-33.34%)` | :arrow_down: |
   | [...metadata/bookkeeper/PulsarRegistrationManager.java](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLW1ldGFkYXRhL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9wdWxzYXIvbWV0YWRhdGEvYm9va2tlZXBlci9QdWxzYXJSZWdpc3RyYXRpb25NYW5hZ2VyLmphdmE=) | `31.49% <0.00%> (-22.66%)` | :arrow_down: |
   | [.../apache/pulsar/metadata/impl/ZKSessionWatcher.java](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLW1ldGFkYXRhL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9wdWxzYXIvbWV0YWRhdGEvaW1wbC9aS1Nlc3Npb25XYXRjaGVyLmphdmE=) | `73.91% <0.00%> (-21.74%)` | :arrow_down: |
   | [...istentStreamingDispatcherSingleActiveConsumer.java](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3BlcnNpc3RlbnQvUGVyc2lzdGVudFN0cmVhbWluZ0Rpc3BhdGNoZXJTaW5nbGVBY3RpdmVDb25zdW1lci5qYXZh) | `42.26% <0.00%> (-8.25%)` | :arrow_down: |
   | [...ervice/streamingdispatch/StreamingEntryReader.java](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3N0cmVhbWluZ2Rpc3BhdGNoL1N0cmVhbWluZ0VudHJ5UmVhZGVyLmphdmE=) | `52.04% <0.00%> (-7.61%)` | :arrow_down: |
   | [...ersistentStreamingDispatcherMultipleConsumers.java](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL3BlcnNpc3RlbnQvUGVyc2lzdGVudFN0cmVhbWluZ0Rpc3BhdGNoZXJNdWx0aXBsZUNvbnN1bWVycy5qYXZh) | `41.58% <0.00%> (-3.97%)` | :arrow_down: |
   | [...ookkeeper/mledger/impl/ManagedCursorContainer.java](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-bWFuYWdlZC1sZWRnZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2Jvb2trZWVwZXIvbWxlZGdlci9pbXBsL01hbmFnZWRDdXJzb3JDb250YWluZXIuamF2YQ==) | `88.80% <0.00%> (-3.21%)` | :arrow_down: |
   | [...ta/impl/batching/AbstractBatchedMetadataStore.java](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLW1ldGFkYXRhL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9wdWxzYXIvbWV0YWRhdGEvaW1wbC9iYXRjaGluZy9BYnN0cmFjdEJhdGNoZWRNZXRhZGF0YVN0b3JlLmphdmE=) | `84.21% <0.00%> (-2.64%)` | :arrow_down: |
   | [...g/apache/pulsar/metadata/impl/ZKMetadataStore.java](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLW1ldGFkYXRhL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9wdWxzYXIvbWV0YWRhdGEvaW1wbC9aS01ldGFkYXRhU3RvcmUuamF2YQ==) | `64.91% <0.00%> (-2.63%)` | :arrow_down: |
   | [.../pulsar/broker/service/BrokerServiceException.java](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL0Jyb2tlclNlcnZpY2VFeGNlcHRpb24uamF2YQ==) | `50.90% <0.00%> (-1.82%)` | :arrow_down: |
   | ... and [83 more](https://codecov.io/gh/apache/pulsar/pull/19252?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on a diff in pull request #19252: [improve][ci] Improve test fail fast: disable for non-PR builds, fail parallel forked test procs

Posted by GitBox <gi...@apache.org>.
lhotari commented on code in PR #19252:
URL: https://github.com/apache/pulsar/pull/19252#discussion_r1072223615


##########
build/run_integration_group.sh:
##########
@@ -64,6 +64,20 @@ mvn_run_integration_test() {
       clean_arg="clean"
       shift
   fi
+  local use_fail_fast=1
+  if [[ "$GITHUB_ACTIONS" == "true" && "$GITHUB_EVENT_NAME" != "pull_request" ]]; then
+    use_fail_fast=0
+  fi
+  if [[ "$1" == "--no-fail-fast" ]]; then
+    use_fail_fast=0
+    shift;
+  fi
+  local failfast_args
+  if [ $use_fail_fast -eq 1 ]; then
+    failfast_args="-DtestFailFast=true -DtestFailFastFile=/tmp/test_fail_fast_killswitch.$$.$RANDOM.$(date +%s) --fail-fast"
+  else
+    failfast_args="-DtestFailFast=false --fail-at-end"

Review Comment:
   that's the default mode in maven. I just added it there to be explicit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on pull request #19252: [improve][ci] Improve test fail fast: disable for non-PR builds, fail parallel forked test procs

Posted by GitBox <gi...@apache.org>.
lhotari commented on PR #19252:
URL: https://github.com/apache/pulsar/pull/19252#issuecomment-1385293622

   /pulsarbot rerun-failure-checks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] nicoloboschi commented on a diff in pull request #19252: [improve][ci] Improve test fail fast: disable for non-PR builds, fail parallel forked test procs

Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on code in PR #19252:
URL: https://github.com/apache/pulsar/pull/19252#discussion_r1072214554


##########
build/run_integration_group.sh:
##########
@@ -64,6 +64,20 @@ mvn_run_integration_test() {
       clean_arg="clean"
       shift
   fi
+  local use_fail_fast=1
+  if [[ "$GITHUB_ACTIONS" == "true" && "$GITHUB_EVENT_NAME" != "pull_request" ]]; then
+    use_fail_fast=0
+  fi
+  if [[ "$1" == "--no-fail-fast" ]]; then
+    use_fail_fast=0
+    shift;
+  fi
+  local failfast_args
+  if [ $use_fail_fast -eq 1 ]; then
+    failfast_args="-DtestFailFast=true -DtestFailFastFile=/tmp/test_fail_fast_killswitch.$$.$RANDOM.$(date +%s) --fail-fast"
+  else
+    failfast_args="-DtestFailFast=false --fail-at-end"

Review Comment:
   what is this `--fail-at-end` ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on a diff in pull request #19252: [improve][ci] Improve test fail fast: disable for non-PR builds, fail parallel forked test procs

Posted by GitBox <gi...@apache.org>.
lhotari commented on code in PR #19252:
URL: https://github.com/apache/pulsar/pull/19252#discussion_r1072224895


##########
build/run_integration_group.sh:
##########
@@ -64,6 +64,20 @@ mvn_run_integration_test() {
       clean_arg="clean"
       shift
   fi
+  local use_fail_fast=1
+  if [[ "$GITHUB_ACTIONS" == "true" && "$GITHUB_EVENT_NAME" != "pull_request" ]]; then
+    use_fail_fast=0
+  fi
+  if [[ "$1" == "--no-fail-fast" ]]; then
+    use_fail_fast=0
+    shift;
+  fi
+  local failfast_args
+  if [ $use_fail_fast -eq 1 ]; then
+    failfast_args="-DtestFailFast=true -DtestFailFastFile=/tmp/test_fail_fast_killswitch.$$.$RANDOM.$(date +%s) --fail-fast"
+  else
+    failfast_args="-DtestFailFast=false --fail-at-end"

Review Comment:
   from `mvn --help`
   ```
    -fae,--fail-at-end                     Only fail the build afterwards;
                                           allow all non-impacted builds to
                                           continue
    -ff,--fail-fast                        Stop at first failure in
                                           reactorized builds
    -fn,--fail-never                       NEVER fail the build, regardless
                                           of project result
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org