You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "xiangfu0 (via GitHub)" <gi...@apache.org> on 2025/12/03 21:53:47 UTC

[PR] Add retry wrapper for Maven steps in workflows [pinot]

xiangfu0 opened a new pull request, #17316:
URL: https://github.com/apache/pinot/pull/17316

   Hope this will improve the CI pass rate for maven transient error and some flaky tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Add retry wrapper for Maven steps in workflows [pinot]

Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 merged PR #17316:
URL: https://github.com/apache/pinot/pull/17316


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Add retry wrapper for Maven steps in workflows [pinot]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #17316:
URL: https://github.com/apache/pinot/pull/17316#issuecomment-3609161606

   ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/17316?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 55.61%. Comparing base ([`8bfb907`](https://app.codecov.io/gh/apache/pinot/commit/8bfb907db9eeb29ea3f494f435aa2fa781378db1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f14e054`](https://app.codecov.io/gh/apache/pinot/commit/f14e054e4197f0bbcd0afdccadfdce1d57ec1f36?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   > :exclamation:  There is a different number of reports uploaded between BASE (8bfb907) and HEAD (f14e054). Click for more details.
   > 
   > <details><summary>HEAD has 12 uploads less than BASE</summary>
   >
   >| Flag | BASE (8bfb907) | HEAD (f14e054) |
   >|------|------|------|
   >|unittests|3|1|
   >|java-11|3|0|
   >|temurin|4|1|
   >|unittests2|2|0|
   >|integration|1|0|
   >|integration2|1|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #17316      +/-   ##
   ============================================
   - Coverage     63.24%   55.61%   -7.64%     
   + Complexity     1424      702     -722     
   ============================================
     Files          3133     2446     -687     
     Lines        186150   137784   -48366     
     Branches      28412    21894    -6518     
   ============================================
   - Hits         117737    76631   -41106     
   + Misses        59351    54730    -4621     
   + Partials       9062     6423    -2639     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/pinot/pull/17316/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [integration](https://app.codecov.io/gh/apache/pinot/pull/17316/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration2](https://app.codecov.io/gh/apache/pinot/pull/17316/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-11](https://app.codecov.io/gh/apache/pinot/pull/17316/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-21](https://app.codecov.io/gh/apache/pinot/pull/17316/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.61% <ø> (+21.73%)` | :arrow_up: |
   | [temurin](https://app.codecov.io/gh/apache/pinot/pull/17316/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.61% <ø> (-7.64%)` | :arrow_down: |
   | [unittests](https://app.codecov.io/gh/apache/pinot/pull/17316/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.61% <ø> (-7.64%)` | :arrow_down: |
   | [unittests1](https://app.codecov.io/gh/apache/pinot/pull/17316/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.61% <ø> (-0.04%)` | :arrow_down: |
   | [unittests2](https://app.codecov.io/gh/apache/pinot/pull/17316/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/17316?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Add retry wrapper for Maven steps in workflows [pinot]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #17316:
URL: https://github.com/apache/pinot/pull/17316#discussion_r2586736526


##########
.github/workflows/pinot_tests.yml:
##########
@@ -235,7 +265,12 @@ jobs:
           verbose: true
       - name: Generate Surefire Report
         if: ${{ !cancelled() && github.ref == 'refs/heads/master' }}
-        run: mvn surefire-report:report-only -P github-actions,codecoverage,no-integration-tests || echo "Surefire report generation failed, but continuing..."
+        uses: nick-fields/retry@v3
+        with:
+          max_attempts: 3
+          retry_wait_seconds: 20
+          timeout_minutes: 120
+          command: mvn surefire-report:report-only -P github-actions,codecoverage,no-integration-tests || echo "Surefire report generation failed, but continuing..."

Review Comment:
   The command includes `|| echo \"Surefire report generation failed, but continuing...\"` which will cause the retry action to consider any Maven failure as a success. This defeats the purpose of adding retry logic. Either remove the `|| echo` fallback to allow retries on failure, or add `continue-on-error: true` to the step configuration if failure should be tolerated without retries.
   ```suggestion
             command: mvn surefire-report:report-only -P github-actions,codecoverage,no-integration-tests
   ```



##########
.github/workflows/pinot_tests.yml:
##########
@@ -100,22 +110,32 @@ jobs:
             ${{ runner.os }}-maven-
       # Build the PR branch's module to get a new JAR to compare
       - name: Build SPI on PR branch
-        run: |
-          for mod in pinot-spi pinot-segment-spi; do
-            mvn clean package -T1C -pl $mod -am -DskipTests -Dproject.build.finalName="$mod"
-          done
+        uses: nick-fields/retry@v3
+        with:
+          max_attempts: 3
+          retry_wait_seconds: 20
+          timeout_minutes: 120
+          command: |
+            for mod in pinot-spi pinot-segment-spi; do
+              mvn clean package -T1C -pl $mod -am -DskipTests -Dproject.build.finalName="$mod"
+            done
       - name: Checkout master into a subfolder
         uses: actions/checkout@v4
         with:
           ref: ${{ github.event.pull_request.base.sha }}
           path: master-head
       # Build the same module on master to produce the baseline JAR
       - name: Build SPI on master
-        working-directory: master-head
-        run: |
-          for mod in pinot-spi pinot-segment-spi; do
-            mvn clean package -T1C -pl $mod -am -DskipTests -Dproject.build.finalName="$mod"
-          done
+        uses: nick-fields/retry@v3
+        with:
+          max_attempts: 3
+          retry_wait_seconds: 20
+          timeout_minutes: 120
+          command: |
+            cd master-head

Review Comment:
   The `working-directory` context has been removed but the command now includes `cd master-head`. This change is correct, but the command will fail if subsequent Maven commands in lines 136-138 don't execute in that directory. Since the `command` field executes as a single shell script, this should work, but verify that the `for` loop and Maven commands execute within the `master-head` directory context.



##########
.github/workflows/pinot_tests.yml:
##########
@@ -366,7 +416,12 @@ jobs:
           verbose: true
       - name: Generate Surefire Report
         if: ${{ !cancelled() && github.ref == 'refs/heads/master' }}
-        run: mvn surefire-report:report-only -P github-actions,codecoverage || echo "Surefire report generation failed, but continuing..."
+        uses: nick-fields/retry@v3
+        with:
+          max_attempts: 3
+          retry_wait_seconds: 20
+          timeout_minutes: 120
+          command: mvn surefire-report:report-only -P github-actions,codecoverage || echo "Surefire report generation failed, but continuing..."

Review Comment:
   Same issue as Comment 2: the `|| echo` fallback prevents the retry mechanism from working. The command will always succeed even if Maven fails, so retries will never trigger. Consider removing the fallback or using `continue-on-error: true` instead.
   ```suggestion
             command: mvn surefire-report:report-only -P github-actions,codecoverage
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Add retry wrapper for Maven steps in workflows [pinot]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #17316:
URL: https://github.com/apache/pinot/pull/17316#discussion_r2586896167


##########
.github/workflows/pinot_tests.yml:
##########
@@ -235,7 +265,13 @@ jobs:
           verbose: true
       - name: Generate Surefire Report
         if: ${{ !cancelled() && github.ref == 'refs/heads/master' }}
-        run: mvn surefire-report:report-only -P github-actions,codecoverage,no-integration-tests || echo "Surefire report generation failed, but continuing..."
+        continue-on-error: true
+        uses: nick-fields/retry@v3
+        with:
+          max_attempts: 3
+          retry_wait_seconds: 20
+          timeout_minutes: 120
+          command: mvn surefire-report:report-only -P github-actions,codecoverage,no-integration-tests

Review Comment:
   Using both `continue-on-error: true` and retry logic together may mask persistent failures. The retry mechanism should handle transient errors, while `continue-on-error: true` will suppress all failures including legitimate ones. If the surefire report generation consistently fails after retries, this configuration will hide that fact. Consider removing `continue-on-error: true` to let the retry mechanism handle transient failures while surfacing persistent issues.



##########
.github/workflows/pinot_tests.yml:
##########
@@ -57,11 +57,16 @@ jobs:
           distribution: 'temurin'
           cache: 'maven'
       - name: Install pinot-dependency-verifier into repo
-        run: |
-          mvn clean install \
-          -pl pinot-dependency-verifier \
-          -am \
-          -DskipTests
+        uses: nick-fields/retry@v3
+        with:
+          max_attempts: 3
+          retry_wait_seconds: 20
+          timeout_minutes: 120

Review Comment:
   The retry configuration values (max_attempts: 3, retry_wait_seconds: 20, timeout_minutes: 120) are duplicated across 13 different steps in this file and across multiple workflow files. Consider extracting these as workflow-level environment variables or constants to make future adjustments easier and prevent inconsistencies.



##########
.github/workflows/pinot_tests.yml:
##########
@@ -366,7 +417,13 @@ jobs:
           verbose: true
       - name: Generate Surefire Report
         if: ${{ !cancelled() && github.ref == 'refs/heads/master' }}
-        run: mvn surefire-report:report-only -P github-actions,codecoverage || echo "Surefire report generation failed, but continuing..."
+        continue-on-error: true
+        uses: nick-fields/retry@v3
+        with:
+          max_attempts: 3
+          retry_wait_seconds: 20
+          timeout_minutes: 120
+          command: mvn surefire-report:report-only -P github-actions,codecoverage

Review Comment:
   Using both `continue-on-error: true` and retry logic together may mask persistent failures. The retry mechanism should handle transient errors, while `continue-on-error: true` will suppress all failures including legitimate ones. If the surefire report generation consistently fails after retries, this configuration will hide that fact. Consider removing `continue-on-error: true` to let the retry mechanism handle transient failures while surfacing persistent issues.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org