You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "wombatu-kun (via GitHub)" <gi...@apache.org> on 2026/07/31 02:37:31 UTC

[PR] fix(ci): retry the jacoco download when Maven Central returns 429 [hudi]

wombatu-kun opened a new pull request, #19430:
URL: https://github.com/apache/hudi/pull/19430

   ### Describe the issue this Pull Request addresses
   
   `scripts/jacoco/download_jacoco.sh` downloads the JaCoCo CLI with a bare `wget` and no retry. Maven Central answers HTTP 429 (Too Many Requests) when the shared CI egress IP is rate limited, and `wget` treats every 4xx as fatal, so a single 429 fails the job.
   
   Seen on https://github.com/apache/hudi/pull/19426, job https://github.com/apache/hudi/actions/runs/30583128545/job/91008100450: all 550 tests passed and Maven reported `BUILD SUCCESS`, and only then did the `Generate merged coverage report` step die.
   
   ```
   HTTP request sent, awaiting response... 429 Too Many Requests
   2026-07-30 22:31:00 ERROR 429: Too Many Requests.
   unzip:  cannot find or open jacoco-0.8.12.zip, ...
   ls: cannot access 'jacoco-lib/lib/jacococli.jar': No such file or directory
   ##[error]Process completed with exit code 2.
   ```
   
   The script also has no `set -e`, so the failed `wget` did not stop it and the reported failure came from the trailing `ls`, not from the 429. It runs at the end of 17 jobs in `.github/workflows/bot.yml` and 11 in `azure-pipelines-20230430.yml`, so one unlucky request discards up to an hour of green work and trips matrix fail-fast on sibling jobs.
   
   ### Summary and Changelog
   
   - Added `set -euo pipefail` so a failed download aborts at the point of failure with the real error.
   - Wrapped the download in a retry loop with a 15s / 30s / 60s backoff; on exhaustion it prints `Unable to download <url>` to stderr and exits 1.
   - Extracted `JACOCO_VERSION` so the version appears once instead of three times.
   
   The backoff is explicit rather than `wget --waitretry`, which ramps linearly from one second and so retries too fast to clear a rate-limit window.
   
   No code was copied.
   
   ### Impact
   
   CI only. No production code and no user-facing behavior changes. A transient 429 or 5xx no longer fails an otherwise green job; a persistent one still does, now with an accurate error message.
   
   ### Risk Level
   
   low
   
   Confined to a CI helper script, and `bash -n` passes. Verified against real Maven Central (exit 0, `jacococli.jar` extracted) and against a local server returning 429s: two transient 429s recover in 16s, and a permanent 429 exits 1 after 105s, matching the intended 15 + 30 + 60 schedule, with no misleading unzip/ls errors. The previous bare `wget` exits 8 immediately on the same input. `scripts/jacoco/merge_jacoco_exec_files.sh` already got retries in #12921, so this was the last unguarded step in the coverage chain.
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(ci): retry the jacoco download when Maven Central returns 429 [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19430:
URL: https://github.com/apache/hudi/pull/19430#issuecomment-5138657045

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d8a3ab1b9fdc5a4de1ceb6147d9362c98ec43c78",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d8a3ab1b9fdc5a4de1ceb6147d9362c98ec43c78",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d8a3ab1b9fdc5a4de1ceb6147d9362c98ec43c78 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(ci): retry the jacoco download when Maven Central returns 429 [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19430:
URL: https://github.com/apache/hudi/pull/19430#issuecomment-5138779872

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d8a3ab1b9fdc5a4de1ceb6147d9362c98ec43c78",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15874",
       "triggerID" : "d8a3ab1b9fdc5a4de1ceb6147d9362c98ec43c78",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d8a3ab1b9fdc5a4de1ceb6147d9362c98ec43c78 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15874) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(ci): retry the jacoco download when Maven Central returns 429 [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19430:
URL: https://github.com/apache/hudi/pull/19430#issuecomment-5138689206

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/19430?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 75.56%. Comparing base ([`633d142`](https://app.codecov.io/gh/apache/hudi/commit/633d1427f35fd2d4a21c6e02363dc8dd90239237?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`d8a3ab1`](https://app.codecov.io/gh/apache/hudi/commit/d8a3ab1b9fdc5a4de1ceb6147d9362c98ec43c78?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@            Coverage Diff            @@
   ##             master   #19430   +/-   ##
   =========================================
     Coverage     75.56%   75.56%           
   - Complexity    32651    32652    +1     
   =========================================
     Files          2574     2574           
     Lines        142995   142995           
     Branches      17530    17530           
   =========================================
   + Hits         108050   108052    +2     
   + Misses        26908    26907    -1     
   + Partials       8037     8036    -1     
   ```
   
   | [Components](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `82.28% <ø> (+<0.01%)` | :arrow_up: |
   | [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.83% <ø> (ø)` | |
   | [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `84.03% <ø> (ø)` | |
   | [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `68.33% <ø> (ø)` | |
   | [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `71.19% <ø> (ø)` | |
   | [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `15.32% <ø> (ø)` | |
   | [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.49% <ø> (ø)` | |
   | [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.72% <ø> (ø)` | |
   | [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `79.57% <ø> (ø)` | |
   | [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.44% <ø> (ø)` | |
   | [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `64.00% <ø> (ø)` | |
   | [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/19430/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.96% <ø> (ø)` | |
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/19430/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/19430/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.39% <ø> (+0.07%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   [see 1 file with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/19430/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(ci): retry the jacoco download when Maven Central returns 429 [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19430:
URL: https://github.com/apache/hudi/pull/19430#issuecomment-5139228126

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d8a3ab1b9fdc5a4de1ceb6147d9362c98ec43c78",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15874",
       "triggerID" : "d8a3ab1b9fdc5a4de1ceb6147d9362c98ec43c78",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d8a3ab1b9fdc5a4de1ceb6147d9362c98ec43c78 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15874) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(ci): retry the jacoco download when Maven Central returns 429 [hudi]

Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua merged PR #19430:
URL: https://github.com/apache/hudi/pull/19430


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org