You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "dweiss (via GitHub)" <gi...@apache.org> on 2025/12/30 07:36:37 UTC
[I] Builds on github reporting http 429 [lucene]
dweiss opened a new issue, #15541:
URL: https://github.com/apache/lucene/issues/15541
### Description
There have been failures on certain builds on github recently:
https://github.com/apache/lucene/actions/workflows/run-nightly-smoketester.yml
They mostly fail because of request ratio limits when resolving references. This must be something new (?). I am also a bit puzzled at the address these requests get resolved from: repo.maven.apache.org (?).
Something to look into.
```
* What went wrong:
Execution failed for task ':build-infra:compileJava'.
> Could not resolve all files for configuration ':build-infra:compileClasspath'.
> Could not download org.eclipse.jgit-7.5.0.202512021534-r.jar (org.eclipse.jgit:org.eclipse.jgit:7.5.0.202512021534-r)
> Could not get resource 'https://repo.maven.apache.org/maven2/org/eclipse/jgit/org.eclipse.jgit/7.5.0.202512021534-r/org.eclipse.jgit-7.5.0.202512021534-r.jar'.
> Could not GET 'https://repo.maven.apache.org/maven2/org/eclipse/jgit/org.eclipse.jgit/7.5.0.202512021534-r/org.eclipse.jgit-7.5.0.202512021534-r.jar'. Received status code 429 from server: Too Many Requests
> Could not download randomizedtesting-runner-2.8.3.jar (com.carrotsearch.randomizedtesting:randomizedtesting-runner:2.8.3)
> Could not get resource 'https://repo.maven.apache.org/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.8.3/randomizedtesting-runner-2.8.3.jar'.
> Could not GET 'https://repo.maven.apache.org/maven2/com/carrotsearch/randomizedtesting/randomizedtesting-runner/2.8.3/randomizedtesting-runner-2.8.3.jar'. Received status code 429 from server: Too Many Requests
> Could not download zstd-jni-1.5.7-6.jar (com.github.luben:zstd-jni:1.5.7-6)
> Could not get resource 'https://repo.maven.apache.org/maven2/com/github/luben/zstd-jni/1.5.7-6/zstd-jni-1.5.7-6.jar'.
> Could not GET 'https://repo.maven.apache.org/maven2/com/github/luben/zstd-jni/1.5.7-6/zstd-jni-1.5.7-6.jar'. Received status code 429 from server: Too Many Requests
> Could not download software-and-algorithms-1.0.jar (com.github.kevinstern:software-and-algorithms:1.0)
> Could not get resource 'https://repo.maven.apache.org/maven2/com/github/kevinstern/software-and-algorithms/1.0/software-and-algorithms-1.0.jar'.
> Could not GET 'https://repo.maven.apache.org/maven2/com/github/kevinstern/software-and-algorithms/1.0/software-and-algorithms-1.0.jar'. Received status code 429 from server: Too Many Requests
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss closed issue #15541: Builds on github reporting http 429
URL: https://github.com/apache/lucene/issues/15541
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3709258184
I don't think you can fix it by setting up alternative repository location. It seems like an intermittent problem that can occasionally happen between metadata reading and binary artifact resolution - if this happens, gradle won't retry from a different repository.
I believe an alternative would be to force this particular build to run against google's mirror, which hopefully has higher usage limits (or no limits at all).
I'll take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3716048127
I did it slightly differently: use google's mirrors on github but leave maven central everywhere else. Let's see if this helps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3702156023
Ok, https://github.com/apache/lucene/actions/runs/20619145757/job/59217574234 passed. I'll keep an eye on that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3722944368
FYI: https://status.maven.org/
<img width="1812" height="552" alt="Image" src="https://github.com/user-attachments/assets/6227cbf7-3425-4d14-b2d5-8ea23e65d1bc" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3716075223
Interesting, the default google() repository in gradle points to https://dl.google.com/dl/android/maven2/ but it doesn't seem this is in sync with maven central:
```
404 https://dl.google.com/dl/android/maven2/com/ibm/icu/icu4j/78.1/icu4j-78.1.pom
200 https://maven-central.storage-download.googleapis.com/maven2/com/ibm/icu/icu4j/78.1/icu4j-78.1.pom
200 https://repo1.maven.org/maven2/com/ibm/icu/icu4j/78.1/icu4j-78.1.pom
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "prudhvigodithi (via GitHub)" <gi...@apache.org>.
prudhvigodithi commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3711383211
> I believe an alternative would be to force this particular build to run against google's mirror, which hopefully has higher usage limits (or no limits at all).
We can also update the following files to put Google mirror first with Maven Central as fallback. This way, the requests go to Google's mirror 1st (this is the same case as shown in my initial example https://github.com/apache/spark-kubernetes-operator/blob/main/build.gradle#L64-L72).
```
- settings.gradle
- build-tools/build-infra/build.gradle
- build-tools/build-infra-shadow/build.gradle
- LuceneRootConfigurationPlugin.java (covers all Lucene subprojects like lucene:spatial-extras)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3706124721
I believe we already have fine-grained caching in place; this should be handled here:
https://github.com/apache/lucene/blob/main/.github/actions/prepare-for-build/action.yml#L40-L47
The problems I've seen all apply to the "nightly" smoke tester check where we deliberately do not use any caches (to ensure all artifacts and resources are indeed available, have the same checksums, etc.).
https://github.com/apache/lucene/blob/main/.github/workflows/run-nightly-smoketester.yml
This has been working fine for quite some time now (and caught a few regressions). It is only recently that it started failing on dependency resolution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss closed issue #15541: Builds on github reporting http 429
URL: https://github.com/apache/lucene/issues/15541
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3698581750
The only thing that comes to my mind is using an alternative mirror like https://maven-central.storage-download.googleapis.com/maven2/. My guess is github runners have a small set of outbound IPs and those 429 errors will only happen more and more frequently (we do have artifact caching but whenever deps change, it's potentially a problem).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "prudhvigodithi (via GitHub)" <gi...@apache.org>.
prudhvigodithi commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3700259344
Along with `mavenCentral()` we can declare the mirror as if I'm not wrong Gradle resolves dependencies by trying repositories in the order they're declared. Example https://github.com/apache/spark-kubernetes-operator/blob/main/build.gradle#L64-L72. To begin with we can update the following files:
```
build-tools/build-infra-shadow/build.gradle
build-tools/build-infra/build.gradle
settings.gradle
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3702775009
@dweiss I'll try to find some time to look into the caching too.
often times it is configured wrong, gets trashed by PRs, or invalidates entirely if a single dep changes.
I use this pattern for other programming languages (example here: python/uv) at work:
```yaml
# restore exact or inexact cache for pull requests and merge queue. do not save.
- name: Read-only package cache
if: ${{ contains(fromJSON('["pull_request", "merge_group"]'), github.event_name) }}
uses: actions/cache/restore@9255dc7a253b0ccc959486e2bca901246202afeb # v5.0.1
with:
path: ~/.cache/uv
key: pypi-${{ github.job }}-${{ inputs.subdirectory }}-${{ runner.os }}-${{ runner.arch }}-${{ steps.cache-key.outputs.hash }}
restore-keys: pypi-${{ github.job }}-${{ inputs.subdirectory }}-${{ runner.os }}-${{ runner.arch }}-
# save exact cache on the default branch
- name: Read-write package cache
if: ${{ !contains(fromJSON('["pull_request", "merge_group"]'), github.event_name) }}
uses: actions/cache@9255dc7a253b0ccc959486e2bca901246202afeb # v5.0.1
with:
path: ~/.cache/uv
key: pypi-${{ github.job }}-${{ inputs.subdirectory }}-${{ runner.os }}-${{ runner.arch }}-${{ steps.cache-key.outputs.hash }}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "prudhvigodithi (via GitHub)" <gi...@apache.org>.
prudhvigodithi commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3708384797
> Nope, still an issue
Looks like this is same issue but from different module, coming from https://github.com/apache/lucene/issues/15541#issuecomment-3700259344 we should extend this across all the code. This time I see the error is for `lucene:spatial-extras` https://github.com/apache/lucene/tree/main/lucene/spatial-extras.
Seeing the Gradle code updating the https://github.com/apache/lucene/blob/main/build-tools/build-infra/src/main/java/org/apache/lucene/gradle/plugins/misc/LuceneRootConfigurationPlugin.java#L117 should apply to all projects.
Let me take a look at `applyCommonConfiguration()` to see if we can add a alternate maven for across the projects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3708137256
Nope, still an issue -
https://github.com/apache/lucene/actions/runs/20686612348
<img width="2750" height="1058" alt="Image" src="https://github.com/user-attachments/assets/755e5db5-5249-4f92-97ab-6538e6f55fc0" />
[smoke-tester-logs-jdk-25.zip](https://github.com/user-attachments/files/24422246/smoke-tester-logs-jdk-25.zip)
Could be that gradle fetches part of the descriptor (pom file, for example), then the repo for that artifact becomes "sticky" but an attempt to download the rest of it fails. Don't know how to make it more verbose in terms of where it attempts to download artifacts from.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3716085572
Ok, this is google's android repository. It is confusing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "prudhvigodithi (via GitHub)" <gi...@apache.org>.
prudhvigodithi commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3700621701
I have created a draft PR https://github.com/apache/lucene/pull/15542, Let me know if ok to try with this alternate mirror and observe the nightlies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Builds on github reporting http 429 [lucene]
Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #15541:
URL: https://github.com/apache/lucene/issues/15541#issuecomment-3698554286
ok, both `repo1.maven.org` and `repo.maven.apache.org` point to the same content at cloudflare. Those limits must be something new at cloudflare then (or maybe they started throttling github runners?).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org