You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "adoroszlai (via GitHub)" <gi...@apache.org> on 2024/05/09 14:56:00 UTC
[PR] HDDS-10273. Intermittent build failure while downloading nodejs [ozone]
adoroszlai opened a new pull request, #6664:
URL: https://github.com/apache/ozone/pull/6664
## What changes were proposed in this pull request?
Download from `nodejs.org` seems to be unreliable.
This PR fixes intermittent failures in:
- `compile (8, macos-12)`, which previously had to download Node.js for each run, since the binary tarball is platform-specific, and we only cached the one for Linux
- various [checks](https://github.com/apache/ozone/actions/runs/9013830936) in the rare case that `populate-cache` runs into [download problem](https://github.com/apache/ozone/actions/runs/8968724893/job/24628735903#step:5:3429)
Changes:
1. Download both Linux and Mac versions of Node.js. Use `curl`, with retry enabled.
2. Indicate partially built cache by creating a marker file, which is deleted if all downloads succeed.
3. Log cache contents even in case of cache hit, making it easier to check contents.
4. Allow triggering `populate-cache` workflow manually for easier testing.
https://issues.apache.org/jira/browse/HDDS-10273
## How was this patch tested?
Triggered the workflow manually in my fork:
https://github.com/adoroszlai/ozone/actions/workflows/populate-cache.yml
Node.js tarballs from latest run:
https://github.com/adoroszlai/ozone/actions/runs/9018931552/job/24780630992#step:8:300
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10273. Intermittent build failure while downloading nodejs [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #6664:
URL: https://github.com/apache/ozone/pull/6664
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10273. Intermittent build failure while downloading nodejs [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6664:
URL: https://github.com/apache/ozone/pull/6664#issuecomment-2109703079
It turns out `cache/save` cannot overwrite existing cache, so I reverted this. Will post new PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10273. Intermittent build failure while downloading nodejs [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6664:
URL: https://github.com/apache/ozone/pull/6664#issuecomment-2109381887
Thanks @errose28, @smengcl for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10273. Intermittent build failure while downloading nodejs [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #6664:
URL: https://github.com/apache/ozone/pull/6664#issuecomment-2103445071
Overall idea makes sense, but why do we do so much manual cache management? It seems like https://github.com/actions/setup-java and https://github.com/actions/setup-node can handle it for us. Are there extra requirements we have?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10273. Intermittent build failure while downloading nodejs [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6664:
URL: https://github.com/apache/ozone/pull/6664#issuecomment-2104137625
> Overall idea makes sense, but why do we do so much manual cache management? It seems like https://github.com/actions/setup-java and https://github.com/actions/setup-node can handle it for us. Are there extra requirements we have?
1. `setup-java` didn't have this feature when we first started using `cache`. Its simple config seems to correspond to our initial implementation with `cache`, so if this was available at the time, we could have chosen it.
2. `setup-java` does not give control over building a cache vs. only using it. Ozone has many concurrent checks which execute different Maven plugins. Basic checks like `checkstyle` can benefit from using the cache, but shouldn't build it, since they do not download many of the dependencies. So we rely on `cache/save` and `cache/restore`.
3. Building the cache from scratch in a separate workflow:
- can produce a smaller cache, by not having to carry old dependencies forever,
- helps avoid cache expiry, which may be a problem in forks with little activity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org