You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datasketches.apache.org by "notfilippo (via GitHub)" <gi...@apache.org> on 2025/12/31 14:35:06 UTC
[I] Cache generate_serialization_test_data.py's output to speed up CI (datasketches-rust)
notfilippo opened a new issue, #54:
URL: https://github.com/apache/datasketches-rust/issues/54
To speed up the CI times we should [cache the sketch files in CI](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching), keying them by commit hash of the repositories that generated them.
If datasketches-cpp or datasketches-java weren't updated in between CI runs there is no value in re-generating the serialised files so we should leverage the CI caching logic to make it more efficient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [I] Cache generate_serialization_test_data.py's output to speed up CI (datasketches-rust)
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
tisonkun closed issue #54: Cache generate_serialization_test_data.py's output to speed up CI
URL: https://github.com/apache/datasketches-rust/issues/54
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [I] Cache generate_serialization_test_data.py's output to speed up CI (datasketches-rust)
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
tisonkun commented on issue #54:
URL: https://github.com/apache/datasketches-rust/issues/54#issuecomment-5070401449
@Phoenix500526 Go ahead!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [I] Cache generate_serialization_test_data.py's output to speed up CI (datasketches-rust)
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
tisonkun commented on issue #54:
URL: https://github.com/apache/datasketches-rust/issues/54#issuecomment-5066765676
Now we have https://github.com/apache/datasketches-tck and we can change the [`tools/generate_serialization_test_data.py`](https://github.com/apache/datasketches-rust/blob/3668658f3d0a415f4ee157cf769c8db024364603/tools/generate_serialization_test_data.py) script to download snapshots from `datasketches-tck`.
This would reduce the maintenance burden of the generator script for other impls, and speed up CI as download snapshots should be faster than re-generate them every time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [I] Cache generate_serialization_test_data.py's output to speed up CI (datasketches-rust)
Posted by "Phoenix500526 (via GitHub)" <gi...@apache.org>.
Phoenix500526 commented on issue #54:
URL: https://github.com/apache/datasketches-rust/issues/54#issuecomment-5070365601
Lol, me again. I'll do it. 😜
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [I] Cache generate_serialization_test_data.py's output to speed up CI (datasketches-rust)
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
tisonkun commented on issue #54:
URL: https://github.com/apache/datasketches-rust/issues/54#issuecomment-3723983449
You can follow https://lists.apache.org/thread/5ppjzrc151n387n4xfgpx1cxnhcv2sbk where I'm trying to use a shared repo to hold these snapshots so that we can simply download (or `git clone`) the files, and it would significantly reduce CI time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org