You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@buildstream.apache.org by GitBox <gi...@apache.org> on 2021/10/17 00:41:07 UTC
[GitHub] [buildstream] nanonyme opened a new issue #1527: Support compression of logs through user configuration
nanonyme opened a new issue #1527:
URL: https://github.com/apache/buildstream/issues/1527
BuildStream element full logs can be rather large, especially when it comes to CI. It would be nice to have an option that would compress them by default so they would be smaller artifacts from CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] gtristan commented on issue #1527: Support compression of logs through user configuration
Posted by GitBox <gi...@apache.org>.
gtristan commented on issue #1527:
URL: https://github.com/apache/buildstream/issues/1527#issuecomment-945078490
I'm a bit torn here..
* One one hand, I think that compressing the logs in the log directory is not mission specific. The project benefits from avoiding handling use cases that can be handled separately
* On the other hand, log files are actually cached in the artifact (e.g. `bst artifact log` displays them, even if you `rm -rf` your log directory)
Instead, I'm more inclined to suggest that:
* We stop saving logs to the log directory, and remove the concept of the *logs* directory completely
* This would require a bit of a change in the UI, which refers to log paths, which instead could be delivered through `bst artifact log` on demand (and is already suggested on the command prompt in the case of interactive failed builds)
* We compress logs stored in the artifact unconditionally
* If you want to capture the logs for a specific build graph after a successful build, you can use `bst artifact log` to capture them and compress them in whatever format you want (for the purpose of say, gitlab pipeline artifacts)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] gtristan commented on issue #1527: Support compression of logs through user configuration
Posted by GitBox <gi...@apache.org>.
gtristan commented on issue #1527:
URL: https://github.com/apache/buildstream/issues/1527#issuecomment-945364422
> To be fair, failure logs are at least as important if not more so than logs for successful build.
Indeed, and artifacts for failed builds are only optionally cached.
probably we should ensure that we cache the failed build artifacts unconditionally, and only make pushing failed build artifacts optional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] gtristan commented on issue #1527: Support compression of logs through user configuration
Posted by GitBox <gi...@apache.org>.
gtristan commented on issue #1527:
URL: https://github.com/apache/buildstream/issues/1527#issuecomment-945065844
Is there a reason why such a feature should be baked into the tool itself, instead of simply compressing (and possibly tarring up) the logs directory after a build ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] nanonyme commented on issue #1527: Support compression of logs through user configuration
Posted by GitBox <gi...@apache.org>.
nanonyme commented on issue #1527:
URL: https://github.com/apache/buildstream/issues/1527#issuecomment-945081095
To be fair, failure logs are at least as important if not more so than logs for successful build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] nanonyme commented on issue #1527: Support compression of logs through user configuration
Posted by GitBox <gi...@apache.org>.
nanonyme commented on issue #1527:
URL: https://github.com/apache/buildstream/issues/1527#issuecomment-945071046
That is precisely what I'm planning to do but it seems as if this might benefit other workloads too. Eg Mesa build log seemed to be around 10 megabytes. When you multiply this with a lot of elements, you get large space usage. There's many toolings like xzless which you can use to read through compressed logs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] nanonyme commented on issue #1527: Support compression of logs through user configuration
Posted by GitBox <gi...@apache.org>.
nanonyme commented on issue #1527:
URL: https://github.com/apache/buildstream/issues/1527#issuecomment-1022593893
Maybe it's better just doing this by consumer after bst.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] nanonyme closed issue #1527: Support compression of logs through user configuration
Posted by GitBox <gi...@apache.org>.
nanonyme closed issue #1527:
URL: https://github.com/apache/buildstream/issues/1527
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org