You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "raulcd (via GitHub)" <gi...@apache.org> on 2023/06/07 14:36:19 UTC

[GitHub] [arrow] raulcd opened a new issue, #35964: [Packaging][Release] Reduce disk requirements for linux packaging jobs

raulcd opened a new issue, #35964:
URL: https://github.com/apache/arrow/issues/35964

   ### Describe the enhancement requested
   
   Currently we have to perform some disk cleanup on the default GH runners system in order to build Linux packaging jobs, otherwise we get out of space.
   Example of occurrences in the past:
   * https://github.com/apache/arrow/pull/34349#issuecomment-1445430510
   * https://github.com/apache/arrow/issues/35066
   * https://github.com/apache/arrow/issues/35946
   
   On the last issue we temporary removed several things from `/usr/local/bin`, `/usr/local/share` and `/opt/hostedtoolcache`.
   
   There are some ideas for improvements proposed on this comment https://github.com/apache/arrow/pull/35947#issuecomment-1580718883
   _From the comment_
   
   Short term workarounds:
   1. Use self-hosted runner like arm64
   2. Disable some large components such as S3 and GCP
   
   Long term solution:
   
   1. Build deb packages for large dependencies such as aws-sdk-cpp and google-cloud-cpp separately and use them to build our deb packages
   
   FYI: https://github.com/datafusion-contrib/datafusion-c/ provides deb packages at https://apache.jfrog.io/ui/native/arrow/ubuntu/pool/jammy/main/d/datafusion-c/ . Apache Arrow C++ dependencies can use the same approach.
   
   One more idea: https://github.com/apache/arrow/issues/35961
   
   ### Component(s)
   
   Packaging, Release


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Packaging][Release] Reduce disk requirements for linux packaging jobs [arrow]

Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #35964:
URL: https://github.com/apache/arrow/issues/35964#issuecomment-3921751237

   This issue has been marked as stale because it has had no activity in the past 2 years. Please remove the stale label or comment below, or this issue will be closed in 14 days.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Packaging][Release] Reduce disk requirements for linux packaging jobs [arrow]

Posted by "lriggs (via GitHub)" <gi...@apache.org>.
lriggs commented on issue #35964:
URL: https://github.com/apache/arrow/issues/35964#issuecomment-1745525445

   Are there any updates on work towards a long term solution? I'm running into this problem trying to build arrow jars using the default ubuntu github runner. The problem started recently and I imagine would start to affect other users as well.
   
   Are there any other temporary workaround I could try in the meantime?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Packaging][Release] Reduce disk requirements for linux packaging jobs [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #35964:
URL: https://github.com/apache/arrow/issues/35964#issuecomment-1745584476

   There is not much update yet, as a temporary workaround we do clean some unused binaries and cached packages on the GitHub runner by using the following script: https://github.com/apache/arrow/blob/main/ci/scripts/util_free_space.sh


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Packaging][Release] Reduce disk requirements for linux packaging jobs [arrow]

Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic closed issue #35964: [Packaging][Release] Reduce disk requirements for linux packaging jobs
URL: https://github.com/apache/arrow/issues/35964


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org