You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@impala.apache.org by "Michael Smith (Code Review)" <ge...@cloudera.org> on 2026/10/06 18:18:38 UTC

[Impala-ASF-CR] Use sccache multi-level (disk/S3) config

Michael Smith has uploaded this change for review. ( http://gerrit.cloudera.org:8080/25031


Change subject: Use sccache multi-level (disk/S3) config
......................................................................

Use sccache multi-level (disk/S3) config

Adds sccache - installed in bootstrap_system.sh - configured to use
multi-level tiering with local disk and S3 to speed up CI and fresh
local builds.

Preserves ccache support as a fallback if sccache isn't installed.

Change-Id: Ia58f2d2367ddf143145f826037e50b08ef16dde9
---
M be/CMakeLists.txt
M bin/bootstrap_system.sh
M bin/impala-config.sh
M bin/jenkins/build-all-flag-combinations.sh
M docker/entrypoint.sh
M docker/test-with-docker.py
6 files changed, 104 insertions(+), 47 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/25031/1
-- 
To view, visit http://gerrit.cloudera.org:8080/25031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia58f2d2367ddf143145f826037e50b08ef16dde9
Gerrit-Change-Number: 25031
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] Use sccache multi-level (disk/S3) config

Posted by "Laszlo Gaal (Cloudera) (Code Review)" <ge...@cloudera.org>.

Laszlo Gaal (Cloudera) has posted comments on this change. ( http://gerrit.cloudera.org:8080/25031 )

Change subject: Use sccache multi-level (disk/S3) config
......................................................................


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/25031/1/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/25031/1/bin/impala-config.sh@856
PS1, Line 856: export IMPALA_SCCACHE_S3_BUCKET="${IMPALA_SCCACHE_S3_BUCKET-native-toolchain-us-west-2}"
> This may not be the final location.
Yeah, the sccache content is ephemeral (or at least heavily churning) temporary cache content, for which the exact storage location doesn't matter.
I would not like to mix this content/workload with the static, archival content in the toolchain buckets.
The best would be to use a neutral placeholder name here that does not point to any usable bucket, and allow overriding it in a preamble script, or bin/impala-config-local.sh or similar.
Using the toolchain bucket is also privileged: not all contributors have the powers to write to it, so it is probably a bad example here.


http://gerrit.cloudera.org:8080/#/c/25031/1/bin/jenkins/build-all-flag-combinations.sh
File bin/jenkins/build-all-flag-combinations.sh:

http://gerrit.cloudera.org:8080/#/c/25031/1/bin/jenkins/build-all-flag-combinations.sh@116
PS1, Line 116: env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY
Add AWS_SESSION_TOKEN as well


http://gerrit.cloudera.org:8080/#/c/25031/1/bin/jenkins/build-all-flag-combinations.sh@166
PS1, Line 166: env -u AWS_ACCESS_KEY_ID -u AWS_SECRET_ACCESS_KEY 
... and here too



-- 
To view, visit http://gerrit.cloudera.org:8080/25031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia58f2d2367ddf143145f826037e50b08ef16dde9
Gerrit-Change-Number: 25031
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal (Cloudera) <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 07 Oct 2026 22:41:42 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] Use sccache multi-level (disk/S3) config

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/25031 )

Change subject: Use sccache multi-level (disk/S3) config
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/24359/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/25031
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia58f2d2367ddf143145f826037e50b08ef16dde9
Gerrit-Change-Number: 25031
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Laszlo Gaal (Cloudera) <la...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Oct 2026 20:33:14 +0000
Gerrit-HasComments: No