You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Lars Volker (Code Review)" <ge...@cloudera.org> on 2018/06/05 17:20:35 UTC
[Impala-ASF-CR] Guard S3 access check with variable
Lars Volker has uploaded this change for review. ( http://gerrit.cloudera.org:8080/10607
Change subject: Guard S3 access check with variable
......................................................................
Guard S3 access check with variable
We've recently seen automated build failures where multiple inclusions
of impala-config.sh led to the S3 access check being performed multiple
times. This would trigger rate limiting on S3's side, which then caused
the jobs to fail.
This change adds a guard to only perform the check once, even when
including impala-config.sh multiple times.
To test this I included impala-config.sh multiple times on my local
shell and observed that the check was only performed once.
Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
---
M bin/impala-config.sh
1 file changed, 9 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/10607/1
--
To view, visit http://gerrit.cloudera.org:8080/10607
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
Gerrit-Change-Number: 10607
Gerrit-PatchSet: 1
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
[Impala-ASF-CR] Guard S3 access check with variable
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10607 )
Change subject: Guard S3 access check with variable
......................................................................
Patch Set 1: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/10607/1/bin/impala-config.sh
File bin/impala-config.sh:
http://gerrit.cloudera.org:8080/#/c/10607/1/bin/impala-config.sh@401
PS1, Line 401: if ${IMPALA_HOME}/bin/check-s3-access.sh; then
Maybe add a brief comment here to explain the motivation (the commit message did a good job)
--
To view, visit http://gerrit.cloudera.org:8080/10607
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
Gerrit-Change-Number: 10607
Gerrit-PatchSet: 1
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 Jun 2018 17:26:31 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Guard S3 access check with variable
Posted by "Lars Volker (Code Review)" <ge...@cloudera.org>.
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/10607 )
Change subject: Guard S3 access check with variable
......................................................................
Patch Set 2: Code-Review+2
(1 comment)
Thanks for the review. I addressed the comment in PS2.
Carrying Tim's +2.
http://gerrit.cloudera.org:8080/#/c/10607/1/bin/impala-config.sh
File bin/impala-config.sh:
http://gerrit.cloudera.org:8080/#/c/10607/1/bin/impala-config.sh@401
PS1, Line 401: # on AWS and multiple inclusions of S3 can exceed the limit, causing the check to fail.
> Maybe add a brief comment here to explain the motivation (the commit messag
Done
--
To view, visit http://gerrit.cloudera.org:8080/10607
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
Gerrit-Change-Number: 10607
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 Jun 2018 18:21:46 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] Guard S3 access check with variable
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10607 )
Change subject: Guard S3 access check with variable
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/10607
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
Gerrit-Change-Number: 10607
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 Jun 2018 21:47:01 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] Guard S3 access check with variable
Posted by "Lars Volker (Code Review)" <ge...@cloudera.org>.
Hello Tim Armstrong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/10607
to look at the new patch set (#2).
Change subject: Guard S3 access check with variable
......................................................................
Guard S3 access check with variable
We've recently seen automated build failures where multiple inclusions
of impala-config.sh led to the S3 access check being performed multiple
times. This would trigger rate limiting on S3's side, which then caused
the jobs to fail.
This change adds a guard to only perform the check once, even when
including impala-config.sh multiple times.
To test this I included impala-config.sh multiple times on my local
shell and observed that the check was only performed once.
Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
---
M bin/impala-config.sh
1 file changed, 11 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/10607/2
--
To view, visit http://gerrit.cloudera.org:8080/10607
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
Gerrit-Change-Number: 10607
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] Guard S3 access check with variable
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10607 )
Change subject: Guard S3 access check with variable
......................................................................
Guard S3 access check with variable
We've recently seen automated build failures where multiple inclusions
of impala-config.sh led to the S3 access check being performed multiple
times. This would trigger rate limiting on S3's side, which then caused
the jobs to fail.
This change adds a guard to only perform the check once, even when
including impala-config.sh multiple times.
To test this I included impala-config.sh multiple times on my local
shell and observed that the check was only performed once.
Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
Reviewed-on: http://gerrit.cloudera.org:8080/10607
Reviewed-by: Lars Volker <lv...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M bin/impala-config.sh
1 file changed, 11 insertions(+), 4 deletions(-)
Approvals:
Lars Volker: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/10607
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
Gerrit-Change-Number: 10607
Gerrit-PatchSet: 3
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] Guard S3 access check with variable
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/10607 )
Change subject: Guard S3 access check with variable
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2596/
--
To view, visit http://gerrit.cloudera.org:8080/10607
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I1df16b2d037afafb7748f52b64732f680883f16c
Gerrit-Change-Number: 10607
Gerrit-PatchSet: 2
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 05 Jun 2018 18:21:55 +0000
Gerrit-HasComments: No