You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Andrew Sherman (Code Review)" <ge...@cloudera.org> on 2020/11/10 17:21:00 UTC
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Andrew Sherman has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16702
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
When Impala Shell receives an http error message (that is a message with
http code greater than or equal to 300), it may sleep for a time before
retrying. If the message contains a 'Retry-After' header that has an
integer value, then this will be used as the time for which to sleep.
The implementation is to use a new HttpError exception (similar to that
used in Impyla) which includes more information from the error message
(including the headers) so that catchers of the exception can use the
'Retry-After' header if appropriate.
TESTING:
Hand testing with a proxy that uses the 'Retry-After' header.
Added new tests that use the fault injection framework in
test_hs2_fault_injection.py
Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
---
M shell/ImpalaHttpClient.py
M shell/impala_client.py
M shell/shell_exceptions.py
M tests/custom_cluster/test_hs2_fault_injection.py
4 files changed, 140 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/16702/2
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Nov 2020 01:37:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/7629/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Nov 2020 17:53:02 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
When Impala Shell receives an http error message (that is a message with
http code greater than or equal to 300), it may sleep for a time before
retrying. If the message contains a 'Retry-After' header that has an
integer value, then this will be used as the time for which to sleep.
The implementation is to use a new HttpError exception (similar to that
used in Impyla) which includes more information from the error message
(including the headers) so that catchers of the exception can use the
'Retry-After' header if appropriate.
TESTING:
Hand testing with a proxy that uses the 'Retry-After' header.
Added new tests that use the fault injection framework in
test_hs2_fault_injection.py
Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
---
M shell/ImpalaHttpClient.py
M shell/impala_client.py
M shell/shell_exceptions.py
M tests/custom_cluster/test_hs2_fault_injection.py
4 files changed, 142 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/16702/3
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Abhishek Rawat (Code Review)" <ge...@cloudera.org>.
Abhishek Rawat has posted comments on this change. ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
Patch Set 3: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/16702/3/shell/impala_client.py
File shell/impala_client.py:
http://gerrit.cloudera.org:8080/#/c/16702/3/shell/impala_client.py@986
PS3, Line 986: print('Caught exception {0}, type={1} in {2}. {3}, retry after {4} secs'
For the last try (with no more retries left), we will still have a message "retry after {4} secs" at the end. That could be confusing?
We could probably update the condition in L978 so that we have a non None 'retry_secs' for all but the last try.
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Nov 2020 17:58:16 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Nov 2020 19:43:23 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
Patch Set 5: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Nov 2020 07:08:10 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
Patch Set 5:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6640/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Wed, 11 Nov 2020 01:37:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/7628/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Nov 2020 17:41:49 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/16702/2/shell/ImpalaHttpClient.py
File shell/ImpalaHttpClient.py:
http://gerrit.cloudera.org:8080/#/c/16702/2/shell/ImpalaHttpClient.py@31
PS2, Line 31: from shell_exceptions import RPCException, HttpError
flake8: F401 'shell_exceptions.RPCException' imported but unused
http://gerrit.cloudera.org:8080/#/c/16702/2/shell/shell_exceptions.py
File shell/shell_exceptions.py:
http://gerrit.cloudera.org:8080/#/c/16702/2/shell/shell_exceptions.py@55
PS2, Line 55: class HttpError(Exception):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/16702/2/tests/custom_cluster/test_hs2_fault_injection.py
File tests/custom_cluster/test_hs2_fault_injection.py:
http://gerrit.cloudera.org:8080/#/c/16702/2/tests/custom_cluster/test_hs2_fault_injection.py@194
PS2, Line 194: }
flake8: E501 line too long (92 > 90 characters)
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Nov 2020 17:21:44 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
When Impala Shell receives an http error message (that is a message with
http code greater than or equal to 300), it may sleep for a time before
retrying. If the message contains a 'Retry-After' header that has an
integer value, then this will be used as the time for which to sleep.
The implementation is to use a new HttpError exception (similar to that
used in Impyla) which includes more information from the error message
(including the headers) so that catchers of the exception can use the
'Retry-After' header if appropriate.
TESTING:
Hand testing with a proxy that uses the 'Retry-After' header.
Added new tests that use the fault injection framework in
test_hs2_fault_injection.py
Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
---
M shell/ImpalaHttpClient.py
M shell/impala_client.py
M shell/shell_exceptions.py
M tests/custom_cluster/test_hs2_fault_injection.py
4 files changed, 143 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/16702/4
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/7630/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Tue, 10 Nov 2020 19:06:56 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16702 )
Change subject: IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
......................................................................
IMPALA-10309: Use sleep time from a Retry-After header in Impala Shell
When Impala Shell receives an http error message (that is a message with
http code greater than or equal to 300), it may sleep for a time before
retrying. If the message contains a 'Retry-After' header that has an
integer value, then this will be used as the time for which to sleep.
The implementation is to use a new HttpError exception (similar to that
used in Impyla) which includes more information from the error message
(including the headers) so that catchers of the exception can use the
'Retry-After' header if appropriate.
TESTING:
Hand testing with a proxy that uses the 'Retry-After' header.
Added new tests that use the fault injection framework in
test_hs2_fault_injection.py
Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Reviewed-on: http://gerrit.cloudera.org:8080/16702
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M shell/ImpalaHttpClient.py
M shell/impala_client.py
M shell/shell_exceptions.py
M tests/custom_cluster/test_hs2_fault_injection.py
4 files changed, 143 insertions(+), 12 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/16702
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I2b4226e7723d585d61deb4d1d6777aac901bfd93
Gerrit-Change-Number: 16702
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>