You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Xuebin Su (Code Review)" <ge...@cloudera.org> on 2024/07/12 08:25:06 UTC
[Impala-ASF-CR] IMPALA-13115: Add query id to message of UNREACHABLE IMPALADS
Xuebin Su has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21581
Change subject: IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
......................................................................
IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
This patch adds the query id to the message of error
UNREACHABLE_IMPALADS so that user can easily figure out the affected
queries.
This patch also adds two empty __init__.py files in common/thrift
so that the error code definition can be used in the tests without
copy.
Testing:
- Added an assertion to the case of killing and restarting a worker
in the custom cluster test to ensure that the error message is in
both the query profile web page and the log got by the client.
Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
---
M be/src/service/impala-server.cc
A common/__init__.py
A common/thrift/__init__.py
M common/thrift/generate_error_codes.py
M tests/custom_cluster/test_process_failures.py
5 files changed, 37 insertions(+), 24 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/21581/1
--
To view, visit http://gerrit.cloudera.org:8080/21581
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
Gerrit-Change-Number: 21581
Gerrit-PatchSet: 1
Gerrit-Owner: Xuebin Su <xs...@cloudera.com>
[Impala-ASF-CR] IMPALA-13115: Add query id to message of UNREACHABLE IMPALADS
Posted by "Xuebin Su (Code Review)" <ge...@cloudera.org>.
Xuebin Su has posted comments on this change. ( http://gerrit.cloudera.org:8080/21581 )
Change subject: IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
......................................................................
Patch Set 3:
(2 comments)
> Patch Set 2:
>
> (2 comments)
>
> This is something we could be more consistent about. Thanks.
Hi Michael! Thanks for reviewing!
http://gerrit.cloudera.org:8080/#/c/21581/2/tests/custom_cluster/test_process_failures.py
File tests/custom_cluster/test_process_failures.py:
http://gerrit.cloudera.org:8080/#/c/21581/2/tests/custom_cluster/test_process_failures.py@156
PS2, Line 156: query_id = handle.get_handle().id
> You can get this with
Thank you! I didn't noticed this module before. I think it might be better to import the `ErrorCode` module than to make `generate_error_codes.py` import-able. So in the new patch set (patch set 3), I
- refactored both `get_error_formatter()` and `test_kill_restart_worker()` to use the `ErrorCode` module, and
- moved `get_error_formatter()` from `generate_error_codes.py` to be a static method of `ImpalaTestSuite`.
Hope that can make the code cleaner.
What do you think? Thanks!
http://gerrit.cloudera.org:8080/#/c/21581/2/tests/custom_cluster/test_process_failures.py@157
PS2, Line 157: error_code = TErrorCode.UNREACHABLE_IMPALADS
> A part of me prefers having the explicit string here, as it makes it easier
Thank you!
--
To view, visit http://gerrit.cloudera.org:8080/21581
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
Gerrit-Change-Number: 21581
Gerrit-PatchSet: 3
Gerrit-Owner: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 15 Jul 2024 07:35:43 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13115: Add query id to message of UNREACHABLE IMPALADS
Posted by "Xuebin Su (Code Review)" <ge...@cloudera.org>.
Xuebin Su has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/21581 )
Change subject: IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
......................................................................
IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
This patch adds the query id to the message of error
UNREACHABLE_IMPALADS so that user can easily figure out the affected
queries.
This patch also adds two empty __init__.py files in common/thrift
so that the error code definition can be used in the tests without
copy.
Testing:
- Added an assertion to the case of killing and restarting a worker
in the custom cluster test to ensure that the error message is in
both the query profile web page and the log got by the client.
Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
---
M be/src/service/impala-server.cc
M bin/rat_exclude_files.txt
A common/__init__.py
A common/thrift/__init__.py
M common/thrift/generate_error_codes.py
M tests/custom_cluster/test_process_failures.py
6 files changed, 44 insertions(+), 24 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/21581/2
--
To view, visit http://gerrit.cloudera.org:8080/21581
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
Gerrit-Change-Number: 21581
Gerrit-PatchSet: 2
Gerrit-Owner: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-13115: Add query id to message of UNREACHABLE IMPALADS
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21581 )
Change subject: IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
......................................................................
Patch Set 1:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/21581/1/common/thrift/generate_error_codes.py
File common/thrift/generate_error_codes.py:
http://gerrit.cloudera.org:8080/#/c/21581/1/common/thrift/generate_error_codes.py@495
PS1, Line 495: import re
flake8: E402 module level import not at top of file
http://gerrit.cloudera.org:8080/#/c/21581/1/common/thrift/generate_error_codes.py@497
PS1, Line 497: def get_error_formatter(code):
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/21581/1/common/thrift/generate_error_codes.py@519
PS1, Line 519: def main():
flake8: E302 expected 2 blank lines, found 1
http://gerrit.cloudera.org:8080/#/c/21581/1/common/thrift/generate_error_codes.py@562
PS1, Line 562: (
flake8: E225 missing whitespace around operator
http://gerrit.cloudera.org:8080/#/c/21581/1/common/thrift/generate_error_codes.py@569
PS1, Line 569: if __name__ == "__main__":
flake8: E305 expected 2 blank lines after class or function definition, found 1
--
To view, visit http://gerrit.cloudera.org:8080/21581
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
Gerrit-Change-Number: 21581
Gerrit-PatchSet: 1
Gerrit-Owner: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Jul 2024 08:26:18 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13115: Add query id to message of UNREACHABLE IMPALADS
Posted by "Xuebin Su (Code Review)" <ge...@cloudera.org>.
Xuebin Su has abandoned this change. ( http://gerrit.cloudera.org:8080/21581 )
Change subject: IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
......................................................................
Abandoned
Hi reviwers, this Change is abandoned as it only handles one kind of error. Please see http://gerrit.cloudera.org:8080/21587 for a fix. Thanks!
--
To view, visit http://gerrit.cloudera.org:8080/21581
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
Gerrit-Change-Number: 21581
Gerrit-PatchSet: 3
Gerrit-Owner: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
[Impala-ASF-CR] IMPALA-13115: Add query id to message of UNREACHABLE IMPALADS
Posted by "Xuebin Su (Code Review)" <ge...@cloudera.org>.
Xuebin Su has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/21581 )
Change subject: IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
......................................................................
IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
This patch adds the query id to the message of error
UNREACHABLE_IMPALADS so that user can easily figure out the affected
queries.
This patch also adds a helper method `get_error_formatter()` to
translate error message in the error code definition to Python-
style format string so that we don't have to do it manually.
Testing:
- Added an assertion to the case of killing and restarting a worker
in the custom cluster test to ensure that the error message is in
both the query profile web page and the log got by the client.
Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
---
M be/src/service/impala-server.cc
M common/thrift/generate_error_codes.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_process_failures.py
4 files changed, 47 insertions(+), 24 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/21581/3
--
To view, visit http://gerrit.cloudera.org:8080/21581
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
Gerrit-Change-Number: 21581
Gerrit-PatchSet: 3
Gerrit-Owner: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
[Impala-ASF-CR] IMPALA-13115: Add query id to message of UNREACHABLE IMPALADS
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21581 )
Change subject: IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
......................................................................
Patch Set 1:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/16499/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/21581
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
Gerrit-Change-Number: 21581
Gerrit-PatchSet: 1
Gerrit-Owner: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Jul 2024 08:50:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13115: Add query id to message of UNREACHABLE IMPALADS
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21581 )
Change subject: IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
......................................................................
Patch Set 2:
(2 comments)
This is something we could be more consistent about. Thanks.
http://gerrit.cloudera.org:8080/#/c/21581/2/tests/custom_cluster/test_process_failures.py
File tests/custom_cluster/test_process_failures.py:
http://gerrit.cloudera.org:8080/#/c/21581/2/tests/custom_cluster/test_process_failures.py@156
PS2, Line 156: UNREACHABLE_IMPALADS = 125
You can get this with
from ErrorCodes.ttypes import TErrorCode
...
TErrorCode.UNREACHABLE_IMPALADS
as seen in test_custom_statestore.py.
http://gerrit.cloudera.org:8080/#/c/21581/2/tests/custom_cluster/test_process_failures.py@157
PS2, Line 157: error_state = get_error_formatter(UNREACHABLE_IMPALADS).format(query_id, "")
A part of me prefers having the explicit string here, as it makes it easier to visually confirm that the format is what we expect. I'm not opposed to this change however, as get_error_formatter at least provides a separate method of constructing the error string so unlikely to make the same kind of mistakes as we might make in C++ or Java formatting.
--
To view, visit http://gerrit.cloudera.org:8080/21581
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
Gerrit-Change-Number: 21581
Gerrit-PatchSet: 2
Gerrit-Owner: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Jul 2024 17:52:52 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-13115: Add query id to message of UNREACHABLE IMPALADS
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21581 )
Change subject: IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/16501/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/21581
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
Gerrit-Change-Number: 21581
Gerrit-PatchSet: 2
Gerrit-Owner: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 12 Jul 2024 10:32:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-13115: Add query id to message of UNREACHABLE IMPALADS
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21581 )
Change subject: IMPALA-13115: Add query id to message of UNREACHABLE_IMPALADS
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/16512/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/21581
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72260687ad395f97972e65ffcb47846e9de75bfb
Gerrit-Change-Number: 21581
Gerrit-PatchSet: 3
Gerrit-Owner: Xuebin Su <xs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Xuebin Su <xs...@cloudera.com>
Gerrit-Comment-Date: Mon, 15 Jul 2024 07:41:28 +0000
Gerrit-HasComments: No