You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "ttttttz (Code Review)" <ge...@cloudera.org> on 2023/07/16 07:46:18 UTC
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
ttttttz has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20201
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
IMPALA-12278: Adding query option max_error_message_length
Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
---
M be/src/runtime/runtime-state.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/util/error-util-internal.h
M be/src/util/error-util.cc
M be/src/util/error-util.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M tests/custom_cluster/test_logging.py
9 files changed, 61 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/20201/1
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 1
Gerrit-Owner: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20201
to look at the new patch set (#7).
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
IMPALA-12278: Adding query option max_error_message_length
RuntimeState::LogError() does both error aggregation to the
coordinator and logging the error to the log file depending on
the vlog_level. If too many error logs are sent to the coordinator,
it is likely that the coordinator's memory usage will exceed the
limit. This patch limits the length of error message based on
max_error_message_length query option(default is 1024).
Testing:
- Add a test in test_logging.py::TestLoggingCore
Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
---
M be/src/runtime/runtime-state.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/util/error-util-internal.h
M be/src/util/error-util.cc
M be/src/util/error-util.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M tests/custom_cluster/test_logging.py
9 files changed, 68 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/20201/7
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 7
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 8:
(8 comments)
http://gerrit.cloudera.org:8080/#/c/20201/8/be/src/runtime/runtime-state.cc
File be/src/runtime/runtime-state.cc:
http://gerrit.cloudera.org:8080/#/c/20201/8/be/src/runtime/runtime-state.cc@208
PS8, Line 208: message.msg()
I thik we should truncate this as well. We can use the new query option or add a startup flag similar to max_error_logs_per_instance.
http://gerrit.cloudera.org:8080/#/c/20201/8/be/src/runtime/runtime-state.cc@225
PS8, Line 225: error content beyond limit
nit: "error message exceeds the limit"
http://gerrit.cloudera.org:8080/#/c/20201/8/be/src/util/error-util.cc
File be/src/util/error-util.cc:
http://gerrit.cloudera.org:8080/#/c/20201/8/be/src/util/error-util.cc@235
PS8, Line 235: DCHECK_OK(status);
What about 'details_' ? Should we also trancate the total length of strings in it?
http://gerrit.cloudera.org:8080/#/c/20201/8/be/src/util/error-util.cc@261
PS8, Line 261: ErrorMsg GetTruncatedErrorMessage(const ErrorMsg& e, int max_error_message_length) {
Passing the result as a pointer parameter can save some string copies
void GetTruncatedErrorMessage(const ErrorMsg& e, int max_error_message_length, ErrorMsg* result)
http://gerrit.cloudera.org:8080/#/c/20201/8/common/thrift/ImpalaService.thrift
File common/thrift/ImpalaService.thrift:
http://gerrit.cloudera.org:8080/#/c/20201/8/common/thrift/ImpalaService.thrift@837
PS8, Line 837: ;
nit: Use "."
BTW, is it the limit on the aggregated error message? Or is it a limit on each error reported by each executor?
http://gerrit.cloudera.org:8080/#/c/20201/8/tests/custom_cluster/test_logging.py
File tests/custom_cluster/test_logging.py:
http://gerrit.cloudera.org:8080/#/c/20201/8/tests/custom_cluster/test_logging.py@65
PS8, Line 65: def test_max_error_message_length(self):
This doesn't need customized startup flags. We can add it in a non custom-cluster test, e.g. tests/query_test/test_errorlog.py.
http://gerrit.cloudera.org:8080/#/c/20201/8/tests/custom_cluster/test_logging.py@76
PS8, Line 76: assert re.search("UDF WARNING: Failed to parse json at position 0 since: "
nit: can be
"UDF WARNING: Failed to parse json at position 0 since: Invalid value.. Json string:\njust_\n" in profile
http://gerrit.cloudera.org:8080/#/c/20201/8/tests/custom_cluster/test_logging.py@81
PS8, Line 81: assert not re.search("UDF WARNING:", profile)
nit: can be
assert "UDF WARNING:" not in profile
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 8
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Mon, 14 Aug 2023 02:03:56 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13558/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 4
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Mon, 17 Jul 2023 06:26:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20201
to look at the new patch set (#2).
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
IMPALA-12278: Adding query option max_error_message_length
RuntimeState::LogError() does both error aggregation to the
coordinator and logging the error to the log file depending on
the vlog_level. If too many error logs are sent to the coordinator,
it is likely that the coordinator's memory usage will exceed the
limit. This patch limits the length of error message based on
max_error_message_length query option.
Testing:
- Add a test in test_logging.py::TestLoggingCore
Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
---
M be/src/runtime/runtime-state.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/util/error-util-internal.h
M be/src/util/error-util.cc
M be/src/util/error-util.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M tests/custom_cluster/test_logging.py
9 files changed, 61 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/20201/2
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 2
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20201
to look at the new patch set (#8).
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
IMPALA-12278: Adding query option max_error_message_length
RuntimeState::LogError() does both error aggregation to the
coordinator and logging the error to the log file depending on
the vlog_level. If too many error logs are sent to the coordinator,
it is likely that the coordinator's memory usage will exceed the
limit. This patch limits the length of error message based on
max_error_message_length query option(default is 1024).
Testing:
- Add a test in test_logging.py::TestLoggingCore
Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
---
M be/src/runtime/runtime-state.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/util/error-util-internal.h
M be/src/util/error-util.cc
M be/src/util/error-util.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M tests/custom_cluster/test_logging.py
9 files changed, 68 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/20201/8
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 8
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13556/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 1
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sun, 16 Jul 2023 08:09:31 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20201
to look at the new patch set (#4).
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
IMPALA-12278: Adding query option max_error_message_length
RuntimeState::LogError() does both error aggregation to the
coordinator and logging the error to the log file depending on
the vlog_level. If too many error logs are sent to the coordinator,
it is likely that the coordinator's memory usage will exceed the
limit. This patch limits the length of error message based on
max_error_message_length query option.
Testing:
- Add a test in test_logging.py::TestLoggingCore
Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
---
M be/src/runtime/runtime-state.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/util/error-util-internal.h
M be/src/util/error-util.cc
M be/src/util/error-util.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M tests/custom_cluster/test_logging.py
9 files changed, 69 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/20201/4
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 4
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 7:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13561/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 7
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Mon, 17 Jul 2023 06:50:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
ttttttz has removed Quanlong Huang from this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Removed reviewer Quanlong Huang.
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 8
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20201/4/tests/custom_cluster/test_logging.py
File tests/custom_cluster/test_logging.py:
http://gerrit.cloudera.org:8080/#/c/20201/4/tests/custom_cluster/test_logging.py@82
PS4, Line 82:
flake8: W391 blank line at end of file
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 4
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Mon, 17 Jul 2023 06:03:53 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20201/7/tests/custom_cluster/test_logging.py
File tests/custom_cluster/test_logging.py:
http://gerrit.cloudera.org:8080/#/c/20201/7/tests/custom_cluster/test_logging.py@81
PS7, Line 81:
flake8: W292 no newline at end of file
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 7
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Mon, 17 Jul 2023 06:28:57 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 8:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13562/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 8
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Mon, 17 Jul 2023 06:57:56 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20201
to look at the new patch set (#3).
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
IMPALA-12278: Adding query option max_error_message_length
RuntimeState::LogError() does both error aggregation to the
coordinator and logging the error to the log file depending on
the vlog_level. If too many error logs are sent to the coordinator,
it is likely that the coordinator's memory usage will exceed the
limit. This patch limits the length of error message based on
max_error_message_length query option.
Testing:
- Add a test in test_logging.py::TestLoggingCore
Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
---
M be/src/runtime/runtime-state.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/util/error-util-internal.h
M be/src/util/error-util.cc
M be/src/util/error-util.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M tests/custom_cluster/test_logging.py
9 files changed, 67 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/20201/3
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 3
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20201
to look at the new patch set (#5).
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
IMPALA-12278: Adding query option max_error_message_length
RuntimeState::LogError() does both error aggregation to the
coordinator and logging the error to the log file depending on
the vlog_level. If too many error logs are sent to the coordinator,
it is likely that the coordinator's memory usage will exceed the
limit. This patch limits the length of error message based on
max_error_message_length query option.
Testing:
- Add a test in test_logging.py::TestLoggingCore
Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
---
M be/src/runtime/runtime-state.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/util/error-util-internal.h
M be/src/util/error-util.cc
M be/src/util/error-util.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M tests/custom_cluster/test_logging.py
9 files changed, 68 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/20201/5
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 5
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13557/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 3
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Mon, 17 Jul 2023 06:12:43 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 3:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/20201/3/tests/custom_cluster/test_logging.py
File tests/custom_cluster/test_logging.py:
http://gerrit.cloudera.org:8080/#/c/20201/3/tests/custom_cluster/test_logging.py@71
PS3, Line 71: .
flake8: E501 line too long (94 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/20201/3/tests/custom_cluster/test_logging.py@74
PS3, Line 74: :
flake8: E501 line too long (112 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/20201/3/tests/custom_cluster/test_logging.py@75
PS3, Line 75: .
flake8: E501 line too long (94 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/20201/3/tests/custom_cluster/test_logging.py@78
PS3, Line 78: :
flake8: E501 line too long (111 > 90 characters)
http://gerrit.cloudera.org:8080/#/c/20201/3/tests/custom_cluster/test_logging.py@80
PS3, Line 80:
flake8: W391 blank line at end of file
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 3
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Mon, 17 Jul 2023 05:50:45 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13559/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 5
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Mon, 17 Jul 2023 06:26:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "ttttttz (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20201
to look at the new patch set (#6).
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
IMPALA-12278: Adding query option max_error_message_length
RuntimeState::LogError() does both error aggregation to the
coordinator and logging the error to the log file depending on
the vlog_level. If too many error logs are sent to the coordinator,
it is likely that the coordinator's memory usage will exceed the
limit. This patch limits the length of error message based on
max_error_message_length query option(default is 1024).
Testing:
- Add a test in test_logging.py::TestLoggingCore
Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
---
M be/src/runtime/runtime-state.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/util/error-util-internal.h
M be/src/util/error-util.cc
M be/src/util/error-util.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M tests/custom_cluster/test_logging.py
9 files changed, 68 insertions(+), 5 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/01/20201/6
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 6
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
[Impala-ASF-CR] IMPALA-12278: Adding query option max error message length
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20201 )
Change subject: IMPALA-12278: Adding query option max_error_message_length
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13560/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20201
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22bd1fe8b15ab83fe5f6fd4d5455c5f68e247ba6
Gerrit-Change-Number: 20201
Gerrit-PatchSet: 6
Gerrit-Owner: ttttttz <24...@qq.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: ttttttz <24...@qq.com>
Gerrit-Comment-Date: Mon, 17 Jul 2023 06:32:58 +0000
Gerrit-HasComments: No