[Impala-ASF-CR] IMPALA-5845: Limit the number of non-fatal errors logging to INFO

["Riza Suminto (Code Review)" <ge...@cloudera.org>]

Hello Quanlong Huang, Kurt Deschler, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18565

to look at the new patch set (#4).

Change subject: IMPALA-5845: Limit the number of non-fatal errors logging to INFO
......................................................................

IMPALA-5845: Limit the number of non-fatal errors logging to INFO

RuntimeState::LogError() does both error aggregation to the coordinator
and logging the error to the log file depending on the vlog_level. This
can flood INFO log if the specified vlog_level is 1 and makes it
difficult to analyze other more significant log lines. This patch limits
the number of errors logged to INFO based on max_error_logs_per_instance
flag (default is 2000). When this number is exceeded, vlog_level=1 will
be downgraded to vlog_level=2.

To allow easy debugging in the future, this flag will be ignored if
users sets query option max_errors < 0, which in that case all errors
targetting vlog_level 1 will be logged.

This patch also fixes a bug where the error count is not increased for
non-general error code that is already in 'error_log_' map.

Testing:
- Add test_logging.py::TestLoggingCore

Change-Id: I924768ec461735c172fbf75d6415033bbdb77f9b
---
M be/src/runtime/query-state.cc
M be/src/runtime/runtime-state.cc
M be/src/runtime/runtime-state.h
A tests/custom_cluster/test_logging.py
4 files changed, 104 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/18565/4
-- 
To view, visit http://gerrit.cloudera.org:8080/18565
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I924768ec461735c172fbf75d6415033bbdb77f9b
Gerrit-Change-Number: 18565
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>