You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/08/23 21:00:00 UTC
[jira] [Commented] (IMPALA-7433) Reduce volume of -v=1 logs on
executors
[ https://issues.apache.org/jira/browse/IMPALA-7433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590804#comment-16590804 ]
ASF subversion and git services commented on IMPALA-7433:
---------------------------------------------------------
Commit df1865856a7a9988f946c38deeb39cbb3fff80da in impala's branch refs/heads/master from [~tarmstrong@cloudera.com]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=df18658 ]
IMPALA-7433: reduce logging on executors
Moved logs to -v=2 for reasons described in the JIRA. Added
more details to some existing log messages or new
less-frequent log messages so that useful information is not
removed.
Sample logging for an executor after the change:
I0813 12:10:50.249850 31250 impala-internal-service.cc:49] ExecQueryFInstances(): query_id=fd4ae28bc993236e:27343be100000000 coord=tarmstrong-box:22000 #instances=2
I0813 12:10:50.250722 31256 query-state.cc:477] Executing instance. instance_id=fd4ae28bc993236e:27343be100000006 fragment_idx=1 per_fragment_instance_idx=2 coord_state_idx=1 #in-flight=1
I0813 12:10:50.250804 31259 query-state.cc:477] Executing instance. instance_id=fd4ae28bc993236e:27343be100000003 fragment_idx=2 per_fragment_instance_idx=2 coord_state_idx=1 #in-flight=2
I0813 12:10:50.374167 31259 query-state.cc:485] Instance completed. instance_id=fd4ae28bc993236e:27343be100000003 #in-flight=1 status=OK
I0813 12:10:50.375370 31269 krpc-data-stream-mgr.cc:294] DeregisterRecvr(): fragment_instance_id=fd4ae28bc993236e:27343be100000006, node=3
I0813 12:10:50.417552 31256 query-state.cc:485] Instance completed. instance_id=fd4ae28bc993236e:27343be100000006 #in-flight=0 status=OK
I0813 12:10:50.418007 31256 query-exec-mgr.cc:179] ReleaseQueryState(): deleted query_id=fd4ae28bc993236e:27343be100000000
Change-Id: I6c1db44acc6def2b05a4fd032c63716e08cdf5ff
Reviewed-on: http://gerrit.cloudera.org:8080/11202
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Reduce volume of -v=1 logs on executors
> ---------------------------------------
>
> Key: IMPALA-7433
> URL: https://issues.apache.org/jira/browse/IMPALA-7433
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.1.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Fix For: Impala 3.1.0
>
>
> We're currently logging a lot of things per query on executors that is redundant or unnecessary, e.g.:
> * ExecQueryFInstances() and StartQueryFInstances() are both logged but are redundant
> * Some random low-level details are logged, e.g. "Max row batch queue size for scan node '0' in fragment instance '940cdcd50664a51:368ab6690000001e': 180"
> I think we should cut back the logging at -v=1, which is the default log level most deployments run at:
> * Do log key finstance lifecycle events that are useful to understand what finstances are running and if/when failures occurred
> * Don't log things redundantly
> * Don't log lower-level events that occur in the normal flow of query execution (e.g. memory limits, queue lengths). These things have limited diagnostic value and should be in the profile, if anywhere
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org