[jira] [Commented] (IMPALA-2343) Capture operator timing information covering open/close & first/last batch close

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708084#comment-16708084 ] 

ASF subversion and git services commented on IMPALA-2343:
---------------------------------------------------------

Commit c23c852bd0d019df670109fa7420e107a757e6dd in impala's branch refs/heads/master from [~tarmstrong@cloudera.com]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=c23c852 ]

IMPALA-2343: Add lifecycle timeline to plan nodes

Track the time when various significant events in the lifecycle
of an ExecNode occur the timeline of fragment execution. The events
tracked are: 'Open Started', 'Open Finished', 'First Batch Fetched',
'First Batch Returned', 'Last Batch Returned', 'Closed'.

This uses the existing EventSequence infrastructure so that time will
correspond to the fragment instance lifecycle timelines. It's
implemented mostly using scoped objects that add the events when
entering or exiting Open() and GetNext().

These times are not set inside subplans because it would be mostly
redundant with the counters in the containing subplan node.

Also fix MaterializeTupleTime for Kudu scan nodes to match the timing of
other scanners, where it measures the time spent in the scanner threads,
not the time in the main fragment execution thread.

Testing:
Added a basic test to verify that the event sequence is present.

Manually inspected some profiles.

Verified the Kudu timer by running a query with scanner parallelism and
checking that MaterializeTupleTime was > the wallclock time.

  set num_nodes=1;
  select * from tpch_kudu.lineitem
  where lower(l_comment) = 'foo';

Perf:
Ran TPC-H 10 locally. There was no significant perf change.

Ran TPC-H Nested locally. There was no significant perf change.

Change-Id: I15341bdb15022bad9814882689ce5cb2939f4653
Reviewed-on: http://gerrit.cloudera.org:8080/11992
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Capture operator timing information covering open/close & first/last batch close
> --------------------------------------------------------------------------------
>
>                 Key: IMPALA-2343
>                 URL: https://issues.apache.org/jira/browse/IMPALA-2343
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>    Affects Versions: Impala 2.2.4
>            Reporter: Mostafa Mokhtar
>            Assignee: Tim Armstrong
>            Priority: Minor
>              Labels: performance, supportability
>         Attachments: 0001-Add-start-and-end-time-to-a-bunch-of-plan-nodes.patch
>
>
> Currently Impala query profile doesn't cover operator level timeline, which makes it difficult to understand the query timeline and fragment dependencies.
> Such information will allow us to provide query swim lanes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org