You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2018/12/07 15:51:00 UTC
[jira] [Updated] (IMPALA-5285) Account for missing significant
contributors to query time in exec summary
[ https://issues.apache.org/jira/browse/IMPALA-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong updated IMPALA-5285:
----------------------------------
Labels: resource-management (was: )
> Account for missing significant contributors to query time in exec summary
> --------------------------------------------------------------------------
>
> Key: IMPALA-5285
> URL: https://issues.apache.org/jira/browse/IMPALA-5285
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Mala Chikka Kempanna
> Priority: Major
> Labels: resource-management
>
> Entire time of query begin-end duration is not accounted for in summary(This is excluding client fetch wait timer)
> For eg
> 1) Start end time is
> Start Time: 2017-03-24 09:23:03.131144000
> End Time: 2017-03-24 10:54:57.702228000
> 2) Query exec summary and timelines look like below
> {code}
> Admission result: Admitted immediately
> ExecSummary:
> Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
> ---------------------------------------------------------------------------------------------------------------------
> 04:EXCHANGE 1 17.320us 17.320us 0 1.18B 0 -1.00 B UNPARTITIONED
> 03:AGGREGATE 18 4m35s 5m54s 0 1.18B 5.01 GB 5.99 GB FINALIZE
> 02:EXCHANGE 18 5s530ms 7s483ms 1.18B 1.18B 0 0 HASH(TRADEDATE,SYMB,EVENTID)
> 01:AGGREGATE 18 39s865ms 59s336ms 1.18B 1.18B 170.09 MB 56.95 GB STREAMING
> 00:SCAN HDFS 18 7s721ms 18s678ms 1.18B 1.18B 213.28 MB 264.00 MB cif20.dlereference
> - ComputeScanRangeAssignmentTimer: 835228
> - InactiveTotalTime: 0
> - TotalTime: 0
> Planner Timeline
> Analysis finished: 1768171
> Equivalence classes computed: 1953496
> Single node plan created: 2636132
> Runtime filters computed: 2764237
> Distributed plan created: 3178894
> Lineage info computed: 3297595
> Planning finished: 4567814
> Query Timeline
> Query submitted: 315704
> Planning finished: 8032272
> Submit for admission: 10787376
> Completed admission: 11053424
> Ready to start 37 fragment instances: 12622968
> All 37 fragment instances started: 591275200
> Rows available: 5514496400348
> First row fetched: 5514558703092
> Unregister query: 5514571011444
> ImpalaServer
> - ClientFetchWaitTimer: 68269916
> - InactiveTotalTime: 0
> - RowMaterializationTimer: 6332212
> - TotalTime: 0
> {code}
> I want to call attention to large discrepancy in total time accounted for in summary and timeline to one recorded ad start-end time
> This I believe is happening due to not recording fragment receive time in query profiles, which will remove the confusion when reading such profiles.
> This large time is listed under each fragment completion times in same profile
> {code}
> Averaged Fragment F01
> split sizes: min: 0, max: 0, avg: 0, stddev: 0
> completion times: min:1h31m max:1h31m mean: 1h31m stddev:2s911ms
> execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
> num instances: 18
>
> Averaged Fragment F00
> split sizes: min: 628.43 MB, max: 1.00 GB, avg: 827.75 MB, stddev: 104.93 MB
> completion times: min:35m18s max:1h31m mean: 1h4m stddev:13m10s
> execution rates: min:184.88 KB/sec max:303.77 KB/sec mean:224.86 KB/sec stddev:30.22 KB/sec
> num instances: 18
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org