You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2018/03/29 22:31:00 UTC
[jira] [Resolved] (IMPALA-6694) BufferPool appears misaligned in
query profile
[ https://issues.apache.org/jira/browse/IMPALA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-6694.
-----------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.12.0
Impala 3.0
> BufferPool appears misaligned in query profile
> ----------------------------------------------
>
> Key: IMPALA-6694
> URL: https://issues.apache.org/jira/browse/IMPALA-6694
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.0, Impala 2.12.0
> Reporter: Michael Ho
> Assignee: Tim Armstrong
> Priority: Critical
> Fix For: Impala 3.0, Impala 2.12.0
>
> Attachments: bad_profile.txt
>
>
> It appears that the buffer pool statistics of exec node is sometimes misaligned in the profile. For instance, the aggregation node's buffer pool appears after the exchange node below:
> {noformat}
> AGGREGATION_NODE (id=101):(Total: 120.713ms, non-child: 7.815ms, % non-child: 6.47%)
> - BuildTime: 934.375us
> - GetResultsTime: 4.245ms
> - HTResizeTime: 2.644us
> - HashBuckets: 16.38K (16384)
> - LargestPartitionPercent: 7 (7)
> - MaxPartitionLevel: 0 (0)
> - NumRepartitions: 0 (0)
> - PartitionsCreated: 16 (16)
> - PeakMemoryUsage: 1.99 MB (2082944)
> - RowsRepartitioned: 0 (0)
> - RowsReturned: 399 (399)
> - RowsReturnedRate: 3.32 K/sec
> - SpilledPartitions: 0 (0)
> EXCHANGE_NODE (id=100):(Total: 112.897ms, non-child: 112.897ms, % non-child: 100.00%)
> - ConvertRowBatchTime: 69.267us
> - PeakMemoryUsage: 160.00 KB (163840)
> - RowsReturned: 421 (421)
> - RowsReturnedRate: 3.74 K/sec
> Buffer pool:
> - AllocTime: 37.231us
> - CumulativeAllocationBytes: 202.67 KB (207530)
> - CumulativeAllocations: 12 (12)
> - PeakReservation: 160.00 KB (163840)
> - PeakUnpinnedBytes: 0
> - PeakUsedReservation: 160.00 KB (163840)
> - ReadIoBytes: 0
> - ReadIoOps: 0 (0)
> - ReadIoWaitTime: 0.000ns
> - WriteIoBytes: 0
> - WriteIoOps: 0 (0)
> - WriteIoWaitTime: 0.000ns
> RecvrSide:
> - FirstBatchArrivalWaitTime: 64.009ms
> - TotalBytesReceived: 98.23 KB (100588)
> - TotalGetBatchTime: 112.773ms
> - DataArrivalTimer: 112.732ms
> SenderSide:
> - DeserializeRowBatchTime: 261.039us
> - NumBatchesArrived: 6 (6)
> - NumBatchesDeferred: 0 (0)
> - NumBatchesEnqueued: 6 (6)
> - NumBatchesReceived: 6 (6)
> - NumEarlySenders: 0 (0)
> - NumEosReceived: 3 (3)
> Buffer pool:
> - AllocTime: 28.534us
> - CumulativeAllocationBytes: 1.25 MB (1310720)
> - CumulativeAllocations: 20 (20)
> - PeakReservation: 1.94 MB (2031616)
> - PeakUnpinnedBytes: 0
> - PeakUsedReservation: 1.25 MB (1310720)
> - ReadIoBytes: 0
> - ReadIoOps: 0 (0)
> - ReadIoWaitTime: 0.000ns
> - WriteIoBytes: 0
> - WriteIoOps: 0 (0)
> - WriteIoWaitTime: 0.000ns
> AGGREGATION_NODE (id=111):(Total: 7.688ms, non-child: 7.587ms, % non-child: 98.68%)
> - BuildTime: 788.577us
> - GetResultsTime: 4.245ms
> - HTResizeTime: 959.000ns
> - HashBuckets: 16.38K (16384)
> - LargestPartitionPercent: 7 (7)
> - MaxPartitionLevel: 0 (0)
> - NumRepartitions: 0 (0)
> - PartitionsCreated: 16 (16)
> - PeakMemoryUsage: 1.97 MB (2065536)
> - RowsRepartitioned: 0 (0)
> - RowsReturned: 399 (399)
> - RowsReturnedRate: 53.59 K/sec
> - SpilledPartitions: 0 (0)
> EXCHANGE_NODE (id=110):(Total: 101.784us, non-child: 101.784us, % non-child: 100.00%)
> - ConvertRowBatchTime: 56.804us
> - PeakMemoryUsage: 186.67 KB (191146)
> - RowsReturned: 421 (421)
> - RowsReturnedRate: 4.40 M/sec
> Buffer pool:
> - AllocTime: 32.337us
> - CumulativeAllocationBytes: 186.67 KB (191146)
> - CumulativeAllocations: 12 (12)
> - PeakReservation: 186.67 KB (191146)
> - PeakUnpinnedBytes: 0
> - PeakUsedReservation: 186.67 KB (191146)
> - ReadIoBytes: 0
> - ReadIoOps: 0 (0)
> - ReadIoWaitTime: 0.000ns
> - WriteIoBytes: 0
> - WriteIoOps: 0 (0)
> - WriteIoWaitTime: 0.000ns
> RecvrSide:
> - FirstBatchArrivalWaitTime: 0.000ns
> - TotalBytesReceived: 85.93 KB (87989)
> - TotalGetBatchTime: 21.294us
> - DataArrivalTimer: 0.000ns
> SenderSide:
> - DeserializeRowBatchTime: 231.593us
> - NumBatchesArrived: 6 (6)
> - NumBatchesDeferred: 0 (0)
> - NumBatchesEnqueued: 6 (6)
> - NumBatchesReceived: 6 (6)
> - NumEarlySenders: 0 (0)
> - NumEosReceived: 3 (3)
> Buffer pool:
> - AllocTime: 28.224us
> - CumulativeAllocationBytes: 1.25 MB (1310720)
> - CumulativeAllocations: 20 (20)
> - PeakReservation: 1.94 MB (2031616)
> - PeakUnpinnedBytes: 0
> - PeakUsedReservation: 1.25 MB (1310720)
> - ReadIoBytes: 0
> - ReadIoOps: 0 (0)
> - ReadIoWaitTime: 0.000ns
> - WriteIoBytes: 0
> - WriteIoOps: 0 (0)
> - WriteIoWaitTime: 0.000ns
> {noformat}
> cc'ing [~tarmstrong@cloudera.com]], [~mmokhtar]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)