You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Huang Ning (JIRA)" <ji...@apache.org> on 2019/03/19 08:24:00 UTC
[jira] [Created] (IMPALA-8321) The Impala query is slow.
Huang Ning created IMPALA-8321:
----------------------------------
Summary: The Impala query is slow.
Key: IMPALA-8321
URL: https://issues.apache.org/jira/browse/IMPALA-8321
Project: IMPALA
Issue Type: Improvement
Components: Frontend
Affects Versions: 2.2.0
Environment: linux suse 11.3
Reporter: Huang Ning
Impala query is very slow. What is the performance bottleneck? Can someone please help me
Plan:
----------------
Estimated Per-Host Requirements: Memory=4.73GB VCores=3
WARNING: The following tables are missing relevant table and/or column statistics.
nethouse.oc_whitelist_cfg, ps.detail_cdr_s1mme_17969
F05:PLAN FRAGMENT [UNPARTITIONED]
13:EXCHANGE [UNPARTITIONED]
limit: 5000
hosts=219 per-host-mem=unavailable
tuple-ids=0,2N,5N row-size=379B cardinality=5000
F00:PLAN FRAGMENT [RANDOM]
DATASTREAM SINK [FRAGMENT=F05, EXCHANGE=13, UNPARTITIONED]
06:HASH JOIN [LEFT OUTER JOIN, BROADCAST]
| hash predicates: (concat(coalesce(CAST(MSISDN AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST('4' AS STRING), ''))) = concat(coalesce(CAST(MSISDN AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), ''))
| other predicates: concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), '')) IS NULL
| limit: 5000
| hosts=219 per-host-mem=2.00GB
| tuple-ids=0,2N,5N row-size=379B cardinality=unavailable
|
|--12:EXCHANGE [BROADCAST]
| hosts=2 per-host-mem=0B
| tuple-ids=5 row-size=32B cardinality=unavailable
|
05:HASH JOIN [LEFT OUTER JOIN, BROADCAST]
| hash predicates: (concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST('4' AS STRING), ''))) = concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), ''))
| other predicates: concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), '')) IS NULL
| hosts=219 per-host-mem=2.00GB
| tuple-ids=0,2N row-size=347B cardinality=unavailable
|
|--09:EXCHANGE [BROADCAST]
| hosts=2 per-host-mem=0B
| tuple-ids=2 row-size=32B cardinality=unavailable
|
00:SCAN HDFS [ps.detail_cdr_s1mme_17969, RANDOM]
partitions=6/56 files=32281 size=93.86GB
predicates: PROC_STARTTIME >= 1552512807, PROC_STARTTIME < 1552593600, PROC_TYPE != 254, MSISDN = '508968813'
table stats: unavailable
columns missing stats: imsi, msisdn, ms_ip, s1ap_cause, rat, proc_type, arp_neg, procedure_id, esm_ue_rej_cause, auth_rej_emm_cause, esm_nw_rej_cause, interfaceid, proc_starttime, identity_cause, mbr_ul_neg, mme_sig_ip, security_mode_cause, proc_starttime_msec, proc_succed_flag, sv, ran_ne_id, sai_cgi_ecgi, proc_endtime, imei, mbr_dl_neg, proc_endtime_msec, rai_tai, refid, sid, probeid, mme_id, qci_neg, working_mode, emm_cause, csfb_ind, ran_ne_sig_ip, pdn_type, apn, protocol_id, esm_nw_trans_type, esm_ue_trans_type, fir_fail_trans_prot, fir_fail_cause, fir_fail_trans, s1ap_cause_type, s1ap_trans_type, emm_trans_type
hosts=219 per-host-mem=752.00MB
tuple-ids=0 row-size=315B cardinality=unavailable
F04:PLAN FRAGMENT [HASH(concat(coalesce(CAST(MSISDN AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), '')),concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), '')))]
DATASTREAM SINK [FRAGMENT=F00, EXCHANGE=12, BROADCAST]
11:AGGREGATE [FINALIZE]
| group by: concat(coalesce(CAST(MSISDN AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), '')), concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), ''))
| hosts=2 per-host-mem=128.00MB
| tuple-ids=5 row-size=32B cardinality=unavailable
|
10:EXCHANGE [HASH(concat(coalesce(CAST(MSISDN AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), '')),concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), '')))]
hosts=2 per-host-mem=0B
tuple-ids=5 row-size=32B cardinality=unavailable
F03:PLAN FRAGMENT [RANDOM]
DATASTREAM SINK [FRAGMENT=F04, EXCHANGE=10, HASH(concat(coalesce(CAST(MSISDN AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), '')),concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), '')))]
04:AGGREGATE
| group by: concat(coalesce(CAST(MSISDN AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), '')), concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), ''))
| hosts=2 per-host-mem=128.00MB
| tuple-ids=5 row-size=32B cardinality=unavailable
|
03:SCAN HDFS [nethouse.oc_whitelist_cfg, RANDOM]
partitions=1/1 files=1 size=703B
table stats: unavailable
column stats: unavailable
hosts=2 per-host-mem=48.00MB
tuple-ids=4 row-size=38B cardinality=unavailable
F02:PLAN FRAGMENT [HASH(concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), '')),concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), '')))]
DATASTREAM SINK [FRAGMENT=F00, EXCHANGE=09, BROADCAST]
08:AGGREGATE [FINALIZE]
| group by: concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), '')), concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), ''))
| hosts=2 per-host-mem=128.00MB
| tuple-ids=2 row-size=32B cardinality=unavailable
|
07:EXCHANGE [HASH(concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), '')),concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), '')))]
hosts=2 per-host-mem=0B
tuple-ids=2 row-size=32B cardinality=unavailable
F01:PLAN FRAGMENT [RANDOM]
DATASTREAM SINK [FRAGMENT=F02, EXCHANGE=07, HASH(concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), '')),concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), '')))]
02:AGGREGATE
| group by: concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('-' AS STRING), ''), coalesce(CAST(SERVICETYPE AS STRING), '')), concat(coalesce(CAST(IMSI AS STRING), ''), coalesce(CAST('_' AS STRING), ''), coalesce(CAST(MSISDN AS STRING), ''))
| hosts=2 per-host-mem=128.00MB
| tuple-ids=2 row-size=32B cardinality=unavailable
|
01:SCAN HDFS [nethouse.oc_whitelist_cfg, RANDOM]
partitions=1/1 files=1 size=703B
table stats: unavailable
column stats: unavailable
hosts=2 per-host-mem=48.00MB
tuple-ids=1 row-size=38B cardinality=unavailable
----------------
Estimated Per-Host Mem: 5083496448
Estimated Per-Host VCores: 3
Tables Missing Stats: nethouse.oc_whitelist_cfg,ps.detail_cdr_s1mme_17969
Request Pool: default-pool
ExecSummary:
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
---------------------------------------------------------------------------------------------------------------------------
13:EXCHANGE 1 7.565ms 7.565ms 816 5.00K 0 -1.00 B UNPARTITIONED
06:HASH JOIN 57 226.188ms 854.628ms 816 -1 29.40 MB 2.00 GB LEFT OUTER JOIN, BROADCAST
|--12:EXCHANGE 57 28.20us 62.365us 2 -1 0 0 BROADCAST
| 11:AGGREGATE 1 196.363ms 196.363ms 2 -1 3.14 MB 128.00 MB FINALIZE
| 10:EXCHANGE 1 19.428us 19.428us 2 -1 0 0 HASH(concat(coalesce(CAST(M...
| 04:AGGREGATE 1 401.535ms 401.535ms 2 -1 3.21 MB 128.00 MB
| 03:SCAN HDFS 1 2s029ms 2s029ms 2 -1 76.00 KB 48.00 MB nethouse.oc_whitelist_cfg
05:HASH JOIN 57 2s897ms 3s466ms 816 -1 20.95 MB 2.00 GB LEFT OUTER JOIN, BROADCAST
|--09:EXCHANGE 57 27.747us 46.394us 2 -1 0 0 BROADCAST
| 08:AGGREGATE 1 200.119ms 200.119ms 2 -1 3.14 MB 128.00 MB FINALIZE
| 07:EXCHANGE 1 24.890us 24.890us 2 -1 0 0 HASH(concat(coalesce(CAST(I...
| 02:AGGREGATE 1 341.394ms 341.394ms 2 -1 3.21 MB 128.00 MB
| 01:SCAN HDFS 1 1s935ms 1s935ms 2 -1 76.00 KB 48.00 MB nethouse.oc_whitelist_cfg
00:SCAN HDFS 57 9m49s 13m21s 816 -1 98.55 MB 752.00 MB ps.detail_cdr_s1mme_17969
Planner Timeline: 106.658ms
- Analysis finished: 52.699ms (52.699ms)
- Equivalence classes computed: 54.425ms (1.725ms)
- Single node plan created: 100.304ms (45.878ms)
- Distributed plan created: 103.693ms (3.389ms)
- Planning finished: 106.658ms (2.964ms)
Query Timeline: 13m27s
- Start execution: 45.864us (45.864us)
- Planning finished: 189.956ms (189.910ms)
- Ready to start remote fragments: 243.549ms (53.592ms)
- Remote fragments started: 1s397ms (1s153ms)
- Rows available: 14s107ms (12s709ms)
- First row fetched: 14s470ms (363.923ms)
- Unregister query: 13m27s (13m13s)
ImpalaServer:
- ClientFetchWaitTimer: 3s450ms
- RowMaterializationTimer: 49.78ms
Execution Profile c14f7b9b17987c60:d5379fb954101693:(Total: 13m24s, non-child: 0ns, % non-child: 0.00%)
Fragment start latencies: count: 61, last: 0.222159ns, min: 0.200340ns, max: 0.292795ns, mean: 0.226639ns, stddev: 0.015480ns
Per Node Peak Memory Usage: hadoopc1h8:28002(74.45 MB) hadoopc1h21:28002(74.02 MB) hadoopc1h66:28002(106.20 MB) hadoopc1h81:28002(85.18 MB) hadoopc1h85:28002(87.55 MB) hadoopc1h23:28002(70.99 MB) hadoopc1h4:28002(82.84 MB) hadoopc1h25:28002(79.28 MB) hadoopc1h22:28002(78.90 MB) hadoopc1h24:28002(72.33 MB) hadoopc1h77:28002(87.46 MB) hadoopc1h73:28002(83.32 MB) hadoopc1h90:28002(72.74 MB) hadoopc1h76:28002(81.06 MB) hadoopc1h142:28002(88.48 MB) hadoopc1h6:28002(78.39 MB) hadoopc1h91:28002(88.59 MB) hadoopc1h26:28002(71.41 MB) hadoopc1h88:28002(76.26 MB) hadoopc1h69:28002(83.41 MB) hadoopc1h16:28002(87.57 MB) hadoopc1h75:28002(81.51 MB) hadoopc1h87:28002(80.06 MB) hadoopc1h31:28002(84.32 MB) hadoopc1h84:28002(83.81 MB) hadoopc1h83:28002(79.82 MB) hadoopc1h68:28002(86.55 MB) hadoopc1h19:28002(84.76 MB) hadoopc1h86:28002(82.18 MB) hadoopc1h65:28002(93.12 MB) hadoopc1h15:28002(87.07 MB) hadoopc1h17:28002(73.77 MB) hadoopc1h79:28002(76.09 MB) hadoopc1h74:28002(86.46 MB) hadoopc1h20:28002(69.40 MB) hadoopc1h27:28002(83.99 MB) hadoopc1h89:28002(85.85 MB) hadoopc1h12:28002(83.48 MB) hadoopc1h5:28002(95.79 MB) hadoopc1h13:28002(73.71 MB) hadoopc1h28:28002(69.96 MB) hadoopc1h9:28002(79.79 MB) hadoopc1h67:28002(88.58 MB) hadoopc1h11:28002(81.24 MB) hadoopc1h140:28002(77.19 MB) hadoopc1h30:28002(81.52 MB) hadoopc1h14:28002(75.93 MB) hadoopc1h80:28002(79.66 MB) hadoopc1h82:28002(90.40 MB) hadoopc1h143:28002(82.98 MB) hadoopc1h71:28002(74.65 MB) hadoopc1h29:28002(72.19 MB) hadoopc1h70:28002(72.44 MB) hadoopc1h141:28002(85.72 MB) hadoopc1h78:28002(84.12 MB) hadoopc1h10:28002(95.96 MB) hadoopc1h72:28002(85.81 MB)
- FinalizationTimer: 0ns
Coordinator Fragment F05:(Total: 13m22s, non-child: 1.547ms, % non-child: 0.00%)
MemoryUsage(16s000ms): 26.75 KB, 92.88 KB, 93.00 KB, 88.88 KB, 88.12 KB, 88.62 KB, 88.25 KB, 100.00 KB, 88.50 KB, 88.00 KB, 90.75 KB, 90.50 KB, 88.00 KB, 88.62 KB, 89.00 KB, 88.75 KB, 88.12 KB, 88.62 KB, 88.00 KB, 88.00 KB, 89.50 KB, 89.00 KB, 89.00 KB, 88.38 KB, 108.25 KB, 92.75 KB, 88.88 KB, 89.38 KB, 88.50 KB, 88.25 KB, 88.12 KB, 92.25 KB, 94.12 KB, 100.38 KB, 163.75 KB, 147.75 KB, 90.62 KB, 88.00 KB, 89.75 KB, 93.38 KB, 96.00 KB, 92.62 KB, 88.00 KB, 88.00 KB, 88.00 KB, 88.00 KB, 88.12 KB, 88.75 KB, 88.00 KB, 88.00 KB
ThreadUsage(500.0ms): 1, 1
- AverageThreadTokens: 1.00
- PeakMemoryUsage: 192.00 KB (196608)
- PerHostPeakMemUsage: 0
- PrepareTime: 41.776us
- RowsProduced: 816 (816)
- TotalCpuTime: 2s767ms
- TotalNetworkReceiveTime: 13m22s
- TotalNetworkSendTime: 0ns
- TotalStorageWaitTime: 0ns
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
EXCHANGE_NODE (id=13):(Total: 13m22s, non-child: 7.565ms, % non-child: 0.00%)
BytesReceived(16s000ms): 160.00 B, 8.64 KB, 17.69 KB, 23.31 KB, 29.96 KB, 35.41 KB, 40.13 KB, 43.08 KB, 48.85 KB, 54.40 KB, 59.38 KB, 68.40 KB, 77.12 KB, 85.39 KB, 92.58 KB, 102.02 KB, 112.34 KB, 121.84 KB, 127.24 KB, 130.69 KB, 138.72 KB, 149.68 KB, 157.08 KB, 162.23 KB, 167.88 KB, 172.20 KB, 175.25 KB, 181.80 KB, 189.07 KB, 195.29 KB, 201.70 KB, 207.36 KB, 214.90 KB, 221.89 KB, 229.79 KB, 233.14 KB, 234.87 KB, 236.87 KB, 238.01 KB, 238.54 KB, 238.79 KB, 239.45 KB, 239.81 KB, 239.81 KB, 240.05 KB, 240.20 KB, 240.21 KB, 241.01 KB, 241.08 KB, 241.08 KB
- BytesReceived: 241.08 KB (246866)
- ConvertRowBatchTime: 1.351ms
- DeserializeRowBatchTimer: 6.345ms
- FirstBatchArrivalWaitTime: 12s704ms
- PeakMemoryUsage: 0
- RowsReturned: 816 (816)
- RowsReturnedRate: 1.00 /sec
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
Averaged Fragment F00:(Total: 9m52s, non-child: 0ns, % non-child: 0.00%)
split sizes: min: 1.57 GB, max: 1.70 GB, avg: 1.65 GB, stddev: 23.27 MB
completion times: min:7m37s max:13m25s mean: 9m52s stddev:1m3s
execution rates: min:2.07 MB/sec max:3.73 MB/sec mean:2.88 MB/sec stddev:313.77 KB/sec
num instances: 57
- AverageThreadTokens: 10.79
- PeakMemoryUsage: 81.76 MB (85730257)
- PerHostPeakMemUsage: 81.76 MB (85731982)
- PrepareTime: 199.431ms
- RowsProduced: 14 (14)
- TotalCpuTime: 1h36m
- TotalNetworkReceiveTime: 5s402ms
- TotalNetworkSendTime: 5.535us
- TotalStorageWaitTime: 1h35m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 38 (38)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 145.96 GB (156718670994)
- PeakMemoryUsage: 9.20 KB (9425)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 698.440ms, non-child: 698.440ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 141.232ms
- LoadTime: 146.598us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 370.904ms
- PrepareTime: 184.461ms
DataStreamSender (dst_id=13):(Total: 262.310us, non-child: 262.310us, % non-child: 100.00%)
- BytesSent: 4.23 KB (4330)
- NetworkThroughput(*): 1.64 MB/sec
- OverallThroughput: 15.72 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 186.355us
- ThriftTransmitTime(*): 2.701ms
- UncompressedRowBatchSize: 6.43 KB (6581)
HASH_JOIN_NODE (id=6):(Total: 9m52s, non-child: 0ns, % non-child: 0.00%)
- BuildPartitionTime: 8.850us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.788us
- GetNewBlockTime: 80.966us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.13 MB (25297060)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 178.78us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 924ns
EXCHANGE_NODE (id=12):(Total: 2s575ms, non-child: 2s575ms, % non-child: 100.00%)
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.41us
- DeserializeRowBatchTimer: 16.809us
- FirstBatchArrivalWaitTime: 2s568ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m52s, non-child: 69.360ms, % non-child: 0.01%)
- BuildPartitionTime: 8.867us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.30us
- GetNewBlockTime: 72.912us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.65 MB (16411697)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 174.794us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 973ns
EXCHANGE_NODE (id=9):(Total: 2s827ms, non-child: 2s827ms, % non-child: 100.00%)
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.995us
- DeserializeRowBatchTimer: 17.328us
- FirstBatchArrivalWaitTime: 2s820ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m49s, non-child: 9m49s, % non-child: 100.00%)
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 9.79
- BytesRead: 828.22 MB (868454598)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 828.22 MB (868454598)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 828.22 MB (868454598)
- DecompressionTime: 927.94ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 11 (11)
- NumScannerThreadsStarted: 132 (132)
- PeakMemoryUsage: 75.98 MB (79671325)
- PerReadThreadRawHdfsThroughput: 68.70 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.57M (11568980)
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- ScanRangesComplete: 566 (566)
- ScannerThreadsInvoluntaryContextSwitches: 829 (829)
- ScannerThreadsTotalWallClockTime: 1h36m
- MaterializeTupleTime(*): 16s101ms
- ScannerThreadsSysTime: 543.167ms
- ScannerThreadsUserTime: 18s902ms
- ScannerThreadsVoluntaryContextSwitches: 46.68K (46678)
- TotalRawHdfsReadTime(*): 13s097ms
- TotalReadThroughput: 1.41 MB/sec
Averaged Fragment F04:(Total: 2s713ms, non-child: 0ns, % non-child: 0.00%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:2s825ms max:2s825ms mean: 2s825ms stddev:0.000000ns
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 1
- AverageThreadTokens: 1.00
- PeakMemoryUsage: 3.38 MB (3538944)
- PerHostPeakMemUsage: 13.19 MB (13835264)
- PrepareTime: 195.945ms
- RowsProduced: 2 (2)
- TotalCpuTime: 515.965ms
- TotalNetworkReceiveTime: 2s510ms
- TotalNetworkSendTime: 391.855us
- TotalStorageWaitTime: 0ns
CodeGen:(Total: 490.990ms, non-child: 490.990ms, % non-child: 100.00%)
- CodegenTime: 2.886ms
- CompileTime: 45.696ms
- LoadTime: 133.49us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 252.637ms
- PrepareTime: 190.687ms
DataStreamSender (dst_id=12):(Total: 877.949us, non-child: 877.949us, % non-child: 100.00%)
- BytesSent: 6.68 KB (6840)
- NetworkThroughput(*): 247.41 KB/sec
- OverallThroughput: 7.43 MB/sec
- PeakMemoryUsage: 228.00 KB (233472)
- SerializeBatchTime: 14.832us
- ThriftTransmitTime(*): 26.998ms
- UncompressedRowBatchSize: 8.68 KB (8892)
AGGREGATION_NODE (id=11):(Total: 2s706ms, non-child: 196.363ms, % non-child: 7.26%)
- BuildTime: 9.582us
- GetNewBlockTime: 129.749us
- GetResultsTime: 4.355us
- HashBuckets: 2.05K (2048)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 3.14 MB (3293184)
- PinTime: 0ns
- RowsRepartitioned: 0 (0)
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 784ns
EXCHANGE_NODE (id=10):(Total: 2s510ms, non-child: 2s510ms, % non-child: 100.00%)
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.22us
- DeserializeRowBatchTimer: 14.66us
- FirstBatchArrivalWaitTime: 2s509ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
Averaged Fragment F03:(Total: 2s431ms, non-child: 0ns, % non-child: 0.00%)
split sizes: min: 703.00 B, max: 703.00 B, avg: 703.00 B, stddev: 0
completion times: min:2s610ms max:2s610ms mean: 2s610ms stddev:0.000000ns
execution rates: min:269.00 B/sec max:269.00 B/sec mean:269.00 B/sec stddev:0.00 /sec
num instances: 1
- AverageThreadTokens: 1.67
- PeakMemoryUsage: 3.22 MB (3380224)
- PerHostPeakMemUsage: 13.19 MB (13835264)
- PrepareTime: 193.971ms
- RowsProduced: 2 (2)
- TotalCpuTime: 3s049ms
- TotalNetworkReceiveTime: 0ns
- TotalNetworkSendTime: 298.890us
- TotalStorageWaitTime: 1s782ms
CodeGen:(Total: 560.333ms, non-child: 560.333ms, % non-child: 100.00%)
- CodegenTime: 3.495ms
- CompileTime: 63.686ms
- LoadTime: 95.195us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 307.18ms
- PrepareTime: 187.870ms
DataStreamSender (dst_id=10):(Total: 240.99us, non-child: 240.99us, % non-child: 100.00%)
- BytesSent: 120.00 B (120)
- NetworkThroughput(*): 414.71 KB/sec
- OverallThroughput: 488.08 KB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 16.461us
- ThriftTransmitTime(*): 282.576us
- UncompressedRowBatchSize: 156.00 B (156)
AGGREGATION_NODE (id=4):(Total: 2s430ms, non-child: 401.535ms, % non-child: 16.52%)
- BuildTime: 31.444us
- GetNewBlockTime: 48.844us
- GetResultsTime: 1.821us
- HashBuckets: 2.05K (2048)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 3.21 MB (3367936)
- PinTime: 0ns
- RowsRepartitioned: 0 (0)
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.185us
HDFS_SCAN_NODE (id=3):(Total: 2s029ms, non-child: 2s029ms, % non-child: 100.00%)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 892.00 B (892)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 892.00 B (892)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 892.00 B (892)
- DecompressionTime: 7.57us
- MaxCompressedTextFileLength: 0
- NumColumns: 3 (3)
- NumDisksAccessed: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 76.00 KB (77824)
- PerReadThreadRawHdfsThroughput: 31.37 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 2 (2)
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 2s028ms
- MaterializeTupleTime(*): 7.593us
- ScannerThreadsSysTime: 0ns
- ScannerThreadsUserTime: 0ns
- ScannerThreadsVoluntaryContextSwitches: 9 (9)
- TotalRawHdfsReadTime(*): 27.120us
- TotalReadThroughput: 326.00 B/sec
Averaged Fragment F02:(Total: 2s561ms, non-child: 0ns, % non-child: 0.00%)
split sizes: min: 0, max: 0, avg: 0, stddev: 0
completion times: min:2s660ms max:2s660ms mean: 2s660ms stddev:0.000000ns
execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
num instances: 1
- AverageThreadTokens: 1.00
- PeakMemoryUsage: 3.38 MB (3538944)
- PerHostPeakMemUsage: 13.19 MB (13835264)
- PrepareTime: 199.748ms
- RowsProduced: 2 (2)
- TotalCpuTime: 515.267ms
- TotalNetworkReceiveTime: 2s352ms
- TotalNetworkSendTime: 366.750us
- TotalStorageWaitTime: 0ns
CodeGen:(Total: 488.184ms, non-child: 488.184ms, % non-child: 100.00%)
- CodegenTime: 2.562ms
- CompileTime: 43.837ms
- LoadTime: 114.82us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 247.491ms
- PrepareTime: 195.74ms
DataStreamSender (dst_id=9):(Total: 1.139ms, non-child: 1.139ms, % non-child: 100.00%)
- BytesSent: 6.40 KB (6555)
- NetworkThroughput(*): 239.69 KB/sec
- OverallThroughput: 5.49 MB/sec
- PeakMemoryUsage: 228.00 KB (233472)
- SerializeBatchTime: 15.35us
- ThriftTransmitTime(*): 26.706ms
- UncompressedRowBatchSize: 9.69 KB (9918)
AGGREGATION_NODE (id=8):(Total: 2s552ms, non-child: 200.119ms, % non-child: 7.84%)
- BuildTime: 7.646us
- GetNewBlockTime: 127.195us
- GetResultsTime: 4.457us
- HashBuckets: 2.05K (2048)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 3.14 MB (3293184)
- PinTime: 0ns
- RowsRepartitioned: 0 (0)
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 603ns
EXCHANGE_NODE (id=7):(Total: 2s352ms, non-child: 2s352ms, % non-child: 100.00%)
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.52us
- DeserializeRowBatchTimer: 13.114us
- FirstBatchArrivalWaitTime: 2s351ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
Averaged Fragment F01:(Total: 2s278ms, non-child: 0ns, % non-child: 0.00%)
split sizes: min: 703.00 B, max: 703.00 B, avg: 703.00 B, stddev: 0
completion times: min:2s422ms max:2s422ms mean: 2s422ms stddev:0.000000ns
execution rates: min:290.00 B/sec max:290.00 B/sec mean:290.00 B/sec stddev:0.00 /sec
num instances: 1
- AverageThreadTokens: 1.67
- PeakMemoryUsage: 3.22 MB (3380224)
- PerHostPeakMemUsage: 13.19 MB (13835264)
- PrepareTime: 216.361ms
- RowsProduced: 2 (2)
- TotalCpuTime: 3s238ms
- TotalNetworkReceiveTime: 0ns
- TotalNetworkSendTime: 263.511us
- TotalStorageWaitTime: 1s335ms
CodeGen:(Total: 571.640ms, non-child: 571.640ms, % non-child: 100.00%)
- CodegenTime: 3.236ms
- CompileTime: 60.668ms
- LoadTime: 102.841us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 298.651ms
- PrepareTime: 210.222ms
DataStreamSender (dst_id=7):(Total: 167.531us, non-child: 167.531us, % non-child: 100.00%)
- BytesSent: 115.00 B (115)
- NetworkThroughput(*): 438.26 KB/sec
- OverallThroughput: 670.35 KB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 9.922us
- ThriftTransmitTime(*): 256.252us
- UncompressedRowBatchSize: 174.00 B (174)
AGGREGATION_NODE (id=2):(Total: 2s277ms, non-child: 341.394ms, % non-child: 14.99%)
- BuildTime: 93.963us
- GetNewBlockTime: 38.841us
- GetResultsTime: 3.265us
- HashBuckets: 2.05K (2048)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 3.21 MB (3367936)
- PinTime: 0ns
- RowsRepartitioned: 0 (0)
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 756ns
HDFS_SCAN_NODE (id=1):(Total: 1s935ms, non-child: 1s935ms, % non-child: 100.00%)
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 892.00 B (892)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 892.00 B (892)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 892.00 B (892)
- DecompressionTime: 8.123us
- MaxCompressedTextFileLength: 0
- NumColumns: 3 (3)
- NumDisksAccessed: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 76.00 KB (77824)
- PerReadThreadRawHdfsThroughput: 30.61 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 2 (2)
- RowsReturned: 2 (2)
- RowsReturnedRate: 1.00 /sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 1s935ms
- MaterializeTupleTime(*): 8.595us
- ScannerThreadsSysTime: 0ns
- ScannerThreadsUserTime: 0ns
- ScannerThreadsVoluntaryContextSwitches: 7 (7)
- TotalRawHdfsReadTime(*): 27.789us
- TotalReadThroughput: 377.00 B/sec
Fragment F00:
Instance c14f7b9b17987c60:d5379fb9541016ba (host=hadoopc1h12:28002):(Total: 13m24s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:45/128.14 MB 1:45/118.86 MB 2:51/158.85 MB 3:35/101.19 MB 4:47/148.12 MB 5:39/136.87 MB 6:43/138.53 MB 7:34/86.87 MB 8:43/138.08 MB 9:50/175.71 MB 10:54/212.16 MB 11:41/122.78 MB
MemoryUsage(16s000ms): 19.97 MB, 30.24 MB, 37.72 MB, 53.56 MB, 39.44 MB, 34.84 MB, 46.37 MB, 33.93 MB, 35.05 MB, 36.46 MB, 33.71 MB, 18.23 MB, 13.28 MB, 22.84 MB, 24.39 MB, 18.08 MB, 18.91 MB, 15.94 MB, 18.18 MB, 19.23 MB, 31.06 MB, 24.24 MB, 31.66 MB, 27.24 MB, 22.65 MB, 37.29 MB, 37.24 MB, 39.25 MB, 32.42 MB, 31.66 MB, 35.12 MB, 21.15 MB, 25.65 MB, 39.10 MB, 30.45 MB, 31.78 MB, 34.71 MB, 43.29 MB, 20.59 MB, 24.63 MB, 19.06 MB, 19.42 MB, 22.20 MB, 17.20 MB, 32.97 MB, 30.35 MB, 26.65 MB, 17.98 MB, 13.73 MB, 14.05 MB
ThreadUsage(16s000ms): 8, 9, 10, 11, 11, 10, 11, 10, 10, 8, 8, 6, 6, 6, 7, 6, 6, 7, 8, 7, 8, 8, 9, 8, 8, 7, 9, 8, 7, 7, 6, 6, 6, 8, 8, 8, 7, 9, 7, 8, 5, 5, 5, 5, 6, 7, 7, 7, 6, 7
- AverageThreadTokens: 8.11
- PeakMemoryUsage: 83.48 MB (87533720)
- PerHostPeakMemUsage: 83.48 MB (87533720)
- PrepareTime: 213.490ms
- RowsProduced: 18 (18)
- TotalCpuTime: 1h35m
- TotalNetworkReceiveTime: 5s243ms
- TotalNetworkSendTime: 2.436us
- TotalStorageWaitTime: 1h34m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 774.211ms, non-child: 774.211ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 157.285ms
- LoadTime: 135.76us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 417.767ms
- PrepareTime: 197.319ms
DataStreamSender (dst_id=13):(Total: 344.418us, non-child: 344.418us, % non-child: 100.00%)
- BytesSent: 5.19 KB (5315)
- NetworkThroughput(*): 1.01 MB/sec
- OverallThroughput: 14.72 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 250.986us
- ThriftTransmitTime(*): 4.995ms
- UncompressedRowBatchSize: 8.12 KB (8311)
HASH_JOIN_NODE (id=6):(Total: 13m24s, non-child: 214.395ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.733us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.592us
- GetNewBlockTime: 68.159us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.33 MB (26564800)
- PinTime: 0ns
- ProbeRows: 18 (18)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 223.550us
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 879ns
EXCHANGE_NODE (id=12):(Total: 2s500ms, non-child: 28.527us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.259us
- DeserializeRowBatchTimer: 15.831us
- FirstBatchArrivalWaitTime: 2s487ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 13m24s, non-child: 2s760ms, % non-child: 0.34%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.7us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.998us
- GetNewBlockTime: 49.229us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.97 MB (16741216)
- PinTime: 0ns
- ProbeRows: 18 (18)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 180.768us
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 905ns
EXCHANGE_NODE (id=9):(Total: 2s743ms, non-child: 23.759us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.663us
- DeserializeRowBatchTimer: 18.194us
- FirstBatchArrivalWaitTime: 2s739ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 13m21s, non-child: 13m21s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:45/128.14 MB 1:45/118.86 MB 2:51/158.85 MB 3:35/101.19 MB 4:47/148.12 MB 5:39/136.87 MB 6:43/138.53 MB 7:34/86.87 MB 8:43/138.08 MB 9:50/175.71 MB 10:54/212.16 MB 11:41/122.78 MB
Hdfs Read Thread Concurrency Bucket: 0:96.14% 1:3.731% 2:0.06219% 3:0.06219% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:24769
ExecOption: Codegen enabled: 0 out of 527
BytesRead(16s000ms): 8.01 MB, 30.63 MB, 57.84 MB, 82.41 MB, 92.61 MB, 105.63 MB, 129.73 MB, 146.20 MB, 163.01 MB, 181.26 MB, 191.10 MB, 198.88 MB, 211.04 MB, 225.97 MB, 252.40 MB, 277.46 MB, 290.24 MB, 300.54 MB, 307.12 MB, 311.54 MB, 323.73 MB, 338.49 MB, 352.28 MB, 366.01 MB, 377.45 MB, 390.06 MB, 404.38 MB, 424.29 MB, 441.48 MB, 455.79 MB, 473.81 MB, 488.47 MB, 501.50 MB, 528.20 MB, 554.89 MB, 573.87 MB, 598.99 MB, 619.60 MB, 633.77 MB, 645.84 MB, 654.73 MB, 665.50 MB, 681.25 MB, 692.63 MB, 716.78 MB, 741.76 MB, 771.16 MB, 793.13 MB, 796.87 MB, 806.39 MB
- AverageHdfsReadThreadConcurrency: 0.04
- AverageScannerThreadConcurrency: 7.11
- BytesRead: 816.28 MB (855926616)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 816.28 MB (855926616)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 816.28 MB (855926616)
- DecompressionTime: 1s122ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 160 (160)
- PeakMemoryUsage: 75.61 MB (79285816)
- PerReadThreadRawHdfsThroughput: 25.15 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.43M (11434671)
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- ScanRangesComplete: 527 (527)
- ScannerThreadsInvoluntaryContextSwitches: 1.64K (1641)
- ScannerThreadsTotalWallClockTime: 1h35m
- MaterializeTupleTime(*): 19s076ms
- ScannerThreadsSysTime: 532.10ms
- ScannerThreadsUserTime: 22s069ms
- ScannerThreadsVoluntaryContextSwitches: 49.18K (49184)
- TotalRawHdfsReadTime(*): 32s454ms
- TotalReadThroughput: 1.01 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016a1 (host=hadoopc1h90:28002):(Total: 12m2s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:50/133.57 MB 1:55/170.37 MB 2:61/143.38 MB 3:57/186.69 MB 4:33/74.70 MB 5:58/127.46 MB 6:52/144.45 MB 7:40/127.70 MB 8:48/117.54 MB 9:54/133.31 MB 10:47/111.00 MB 11:66/174.66 MB
MemoryUsage(16s000ms): 11.04 MB, 19.16 MB, 33.01 MB, 39.84 MB, 27.69 MB, 26.17 MB, 38.79 MB, 41.58 MB, 43.30 MB, 35.00 MB, 48.73 MB, 25.60 MB, 27.04 MB, 22.51 MB, 27.46 MB, 25.08 MB, 21.35 MB, 22.77 MB, 37.42 MB, 27.20 MB, 27.45 MB, 32.71 MB, 35.69 MB, 39.22 MB, 33.13 MB, 21.39 MB, 33.39 MB, 25.58 MB, 20.81 MB, 16.72 MB, 27.75 MB, 36.07 MB, 34.85 MB, 28.81 MB, 34.90 MB, 30.14 MB, 28.38 MB, 34.91 MB, 23.87 MB, 24.14 MB, 19.89 MB, 13.85 MB, 10.86 MB, 12.01 MB, 14.54 MB
ThreadUsage(16s000ms): 7, 8, 8, 9, 10, 10, 10, 10, 11, 10, 8, 7, 6, 7, 7, 7, 5, 6, 8, 8, 8, 8, 9, 8, 8, 8, 8, 7, 5, 6, 6, 7, 7, 8, 9, 10, 10, 12, 11, 9, 7, 7, 6, 8, 6
- AverageThreadTokens: 8.51
- PeakMemoryUsage: 72.74 MB (76275224)
- PerHostPeakMemUsage: 72.74 MB (76275224)
- PrepareTime: 204.450ms
- RowsProduced: 20 (20)
- TotalCpuTime: 1h30m
- TotalNetworkReceiveTime: 5s379ms
- TotalNetworkSendTime: 3.218us
- TotalStorageWaitTime: 1h30m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 707.595ms, non-child: 707.595ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 142.922ms
- LoadTime: 159.583us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 373.553ms
- PrepareTime: 189.129ms
DataStreamSender (dst_id=13):(Total: 445.381us, non-child: 445.381us, % non-child: 100.00%)
- BytesSent: 6.55 KB (6710)
- NetworkThroughput(*): 1.57 MB/sec
- OverallThroughput: 14.37 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 314.765us
- ThriftTransmitTime(*): 4.72ms
- UncompressedRowBatchSize: 9.08 KB (9294)
HASH_JOIN_NODE (id=6):(Total: 12m2s, non-child: 204.362ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 9.334us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.590us
- GetNewBlockTime: 63.346us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.92 MB (22983712)
- PinTime: 0ns
- ProbeRows: 20 (20)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 226.198us
- RowsReturned: 20 (20)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 832ns
EXCHANGE_NODE (id=12):(Total: 2s566ms, non-child: 20.264us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.337us
- DeserializeRowBatchTimer: 16.630us
- FirstBatchArrivalWaitTime: 2s556ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 12m2s, non-child: 2s836ms, % non-child: 0.39%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.926us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.421us
- GetNewBlockTime: 57.292us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 17.01 MB (17832776)
- PinTime: 0ns
- ProbeRows: 20 (20)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 221.678us
- RowsReturned: 20 (20)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 860ns
EXCHANGE_NODE (id=9):(Total: 2s812ms, non-child: 22.856us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.665us
- DeserializeRowBatchTimer: 14.257us
- FirstBatchArrivalWaitTime: 2s807ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 11m59s, non-child: 11m59s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:50/133.57 MB 1:55/170.37 MB 2:61/143.38 MB 3:57/186.69 MB 4:33/74.70 MB 5:58/127.46 MB 6:52/144.45 MB 7:40/127.70 MB 8:48/117.54 MB 9:54/133.31 MB 10:47/111.00 MB 11:66/174.66 MB
Hdfs Read Thread Concurrency Bucket: 0:97.58% 1:2.422% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:29187
ExecOption: Codegen enabled: 0 out of 621
BytesRead(16s000ms): 3.33 MB, 15.17 MB, 36.43 MB, 62.11 MB, 77.13 MB, 93.94 MB, 116.38 MB, 134.52 MB, 155.45 MB, 171.58 MB, 194.98 MB, 212.89 MB, 240.33 MB, 259.35 MB, 278.99 MB, 307.41 MB, 345.59 MB, 370.33 MB, 382.73 MB, 393.12 MB, 412.40 MB, 432.59 MB, 451.25 MB, 470.26 MB, 492.22 MB, 503.02 MB, 517.47 MB, 534.45 MB, 550.27 MB, 560.88 MB, 576.84 MB, 599.74 MB, 626.00 MB, 650.27 MB, 677.86 MB, 697.93 MB, 720.16 MB, 740.24 MB, 754.96 MB, 767.42 MB, 779.75 MB, 787.15 MB, 793.76 MB, 800.58 MB, 808.64 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 7.51
- BytesRead: 813.45 MB (852966824)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 813.45 MB (852966824)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 813.45 MB (852966824)
- DecompressionTime: 850.225ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 90 (90)
- PeakMemoryUsage: 63.01 MB (66075720)
- PerReadThreadRawHdfsThroughput: 59.54 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.24M (11241742)
- RowsReturned: 20 (20)
- RowsReturnedRate: 0
- ScanRangesComplete: 621 (621)
- ScannerThreadsInvoluntaryContextSwitches: 514 (514)
- ScannerThreadsTotalWallClockTime: 1h30m
- MaterializeTupleTime(*): 15s496ms
- ScannerThreadsSysTime: 476.9ms
- ScannerThreadsUserTime: 18s601ms
- ScannerThreadsVoluntaryContextSwitches: 49.76K (49756)
- TotalRawHdfsReadTime(*): 13s663ms
- TotalReadThroughput: 1.13 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016c4 (host=hadoopc1h80:28002):(Total: 12m2s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:44/117.43 MB 1:53/189.11 MB 2:51/167.84 MB 3:44/121.36 MB 4:42/158.63 MB 5:45/95.99 MB 6:37/95.46 MB 7:39/126.49 MB 8:47/158.47 MB 9:57/129.15 MB 10:47/142.05 MB 11:70/196.79 MB
MemoryUsage(16s000ms): 26.97 MB, 39.44 MB, 52.21 MB, 47.91 MB, 44.67 MB, 48.27 MB, 51.04 MB, 35.99 MB, 43.85 MB, 45.11 MB, 36.81 MB, 30.09 MB, 29.14 MB, 42.61 MB, 28.92 MB, 39.25 MB, 40.46 MB, 40.96 MB, 41.13 MB, 53.82 MB, 51.23 MB, 43.81 MB, 48.19 MB, 42.09 MB, 47.35 MB, 38.91 MB, 39.75 MB, 45.54 MB, 34.73 MB, 46.18 MB, 46.09 MB, 30.05 MB, 31.74 MB, 31.50 MB, 21.72 MB, 20.70 MB, 14.88 MB, 17.84 MB, 19.42 MB, 12.08 MB, 16.82 MB, 15.36 MB, 12.03 MB, 9.75 MB, 5.86 MB
ThreadUsage(16s000ms): 11, 12, 13, 13, 13, 15, 17, 14, 12, 10, 10, 9, 9, 8, 8, 9, 8, 9, 10, 11, 12, 11, 12, 10, 10, 10, 11, 10, 8, 10, 10, 10, 10, 11, 14, 14, 12, 16, 15, 13, 12, 10, 8, 6, 4
- AverageThreadTokens: 11.49
- PeakMemoryUsage: 79.66 MB (83525400)
- PerHostPeakMemUsage: 79.66 MB (83525400)
- PrepareTime: 206.492ms
- RowsProduced: 10 (10)
- TotalCpuTime: 2h6m
- TotalNetworkReceiveTime: 5s354ms
- TotalNetworkSendTime: 906ns
- TotalStorageWaitTime: 2h5m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 714.849ms, non-child: 714.849ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 151.495ms
- LoadTime: 134.100us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 370.769ms
- PrepareTime: 190.806ms
DataStreamSender (dst_id=13):(Total: 262.223us, non-child: 262.223us, % non-child: 100.00%)
- BytesSent: 2.91 KB (2983)
- NetworkThroughput(*): 1.87 MB/sec
- OverallThroughput: 10.85 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 199.358us
- ThriftTransmitTime(*): 1.517ms
- UncompressedRowBatchSize: 4.49 KB (4593)
HASH_JOIN_NODE (id=6):(Total: 12m2s, non-child: 206.526ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.830us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.163us
- GetNewBlockTime: 154.447us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.38 MB (26617056)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 179.375us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 914ns
EXCHANGE_NODE (id=12):(Total: 2s551ms, non-child: 18.333us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.486us
- DeserializeRowBatchTimer: 10.409us
- FirstBatchArrivalWaitTime: 2s550ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 12m2s, non-child: 2s821ms, % non-child: 0.39%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.790us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 6.364us
- GetNewBlockTime: 112.255us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.43 MB (16180432)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 182.865us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 978ns
EXCHANGE_NODE (id=9):(Total: 2s803ms, non-child: 18.205us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 990ns
- DeserializeRowBatchTimer: 9.216us
- FirstBatchArrivalWaitTime: 2s802ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 11m59s, non-child: 11m59s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:44/117.43 MB 1:53/189.11 MB 2:51/167.84 MB 3:44/121.36 MB 4:42/158.63 MB 5:45/95.99 MB 6:37/95.46 MB 7:39/126.49 MB 8:47/158.47 MB 9:57/129.15 MB 10:47/142.05 MB 11:70/196.79 MB
Hdfs Read Thread Concurrency Bucket: 0:97.65% 1:2.216% 2:0.1385% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:27072
ExecOption: Codegen enabled: 0 out of 576
BytesRead(16s000ms): 7.46 MB, 29.32 MB, 54.46 MB, 74.08 MB, 87.16 MB, 105.86 MB, 130.07 MB, 142.78 MB, 160.53 MB, 173.91 MB, 191.41 MB, 207.10 MB, 225.76 MB, 250.31 MB, 271.44 MB, 300.83 MB, 338.04 MB, 365.96 MB, 382.21 MB, 401.03 MB, 425.71 MB, 449.59 MB, 469.53 MB, 492.51 MB, 518.98 MB, 537.20 MB, 553.51 MB, 578.44 MB, 615.02 MB, 643.54 MB, 673.49 MB, 698.94 MB, 728.02 MB, 759.82 MB, 782.41 MB, 797.61 MB, 806.84 MB, 811.87 MB, 814.01 MB, 816.67 MB, 819.40 MB, 820.93 MB, 824.44 MB, 829.55 MB, 833.33 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 10.50
- BytesRead: 835.66 MB (876249682)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 835.66 MB (876249682)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 835.66 MB (876249682)
- DecompressionTime: 1s021ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 138 (138)
- PeakMemoryUsage: 71.60 MB (75081576)
- PerReadThreadRawHdfsThroughput: 56.24 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.64M (11636543)
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- ScanRangesComplete: 576 (576)
- ScannerThreadsInvoluntaryContextSwitches: 509 (509)
- ScannerThreadsTotalWallClockTime: 2h6m
- MaterializeTupleTime(*): 15s591ms
- ScannerThreadsSysTime: 544.8ms
- ScannerThreadsUserTime: 18s357ms
- ScannerThreadsVoluntaryContextSwitches: 43.47K (43469)
- TotalRawHdfsReadTime(*): 14s858ms
- TotalReadThroughput: 1.16 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016b6 (host=hadoopc1h79:28002):(Total: 11m39s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:58/188.31 MB 1:49/155.23 MB 2:59/175.62 MB 3:51/143.83 MB 4:41/133.04 MB 5:46/110.32 MB 6:60/153.43 MB 7:56/166.05 MB 8:64/167.49 MB 9:51/117.33 MB 10:47/160.34 MB
MemoryUsage(16s000ms): 41.42 MB, 45.33 MB, 36.59 MB, 49.49 MB, 50.91 MB, 47.62 MB, 37.59 MB, 45.42 MB, 48.31 MB, 47.08 MB, 24.77 MB, 40.73 MB, 29.96 MB, 35.25 MB, 27.66 MB, 32.55 MB, 30.01 MB, 36.30 MB, 42.08 MB, 46.12 MB, 40.48 MB, 39.31 MB, 40.70 MB, 40.99 MB, 37.06 MB, 40.83 MB, 47.22 MB, 33.81 MB, 36.58 MB, 35.29 MB, 37.32 MB, 31.04 MB, 32.68 MB, 26.66 MB, 32.84 MB, 30.63 MB, 22.88 MB, 40.74 MB, 25.46 MB, 26.90 MB, 21.04 MB, 12.47 MB, 13.22 MB
ThreadUsage(16s000ms): 11, 11, 13, 13, 14, 17, 16, 13, 12, 10, 9, 8, 8, 7, 8, 9, 8, 9, 10, 10, 10, 10, 12, 10, 10, 9, 11, 10, 8, 9, 9, 9, 9, 10, 12, 11, 12, 17, 12, 12, 10, 7, 4
- AverageThreadTokens: 10.92
- PeakMemoryUsage: 76.09 MB (79781728)
- PerHostPeakMemUsage: 76.09 MB (79781728)
- PrepareTime: 205.757ms
- RowsProduced: 18 (18)
- TotalCpuTime: 1h55m
- TotalNetworkReceiveTime: 5s389ms
- TotalNetworkSendTime: 2.230us
- TotalStorageWaitTime: 1h55m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 701.989ms, non-child: 701.989ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 138.727ms
- LoadTime: 181.193us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 372.147ms
- PrepareTime: 189.229ms
DataStreamSender (dst_id=13):(Total: 398.30us, non-child: 398.30us, % non-child: 100.00%)
- BytesSent: 5.61 KB (5747)
- NetworkThroughput(*): 1.67 MB/sec
- OverallThroughput: 13.77 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 304.187us
- ThriftTransmitTime(*): 3.280ms
- UncompressedRowBatchSize: 8.08 KB (8275)
HASH_JOIN_NODE (id=6):(Total: 11m39s, non-child: 205.455ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 9.13us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.7us
- GetNewBlockTime: 143.600us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 20.23 MB (21212504)
- PinTime: 0ns
- ProbeRows: 18 (18)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 160.791us
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 828ns
EXCHANGE_NODE (id=12):(Total: 2s570ms, non-child: 29.984us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.968us
- DeserializeRowBatchTimer: 11.142us
- FirstBatchArrivalWaitTime: 2s560ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 11m39s, non-child: 2s835ms, % non-child: 0.41%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.246us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.521us
- GetNewBlockTime: 98.767us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 12.78 MB (13403624)
- PinTime: 0ns
- ProbeRows: 18 (18)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 186.780us
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 959ns
EXCHANGE_NODE (id=9):(Total: 2s818ms, non-child: 22.387us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.461us
- DeserializeRowBatchTimer: 8.845us
- FirstBatchArrivalWaitTime: 2s812ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 11m36s, non-child: 11m36s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:58/188.31 MB 1:49/155.23 MB 2:59/175.62 MB 3:51/143.83 MB 4:41/133.04 MB 5:46/110.32 MB 6:60/153.43 MB 7:56/166.05 MB 8:64/167.49 MB 9:51/117.33 MB 10:47/160.34 MB
Hdfs Read Thread Concurrency Bucket: 0:97.86% 1:2% 2:0.1429% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:27354
ExecOption: Codegen enabled: 0 out of 582
BytesRead(16s000ms): 10.89 MB, 37.23 MB, 54.49 MB, 70.79 MB, 86.38 MB, 102.60 MB, 118.29 MB, 133.90 MB, 151.43 MB, 166.63 MB, 176.92 MB, 200.24 MB, 222.31 MB, 243.83 MB, 268.52 MB, 289.26 MB, 319.75 MB, 344.90 MB, 357.28 MB, 371.60 MB, 394.04 MB, 415.66 MB, 432.58 MB, 450.87 MB, 467.69 MB, 482.84 MB, 499.22 MB, 517.91 MB, 539.88 MB, 561.56 MB, 584.31 MB, 611.91 MB, 639.13 MB, 668.57 MB, 705.75 MB, 729.76 MB, 754.16 MB, 771.02 MB, 785.81 MB, 795.95 MB, 806.96 MB, 814.76 MB, 819.15 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 9.91
- BytesRead: 822.63 MB (862587677)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 822.63 MB (862587677)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 822.63 MB (862587677)
- DecompressionTime: 514.577ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 11 (11)
- NumScannerThreadsStarted: 175 (175)
- PeakMemoryUsage: 74.31 MB (77923792)
- PerReadThreadRawHdfsThroughput: 62.94 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.45M (11447541)
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- ScanRangesComplete: 582 (582)
- ScannerThreadsInvoluntaryContextSwitches: 409 (409)
- ScannerThreadsTotalWallClockTime: 1h55m
- MaterializeTupleTime(*): 15s156ms
- ScannerThreadsSysTime: 544.6ms
- ScannerThreadsUserTime: 17s701ms
- ScannerThreadsVoluntaryContextSwitches: 43.91K (43910)
- TotalRawHdfsReadTime(*): 13s070ms
- TotalReadThroughput: 1.17 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016b4 (host=hadoopc1h17:28002):(Total: 11m21s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:43/163.17 MB 1:46/154.68 MB 2:53/198.57 MB 3:34/120.76 MB 4:39/126.48 MB 5:42/151.05 MB 6:36/99.07 MB 7:50/196.04 MB 8:39/130.19 MB 9:41/101.68 MB 10:44/158.50 MB 11:42/139.61 MB
MemoryUsage(16s000ms): 21.70 MB, 27.36 MB, 36.49 MB, 37.19 MB, 36.82 MB, 45.96 MB, 34.78 MB, 27.86 MB, 42.44 MB, 30.30 MB, 24.33 MB, 24.09 MB, 29.43 MB, 23.10 MB, 22.80 MB, 15.40 MB, 10.38 MB, 12.96 MB, 28.44 MB, 33.31 MB, 27.15 MB, 26.19 MB, 28.58 MB, 24.06 MB, 21.64 MB, 21.74 MB, 23.10 MB, 26.20 MB, 30.32 MB, 28.52 MB, 40.28 MB, 36.73 MB, 28.08 MB, 40.51 MB, 34.44 MB, 41.50 MB, 31.09 MB, 28.68 MB, 24.06 MB, 15.40 MB, 20.54 MB, 12.94 MB
ThreadUsage(16s000ms): 9, 8, 10, 10, 11, 12, 12, 10, 10, 8, 7, 7, 6, 6, 7, 7, 6, 7, 8, 8, 9, 8, 9, 8, 8, 7, 8, 8, 7, 7, 6, 7, 7, 8, 10, 9, 8, 11, 9, 10, 7, 5
- AverageThreadTokens: 8.76
- PeakMemoryUsage: 73.77 MB (77349576)
- PerHostPeakMemUsage: 73.77 MB (77349576)
- PrepareTime: 194.550ms
- RowsProduced: 11 (11)
- TotalCpuTime: 1h28m
- TotalNetworkReceiveTime: 5s435ms
- TotalNetworkSendTime: 1.407us
- TotalStorageWaitTime: 1h27m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 690.182ms, non-child: 690.182ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 144.46ms
- LoadTime: 134.821us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 364.343ms
- PrepareTime: 180.1ms
DataStreamSender (dst_id=13):(Total: 198.346us, non-child: 198.346us, % non-child: 100.00%)
- BytesSent: 3.09 KB (3160)
- NetworkThroughput(*): 1003.00 KB/sec
- OverallThroughput: 15.19 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 149.507us
- ThriftTransmitTime(*): 3.76ms
- UncompressedRowBatchSize: 4.93 KB (5051)
HASH_JOIN_NODE (id=6):(Total: 11m21s, non-child: 194.300ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 11.405us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.790us
- GetNewBlockTime: 64.516us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 26.64 MB (27938944)
- PinTime: 0ns
- ProbeRows: 11 (11)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 142.712us
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 844ns
EXCHANGE_NODE (id=12):(Total: 2s591ms, non-child: 33.466us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.891us
- DeserializeRowBatchTimer: 15.859us
- FirstBatchArrivalWaitTime: 2s583ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 11m21s, non-child: 2s871ms, % non-child: 0.42%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.734us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.958us
- GetNewBlockTime: 81.489us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 20.95 MB (21971600)
- PinTime: 0ns
- ProbeRows: 11 (11)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 126.638us
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.237us
EXCHANGE_NODE (id=9):(Total: 2s843ms, non-child: 30.274us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.845us
- DeserializeRowBatchTimer: 13.749us
- FirstBatchArrivalWaitTime: 2s834ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 11m18s, non-child: 11m18s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:43/163.17 MB 1:46/154.68 MB 2:53/198.57 MB 3:34/120.76 MB 4:39/126.48 MB 5:42/151.05 MB 6:36/99.07 MB 7:50/196.04 MB 8:39/130.19 MB 9:41/101.68 MB 10:44/158.50 MB 11:42/139.61 MB
Hdfs Read Thread Concurrency Bucket: 0:97.21% 1:2.641% 2:0.1467% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:23923
ExecOption: Codegen enabled: 0 out of 509
BytesRead(16s000ms): 8.75 MB, 29.77 MB, 59.48 MB, 80.41 MB, 97.23 MB, 120.64 MB, 143.19 MB, 159.32 MB, 185.09 MB, 203.24 MB, 220.76 MB, 242.84 MB, 268.84 MB, 288.17 MB, 305.69 MB, 322.42 MB, 335.22 MB, 346.58 MB, 357.55 MB, 377.76 MB, 396.64 MB, 416.82 MB, 431.97 MB, 447.63 MB, 458.36 MB, 470.40 MB, 480.02 MB, 495.69 MB, 520.07 MB, 542.07 MB, 564.34 MB, 598.82 MB, 626.39 MB, 668.68 MB, 704.40 MB, 743.51 MB, 783.54 MB, 801.20 MB, 822.03 MB, 829.20 MB, 835.52 MB, 843.32 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 7.76
- BytesRead: 848.70 MB (889922541)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 848.70 MB (889922541)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 848.70 MB (889922541)
- DecompressionTime: 999.883ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 132 (132)
- PeakMemoryUsage: 69.09 MB (72450336)
- PerReadThreadRawHdfsThroughput: 45.08 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.99M (11994920)
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- ScanRangesComplete: 509 (509)
- ScannerThreadsInvoluntaryContextSwitches: 1.34K (1344)
- ScannerThreadsTotalWallClockTime: 1h28m
- MaterializeTupleTime(*): 17s669ms
- ScannerThreadsSysTime: 544.7ms
- ScannerThreadsUserTime: 20s413ms
- ScannerThreadsVoluntaryContextSwitches: 47.04K (47038)
- TotalRawHdfsReadTime(*): 18s825ms
- TotalReadThroughput: 1.24 MB/sec
Instance c14f7b9b17987c60:d5379fb95410169a (host=hadoopc1h23:28002):(Total: 11m10s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:47/155.89 MB 1:60/193.25 MB 2:55/202.36 MB 3:43/146.23 MB 4:37/104.74 MB 5:47/126.06 MB 6:44/124.79 MB 7:44/128.41 MB 8:53/146.51 MB 9:42/160.55 MB 10:49/165.04 MB 11:40/76.00 MB
MemoryUsage(16s000ms): 27.24 MB, 33.63 MB, 47.03 MB, 52.85 MB, 53.06 MB, 48.54 MB, 41.19 MB, 35.03 MB, 41.70 MB, 36.13 MB, 21.29 MB, 25.92 MB, 27.46 MB, 28.99 MB, 29.43 MB, 26.35 MB, 39.16 MB, 31.11 MB, 37.15 MB, 30.84 MB, 41.72 MB, 52.19 MB, 53.83 MB, 30.18 MB, 26.21 MB, 31.42 MB, 41.65 MB, 37.21 MB, 30.14 MB, 36.10 MB, 24.52 MB, 33.74 MB, 29.36 MB, 35.67 MB, 31.86 MB, 27.12 MB, 19.43 MB, 22.18 MB, 17.56 MB, 15.52 MB, 15.89 MB
ThreadUsage(16s000ms): 9, 8, 10, 11, 11, 12, 12, 10, 10, 8, 6, 6, 7, 6, 7, 7, 7, 7, 8, 8, 9, 9, 10, 7, 8, 7, 9, 8, 7, 7, 7, 7, 7, 8, 9, 9, 8, 11, 10, 12, 9
- AverageThreadTokens: 9.08
- PeakMemoryUsage: 70.99 MB (74434816)
- PerHostPeakMemUsage: 70.99 MB (74434816)
- PrepareTime: 256.476ms
- RowsProduced: 24 (24)
- TotalCpuTime: 1h30m
- TotalNetworkReceiveTime: 5s174ms
- TotalNetworkSendTime: 2.207us
- TotalStorageWaitTime: 1h29m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 810.902ms, non-child: 810.902ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 150.447ms
- LoadTime: 165.315us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 416.982ms
- PrepareTime: 241.606ms
DataStreamSender (dst_id=13):(Total: 362.215us, non-child: 362.215us, % non-child: 100.00%)
- BytesSent: 6.73 KB (6894)
- NetworkThroughput(*): 1.39 MB/sec
- OverallThroughput: 18.15 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 265.553us
- ThriftTransmitTime(*): 4.733ms
- UncompressedRowBatchSize: 10.70 KB (10955)
HASH_JOIN_NODE (id=6):(Total: 11m10s, non-child: 257.339ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 16.98us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.924us
- GetNewBlockTime: 100.811us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.87 MB (27122080)
- PinTime: 0ns
- ProbeRows: 24 (24)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 198.135us
- RowsReturned: 24 (24)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.328us
EXCHANGE_NODE (id=12):(Total: 2s458ms, non-child: 41.770us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.627us
- DeserializeRowBatchTimer: 22.935us
- FirstBatchArrivalWaitTime: 2s450ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 11m10s, non-child: 2s733ms, % non-child: 0.41%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 15.75us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.413us
- GetNewBlockTime: 95.702us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 18.49 MB (19390400)
- PinTime: 0ns
- ProbeRows: 24 (24)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 196.415us
- RowsReturned: 24 (24)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.396us
EXCHANGE_NODE (id=9):(Total: 2s716ms, non-child: 42.203us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.594us
- DeserializeRowBatchTimer: 23.881us
- FirstBatchArrivalWaitTime: 2s702ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 11m7s, non-child: 11m7s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:47/155.89 MB 1:60/193.25 MB 2:55/202.36 MB 3:43/146.23 MB 4:37/104.74 MB 5:47/126.06 MB 6:44/124.79 MB 7:44/128.41 MB 8:53/146.51 MB 9:42/160.55 MB 10:49/165.04 MB 11:40/76.00 MB
Hdfs Read Thread Concurrency Bucket: 0:98.36% 1:1.641% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:26367
ExecOption: Codegen enabled: 0 out of 561
BytesRead(16s000ms): 10.18 MB, 35.81 MB, 60.76 MB, 84.43 MB, 105.18 MB, 128.20 MB, 151.58 MB, 168.90 MB, 189.50 MB, 207.64 MB, 218.01 MB, 237.65 MB, 268.95 MB, 292.32 MB, 321.97 MB, 356.08 MB, 394.52 MB, 430.50 MB, 445.89 MB, 458.49 MB, 480.46 MB, 511.27 MB, 534.11 MB, 551.23 MB, 563.74 MB, 578.56 MB, 596.10 MB, 618.33 MB, 643.14 MB, 669.37 MB, 684.33 MB, 712.30 MB, 730.63 MB, 753.55 MB, 781.91 MB, 800.22 MB, 812.78 MB, 820.62 MB, 826.77 MB, 830.18 MB, 834.20 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 8.09
- BytesRead: 848.49 MB (889705531)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 848.49 MB (889705531)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 848.49 MB (889705531)
- DecompressionTime: 347.681ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 119 (119)
- PeakMemoryUsage: 68.17 MB (71477976)
- PerReadThreadRawHdfsThroughput: 79.09 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.88M (11880425)
- RowsReturned: 24 (24)
- RowsReturnedRate: 0
- ScanRangesComplete: 561 (561)
- ScannerThreadsInvoluntaryContextSwitches: 655 (655)
- ScannerThreadsTotalWallClockTime: 1h30m
- MaterializeTupleTime(*): 15s612ms
- ScannerThreadsSysTime: 424.4ms
- ScannerThreadsUserTime: 18s545ms
- ScannerThreadsVoluntaryContextSwitches: 51.09K (51094)
- TotalRawHdfsReadTime(*): 10s728ms
- TotalReadThroughput: 1.26 MB/sec
Instance c14f7b9b17987c60:d5379fb954101695 (host=hadoopc1h8:28002):(Total: 11m9s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:56/132.16 MB 1:66/171.17 MB 2:56/148.04 MB 3:45/133.13 MB 4:48/131.77 MB 5:52/140.85 MB 6:42/132.34 MB 7:60/166.65 MB 8:48/122.73 MB 9:48/122.16 MB 10:44/121.96 MB 11:45/143.41 MB
MemoryUsage(16s000ms): 30.83 MB, 33.09 MB, 45.35 MB, 45.32 MB, 48.88 MB, 43.87 MB, 41.48 MB, 29.51 MB, 38.81 MB, 35.14 MB, 35.40 MB, 24.70 MB, 29.46 MB, 34.27 MB, 25.94 MB, 31.38 MB, 29.32 MB, 24.45 MB, 25.32 MB, 42.16 MB, 44.32 MB, 44.51 MB, 45.07 MB, 33.21 MB, 26.15 MB, 27.06 MB, 29.28 MB, 31.24 MB, 32.56 MB, 21.25 MB, 25.95 MB, 27.92 MB, 26.39 MB, 30.94 MB, 27.11 MB, 21.36 MB, 19.50 MB, 15.87 MB, 14.09 MB, 8.96 MB, 8.88 MB
ThreadUsage(16s000ms): 9, 9, 10, 12, 13, 12, 13, 10, 10, 8, 7, 7, 7, 7, 7, 8, 7, 8, 9, 9, 10, 10, 10, 8, 8, 8, 8, 8, 7, 6, 7, 9, 9, 13, 15, 13, 12, 11, 9, 7, 5
- AverageThreadTokens: 9.70
- PeakMemoryUsage: 74.45 MB (78069728)
- PerHostPeakMemUsage: 74.45 MB (78069728)
- PrepareTime: 206.662ms
- RowsProduced: 14 (14)
- TotalCpuTime: 1h37m
- TotalNetworkReceiveTime: 5s439ms
- TotalNetworkSendTime: 1.748us
- TotalStorageWaitTime: 1h36m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 683.494ms, non-child: 683.494ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 138.916ms
- LoadTime: 141.787us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 353.405ms
- PrepareTime: 189.365ms
DataStreamSender (dst_id=13):(Total: 243.646us, non-child: 243.646us, % non-child: 100.00%)
- BytesSent: 4.46 KB (4572)
- NetworkThroughput(*): 1.52 MB/sec
- OverallThroughput: 17.90 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 161.959us
- ThriftTransmitTime(*): 2.864ms
- UncompressedRowBatchSize: 6.31 KB (6460)
HASH_JOIN_NODE (id=6):(Total: 11m9s, non-child: 206.448ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.676us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 6.267us
- GetNewBlockTime: 76.991us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 23.53 MB (24671904)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 187.417us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 866ns
EXCHANGE_NODE (id=12):(Total: 2s596ms, non-child: 25.958us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.561us
- DeserializeRowBatchTimer: 15.40us
- FirstBatchArrivalWaitTime: 2s587ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 11m9s, non-child: 2s862ms, % non-child: 0.43%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.975us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.940us
- GetNewBlockTime: 53.618us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 13.60 MB (14257568)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 193.321us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 852ns
EXCHANGE_NODE (id=9):(Total: 2s842ms, non-child: 25.371us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.450us
- DeserializeRowBatchTimer: 13.805us
- FirstBatchArrivalWaitTime: 2s839ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 11m6s, non-child: 11m6s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:56/132.16 MB 1:66/171.17 MB 2:56/148.04 MB 3:45/133.13 MB 4:48/131.77 MB 5:52/140.85 MB 6:42/132.34 MB 7:60/166.65 MB 8:48/122.73 MB 9:48/122.16 MB 10:44/121.96 MB 11:45/143.41 MB
Hdfs Read Thread Concurrency Bucket: 0:99.03% 1:0.9709% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28670
ExecOption: Codegen enabled: 0 out of 610
BytesRead(16s000ms): 11.37 MB, 35.13 MB, 64.88 MB, 86.49 MB, 101.58 MB, 119.45 MB, 142.68 MB, 154.84 MB, 172.72 MB, 188.38 MB, 209.01 MB, 226.85 MB, 252.92 MB, 280.24 MB, 305.17 MB, 332.72 MB, 361.51 MB, 386.57 MB, 397.22 MB, 410.58 MB, 439.61 MB, 468.21 MB, 493.05 MB, 512.68 MB, 531.70 MB, 545.11 MB, 558.12 MB, 578.05 MB, 608.27 MB, 632.05 MB, 659.16 MB, 687.10 MB, 712.75 MB, 742.56 MB, 771.65 MB, 785.09 MB, 797.89 MB, 805.58 MB, 809.24 MB, 812.78 MB, 816.43 MB
- AverageHdfsReadThreadConcurrency: 0.01
- AverageScannerThreadConcurrency: 8.70
- BytesRead: 822.50 MB (862455441)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 822.50 MB (862455441)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 822.50 MB (862455441)
- DecompressionTime: 411.985ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 130 (130)
- PeakMemoryUsage: 67.46 MB (70736432)
- PerReadThreadRawHdfsThroughput: 90.25 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.41M (11405673)
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- ScanRangesComplete: 610 (610)
- ScannerThreadsInvoluntaryContextSwitches: 541 (541)
- ScannerThreadsTotalWallClockTime: 1h37m
- MaterializeTupleTime(*): 14s643ms
- ScannerThreadsSysTime: 456.7ms
- ScannerThreadsUserTime: 17s449ms
- ScannerThreadsVoluntaryContextSwitches: 52.75K (52754)
- TotalRawHdfsReadTime(*): 9s113ms
- TotalReadThroughput: 1.23 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016a9 (host=hadoopc1h16:28002):(Total: 11m2s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:45/104.42 MB 1:47/147.47 MB 2:54/186.87 MB 3:44/133.01 MB 4:49/178.12 MB 5:54/173.36 MB 6:52/134.27 MB 7:49/178.90 MB 8:46/160.47 MB 9:44/141.73 MB 10:54/157.76 MB
MemoryUsage(16s000ms): 17.43 MB, 29.76 MB, 41.65 MB, 47.98 MB, 46.31 MB, 41.15 MB, 61.26 MB, 34.44 MB, 41.35 MB, 42.67 MB, 31.26 MB, 26.40 MB, 41.62 MB, 30.71 MB, 32.39 MB, 30.40 MB, 29.53 MB, 30.24 MB, 31.59 MB, 31.71 MB, 39.93 MB, 40.30 MB, 44.08 MB, 29.55 MB, 25.48 MB, 19.89 MB, 28.17 MB, 26.51 MB, 35.44 MB, 35.44 MB, 27.89 MB, 36.45 MB, 29.21 MB, 32.91 MB, 41.27 MB, 35.01 MB, 29.06 MB, 25.81 MB, 11.45 MB, 5.96 MB, 10.18 MB
ThreadUsage(16s000ms): 9, 9, 10, 10, 10, 11, 11, 10, 10, 9, 7, 6, 6, 6, 6, 7, 6, 7, 8, 8, 9, 9, 10, 7, 7, 7, 8, 8, 7, 7, 6, 7, 7, 8, 8, 8, 7, 11, 7, 6, 3
- AverageThreadTokens: 8.56
- PeakMemoryUsage: 87.57 MB (91828112)
- PerHostPeakMemUsage: 87.57 MB (91828112)
- PrepareTime: 200.497ms
- RowsProduced: 14 (14)
- TotalCpuTime: 1h23m
- TotalNetworkReceiveTime: 5s415ms
- TotalNetworkSendTime: 216.284us
- TotalStorageWaitTime: 1h23m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 691.624ms, non-child: 691.624ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 130.862ms
- LoadTime: 142.768us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 372.867ms
- PrepareTime: 186.174ms
DataStreamSender (dst_id=13):(Total: 258.53us, non-child: 258.53us, % non-child: 100.00%)
- BytesSent: 4.54 KB (4649)
- NetworkThroughput(*): 917.32 KB/sec
- OverallThroughput: 17.18 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 161.842us
- ThriftTransmitTime(*): 4.949ms
- UncompressedRowBatchSize: 6.29 KB (6438)
HASH_JOIN_NODE (id=6):(Total: 11m2s, non-child: 200.866ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 10.374us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.910us
- GetNewBlockTime: 60.629us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 23.16 MB (24287736)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 205.888us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.177us
EXCHANGE_NODE (id=12):(Total: 2s584ms, non-child: 21.532us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.479us
- DeserializeRowBatchTimer: 17.139us
- FirstBatchArrivalWaitTime: 2s576ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 11m2s, non-child: 2s850ms, % non-child: 0.43%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.567us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.407us
- GetNewBlockTime: 55.312us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 17.09 MB (17916504)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 173.355us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 881ns
EXCHANGE_NODE (id=9):(Total: 2s831ms, non-child: 27.614us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.66us
- DeserializeRowBatchTimer: 17.613us
- FirstBatchArrivalWaitTime: 2s828ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m59s, non-child: 10m59s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:45/104.42 MB 1:47/147.47 MB 2:54/186.87 MB 3:44/133.01 MB 4:49/178.12 MB 5:54/173.36 MB 6:52/134.27 MB 7:49/178.90 MB 8:46/160.47 MB 9:44/141.73 MB 10:54/157.76 MB
Hdfs Read Thread Concurrency Bucket: 0:98.04% 1:1.962% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0%
File Formats: PARQUET/SNAPPY:25286
ExecOption: Codegen enabled: 0 out of 538
BytesRead(16s000ms): 5.72 MB, 25.79 MB, 49.39 MB, 70.75 MB, 89.23 MB, 109.56 MB, 135.89 MB, 149.80 MB, 169.92 MB, 187.26 MB, 201.62 MB, 218.33 MB, 247.85 MB, 269.79 MB, 298.92 MB, 335.38 MB, 370.65 MB, 402.50 MB, 414.54 MB, 427.24 MB, 447.06 MB, 469.95 MB, 491.51 MB, 506.63 MB, 517.21 MB, 524.68 MB, 534.37 MB, 548.07 MB, 566.71 MB, 585.75 MB, 603.91 MB, 627.36 MB, 653.55 MB, 682.51 MB, 729.23 MB, 760.83 MB, 793.26 MB, 813.34 MB, 822.80 MB, 824.15 MB, 826.35 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 7.56
- BytesRead: 829.83 MB (870134863)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 829.83 MB (870134863)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 829.83 MB (870134863)
- DecompressionTime: 658.477ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 11 (11)
- NumScannerThreadsStarted: 140 (140)
- PeakMemoryUsage: 79.15 MB (82995984)
- PerReadThreadRawHdfsThroughput: 53.40 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.68M (11682101)
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- ScanRangesComplete: 538 (538)
- ScannerThreadsInvoluntaryContextSwitches: 2.03K (2034)
- ScannerThreadsTotalWallClockTime: 1h23m
- MaterializeTupleTime(*): 17s687ms
- ScannerThreadsSysTime: 488.8ms
- ScannerThreadsUserTime: 20s537ms
- ScannerThreadsVoluntaryContextSwitches: 50.09K (50092)
- TotalRawHdfsReadTime(*): 15s539ms
- TotalReadThroughput: 1.25 MB/sec
Instance c14f7b9b17987c60:d5379fb95410169d (host=hadoopc1h22:28002):(Total: 10m54s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:30/86.30 MB 1:49/158.29 MB 2:50/137.67 MB 3:59/146.99 MB 4:56/159.27 MB 5:49/164.38 MB 6:49/178.22 MB 7:39/105.73 MB 8:32/101.42 MB 9:50/173.77 MB 10:45/101.17 MB 11:47/182.35 MB
MemoryUsage(16s000ms): 18.57 MB, 32.39 MB, 31.05 MB, 44.74 MB, 48.05 MB, 55.76 MB, 41.12 MB, 32.08 MB, 47.21 MB, 29.17 MB, 27.17 MB, 23.53 MB, 29.42 MB, 35.68 MB, 29.45 MB, 27.25 MB, 31.44 MB, 37.55 MB, 31.41 MB, 28.02 MB, 27.76 MB, 28.59 MB, 49.83 MB, 36.69 MB, 28.25 MB, 27.03 MB, 49.30 MB, 38.16 MB, 19.84 MB, 17.29 MB, 40.85 MB, 33.11 MB, 27.89 MB, 40.39 MB, 34.35 MB, 21.64 MB, 18.88 MB, 19.62 MB, 14.23 MB, 6.60 MB
ThreadUsage(16s000ms): 9, 9, 10, 11, 11, 12, 12, 10, 10, 8, 7, 6, 6, 6, 7, 7, 7, 7, 8, 8, 9, 8, 10, 8, 8, 7, 9, 8, 7, 7, 7, 8, 7, 9, 10, 10, 8, 10, 7, 4
- AverageThreadTokens: 8.78
- PeakMemoryUsage: 78.90 MB (82732648)
- PerHostPeakMemUsage: 78.90 MB (82732648)
- PrepareTime: 188.846ms
- RowsProduced: 9 (9)
- TotalCpuTime: 1h24m
- TotalNetworkReceiveTime: 5s511ms
- TotalNetworkSendTime: 1.512us
- TotalStorageWaitTime: 1h24m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 649.986ms, non-child: 649.986ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 132.692ms
- LoadTime: 154.256us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 340.599ms
- PrepareTime: 174.906ms
DataStreamSender (dst_id=13):(Total: 180.486us, non-child: 180.486us, % non-child: 100.00%)
- BytesSent: 2.78 KB (2851)
- NetworkThroughput(*): 1.26 MB/sec
- OverallThroughput: 15.06 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 134.588us
- ThriftTransmitTime(*): 2.166ms
- UncompressedRowBatchSize: 4.03 KB (4122)
HASH_JOIN_NODE (id=6):(Total: 10m54s, non-child: 188.951ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.199us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.717us
- GetNewBlockTime: 75.637us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 27.31 MB (28639816)
- PinTime: 0ns
- ProbeRows: 9 (9)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 170.242us
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 725ns
EXCHANGE_NODE (id=12):(Total: 2s626ms, non-child: 22.826us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.814us
- DeserializeRowBatchTimer: 16.601us
- FirstBatchArrivalWaitTime: 2s623ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m53s, non-child: 2s903ms, % non-child: 0.44%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 16.35us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.585us
- GetNewBlockTime: 59.12us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 18.04 MB (18919352)
- PinTime: 0ns
- ProbeRows: 9 (9)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 162.961us
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 709ns
EXCHANGE_NODE (id=9):(Total: 2s884ms, non-child: 40.424us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.373us
- DeserializeRowBatchTimer: 18.71us
- FirstBatchArrivalWaitTime: 2s875ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m51s, non-child: 10m51s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:30/86.30 MB 1:49/158.29 MB 2:50/137.67 MB 3:59/146.99 MB 4:56/159.27 MB 5:49/164.38 MB 6:49/178.22 MB 7:39/105.73 MB 8:32/101.42 MB 9:50/173.77 MB 10:45/101.17 MB 11:47/182.35 MB
Hdfs Read Thread Concurrency Bucket: 0:98.62% 1:1.377% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:26085
ExecOption: Codegen enabled: 0 out of 555
BytesRead(16s000ms): 6.51 MB, 28.73 MB, 47.92 MB, 67.18 MB, 82.80 MB, 106.42 MB, 127.39 MB, 141.05 MB, 163.97 MB, 177.35 MB, 190.33 MB, 212.95 MB, 235.45 MB, 264.38 MB, 292.94 MB, 321.74 MB, 355.91 MB, 393.62 MB, 409.37 MB, 420.36 MB, 438.27 MB, 457.13 MB, 479.19 MB, 500.29 MB, 516.10 MB, 527.74 MB, 546.77 MB, 567.13 MB, 583.09 MB, 595.88 MB, 617.28 MB, 650.35 MB, 671.80 MB, 707.14 MB, 765.21 MB, 796.81 MB, 815.97 MB, 821.09 MB, 827.37 MB, 829.13 MB
- AverageHdfsReadThreadConcurrency: 0.01
- AverageScannerThreadConcurrency: 7.78
- BytesRead: 831.00 MB (871361911)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 831.00 MB (871361911)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 831.00 MB (871361911)
- DecompressionTime: 532.690ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 113 (113)
- PeakMemoryUsage: 73.27 MB (76831264)
- PerReadThreadRawHdfsThroughput: 100.04 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.65M (11651632)
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- ScanRangesComplete: 555 (555)
- ScannerThreadsInvoluntaryContextSwitches: 563 (563)
- ScannerThreadsTotalWallClockTime: 1h24m
- MaterializeTupleTime(*): 15s229ms
- ScannerThreadsSysTime: 468.5ms
- ScannerThreadsUserTime: 18s041ms
- ScannerThreadsVoluntaryContextSwitches: 49.44K (49441)
- TotalRawHdfsReadTime(*): 8s306ms
- TotalReadThroughput: 1.27 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016cc (host=hadoopc1h72:28002):(Total: 10m48s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:50/143.44 MB 1:60/148.85 MB 2:46/108.35 MB 3:41/115.63 MB 4:46/111.34 MB 5:51/146.67 MB 6:50/124.81 MB 7:65/173.61 MB 8:57/172.68 MB 9:50/127.86 MB 10:47/125.99 MB 11:66/175.58 MB
MemoryUsage(16s000ms): 29.80 MB, 43.37 MB, 39.00 MB, 34.66 MB, 46.39 MB, 44.65 MB, 50.97 MB, 52.90 MB, 45.89 MB, 52.15 MB, 38.74 MB, 47.52 MB, 35.99 MB, 43.99 MB, 54.69 MB, 39.97 MB, 36.15 MB, 35.66 MB, 29.31 MB, 35.54 MB, 34.39 MB, 49.33 MB, 40.04 MB, 30.94 MB, 36.40 MB, 60.81 MB, 47.59 MB, 34.03 MB, 32.17 MB, 34.16 MB, 35.32 MB, 36.41 MB, 28.32 MB, 24.54 MB, 27.50 MB, 22.41 MB, 22.46 MB, 16.01 MB, 13.22 MB, 10.50 MB
ThreadUsage(16s000ms): 11, 11, 13, 12, 13, 15, 17, 14, 12, 11, 11, 10, 10, 13, 13, 12, 10, 10, 11, 11, 12, 13, 15, 10, 11, 11, 12, 11, 9, 9, 11, 13, 13, 17, 19, 17, 13, 11, 7, 4
- AverageThreadTokens: 12.45
- PeakMemoryUsage: 85.81 MB (89980512)
- PerHostPeakMemUsage: 85.81 MB (89980512)
- PrepareTime: 205.496ms
- RowsProduced: 21 (21)
- TotalCpuTime: 2h3m
- TotalNetworkReceiveTime: 5s354ms
- TotalNetworkSendTime: 1.824us
- TotalStorageWaitTime: 2h3m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 708.748ms, non-child: 708.748ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 138.777ms
- LoadTime: 149.62us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 378.36ms
- PrepareTime: 190.127ms
DataStreamSender (dst_id=13):(Total: 319.386us, non-child: 319.386us, % non-child: 100.00%)
- BytesSent: 6.07 KB (6211)
- NetworkThroughput(*): 2.17 MB/sec
- OverallThroughput: 18.55 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 227.1us
- ThriftTransmitTime(*): 2.726ms
- UncompressedRowBatchSize: 9.42 KB (9642)
HASH_JOIN_NODE (id=6):(Total: 10m48s, non-child: 205.93ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.87us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.976us
- GetNewBlockTime: 142.770us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.62 MB (25819800)
- PinTime: 0ns
- ProbeRows: 21 (21)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 175.819us
- RowsReturned: 21 (21)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.325us
EXCHANGE_NODE (id=12):(Total: 2s550ms, non-child: 19.641us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.719us
- DeserializeRowBatchTimer: 10.927us
- FirstBatchArrivalWaitTime: 2s550ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m48s, non-child: 2s832ms, % non-child: 0.44%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.244us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.695us
- GetNewBlockTime: 152.480us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 13.65 MB (14308992)
- PinTime: 0ns
- ProbeRows: 21 (21)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 179.80us
- RowsReturned: 21 (21)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.87us
EXCHANGE_NODE (id=9):(Total: 2s804ms, non-child: 20.699us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.304us
- DeserializeRowBatchTimer: 7.880us
- FirstBatchArrivalWaitTime: 2s802ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m45s, non-child: 10m45s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:50/143.44 MB 1:60/148.85 MB 2:46/108.35 MB 3:41/115.63 MB 4:46/111.34 MB 5:51/146.67 MB 6:50/124.81 MB 7:65/173.61 MB 8:57/172.68 MB 9:50/127.86 MB 10:47/125.99 MB 11:66/175.58 MB
Hdfs Read Thread Concurrency Bucket: 0:98.38% 1:1.619% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:29563
ExecOption: Codegen enabled: 0 out of 629
BytesRead(16s000ms): 11.87 MB, 42.24 MB, 63.02 MB, 77.68 MB, 94.62 MB, 112.85 MB, 137.86 MB, 158.79 MB, 180.24 MB, 200.58 MB, 218.08 MB, 242.20 MB, 268.25 MB, 288.65 MB, 315.85 MB, 345.81 MB, 375.92 MB, 403.36 MB, 419.18 MB, 435.90 MB, 463.36 MB, 498.49 MB, 521.67 MB, 545.33 MB, 567.13 MB, 593.76 MB, 615.78 MB, 635.58 MB, 659.02 MB, 680.16 MB, 700.71 MB, 732.04 MB, 756.41 MB, 772.86 MB, 791.36 MB, 802.94 MB, 811.09 MB, 815.71 MB, 821.06 MB, 824.31 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 11.46
- BytesRead: 828.36 MB (868594347)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 828.36 MB (868594347)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 828.36 MB (868594347)
- DecompressionTime: 1s478ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 173 (173)
- PeakMemoryUsage: 75.82 MB (79500416)
- PerReadThreadRawHdfsThroughput: 67.86 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.43M (11433575)
- RowsReturned: 21 (21)
- RowsReturnedRate: 0
- ScanRangesComplete: 629 (629)
- ScannerThreadsInvoluntaryContextSwitches: 536 (536)
- ScannerThreadsTotalWallClockTime: 2h3m
- MaterializeTupleTime(*): 15s439ms
- ScannerThreadsSysTime: 476.7ms
- ScannerThreadsUserTime: 18s257ms
- ScannerThreadsVoluntaryContextSwitches: 48.36K (48358)
- TotalRawHdfsReadTime(*): 12s206ms
- TotalReadThroughput: 1.28 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016b7 (host=hadoopc1h20:28002):(Total: 10m46s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:61/160.39 MB 1:61/164.59 MB 2:48/121.00 MB 3:51/147.86 MB 4:58/155.00 MB 5:56/151.00 MB 6:51/155.48 MB 7:50/134.56 MB 8:54/147.07 MB 9:63/177.06 MB 10:55/153.68 MB
MemoryUsage(16s000ms): 24.05 MB, 45.41 MB, 31.38 MB, 41.65 MB, 40.76 MB, 46.11 MB, 51.47 MB, 44.51 MB, 46.69 MB, 29.65 MB, 42.88 MB, 26.59 MB, 30.21 MB, 32.50 MB, 36.46 MB, 24.78 MB, 29.50 MB, 28.82 MB, 27.13 MB, 31.02 MB, 27.80 MB, 37.07 MB, 50.50 MB, 20.67 MB, 25.74 MB, 25.97 MB, 29.95 MB, 32.42 MB, 36.49 MB, 29.87 MB, 23.47 MB, 36.70 MB, 35.17 MB, 33.96 MB, 29.56 MB, 20.90 MB, 26.40 MB, 21.29 MB, 18.39 MB, 11.17 MB
ThreadUsage(16s000ms): 9, 10, 11, 11, 13, 11, 12, 10, 10, 8, 7, 7, 7, 7, 8, 8, 7, 7, 9, 8, 8, 9, 10, 7, 8, 8, 9, 9, 7, 7, 7, 9, 10, 11, 12, 11, 11, 17, 11, 4
- AverageThreadTokens: 9.61
- PeakMemoryUsage: 69.40 MB (72775744)
- PerHostPeakMemUsage: 69.40 MB (72775744)
- PrepareTime: 181.918ms
- RowsProduced: 17 (17)
- TotalCpuTime: 1h32m
- TotalNetworkReceiveTime: 5s514ms
- TotalNetworkSendTime: 2.231us
- TotalStorageWaitTime: 1h32m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 648.738ms, non-child: 648.738ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 134.444ms
- LoadTime: 147.500us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 344.689ms
- PrepareTime: 167.958ms
DataStreamSender (dst_id=13):(Total: 327.32us, non-child: 327.32us, % non-child: 100.00%)
- BytesSent: 5.19 KB (5316)
- NetworkThroughput(*): 1.40 MB/sec
- OverallThroughput: 15.50 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 239.721us
- ThriftTransmitTime(*): 3.619ms
- UncompressedRowBatchSize: 7.65 KB (7832)
HASH_JOIN_NODE (id=6):(Total: 10m46s, non-child: 182.379ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.721us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.187us
- GetNewBlockTime: 88.357us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 22.38 MB (23467672)
- PinTime: 0ns
- ProbeRows: 17 (17)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 247.977us
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 900ns
EXCHANGE_NODE (id=12):(Total: 2s629ms, non-child: 22.5us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.752us
- DeserializeRowBatchTimer: 19.647us
- FirstBatchArrivalWaitTime: 2s622ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m46s, non-child: 2s901ms, % non-child: 0.45%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.275us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.120us
- GetNewBlockTime: 55.150us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 14.09 MB (14771416)
- PinTime: 0ns
- ProbeRows: 17 (17)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 180.596us
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.30us
EXCHANGE_NODE (id=9):(Total: 2s884ms, non-child: 22.743us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.452us
- DeserializeRowBatchTimer: 14.650us
- FirstBatchArrivalWaitTime: 2s874ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m43s, non-child: 10m43s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:61/160.39 MB 1:61/164.59 MB 2:48/121.00 MB 3:51/147.86 MB 4:58/155.00 MB 5:56/151.00 MB 6:51/155.48 MB 7:50/134.56 MB 8:54/147.07 MB 9:63/177.06 MB 10:55/153.68 MB
Hdfs Read Thread Concurrency Bucket: 0:98.14% 1:1.856% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28576
ExecOption: Codegen enabled: 0 out of 608
BytesRead(16s000ms): 7.78 MB, 32.57 MB, 50.96 MB, 68.31 MB, 81.23 MB, 99.80 MB, 124.35 MB, 142.57 MB, 165.06 MB, 178.54 MB, 198.06 MB, 220.29 MB, 243.91 MB, 267.91 MB, 293.50 MB, 318.09 MB, 350.31 MB, 379.40 MB, 391.30 MB, 403.17 MB, 418.53 MB, 436.33 MB, 463.76 MB, 480.05 MB, 493.34 MB, 507.66 MB, 520.05 MB, 540.25 MB, 575.13 MB, 595.89 MB, 613.51 MB, 640.64 MB, 670.53 MB, 705.88 MB, 749.93 MB, 771.51 MB, 793.28 MB, 806.20 MB, 815.31 MB, 821.55 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 8.61
- BytesRead: 823.13 MB (863116930)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 823.13 MB (863116930)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 823.13 MB (863116930)
- DecompressionTime: 576.926ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 11 (11)
- NumScannerThreadsStarted: 131 (131)
- PeakMemoryUsage: 68.82 MB (72165376)
- PerReadThreadRawHdfsThroughput: 77.71 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.39M (11394863)
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- ScanRangesComplete: 608 (608)
- ScannerThreadsInvoluntaryContextSwitches: 476 (476)
- ScannerThreadsTotalWallClockTime: 1h32m
- MaterializeTupleTime(*): 14s751ms
- ScannerThreadsSysTime: 464.8ms
- ScannerThreadsUserTime: 17s593ms
- ScannerThreadsVoluntaryContextSwitches: 51.64K (51640)
- TotalRawHdfsReadTime(*): 10s592ms
- TotalReadThroughput: 1.27 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016b1 (host=hadoopc1h86:28002):(Total: 10m38s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:54/135.10 MB 1:46/134.39 MB 2:61/152.77 MB 3:45/110.44 MB 4:48/154.66 MB 5:43/132.41 MB 6:54/151.52 MB 7:45/110.10 MB 8:58/143.76 MB 9:49/134.62 MB 10:58/150.68 MB 11:55/137.76 MB
MemoryUsage(16s000ms): 28.63 MB, 43.11 MB, 49.25 MB, 42.18 MB, 39.64 MB, 59.16 MB, 53.12 MB, 47.06 MB, 49.59 MB, 44.64 MB, 42.30 MB, 37.39 MB, 45.30 MB, 59.11 MB, 51.23 MB, 33.62 MB, 33.13 MB, 39.58 MB, 43.23 MB, 44.09 MB, 46.53 MB, 53.01 MB, 38.59 MB, 37.42 MB, 36.56 MB, 36.63 MB, 37.07 MB, 47.26 MB, 37.35 MB, 33.95 MB, 30.84 MB, 37.84 MB, 47.42 MB, 30.90 MB, 30.19 MB, 21.23 MB, 18.95 MB, 14.82 MB, 16.96 MB
ThreadUsage(16s000ms): 11, 14, 13, 14, 14, 15, 15, 12, 13, 11, 11, 10, 11, 12, 13, 11, 9, 13, 14, 13, 14, 14, 13, 13, 11, 11, 12, 13, 10, 10, 11, 13, 14, 17, 19, 17, 15, 11, 6
- AverageThreadTokens: 13.12
- PeakMemoryUsage: 82.18 MB (86176480)
- PerHostPeakMemUsage: 82.18 MB (86176480)
- PrepareTime: 194.188ms
- RowsProduced: 14 (14)
- TotalCpuTime: 2h8m
- TotalNetworkReceiveTime: 5s442ms
- TotalNetworkSendTime: 1.865us
- TotalStorageWaitTime: 2h8m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 683.807ms, non-child: 683.807ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 134.908ms
- LoadTime: 147.652us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 367.983ms
- PrepareTime: 179.40ms
DataStreamSender (dst_id=13):(Total: 286.957us, non-child: 286.957us, % non-child: 100.00%)
- BytesSent: 4.38 KB (4487)
- NetworkThroughput(*): 1.88 MB/sec
- OverallThroughput: 14.91 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 200.833us
- ThriftTransmitTime(*): 2.274ms
- UncompressedRowBatchSize: 6.34 KB (6497)
HASH_JOIN_NODE (id=6):(Total: 10m38s, non-child: 194.316ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.397us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.324us
- GetNewBlockTime: 75.331us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 20.14 MB (21113200)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 182.122us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 801ns
EXCHANGE_NODE (id=12):(Total: 2s594ms, non-child: 24.636us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.514us
- DeserializeRowBatchTimer: 15.786us
- FirstBatchArrivalWaitTime: 2s583ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m37s, non-child: 2s867ms, % non-child: 0.45%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.463us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.614us
- GetNewBlockTime: 66.879us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.03 MB (15755016)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 183.479us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 830ns
EXCHANGE_NODE (id=9):(Total: 2s848ms, non-child: 22.156us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.645us
- DeserializeRowBatchTimer: 18.280us
- FirstBatchArrivalWaitTime: 2s835ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m35s, non-child: 10m35s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:54/135.10 MB 1:46/134.39 MB 2:61/152.77 MB 3:45/110.44 MB 4:48/154.66 MB 5:43/132.41 MB 6:54/151.52 MB 7:45/110.10 MB 8:58/143.76 MB 9:49/134.62 MB 10:58/150.68 MB 11:55/137.76 MB
Hdfs Read Thread Concurrency Bucket: 0:97.57% 1:2.353% 2:0.07843% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28952
ExecOption: Codegen enabled: 0 out of 616
BytesRead(16s000ms): 8.90 MB, 31.14 MB, 53.05 MB, 70.38 MB, 81.14 MB, 102.80 MB, 127.05 MB, 143.83 MB, 167.04 MB, 181.98 MB, 197.87 MB, 219.90 MB, 245.74 MB, 271.22 MB, 297.08 MB, 323.10 MB, 352.16 MB, 376.90 MB, 389.96 MB, 402.27 MB, 425.92 MB, 456.81 MB, 475.88 MB, 496.20 MB, 515.03 MB, 531.00 MB, 548.05 MB, 572.43 MB, 605.14 MB, 629.59 MB, 653.54 MB, 683.59 MB, 719.75 MB, 745.52 MB, 770.25 MB, 785.00 MB, 797.87 MB, 803.86 MB, 811.39 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 12.13
- BytesRead: 816.04 MB (855676887)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 816.04 MB (855676887)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 816.04 MB (855676887)
- DecompressionTime: 1s045ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 163 (163)
- PeakMemoryUsage: 70.81 MB (74246536)
- PerReadThreadRawHdfsThroughput: 64.38 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.24M (11235691)
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- ScanRangesComplete: 616 (616)
- ScannerThreadsInvoluntaryContextSwitches: 422 (422)
- ScannerThreadsTotalWallClockTime: 2h8m
- MaterializeTupleTime(*): 14s821ms
- ScannerThreadsSysTime: 540.5ms
- ScannerThreadsUserTime: 17s529ms
- ScannerThreadsVoluntaryContextSwitches: 45.04K (45038)
- TotalRawHdfsReadTime(*): 12s675ms
- TotalReadThroughput: 1.28 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016b9 (host=hadoopc1h89:28002):(Total: 10m37s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:46/114.28 MB 1:44/115.46 MB 2:53/141.03 MB 3:57/158.88 MB 4:50/126.19 MB 5:52/161.63 MB 6:50/167.85 MB 7:53/150.04 MB 8:53/151.99 MB 9:62/195.16 MB 10:52/140.14 MB 11:36/88.22 MB
MemoryUsage(16s000ms): 21.83 MB, 47.70 MB, 41.45 MB, 43.77 MB, 44.89 MB, 48.17 MB, 52.80 MB, 44.47 MB, 39.03 MB, 31.41 MB, 34.75 MB, 38.90 MB, 36.02 MB, 45.38 MB, 40.59 MB, 44.20 MB, 25.02 MB, 36.56 MB, 30.02 MB, 32.37 MB, 38.56 MB, 30.89 MB, 52.04 MB, 29.28 MB, 34.05 MB, 39.79 MB, 44.70 MB, 52.83 MB, 34.98 MB, 28.92 MB, 26.49 MB, 35.63 MB, 36.41 MB, 34.39 MB, 34.78 MB, 28.44 MB, 13.87 MB, 9.06 MB, 13.81 MB
ThreadUsage(16s000ms): 11, 12, 13, 14, 15, 16, 15, 13, 13, 11, 11, 11, 10, 12, 11, 11, 10, 13, 14, 14, 14, 13, 15, 12, 11, 13, 13, 13, 11, 10, 11, 11, 9, 10, 13, 13, 8, 6, 4
- AverageThreadTokens: 12.23
- PeakMemoryUsage: 85.85 MB (90015816)
- PerHostPeakMemUsage: 85.85 MB (90015816)
- PrepareTime: 195.635ms
- RowsProduced: 3 (3)
- TotalCpuTime: 1h59m
- TotalNetworkReceiveTime: 5s421ms
- TotalNetworkSendTime: 499ns
- TotalStorageWaitTime: 1h58m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 685.728ms, non-child: 685.728ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 139.460ms
- LoadTime: 146.339us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 364.274ms
- PrepareTime: 180.131ms
DataStreamSender (dst_id=13):(Total: 84.901us, non-child: 84.901us, % non-child: 100.00%)
- BytesSent: 893.00 B (893)
- NetworkThroughput(*): 2.61 MB/sec
- OverallThroughput: 10.03 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 57.931us
- ThriftTransmitTime(*): 325.997us
- UncompressedRowBatchSize: 1.35 KB (1384)
HASH_JOIN_NODE (id=6):(Total: 10m37s, non-child: 193.312ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 5.605us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.52us
- GetNewBlockTime: 58.537us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 18.13 MB (19013408)
- PinTime: 0ns
- ProbeRows: 3 (3)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 35.440us
- RowsReturned: 3 (3)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 838ns
EXCHANGE_NODE (id=12):(Total: 2s580ms, non-child: 20.842us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.188us
- DeserializeRowBatchTimer: 15.837us
- FirstBatchArrivalWaitTime: 2s577ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m37s, non-child: 2s897ms, % non-child: 0.45%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 9.23us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 7.82us
- GetNewBlockTime: 60.217us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.09 MB (15818992)
- PinTime: 0ns
- ProbeRows: 3 (3)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 36.885us
- RowsReturned: 3 (3)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 859ns
EXCHANGE_NODE (id=9):(Total: 2s841ms, non-child: 26.254us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.148us
- DeserializeRowBatchTimer: 14.428us
- FirstBatchArrivalWaitTime: 2s829ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m34s, non-child: 10m34s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:46/114.28 MB 1:44/115.46 MB 2:53/141.03 MB 3:57/158.88 MB 4:50/126.19 MB 5:52/161.63 MB 6:50/167.85 MB 7:53/150.04 MB 8:53/151.99 MB 9:62/195.16 MB 10:52/140.14 MB 11:36/88.22 MB
Hdfs Read Thread Concurrency Bucket: 0:97.72% 1:2.2% 2:0.07855% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28576
ExecOption: Codegen enabled: 0 out of 608
BytesRead(16s000ms): 7.14 MB, 37.63 MB, 63.40 MB, 81.64 MB, 96.11 MB, 115.12 MB, 140.25 MB, 157.72 MB, 176.41 MB, 188.36 MB, 200.69 MB, 220.50 MB, 245.95 MB, 275.34 MB, 302.72 MB, 335.59 MB, 367.14 MB, 394.62 MB, 404.66 MB, 413.96 MB, 431.89 MB, 451.59 MB, 476.37 MB, 497.44 MB, 518.26 MB, 537.89 MB, 554.87 MB, 582.95 MB, 617.52 MB, 638.46 MB, 656.65 MB, 688.72 MB, 725.35 MB, 761.27 MB, 801.22 MB, 822.41 MB, 836.20 MB, 838.98 MB, 840.96 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 11.24
- BytesRead: 843.42 MB (884385498)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 843.42 MB (884385498)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 843.42 MB (884385498)
- DecompressionTime: 381.123ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 153 (153)
- PeakMemoryUsage: 85.29 MB (89438216)
- PerReadThreadRawHdfsThroughput: 61.84 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.69M (11688328)
- RowsReturned: 3 (3)
- RowsReturnedRate: 0
- ScanRangesComplete: 608 (608)
- ScannerThreadsInvoluntaryContextSwitches: 860 (860)
- ScannerThreadsTotalWallClockTime: 1h59m
- MaterializeTupleTime(*): 16s987ms
- ScannerThreadsSysTime: 508.7ms
- ScannerThreadsUserTime: 19s877ms
- ScannerThreadsVoluntaryContextSwitches: 47.09K (47090)
- TotalRawHdfsReadTime(*): 13s638ms
- TotalReadThroughput: 1.32 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016bc (host=hadoopc1h13:28002):(Total: 10m32s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:47/157.32 MB 1:47/133.45 MB 2:31/106.47 MB 3:48/135.15 MB 4:53/176.94 MB 5:57/174.59 MB 6:38/91.89 MB 7:46/154.36 MB 8:41/113.52 MB 9:49/149.74 MB 10:43/133.14 MB 11:54/161.05 MB
MemoryUsage(16s000ms): 26.40 MB, 27.34 MB, 45.04 MB, 39.93 MB, 38.86 MB, 50.76 MB, 54.30 MB, 39.94 MB, 38.04 MB, 40.67 MB, 35.00 MB, 34.89 MB, 23.38 MB, 30.86 MB, 33.35 MB, 30.17 MB, 29.89 MB, 31.44 MB, 34.55 MB, 47.78 MB, 41.15 MB, 40.26 MB, 38.28 MB, 37.61 MB, 35.02 MB, 34.10 MB, 40.60 MB, 39.15 MB, 23.18 MB, 32.30 MB, 37.38 MB, 32.76 MB, 30.34 MB, 31.73 MB, 25.27 MB, 19.67 MB, 19.20 MB, 13.09 MB, 11.89 MB
ThreadUsage(16s000ms): 9, 8, 10, 10, 10, 11, 12, 10, 10, 9, 7, 6, 6, 6, 6, 7, 6, 7, 8, 9, 9, 8, 10, 7, 7, 7, 9, 8, 7, 7, 6, 7, 7, 9, 9, 9, 8, 9, 4
- AverageThreadTokens: 8.66
- PeakMemoryUsage: 73.71 MB (77295384)
- PerHostPeakMemUsage: 73.71 MB (77295384)
- PrepareTime: 188.519ms
- RowsProduced: 12 (12)
- TotalCpuTime: 1h20m
- TotalNetworkReceiveTime: 5s545ms
- TotalNetworkSendTime: 1.321us
- TotalStorageWaitTime: 1h20m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 626.510ms, non-child: 626.510ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 121.434ms
- LoadTime: 114.938us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 328.741ms
- PrepareTime: 174.644ms
DataStreamSender (dst_id=13):(Total: 195.304us, non-child: 195.304us, % non-child: 100.00%)
- BytesSent: 3.69 KB (3783)
- NetworkThroughput(*): 1.51 MB/sec
- OverallThroughput: 18.47 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 140.90us
- ThriftTransmitTime(*): 2.381ms
- UncompressedRowBatchSize: 5.38 KB (5507)
HASH_JOIN_NODE (id=6):(Total: 10m32s, non-child: 188.877ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.7us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.254us
- GetNewBlockTime: 56.885us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.72 MB (25922672)
- PinTime: 0ns
- ProbeRows: 12 (12)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 196.239us
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 948ns
EXCHANGE_NODE (id=12):(Total: 2s644ms, non-child: 22.439us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.803us
- DeserializeRowBatchTimer: 10.585us
- FirstBatchArrivalWaitTime: 2s643ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m32s, non-child: 2s917ms, % non-child: 0.46%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.20us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.641us
- GetNewBlockTime: 52.135us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 14.12 MB (14805928)
- PinTime: 0ns
- ProbeRows: 12 (12)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 173.581us
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 732ns
EXCHANGE_NODE (id=9):(Total: 2s901ms, non-child: 22.85us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.558us
- DeserializeRowBatchTimer: 12.357us
- FirstBatchArrivalWaitTime: 2s895ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m29s, non-child: 10m29s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:47/157.32 MB 1:47/133.45 MB 2:31/106.47 MB 3:48/135.15 MB 4:53/176.94 MB 5:57/174.59 MB 6:38/91.89 MB 7:46/154.36 MB 8:41/113.52 MB 9:49/149.74 MB 10:43/133.14 MB 11:54/161.05 MB
Hdfs Read Thread Concurrency Bucket: 0:98.18% 1:1.741% 2:0.07911% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:26038
ExecOption: Codegen enabled: 0 out of 554
BytesRead(16s000ms): 8.50 MB, 28.79 MB, 51.14 MB, 71.36 MB, 86.26 MB, 109.68 MB, 137.02 MB, 154.68 MB, 172.23 MB, 189.56 MB, 209.10 MB, 237.80 MB, 264.08 MB, 285.92 MB, 314.00 MB, 343.60 MB, 373.36 MB, 401.43 MB, 416.96 MB, 436.35 MB, 464.28 MB, 488.74 MB, 507.97 MB, 529.71 MB, 549.55 MB, 566.81 MB, 582.46 MB, 605.65 MB, 623.61 MB, 640.15 MB, 662.27 MB, 695.25 MB, 717.99 MB, 751.91 MB, 791.64 MB, 807.05 MB, 814.72 MB, 820.11 MB, 824.03 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 7.66
- BytesRead: 827.23 MB (867409425)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 827.23 MB (867409425)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 827.23 MB (867409425)
- DecompressionTime: 509.597ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 131 (131)
- PeakMemoryUsage: 67.69 MB (70973888)
- PerReadThreadRawHdfsThroughput: 68.04 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.60M (11595737)
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- ScanRangesComplete: 554 (554)
- ScannerThreadsInvoluntaryContextSwitches: 589 (589)
- ScannerThreadsTotalWallClockTime: 1h20m
- MaterializeTupleTime(*): 15s054ms
- ScannerThreadsSysTime: 464.10ms
- ScannerThreadsUserTime: 17s781ms
- ScannerThreadsVoluntaryContextSwitches: 49.20K (49201)
- TotalRawHdfsReadTime(*): 12s158ms
- TotalReadThroughput: 1.31 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016a3 (host=hadoopc1h142:28002):(Total: 10m32s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:62/152.63 MB 1:46/141.40 MB 2:46/124.86 MB 3:51/182.04 MB 4:31/95.62 MB 5:50/153.50 MB 6:37/105.86 MB 7:57/153.61 MB 8:41/152.90 MB 9:48/151.99 MB 10:43/122.92 MB 11:57/162.84 MB
MemoryUsage(16s000ms): 37.35 MB, 41.86 MB, 36.95 MB, 65.18 MB, 65.16 MB, 52.29 MB, 53.82 MB, 52.57 MB, 41.65 MB, 39.33 MB, 35.34 MB, 34.36 MB, 23.84 MB, 34.82 MB, 32.78 MB, 47.39 MB, 40.37 MB, 33.46 MB, 30.68 MB, 46.27 MB, 35.77 MB, 32.25 MB, 48.46 MB, 34.37 MB, 30.00 MB, 36.88 MB, 41.54 MB, 40.43 MB, 21.85 MB, 31.25 MB, 26.14 MB, 34.08 MB, 36.55 MB, 25.42 MB, 29.17 MB, 25.93 MB, 12.32 MB, 7.34 MB, 8.73 MB
ThreadUsage(16s000ms): 11, 11, 12, 14, 14, 15, 15, 12, 12, 11, 10, 7, 8, 7, 8, 9, 8, 10, 10, 9, 10, 11, 12, 10, 10, 10, 10, 10, 8, 8, 9, 10, 9, 12, 13, 14, 9, 6, 4
- AverageThreadTokens: 10.78
- PeakMemoryUsage: 88.48 MB (92779848)
- PerHostPeakMemUsage: 88.48 MB (92779848)
- PrepareTime: 199.5ms
- RowsProduced: 16 (16)
- TotalCpuTime: 1h43m
- TotalNetworkReceiveTime: 5s284ms
- TotalNetworkSendTime: 1.482us
- TotalStorageWaitTime: 1h42m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 710.340ms, non-child: 710.340ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 151.645ms
- LoadTime: 147.353us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 372.958ms
- PrepareTime: 183.819ms
DataStreamSender (dst_id=13):(Total: 244.722us, non-child: 244.722us, % non-child: 100.00%)
- BytesSent: 4.76 KB (4870)
- NetworkThroughput(*): 2.23 MB/sec
- OverallThroughput: 18.98 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 173.274us
- ThriftTransmitTime(*): 2.79ms
- UncompressedRowBatchSize: 7.18 KB (7349)
HASH_JOIN_NODE (id=6):(Total: 10m32s, non-child: 199.124ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.412us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.606us
- GetNewBlockTime: 59.796us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.23 MB (25407320)
- PinTime: 0ns
- ProbeRows: 16 (16)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 175.211us
- RowsReturned: 16 (16)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 732ns
EXCHANGE_NODE (id=12):(Total: 2s517ms, non-child: 22.948us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.948us
- DeserializeRowBatchTimer: 20.34us
- FirstBatchArrivalWaitTime: 2s513ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m32s, non-child: 2s784ms, % non-child: 0.44%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 11.294us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 6.741us
- GetNewBlockTime: 56.161us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 14.41 MB (15108128)
- PinTime: 0ns
- ProbeRows: 16 (16)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 196.731us
- RowsReturned: 16 (16)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 767ns
EXCHANGE_NODE (id=9):(Total: 2s766ms, non-child: 29.529us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.355us
- DeserializeRowBatchTimer: 17.656us
- FirstBatchArrivalWaitTime: 2s766ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m29s, non-child: 10m29s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:62/152.63 MB 1:46/141.40 MB 2:46/124.86 MB 3:51/182.04 MB 4:31/95.62 MB 5:50/153.50 MB 6:37/105.86 MB 7:57/153.61 MB 8:41/152.90 MB 9:48/151.99 MB 10:43/122.92 MB 11:57/162.84 MB
Hdfs Read Thread Concurrency Bucket: 0:98.42% 1:1.504% 2:0.07918% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:26743
ExecOption: Codegen enabled: 0 out of 569
BytesRead(16s000ms): 10.98 MB, 40.75 MB, 61.37 MB, 82.84 MB, 102.95 MB, 125.37 MB, 150.47 MB, 172.04 MB, 193.11 MB, 208.82 MB, 225.22 MB, 246.83 MB, 262.99 MB, 282.03 MB, 307.25 MB, 344.31 MB, 380.67 MB, 411.91 MB, 424.70 MB, 439.86 MB, 463.31 MB, 486.78 MB, 511.99 MB, 533.64 MB, 554.32 MB, 570.36 MB, 589.38 MB, 616.48 MB, 644.26 MB, 663.17 MB, 686.67 MB, 714.47 MB, 743.25 MB, 773.52 MB, 801.86 MB, 819.07 MB, 829.76 MB, 831.64 MB, 833.54 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 9.79
- BytesRead: 835.26 MB (875833991)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 835.26 MB (875833991)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 835.26 MB (875833991)
- DecompressionTime: 2s250ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 145 (145)
- PeakMemoryUsage: 80.18 MB (84073880)
- PerReadThreadRawHdfsThroughput: 64.98 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.66M (11658834)
- RowsReturned: 16 (16)
- RowsReturnedRate: 0
- ScanRangesComplete: 569 (569)
- ScannerThreadsInvoluntaryContextSwitches: 464 (464)
- ScannerThreadsTotalWallClockTime: 1h43m
- MaterializeTupleTime(*): 16s146ms
- ScannerThreadsSysTime: 564.12ms
- ScannerThreadsUserTime: 18s777ms
- ScannerThreadsVoluntaryContextSwitches: 43.24K (43236)
- TotalRawHdfsReadTime(*): 12s854ms
- TotalReadThroughput: 1.32 MB/sec
Instance c14f7b9b17987c60:d5379fb954101698 (host=hadoopc1h81:28002):(Total: 10m23s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:65/194.97 MB 1:51/136.52 MB 2:44/121.65 MB 3:41/139.52 MB 4:52/131.89 MB 5:52/147.52 MB 6:56/139.41 MB 7:50/152.46 MB 8:52/148.19 MB 9:47/120.82 MB 10:40/112.25 MB 11:50/137.14 MB
MemoryUsage(16s000ms): 32.06 MB, 53.88 MB, 42.57 MB, 48.14 MB, 45.43 MB, 49.68 MB, 62.35 MB, 46.53 MB, 35.24 MB, 43.50 MB, 47.36 MB, 42.89 MB, 41.81 MB, 48.91 MB, 36.54 MB, 38.59 MB, 26.85 MB, 39.98 MB, 32.51 MB, 51.01 MB, 45.02 MB, 39.91 MB, 50.42 MB, 48.61 MB, 47.44 MB, 38.79 MB, 51.64 MB, 34.43 MB, 30.67 MB, 29.02 MB, 45.30 MB, 29.63 MB, 35.62 MB, 21.82 MB, 16.05 MB, 17.27 MB, 12.06 MB, 9.30 MB, 8.12 MB
ThreadUsage(16s000ms): 15, 17, 17, 16, 15, 18, 20, 16, 14, 17, 15, 12, 14, 13, 12, 12, 10, 11, 12, 15, 14, 14, 15, 10, 11, 11, 13, 12, 9, 9, 10, 13, 16, 17, 12, 10, 7, 5, 2
- AverageThreadTokens: 13.59
- PeakMemoryUsage: 85.18 MB (89320928)
- PerHostPeakMemUsage: 85.18 MB (89320928)
- PrepareTime: 202.478ms
- RowsProduced: 15 (15)
- TotalCpuTime: 2h10m
- TotalNetworkReceiveTime: 5s425ms
- TotalNetworkSendTime: 1.577us
- TotalStorageWaitTime: 2h10m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 696.116ms, non-child: 696.116ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 141.159ms
- LoadTime: 138.779us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 365.924ms
- PrepareTime: 187.235ms
DataStreamSender (dst_id=13):(Total: 277.939us, non-child: 277.939us, % non-child: 100.00%)
- BytesSent: 4.41 KB (4511)
- NetworkThroughput(*): 2.14 MB/sec
- OverallThroughput: 15.48 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 200.494us
- ThriftTransmitTime(*): 2.13ms
- UncompressedRowBatchSize: 6.76 KB (6922)
HASH_JOIN_NODE (id=6):(Total: 10m23s, non-child: 202.974ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 13.12us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.419us
- GetNewBlockTime: 68.322us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.83 MB (27081048)
- PinTime: 0ns
- ProbeRows: 15 (15)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 169.51us
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 765ns
EXCHANGE_NODE (id=12):(Total: 2s585ms, non-child: 26.650us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.397us
- DeserializeRowBatchTimer: 12.418us
- FirstBatchArrivalWaitTime: 2s575ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m23s, non-child: 2s962ms, % non-child: 0.47%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.896us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.445us
- GetNewBlockTime: 58.527us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 16.20 MB (16989664)
- PinTime: 0ns
- ProbeRows: 15 (15)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 172.854us
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 707ns
EXCHANGE_NODE (id=9):(Total: 2s839ms, non-child: 29.591us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.543us
- DeserializeRowBatchTimer: 14.981us
- FirstBatchArrivalWaitTime: 2s827ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m20s, non-child: 10m20s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:65/194.97 MB 1:51/136.52 MB 2:44/121.65 MB 3:41/139.52 MB 4:52/131.89 MB 5:52/147.52 MB 6:56/139.41 MB 7:50/152.46 MB 8:52/148.19 MB 9:47/120.82 MB 10:40/112.25 MB 11:50/137.14 MB
Hdfs Read Thread Concurrency Bucket: 0:97.75% 1:2.245% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28200
ExecOption: Codegen enabled: 0 out of 600
BytesRead(16s000ms): 9.43 MB, 39.14 MB, 60.66 MB, 79.15 MB, 95.96 MB, 116.81 MB, 139.49 MB, 155.87 MB, 170.81 MB, 184.61 MB, 198.33 MB, 218.24 MB, 244.92 MB, 268.75 MB, 292.33 MB, 328.61 MB, 357.20 MB, 396.02 MB, 415.59 MB, 436.66 MB, 463.58 MB, 492.80 MB, 519.96 MB, 554.61 MB, 582.21 MB, 597.45 MB, 617.43 MB, 641.43 MB, 670.35 MB, 697.69 MB, 731.53 MB, 764.99 MB, 790.49 MB, 804.97 MB, 813.49 MB, 817.05 MB, 823.82 MB, 826.96 MB, 829.19 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 12.60
- BytesRead: 829.99 MB (870311821)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 829.99 MB (870311821)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 829.99 MB (870311821)
- DecompressionTime: 1s739ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 151 (151)
- PeakMemoryUsage: 84.63 MB (88743328)
- PerReadThreadRawHdfsThroughput: 67.64 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.50M (11501191)
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- ScanRangesComplete: 600 (600)
- ScannerThreadsInvoluntaryContextSwitches: 620 (620)
- ScannerThreadsTotalWallClockTime: 2h10m
- MaterializeTupleTime(*): 15s659ms
- ScannerThreadsSysTime: 580.8ms
- ScannerThreadsUserTime: 18s473ms
- ScannerThreadsVoluntaryContextSwitches: 46.06K (46056)
- TotalRawHdfsReadTime(*): 12s270ms
- TotalReadThroughput: 1.33 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016ab (host=hadoopc1h87:28002):(Total: 10m19s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:44/115.87 MB 1:56/188.39 MB 2:54/164.11 MB 3:43/120.62 MB 4:47/138.30 MB 5:49/140.06 MB 6:44/151.92 MB 7:46/147.00 MB 8:44/108.91 MB 9:61/150.69 MB 10:48/141.44 MB 11:47/116.35 MB
MemoryUsage(16s000ms): 29.70 MB, 52.20 MB, 49.85 MB, 44.80 MB, 49.01 MB, 48.65 MB, 53.52 MB, 41.54 MB, 55.15 MB, 54.25 MB, 31.92 MB, 32.66 MB, 39.37 MB, 48.62 MB, 45.10 MB, 41.78 MB, 34.97 MB, 40.81 MB, 42.93 MB, 34.61 MB, 44.93 MB, 46.01 MB, 45.44 MB, 44.78 MB, 32.37 MB, 33.03 MB, 42.37 MB, 41.27 MB, 33.63 MB, 38.78 MB, 39.76 MB, 47.82 MB, 34.87 MB, 32.49 MB, 24.44 MB, 18.55 MB, 13.05 MB, 12.26 MB
ThreadUsage(16s000ms): 13, 15, 15, 16, 16, 18, 18, 14, 14, 13, 11, 10, 10, 12, 12, 11, 9, 13, 14, 14, 14, 16, 16, 11, 10, 13, 15, 12, 10, 10, 11, 13, 13, 17, 20, 13, 6, 4
- AverageThreadTokens: 13.45
- PeakMemoryUsage: 80.06 MB (83951568)
- PerHostPeakMemUsage: 80.06 MB (83951568)
- PrepareTime: 199.338ms
- RowsProduced: 26 (26)
- TotalCpuTime: 2h8m
- TotalNetworkReceiveTime: 5s369ms
- TotalNetworkSendTime: 3.990us
- TotalStorageWaitTime: 2h8m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 719.795ms, non-child: 719.795ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 144.254ms
- LoadTime: 143.974us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 389.863ms
- PrepareTime: 183.817ms
DataStreamSender (dst_id=13):(Total: 418.312us, non-child: 418.312us, % non-child: 100.00%)
- BytesSent: 7.45 KB (7626)
- NetworkThroughput(*): 2.10 MB/sec
- OverallThroughput: 17.39 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 300.301us
- ThriftTransmitTime(*): 3.460ms
- UncompressedRowBatchSize: 11.62 KB (11903)
HASH_JOIN_NODE (id=6):(Total: 10m19s, non-child: 199.764ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.877us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.718us
- GetNewBlockTime: 60.961us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 22.18 MB (23260152)
- PinTime: 0ns
- ProbeRows: 26 (26)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 213.467us
- RowsReturned: 26 (26)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 734ns
EXCHANGE_NODE (id=12):(Total: 2s554ms, non-child: 27.287us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.739us
- DeserializeRowBatchTimer: 17.872us
- FirstBatchArrivalWaitTime: 2s549ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m19s, non-child: 3s274ms, % non-child: 0.53%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 9.363us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.621us
- GetNewBlockTime: 67.753us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 13.36 MB (14005432)
- PinTime: 0ns
- ProbeRows: 26 (26)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 208.92us
- RowsReturned: 26 (26)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.363us
EXCHANGE_NODE (id=9):(Total: 2s814ms, non-child: 33.988us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.89us
- DeserializeRowBatchTimer: 17.212us
- FirstBatchArrivalWaitTime: 2s801ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m15s, non-child: 10m15s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:44/115.87 MB 1:56/188.39 MB 2:54/164.11 MB 3:43/120.62 MB 4:47/138.30 MB 5:49/140.06 MB 6:44/151.92 MB 7:46/147.00 MB 8:44/108.91 MB 9:61/150.69 MB 10:48/141.44 MB 11:47/116.35 MB
Hdfs Read Thread Concurrency Bucket: 0:97.74% 1:2.262% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:27401
ExecOption: Codegen enabled: 0 out of 583
BytesRead(16s000ms): 9.16 MB, 37.27 MB, 61.88 MB, 76.84 MB, 90.98 MB, 110.89 MB, 131.09 MB, 146.88 MB, 167.00 MB, 186.06 MB, 197.16 MB, 216.27 MB, 243.04 MB, 273.13 MB, 295.24 MB, 322.56 MB, 359.07 MB, 388.33 MB, 401.05 MB, 411.21 MB, 436.39 MB, 467.69 MB, 494.15 MB, 521.09 MB, 543.16 MB, 556.72 MB, 572.37 MB, 593.63 MB, 623.79 MB, 649.03 MB, 675.07 MB, 719.07 MB, 754.62 MB, 781.60 MB, 803.68 MB, 813.87 MB, 822.71 MB, 825.35 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 12.45
- BytesRead: 828.01 MB (868228706)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 828.01 MB (868228706)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 828.01 MB (868228706)
- DecompressionTime: 981.824ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 156 (156)
- PeakMemoryUsage: 70.19 MB (73598632)
- PerReadThreadRawHdfsThroughput: 57.67 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.54M (11538387)
- RowsReturned: 26 (26)
- RowsReturnedRate: 0
- ScanRangesComplete: 583 (583)
- ScannerThreadsInvoluntaryContextSwitches: 594 (594)
- ScannerThreadsTotalWallClockTime: 2h8m
- MaterializeTupleTime(*): 15s805ms
- ScannerThreadsSysTime: 540.5ms
- ScannerThreadsUserTime: 18s565ms
- ScannerThreadsVoluntaryContextSwitches: 44.50K (44496)
- TotalRawHdfsReadTime(*): 14s356ms
- TotalReadThroughput: 1.34 MB/sec
Instance c14f7b9b17987c60:d5379fb954101699 (host=hadoopc1h85:28002):(Total: 10m7s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:57/163.25 MB 1:53/151.64 MB 2:43/126.89 MB 3:42/125.96 MB 4:42/109.54 MB 5:67/169.34 MB 6:48/143.56 MB 7:51/144.14 MB 8:54/144.24 MB 9:42/95.13 MB 10:55/153.53 MB 11:55/156.66 MB
MemoryUsage(16s000ms): 18.77 MB, 33.98 MB, 39.24 MB, 34.60 MB, 55.97 MB, 37.65 MB, 55.13 MB, 38.83 MB, 48.34 MB, 54.19 MB, 44.81 MB, 52.17 MB, 33.30 MB, 29.57 MB, 37.56 MB, 39.07 MB, 41.24 MB, 47.60 MB, 47.13 MB, 59.81 MB, 43.32 MB, 51.32 MB, 48.59 MB, 54.47 MB, 52.16 MB, 51.58 MB, 53.84 MB, 46.74 MB, 30.64 MB, 34.18 MB, 39.05 MB, 37.33 MB, 30.01 MB, 18.46 MB, 11.74 MB, 12.20 MB, 14.45 MB, 8.87 MB
ThreadUsage(16s000ms): 13, 13, 15, 17, 16, 18, 19, 18, 19, 20, 16, 15, 11, 10, 10, 13, 13, 15, 18, 18, 17, 17, 18, 17, 15, 15, 16, 13, 11, 11, 12, 16, 20, 15, 10, 8, 5, 2
- AverageThreadTokens: 15.01
- PeakMemoryUsage: 87.55 MB (91804736)
- PerHostPeakMemUsage: 87.55 MB (91804736)
- PrepareTime: 193.961ms
- RowsProduced: 9 (9)
- TotalCpuTime: 2h22m
- TotalNetworkReceiveTime: 5s463ms
- TotalNetworkSendTime: 1.39us
- TotalStorageWaitTime: 2h21m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 673.679ms, non-child: 673.679ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 133.397ms
- LoadTime: 140.219us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 359.149ms
- PrepareTime: 179.349ms
DataStreamSender (dst_id=13):(Total: 221.916us, non-child: 221.916us, % non-child: 100.00%)
- BytesSent: 3.15 KB (3226)
- NetworkThroughput(*): 1.64 MB/sec
- OverallThroughput: 13.86 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 157.2us
- ThriftTransmitTime(*): 1.880ms
- UncompressedRowBatchSize: 4.12 KB (4219)
HASH_JOIN_NODE (id=6):(Total: 10m7s, non-child: 194.280ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.206us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.367us
- GetNewBlockTime: 97.186us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.50 MB (25690760)
- PinTime: 0ns
- ProbeRows: 9 (9)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 154.484us
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.112us
EXCHANGE_NODE (id=12):(Total: 2s607ms, non-child: 26.641us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.128us
- DeserializeRowBatchTimer: 13.20us
- FirstBatchArrivalWaitTime: 2s601ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m7s, non-child: 3s098ms, % non-child: 0.51%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.458us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.972us
- GetNewBlockTime: 66.477us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.19 MB (15928208)
- PinTime: 0ns
- ProbeRows: 9 (9)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 202.388us
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 955ns
EXCHANGE_NODE (id=9):(Total: 2s855ms, non-child: 27.994us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.67us
- DeserializeRowBatchTimer: 12.667us
- FirstBatchArrivalWaitTime: 2s853ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m4s, non-child: 10m4s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:57/163.25 MB 1:53/151.64 MB 2:43/126.89 MB 3:42/125.96 MB 4:42/109.54 MB 5:67/169.34 MB 6:48/143.56 MB 7:51/144.14 MB 8:54/144.24 MB 9:42/95.13 MB 10:55/153.53 MB 11:55/156.66 MB
Hdfs Read Thread Concurrency Bucket: 0:97.53% 1:2.469% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28623
ExecOption: Codegen enabled: 0 out of 609
BytesRead(16s000ms): 5.28 MB, 25.11 MB, 43.28 MB, 55.31 MB, 70.87 MB, 85.86 MB, 107.23 MB, 120.61 MB, 136.54 MB, 154.25 MB, 170.45 MB, 194.64 MB, 216.99 MB, 234.57 MB, 258.45 MB, 299.18 MB, 357.00 MB, 399.90 MB, 420.25 MB, 443.94 MB, 474.93 MB, 509.35 MB, 536.57 MB, 568.53 MB, 595.31 MB, 615.54 MB, 634.21 MB, 667.92 MB, 692.98 MB, 715.50 MB, 741.07 MB, 775.14 MB, 801.78 MB, 816.17 MB, 819.44 MB, 823.65 MB, 827.60 MB, 830.98 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 14.01
- BytesRead: 831.48 MB (871866086)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 831.48 MB (871866086)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 831.48 MB (871866086)
- DecompressionTime: 2s116ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 174 (174)
- PeakMemoryUsage: 81.29 MB (85243776)
- PerReadThreadRawHdfsThroughput: 59.78 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.52M (11520402)
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- ScanRangesComplete: 609 (609)
- ScannerThreadsInvoluntaryContextSwitches: 509 (509)
- ScannerThreadsTotalWallClockTime: 2h21m
- MaterializeTupleTime(*): 15s149ms
- ScannerThreadsSysTime: 548.10ms
- ScannerThreadsUserTime: 17s761ms
- ScannerThreadsVoluntaryContextSwitches: 43.63K (43626)
- TotalRawHdfsReadTime(*): 13s910ms
- TotalReadThroughput: 1.37 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016a2 (host=hadoopc1h76:28002):(Total: 10m6s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:49/119.30 MB 1:32/107.39 MB 2:51/183.87 MB 3:51/142.68 MB 4:38/108.29 MB 5:39/117.77 MB 6:52/131.35 MB 7:48/170.46 MB 8:50/184.59 MB 9:48/133.83 MB 10:48/136.35 MB 11:54/156.78 MB
MemoryUsage(16s000ms): 31.03 MB, 35.19 MB, 60.50 MB, 49.76 MB, 42.97 MB, 50.46 MB, 51.78 MB, 46.78 MB, 37.77 MB, 31.76 MB, 32.00 MB, 33.45 MB, 30.37 MB, 31.95 MB, 36.79 MB, 41.11 MB, 38.35 MB, 36.44 MB, 31.84 MB, 38.44 MB, 42.52 MB, 23.90 MB, 53.69 MB, 41.13 MB, 52.39 MB, 32.61 MB, 37.97 MB, 30.94 MB, 33.38 MB, 36.29 MB, 28.41 MB, 27.48 MB, 36.35 MB, 36.81 MB, 39.73 MB, 20.21 MB, 15.52 MB
ThreadUsage(16s000ms): 11, 12, 13, 13, 14, 16, 15, 12, 13, 10, 9, 8, 8, 8, 9, 10, 8, 9, 10, 11, 11, 10, 13, 10, 10, 9, 10, 10, 8, 9, 10, 10, 10, 11, 12, 12, 6
- AverageThreadTokens: 11.04
- PeakMemoryUsage: 81.06 MB (85002000)
- PerHostPeakMemUsage: 81.06 MB (85002000)
- PrepareTime: 202.313ms
- RowsProduced: 15 (15)
- TotalCpuTime: 1h41m
- TotalNetworkReceiveTime: 5s378ms
- TotalNetworkSendTime: 1.745us
- TotalStorageWaitTime: 1h41m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 710.802ms, non-child: 710.802ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 148.531ms
- LoadTime: 133.150us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 372.807ms
- PrepareTime: 187.645ms
DataStreamSender (dst_id=13):(Total: 254.616us, non-child: 254.616us, % non-child: 100.00%)
- BytesSent: 4.84 KB (4953)
- NetworkThroughput(*): 1.99 MB/sec
- OverallThroughput: 18.55 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 180.281us
- ThriftTransmitTime(*): 2.375ms
- UncompressedRowBatchSize: 6.77 KB (6929)
HASH_JOIN_NODE (id=6):(Total: 10m6s, non-child: 201.837ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 9.27us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.947us
- GetNewBlockTime: 66.678us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 23.92 MB (25085576)
- PinTime: 0ns
- ProbeRows: 15 (15)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 144.895us
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 869ns
EXCHANGE_NODE (id=12):(Total: 2s560ms, non-child: 34.997us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.976us
- DeserializeRowBatchTimer: 14.27us
- FirstBatchArrivalWaitTime: 2s554ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m6s, non-child: 3s177ms, % non-child: 0.52%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 10.642us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.956us
- GetNewBlockTime: 65.72us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 16.12 MB (16898920)
- PinTime: 0ns
- ProbeRows: 15 (15)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 138.906us
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 774ns
EXCHANGE_NODE (id=9):(Total: 2s818ms, non-child: 24.803us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.808us
- DeserializeRowBatchTimer: 14.510us
- FirstBatchArrivalWaitTime: 2s806ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 10m2s, non-child: 10m2s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:49/119.30 MB 1:32/107.39 MB 2:51/183.87 MB 3:51/142.68 MB 4:38/108.29 MB 5:39/117.77 MB 6:52/131.35 MB 7:48/170.46 MB 8:50/184.59 MB 9:48/133.83 MB 10:48/136.35 MB 11:54/156.78 MB
Hdfs Read Thread Concurrency Bucket: 0:98.1% 1:1.898% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:26320
ExecOption: Codegen enabled: 0 out of 560
BytesRead(16s000ms): 9.32 MB, 33.95 MB, 67.16 MB, 89.11 MB, 104.96 MB, 126.81 MB, 151.38 MB, 170.18 MB, 189.85 MB, 204.76 MB, 221.45 MB, 242.98 MB, 272.86 MB, 298.11 MB, 321.28 MB, 351.47 MB, 387.48 MB, 415.52 MB, 426.08 MB, 439.66 MB, 468.31 MB, 487.53 MB, 507.69 MB, 528.35 MB, 554.29 MB, 567.74 MB, 583.14 MB, 598.97 MB, 623.11 MB, 646.08 MB, 668.05 MB, 693.22 MB, 719.38 MB, 750.19 MB, 790.33 MB, 812.62 MB, 825.68 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 10.04
- BytesRead: 829.92 MB (870229507)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 829.92 MB (870229507)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 829.92 MB (870229507)
- DecompressionTime: 974.609ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 150 (150)
- PeakMemoryUsage: 80.51 MB (84424400)
- PerReadThreadRawHdfsThroughput: 49.82 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.66M (11658778)
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- ScanRangesComplete: 560 (560)
- ScannerThreadsInvoluntaryContextSwitches: 1.12K (1119)
- ScannerThreadsTotalWallClockTime: 1h41m
- MaterializeTupleTime(*): 16s881ms
- ScannerThreadsSysTime: 660.14ms
- ScannerThreadsUserTime: 19s593ms
- ScannerThreadsVoluntaryContextSwitches: 45.64K (45640)
- TotalRawHdfsReadTime(*): 16s657ms
- TotalReadThroughput: 1.37 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016bd (host=hadoopc1h28:28002):(Total: 10m2s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:57/170.75 MB 1:47/144.07 MB 2:43/129.31 MB 3:47/176.39 MB 4:47/132.11 MB 5:39/95.92 MB 6:38/122.71 MB 7:56/185.24 MB 8:38/105.12 MB 9:41/128.27 MB 10:42/116.60 MB 11:53/180.07 MB
MemoryUsage(16s000ms): 16.31 MB, 26.23 MB, 48.34 MB, 49.50 MB, 40.58 MB, 51.21 MB, 36.74 MB, 43.55 MB, 43.77 MB, 32.47 MB, 32.61 MB, 21.35 MB, 32.74 MB, 24.30 MB, 29.45 MB, 30.68 MB, 29.23 MB, 32.28 MB, 36.93 MB, 39.48 MB, 44.90 MB, 34.50 MB, 46.92 MB, 30.39 MB, 32.75 MB, 25.66 MB, 40.13 MB, 44.32 MB, 36.33 MB, 28.66 MB, 23.02 MB, 21.14 MB, 16.99 MB, 20.61 MB, 14.60 MB, 11.76 MB, 12.23 MB
ThreadUsage(16s000ms): 8, 8, 10, 10, 10, 11, 12, 10, 10, 8, 8, 6, 6, 6, 6, 7, 7, 7, 8, 9, 10, 10, 11, 8, 9, 8, 9, 9, 7, 7, 7, 8, 8, 9, 9, 7, 4
- AverageThreadTokens: 8.89
- PeakMemoryUsage: 69.96 MB (73355712)
- PerHostPeakMemUsage: 69.96 MB (73355712)
- PrepareTime: 210.195ms
- RowsProduced: 17 (17)
- TotalCpuTime: 1h19m
- TotalNetworkReceiveTime: 5s331ms
- TotalNetworkSendTime: 2.159us
- TotalStorageWaitTime: 1h18m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 731.223ms, non-child: 731.223ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 153.333ms
- LoadTime: 165.228us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 381.92ms
- PrepareTime: 193.870ms
DataStreamSender (dst_id=13):(Total: 294.829us, non-child: 294.829us, % non-child: 100.00%)
- BytesSent: 5.05 KB (5168)
- NetworkThroughput(*): 1.70 MB/sec
- OverallThroughput: 16.72 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 210.944us
- ThriftTransmitTime(*): 2.902ms
- UncompressedRowBatchSize: 7.61 KB (7790)
HASH_JOIN_NODE (id=6):(Total: 10m2s, non-child: 211.85ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 14.622us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.821us
- GetNewBlockTime: 93.54us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.61 MB (26849312)
- PinTime: 0ns
- ProbeRows: 17 (17)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 188.298us
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.120us
EXCHANGE_NODE (id=12):(Total: 2s541ms, non-child: 37.298us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.515us
- DeserializeRowBatchTimer: 20.67us
- FirstBatchArrivalWaitTime: 2s529ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m2s, non-child: 2s809ms, % non-child: 0.47%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 16.299us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.506us
- GetNewBlockTime: 65.899us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.08 MB (15810312)
- PinTime: 0ns
- ProbeRows: 17 (17)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 160.310us
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.154us
EXCHANGE_NODE (id=9):(Total: 2s790ms, non-child: 31.474us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 3.281us
- DeserializeRowBatchTimer: 27.242us
- FirstBatchArrivalWaitTime: 2s781ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m59s, non-child: 9m59s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:57/170.75 MB 1:47/144.07 MB 2:43/129.31 MB 3:47/176.39 MB 4:47/132.11 MB 5:39/95.92 MB 6:38/122.71 MB 7:56/185.24 MB 8:38/105.12 MB 9:41/128.27 MB 10:42/116.60 MB 11:53/180.07 MB
Hdfs Read Thread Concurrency Bucket: 0:97.84% 1:2.158% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:25756
ExecOption: Codegen enabled: 0 out of 548
BytesRead(16s000ms): 6.41 MB, 22.22 MB, 48.05 MB, 71.14 MB, 87.79 MB, 113.28 MB, 135.06 MB, 154.97 MB, 182.43 MB, 201.21 MB, 214.57 MB, 227.31 MB, 257.24 MB, 280.04 MB, 309.45 MB, 346.64 MB, 386.17 MB, 415.12 MB, 431.24 MB, 450.03 MB, 484.29 MB, 518.88 MB, 546.12 MB, 571.91 MB, 591.33 MB, 608.08 MB, 627.81 MB, 658.81 MB, 697.19 MB, 719.51 MB, 741.83 MB, 766.86 MB, 777.55 MB, 797.14 MB, 813.12 MB, 815.63 MB, 821.34 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 7.89
- BytesRead: 825.87 MB (865987311)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 825.87 MB (865987311)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 825.87 MB (865987311)
- DecompressionTime: 678.501ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 131 (131)
- PeakMemoryUsage: 64.37 MB (67492528)
- PerReadThreadRawHdfsThroughput: 82.44 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.60M (11601764)
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- ScanRangesComplete: 548 (548)
- ScannerThreadsInvoluntaryContextSwitches: 465 (465)
- ScannerThreadsTotalWallClockTime: 1h19m
- MaterializeTupleTime(*): 15s071ms
- ScannerThreadsSysTime: 516.12ms
- ScannerThreadsUserTime: 17s757ms
- ScannerThreadsVoluntaryContextSwitches: 47.83K (47834)
- TotalRawHdfsReadTime(*): 10s018ms
- TotalReadThroughput: 1.37 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016c0 (host=hadoopc1h11:28002):(Total: 10m1s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:42/143.27 MB 1:43/131.60 MB 2:42/130.51 MB 3:45/153.64 MB 4:49/140.75 MB 5:57/174.71 MB 6:50/166.98 MB 7:42/102.44 MB 8:46/136.46 MB 9:39/130.81 MB 10:45/173.34 MB 11:40/126.20 MB
MemoryUsage(16s000ms): 27.76 MB, 39.43 MB, 42.69 MB, 40.05 MB, 37.20 MB, 51.48 MB, 34.08 MB, 42.37 MB, 38.42 MB, 34.68 MB, 39.66 MB, 30.18 MB, 36.90 MB, 33.24 MB, 30.56 MB, 32.47 MB, 30.11 MB, 23.40 MB, 31.96 MB, 35.69 MB, 40.61 MB, 25.20 MB, 47.15 MB, 26.55 MB, 29.95 MB, 33.87 MB, 28.31 MB, 31.26 MB, 35.89 MB, 32.62 MB, 37.83 MB, 37.99 MB, 25.98 MB, 23.17 MB, 16.17 MB, 12.00 MB, 12.37 MB
ThreadUsage(16s000ms): 9, 9, 10, 10, 10, 11, 12, 10, 10, 8, 7, 6, 6, 6, 6, 7, 6, 7, 8, 7, 8, 7, 9, 7, 8, 8, 8, 8, 7, 6, 7, 7, 8, 9, 11, 8, 4
- AverageThreadTokens: 8.64
- PeakMemoryUsage: 81.24 MB (85183776)
- PerHostPeakMemUsage: 81.24 MB (85183776)
- PrepareTime: 184.879ms
- RowsProduced: 10 (10)
- TotalCpuTime: 1h16m
- TotalNetworkReceiveTime: 5s484ms
- TotalNetworkSendTime: 1.441us
- TotalStorageWaitTime: 1h16m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 657.938ms, non-child: 657.938ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 134.216ms
- LoadTime: 156.974us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 351.221ms
- PrepareTime: 170.795ms
DataStreamSender (dst_id=13):(Total: 177.56us, non-child: 177.56us, % non-child: 100.00%)
- BytesSent: 2.91 KB (2976)
- NetworkThroughput(*): 1.16 MB/sec
- OverallThroughput: 16.03 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 113.524us
- ThriftTransmitTime(*): 2.450ms
- UncompressedRowBatchSize: 4.52 KB (4633)
HASH_JOIN_NODE (id=6):(Total: 10m1s, non-child: 854.628ms, % non-child: 0.14%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.696us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.525us
- GetNewBlockTime: 69.448us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.97 MB (26181840)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 131.395us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 983ns
EXCHANGE_NODE (id=12):(Total: 2s617ms, non-child: 25.930us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.441us
- DeserializeRowBatchTimer: 11.947us
- FirstBatchArrivalWaitTime: 2s612ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 10m, non-child: 2s951ms, % non-child: 0.49%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.829us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.130us
- GetNewBlockTime: 53.440us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 14.34 MB (15040000)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 127.868us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 818ns
EXCHANGE_NODE (id=9):(Total: 2s867ms, non-child: 46.394us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 3.804us
- DeserializeRowBatchTimer: 21.227us
- FirstBatchArrivalWaitTime: 2s864ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m57s, non-child: 9m57s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:42/143.27 MB 1:43/131.60 MB 2:42/130.51 MB 3:45/153.64 MB 4:49/140.75 MB 5:57/174.71 MB 6:50/166.98 MB 7:42/102.44 MB 8:46/136.46 MB 9:39/130.81 MB 10:45/173.34 MB 11:40/126.20 MB
Hdfs Read Thread Concurrency Bucket: 0:98.59% 1:1.414% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:25380
ExecOption: Codegen enabled: 0 out of 540
BytesRead(16s000ms): 8.83 MB, 38.00 MB, 66.70 MB, 87.04 MB, 101.70 MB, 126.50 MB, 146.56 MB, 165.53 MB, 189.51 MB, 207.44 MB, 225.64 MB, 248.84 MB, 283.28 MB, 314.66 MB, 343.73 MB, 377.21 MB, 416.03 MB, 441.83 MB, 456.28 MB, 472.84 MB, 502.96 MB, 523.03 MB, 547.05 MB, 568.75 MB, 585.09 MB, 600.13 MB, 613.11 MB, 631.85 MB, 660.58 MB, 685.69 MB, 708.20 MB, 747.84 MB, 777.57 MB, 798.61 MB, 816.09 MB, 822.45 MB, 831.30 MB
- AverageHdfsReadThreadConcurrency: 0.01
- AverageScannerThreadConcurrency: 7.63
- BytesRead: 837.08 MB (877739009)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 837.08 MB (877739009)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 837.08 MB (877739009)
- DecompressionTime: 751.26ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 123 (123)
- PeakMemoryUsage: 80.69 MB (84606176)
- PerReadThreadRawHdfsThroughput: 76.90 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.76M (11763407)
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- ScanRangesComplete: 540 (540)
- ScannerThreadsInvoluntaryContextSwitches: 635 (635)
- ScannerThreadsTotalWallClockTime: 1h16m
- MaterializeTupleTime(*): 15s127ms
- ScannerThreadsSysTime: 360.4ms
- ScannerThreadsUserTime: 17s945ms
- ScannerThreadsVoluntaryContextSwitches: 47.76K (47764)
- TotalRawHdfsReadTime(*): 10s885ms
- TotalReadThroughput: 1.39 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016c7 (host=hadoopc1h71:28002):(Total: 9m59s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:47/112.49 MB 1:45/122.29 MB 2:52/175.16 MB 3:58/136.36 MB 4:48/131.29 MB 5:35/118.65 MB 6:52/142.92 MB 7:49/142.30 MB 8:49/149.12 MB 9:54/175.31 MB 10:47/131.44 MB 11:53/141.11 MB
MemoryUsage(16s000ms): 23.18 MB, 34.61 MB, 45.22 MB, 49.43 MB, 44.31 MB, 48.26 MB, 39.56 MB, 38.20 MB, 50.25 MB, 38.99 MB, 34.08 MB, 33.64 MB, 33.79 MB, 36.92 MB, 39.01 MB, 40.09 MB, 31.34 MB, 57.94 MB, 37.87 MB, 34.84 MB, 45.29 MB, 41.25 MB, 33.90 MB, 39.43 MB, 47.70 MB, 34.84 MB, 41.31 MB, 37.02 MB, 27.27 MB, 34.04 MB, 35.42 MB, 44.10 MB, 29.41 MB, 39.34 MB, 29.94 MB, 21.31 MB, 10.35 MB
ThreadUsage(16s000ms): 10, 12, 11, 14, 14, 14, 13, 11, 11, 10, 9, 9, 9, 9, 10, 10, 9, 9, 10, 9, 11, 11, 10, 9, 9, 9, 10, 10, 8, 9, 10, 9, 9, 10, 13, 11, 4
- AverageThreadTokens: 10.69
- PeakMemoryUsage: 74.65 MB (78273024)
- PerHostPeakMemUsage: 74.65 MB (78273024)
- PrepareTime: 195.528ms
- RowsProduced: 20 (20)
- TotalCpuTime: 1h36m
- TotalNetworkReceiveTime: 5s407ms
- TotalNetworkSendTime: 3.200us
- TotalStorageWaitTime: 1h36m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 695.685ms, non-child: 695.685ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 142.342ms
- LoadTime: 152.16us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 370.881ms
- PrepareTime: 180.551ms
DataStreamSender (dst_id=13):(Total: 298.795us, non-child: 298.795us, % non-child: 100.00%)
- BytesSent: 5.54 KB (5669)
- NetworkThroughput(*): 1.82 MB/sec
- OverallThroughput: 18.09 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 209.678us
- ThriftTransmitTime(*): 2.978ms
- UncompressedRowBatchSize: 8.98 KB (9198)
HASH_JOIN_NODE (id=6):(Total: 9m59s, non-child: 196.50ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 9.491us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 6.406us
- GetNewBlockTime: 75.533us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.72 MB (26974120)
- PinTime: 0ns
- ProbeRows: 20 (20)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 196.344us
- RowsReturned: 20 (20)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 992ns
EXCHANGE_NODE (id=12):(Total: 2s573ms, non-child: 22.61us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.445us
- DeserializeRowBatchTimer: 9.444us
- FirstBatchArrivalWaitTime: 2s566ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m59s, non-child: 2s854ms, % non-child: 0.48%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 16.157us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 7.167us
- GetNewBlockTime: 106.718us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 19.40 MB (20341456)
- PinTime: 0ns
- ProbeRows: 20 (20)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 201.110us
- RowsReturned: 20 (20)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 793ns
EXCHANGE_NODE (id=9):(Total: 2s833ms, non-child: 35.245us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.823us
- DeserializeRowBatchTimer: 10.235us
- FirstBatchArrivalWaitTime: 2s819ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m56s, non-child: 9m56s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:47/112.49 MB 1:45/122.29 MB 2:52/175.16 MB 3:58/136.36 MB 4:48/131.29 MB 5:35/118.65 MB 6:52/142.92 MB 7:49/142.30 MB 8:49/149.12 MB 9:54/175.31 MB 10:47/131.44 MB 11:53/141.11 MB
Hdfs Read Thread Concurrency Bucket: 0:97.5% 1:2.502% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:27683
ExecOption: Codegen enabled: 0 out of 589
BytesRead(16s000ms): 6.38 MB, 25.70 MB, 48.95 MB, 66.09 MB, 79.14 MB, 97.90 MB, 116.90 MB, 132.30 MB, 153.19 MB, 169.75 MB, 186.42 MB, 203.90 MB, 232.33 MB, 254.20 MB, 284.01 MB, 312.82 MB, 337.87 MB, 369.79 MB, 385.28 MB, 396.57 MB, 419.50 MB, 446.12 MB, 463.97 MB, 489.43 MB, 514.92 MB, 534.39 MB, 550.72 MB, 572.33 MB, 600.48 MB, 620.27 MB, 642.31 MB, 681.18 MB, 710.08 MB, 747.02 MB, 795.56 MB, 814.28 MB, 825.97 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 9.69
- BytesRead: 827.47 MB (867669668)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 827.47 MB (867669668)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 827.47 MB (867669668)
- DecompressionTime: 1s714ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 117 (117)
- PeakMemoryUsage: 65.13 MB (68298720)
- PerReadThreadRawHdfsThroughput: 55.33 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.48M (11481064)
- RowsReturned: 20 (20)
- RowsReturnedRate: 0
- ScanRangesComplete: 589 (589)
- ScannerThreadsInvoluntaryContextSwitches: 1.27K (1268)
- ScannerThreadsTotalWallClockTime: 1h36m
- MaterializeTupleTime(*): 17s019ms
- ScannerThreadsSysTime: 584.10ms
- ScannerThreadsUserTime: 20s045ms
- ScannerThreadsVoluntaryContextSwitches: 49.36K (49364)
- TotalRawHdfsReadTime(*): 14s955ms
- TotalReadThroughput: 1.38 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016a0 (host=hadoopc1h73:28002):(Total: 9m59s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:57/187.56 MB 1:53/167.51 MB 2:41/108.49 MB 3:52/155.26 MB 4:50/155.96 MB 5:47/126.65 MB 6:47/128.43 MB 7:49/128.28 MB 8:57/116.01 MB 9:52/148.46 MB 10:47/105.95 MB 11:54/156.02 MB
MemoryUsage(16s000ms): 30.83 MB, 46.72 MB, 44.91 MB, 49.47 MB, 45.85 MB, 45.75 MB, 44.38 MB, 40.66 MB, 55.14 MB, 41.15 MB, 47.09 MB, 42.15 MB, 36.66 MB, 46.52 MB, 30.68 MB, 35.08 MB, 39.54 MB, 42.92 MB, 33.67 MB, 48.76 MB, 35.15 MB, 38.26 MB, 48.96 MB, 29.02 MB, 41.10 MB, 36.78 MB, 36.87 MB, 45.74 MB, 30.81 MB, 36.97 MB, 38.05 MB, 41.06 MB, 38.61 MB, 38.62 MB, 36.39 MB, 33.29 MB, 15.34 MB
ThreadUsage(16s000ms): 11, 13, 12, 15, 14, 15, 17, 14, 13, 10, 10, 8, 9, 9, 8, 10, 9, 9, 10, 11, 11, 11, 12, 10, 9, 10, 10, 11, 9, 9, 11, 11, 11, 15, 18, 17, 7
- AverageThreadTokens: 11.99
- PeakMemoryUsage: 83.32 MB (87366480)
- PerHostPeakMemUsage: 83.32 MB (87366480)
- PrepareTime: 198.323ms
- RowsProduced: 15 (15)
- TotalCpuTime: 1h49m
- TotalNetworkReceiveTime: 5s405ms
- TotalNetworkSendTime: 1.850us
- TotalStorageWaitTime: 1h49m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 695.516ms, non-child: 695.516ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 142.258ms
- LoadTime: 151.161us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 368.772ms
- PrepareTime: 182.470ms
DataStreamSender (dst_id=13):(Total: 231.68us, non-child: 231.68us, % non-child: 100.00%)
- BytesSent: 4.37 KB (4478)
- NetworkThroughput(*): 2.29 MB/sec
- OverallThroughput: 18.48 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 161.371us
- ThriftTransmitTime(*): 1.863ms
- UncompressedRowBatchSize: 6.71 KB (6876)
HASH_JOIN_NODE (id=6):(Total: 9m59s, non-child: 198.348ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.468us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.554us
- GetNewBlockTime: 92.444us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 23.77 MB (24924000)
- PinTime: 0ns
- ProbeRows: 15 (15)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 199.742us
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 794ns
EXCHANGE_NODE (id=12):(Total: 2s577ms, non-child: 24.936us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.699us
- DeserializeRowBatchTimer: 20.749us
- FirstBatchArrivalWaitTime: 2s569ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m59s, non-child: 2s849ms, % non-child: 0.48%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.456us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.215us
- GetNewBlockTime: 88.983us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 13.77 MB (14443656)
- PinTime: 0ns
- ProbeRows: 15 (15)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 221.837us
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 838ns
EXCHANGE_NODE (id=9):(Total: 2s828ms, non-child: 26.727us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.91us
- DeserializeRowBatchTimer: 18.623us
- FirstBatchArrivalWaitTime: 2s820ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m56s, non-child: 9m56s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:57/187.56 MB 1:53/167.51 MB 2:41/108.49 MB 3:52/155.26 MB 4:50/155.96 MB 5:47/126.65 MB 6:47/128.43 MB 7:49/128.28 MB 8:57/116.01 MB 9:52/148.46 MB 10:47/105.95 MB 11:54/156.02 MB
Hdfs Read Thread Concurrency Bucket: 0:96.83% 1:3.169% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28482
ExecOption: Codegen enabled: 0 out of 606
BytesRead(16s000ms): 8.27 MB, 34.34 MB, 60.60 MB, 80.23 MB, 95.82 MB, 111.56 MB, 131.32 MB, 145.85 MB, 166.46 MB, 184.14 MB, 201.16 MB, 225.76 MB, 247.48 MB, 274.03 MB, 294.71 MB, 317.90 MB, 351.88 MB, 386.03 MB, 400.33 MB, 416.17 MB, 434.12 MB, 454.86 MB, 478.06 MB, 501.52 MB, 521.01 MB, 537.10 MB, 548.71 MB, 570.14 MB, 598.57 MB, 620.81 MB, 645.98 MB, 675.30 MB, 709.35 MB, 749.36 MB, 784.87 MB, 808.66 MB, 825.77 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 10.99
- BytesRead: 831.96 MB (872374065)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 831.96 MB (872374065)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 831.96 MB (872374065)
- DecompressionTime: 1s514ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 144 (144)
- PeakMemoryUsage: 75.39 MB (79050080)
- PerReadThreadRawHdfsThroughput: 52.05 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.51M (11505227)
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- ScanRangesComplete: 606 (606)
- ScannerThreadsInvoluntaryContextSwitches: 1.38K (1376)
- ScannerThreadsTotalWallClockTime: 1h49m
- MaterializeTupleTime(*): 17s295ms
- ScannerThreadsSysTime: 748.15ms
- ScannerThreadsUserTime: 20s209ms
- ScannerThreadsVoluntaryContextSwitches: 51.79K (51794)
- TotalRawHdfsReadTime(*): 15s983ms
- TotalReadThroughput: 1.39 MB/sec
Instance c14f7b9b17987c60:d5379fb95410169f (host=hadoopc1h77:28002):(Total: 9m56s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:54/167.95 MB 1:43/118.75 MB 2:46/125.39 MB 3:51/176.33 MB 4:45/149.38 MB 5:32/97.24 MB 6:53/147.05 MB 7:43/136.74 MB 8:55/202.00 MB 9:44/129.89 MB 10:42/118.87 MB 11:39/119.74 MB
MemoryUsage(16s000ms): 18.50 MB, 31.03 MB, 34.45 MB, 63.77 MB, 39.07 MB, 50.07 MB, 52.81 MB, 47.63 MB, 41.31 MB, 51.09 MB, 44.91 MB, 42.01 MB, 32.48 MB, 33.14 MB, 31.06 MB, 41.38 MB, 43.48 MB, 50.55 MB, 44.63 MB, 53.97 MB, 46.57 MB, 32.48 MB, 57.99 MB, 35.48 MB, 39.88 MB, 26.29 MB, 39.42 MB, 33.52 MB, 28.19 MB, 32.42 MB, 28.38 MB, 36.10 MB, 33.12 MB, 32.13 MB, 24.64 MB, 20.88 MB, 13.46 MB
ThreadUsage(16s000ms): 11, 11, 13, 15, 14, 15, 15, 12, 13, 11, 10, 9, 8, 8, 8, 9, 8, 9, 10, 11, 11, 10, 12, 10, 9, 9, 11, 10, 8, 9, 8, 10, 9, 12, 13, 13, 7
- AverageThreadTokens: 11.23
- PeakMemoryUsage: 87.46 MB (91713512)
- PerHostPeakMemUsage: 87.46 MB (91713512)
- PrepareTime: 197.399ms
- RowsProduced: 6 (6)
- TotalCpuTime: 1h41m
- TotalNetworkReceiveTime: 5s353ms
- TotalNetworkSendTime: 662ns
- TotalStorageWaitTime: 1h41m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 726.991ms, non-child: 726.991ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 155.750ms
- LoadTime: 155.541us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 387.171ms
- PrepareTime: 182.206ms
DataStreamSender (dst_id=13):(Total: 198.698us, non-child: 198.698us, % non-child: 100.00%)
- BytesSent: 1.98 KB (2027)
- NetworkThroughput(*): 1.80 MB/sec
- OverallThroughput: 9.73 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 151.776us
- ThriftTransmitTime(*): 1.74ms
- UncompressedRowBatchSize: 2.70 KB (2766)
HASH_JOIN_NODE (id=6):(Total: 9m56s, non-child: 198.116ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 10.205us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.702us
- GetNewBlockTime: 86.816us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 27.21 MB (28526904)
- PinTime: 0ns
- ProbeRows: 6 (6)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 164.118us
- RowsReturned: 6 (6)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.377us
EXCHANGE_NODE (id=12):(Total: 2s551ms, non-child: 34.676us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.844us
- DeserializeRowBatchTimer: 23.329us
- FirstBatchArrivalWaitTime: 2s539ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m56s, non-child: 2s823ms, % non-child: 0.47%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.274us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.556us
- GetNewBlockTime: 77.573us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.85 MB (16624480)
- PinTime: 0ns
- ProbeRows: 6 (6)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 155.431us
- RowsReturned: 6 (6)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.657us
EXCHANGE_NODE (id=9):(Total: 2s802ms, non-child: 31.996us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.970us
- DeserializeRowBatchTimer: 19.107us
- FirstBatchArrivalWaitTime: 2s790ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m53s, non-child: 9m53s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:54/167.95 MB 1:43/118.75 MB 2:46/125.39 MB 3:51/176.33 MB 4:45/149.38 MB 5:32/97.24 MB 6:53/147.05 MB 7:43/136.74 MB 8:55/202.00 MB 9:44/129.89 MB 10:42/118.87 MB 11:39/119.74 MB
Hdfs Read Thread Concurrency Bucket: 0:97.65% 1:2.349% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:25709
ExecOption: Codegen enabled: 0 out of 547
BytesRead(16s000ms): 5.37 MB, 24.96 MB, 43.85 MB, 66.84 MB, 82.91 MB, 106.04 MB, 126.65 MB, 144.82 MB, 163.46 MB, 183.03 MB, 198.48 MB, 229.23 MB, 258.87 MB, 284.12 MB, 313.67 MB, 348.62 MB, 392.57 MB, 439.24 MB, 455.96 MB, 471.01 MB, 495.17 MB, 514.89 MB, 538.72 MB, 559.42 MB, 578.23 MB, 588.61 MB, 603.97 MB, 625.96 MB, 646.50 MB, 668.73 MB, 690.23 MB, 723.82 MB, 758.26 MB, 783.88 MB, 804.35 MB, 815.25 MB, 826.04 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 10.24
- BytesRead: 829.21 MB (869488407)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 829.21 MB (869488407)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 829.21 MB (869488407)
- DecompressionTime: 1s373ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 145 (145)
- PeakMemoryUsage: 80.30 MB (84196544)
- PerReadThreadRawHdfsThroughput: 52.41 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.59M (11594023)
- RowsReturned: 6 (6)
- RowsReturnedRate: 0
- ScanRangesComplete: 547 (547)
- ScannerThreadsInvoluntaryContextSwitches: 500 (500)
- ScannerThreadsTotalWallClockTime: 1h41m
- MaterializeTupleTime(*): 15s485ms
- ScannerThreadsSysTime: 588.11ms
- ScannerThreadsUserTime: 18s089ms
- ScannerThreadsVoluntaryContextSwitches: 40.91K (40914)
- TotalRawHdfsReadTime(*): 15s821ms
- TotalReadThroughput: 1.39 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016aa (host=hadoopc1h75:28002):(Total: 9m56s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:59/167.89 MB 1:41/123.19 MB 2:39/83.56 MB 3:47/125.25 MB 4:41/99.94 MB 5:51/152.26 MB 6:40/123.51 MB 7:57/182.07 MB 8:50/162.24 MB 9:56/168.77 MB 10:59/175.93 MB 11:37/111.28 MB
MemoryUsage(16s000ms): 24.62 MB, 37.79 MB, 57.19 MB, 45.42 MB, 61.78 MB, 60.04 MB, 56.12 MB, 43.20 MB, 56.58 MB, 40.77 MB, 46.22 MB, 35.90 MB, 38.35 MB, 37.28 MB, 29.94 MB, 36.30 MB, 34.39 MB, 33.54 MB, 28.03 MB, 41.04 MB, 37.69 MB, 35.05 MB, 37.00 MB, 27.22 MB, 35.63 MB, 45.78 MB, 49.64 MB, 34.97 MB, 32.49 MB, 29.20 MB, 44.18 MB, 34.56 MB, 25.33 MB, 27.75 MB, 21.79 MB, 15.60 MB, 8.76 MB
ThreadUsage(16s000ms): 11, 11, 13, 14, 17, 15, 17, 13, 12, 10, 9, 8, 9, 9, 9, 9, 8, 9, 10, 11, 11, 11, 12, 10, 10, 10, 13, 11, 9, 9, 11, 10, 9, 12, 15, 10, 3
- AverageThreadTokens: 11.49
- PeakMemoryUsage: 81.51 MB (85474176)
- PerHostPeakMemUsage: 81.51 MB (85474176)
- PrepareTime: 194.184ms
- RowsProduced: 22 (22)
- TotalCpuTime: 1h44m
- TotalNetworkReceiveTime: 5s421ms
- TotalNetworkSendTime: 1.484us
- TotalStorageWaitTime: 1h43m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 688.307ms, non-child: 688.307ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 136.933ms
- LoadTime: 138.890us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 370.688ms
- PrepareTime: 178.804ms
DataStreamSender (dst_id=13):(Total: 319.663us, non-child: 319.663us, % non-child: 100.00%)
- BytesSent: 5.99 KB (6134)
- NetworkThroughput(*): 2.19 MB/sec
- OverallThroughput: 18.30 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 232.161us
- ThriftTransmitTime(*): 2.676ms
- UncompressedRowBatchSize: 9.82 KB (10052)
HASH_JOIN_NODE (id=6):(Total: 9m56s, non-child: 194.227ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.250us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.498us
- GetNewBlockTime: 69.156us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 22.30 MB (23385568)
- PinTime: 0ns
- ProbeRows: 22 (22)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 190.487us
- RowsReturned: 22 (22)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 962ns
EXCHANGE_NODE (id=12):(Total: 2s582ms, non-child: 23.164us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.920us
- DeserializeRowBatchTimer: 17.641us
- FirstBatchArrivalWaitTime: 2s579ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m56s, non-child: 2s858ms, % non-child: 0.48%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.701us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.585us
- GetNewBlockTime: 56.170us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 13.43 MB (14084624)
- PinTime: 0ns
- ProbeRows: 22 (22)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 181.922us
- RowsReturned: 22 (22)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 684ns
EXCHANGE_NODE (id=9):(Total: 2s839ms, non-child: 24.122us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.885us
- DeserializeRowBatchTimer: 16.970us
- FirstBatchArrivalWaitTime: 2s831ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m53s, non-child: 9m53s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:59/167.89 MB 1:41/123.19 MB 2:39/83.56 MB 3:47/125.25 MB 4:41/99.94 MB 5:51/152.26 MB 6:40/123.51 MB 7:57/182.07 MB 8:50/162.24 MB 9:56/168.77 MB 10:59/175.93 MB 11:37/111.28 MB
Hdfs Read Thread Concurrency Bucket: 0:97.65% 1:2.349% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:27119
ExecOption: Codegen enabled: 0 out of 577
BytesRead(16s000ms): 7.57 MB, 31.71 MB, 57.71 MB, 75.44 MB, 94.55 MB, 118.91 MB, 147.84 MB, 166.50 MB, 192.25 MB, 212.82 MB, 232.91 MB, 256.20 MB, 281.31 MB, 306.04 MB, 328.30 MB, 360.88 MB, 396.07 MB, 429.43 MB, 440.82 MB, 457.26 MB, 484.99 MB, 514.66 MB, 542.27 MB, 564.58 MB, 583.85 MB, 603.35 MB, 624.11 MB, 645.73 MB, 673.40 MB, 691.60 MB, 713.46 MB, 742.43 MB, 764.48 MB, 786.82 MB, 808.14 MB, 817.12 MB, 821.31 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 10.50
- BytesRead: 822.69 MB (862654526)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 822.69 MB (862654526)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 822.69 MB (862654526)
- DecompressionTime: 352.912ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 142 (142)
- PeakMemoryUsage: 76.15 MB (79854120)
- PerReadThreadRawHdfsThroughput: 61.79 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.50M (11502364)
- RowsReturned: 22 (22)
- RowsReturnedRate: 0
- ScanRangesComplete: 577 (577)
- ScannerThreadsInvoluntaryContextSwitches: 518 (518)
- ScannerThreadsTotalWallClockTime: 1h44m
- MaterializeTupleTime(*): 15s775ms
- ScannerThreadsSysTime: 600.14ms
- ScannerThreadsUserTime: 18s393ms
- ScannerThreadsVoluntaryContextSwitches: 43.68K (43681)
- TotalRawHdfsReadTime(*): 13s313ms
- TotalReadThroughput: 1.38 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016b5 (host=hadoopc1h74:28002):(Total: 9m55s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:43/129.61 MB 1:51/142.97 MB 2:49/140.18 MB 3:57/152.98 MB 4:45/120.54 MB 5:48/136.81 MB 6:47/166.11 MB 7:50/116.47 MB 8:38/122.59 MB 9:61/174.27 MB 10:52/149.65 MB 11:46/135.44 MB
MemoryUsage(16s000ms): 27.84 MB, 37.29 MB, 30.91 MB, 52.84 MB, 42.17 MB, 43.61 MB, 50.01 MB, 39.12 MB, 35.55 MB, 32.39 MB, 47.19 MB, 31.91 MB, 32.91 MB, 31.60 MB, 38.82 MB, 31.85 MB, 41.01 MB, 45.59 MB, 50.45 MB, 37.12 MB, 51.44 MB, 48.72 MB, 55.73 MB, 34.13 MB, 31.55 MB, 35.30 MB, 58.10 MB, 43.65 MB, 49.50 MB, 30.43 MB, 31.36 MB, 24.94 MB, 23.79 MB, 20.73 MB, 17.77 MB, 14.43 MB, 6.32 MB
ThreadUsage(16s000ms): 10, 11, 11, 11, 12, 13, 14, 11, 11, 11, 10, 9, 9, 10, 9, 9, 9, 10, 10, 10, 12, 13, 13, 10, 10, 12, 11, 10, 8, 8, 10, 9, 9, 11, 12, 8, 3
- AverageThreadTokens: 10.86
- PeakMemoryUsage: 86.46 MB (90655240)
- PerHostPeakMemUsage: 86.46 MB (90655240)
- PrepareTime: 195.471ms
- RowsProduced: 23 (23)
- TotalCpuTime: 1h38m
- TotalNetworkReceiveTime: 5s421ms
- TotalNetworkSendTime: 1.715us
- TotalStorageWaitTime: 1h37m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 680.724ms, non-child: 680.724ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 138.23ms
- LoadTime: 41.63us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 360.442ms
- PrepareTime: 180.493ms
DataStreamSender (dst_id=13):(Total: 379.410us, non-child: 379.410us, % non-child: 100.00%)
- BytesSent: 6.31 KB (6466)
- NetworkThroughput(*): 1.75 MB/sec
- OverallThroughput: 16.25 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 267.419us
- ThriftTransmitTime(*): 3.531ms
- UncompressedRowBatchSize: 10.29 KB (10532)
HASH_JOIN_NODE (id=6):(Total: 9m55s, non-child: 569.168ms, % non-child: 0.10%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.487us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.434us
- GetNewBlockTime: 82.103us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.53 MB (26773856)
- PinTime: 0ns
- ProbeRows: 23 (23)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 191.185us
- RowsReturned: 23 (23)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 858ns
EXCHANGE_NODE (id=12):(Total: 2s587ms, non-child: 28.358us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.728us
- DeserializeRowBatchTimer: 14.896us
- FirstBatchArrivalWaitTime: 2s579ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m54s, non-child: 2s852ms, % non-child: 0.48%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 12.310us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.646us
- GetNewBlockTime: 57.557us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 16.11 MB (16896968)
- PinTime: 0ns
- ProbeRows: 23 (23)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 203.711us
- RowsReturned: 23 (23)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 855ns
EXCHANGE_NODE (id=9):(Total: 2s833ms, non-child: 21.382us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.629us
- DeserializeRowBatchTimer: 11.400us
- FirstBatchArrivalWaitTime: 2s831ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m51s, non-child: 9m51s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:43/129.61 MB 1:51/142.97 MB 2:49/140.18 MB 3:57/152.98 MB 4:45/120.54 MB 5:48/136.81 MB 6:47/166.11 MB 7:50/116.47 MB 8:38/122.59 MB 9:61/174.27 MB 10:52/149.65 MB 11:46/135.44 MB
Hdfs Read Thread Concurrency Bucket: 0:97.82% 1:2.101% 2:0.08403% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:27589
ExecOption: Codegen enabled: 0 out of 587
BytesRead(16s000ms): 9.79 MB, 34.25 MB, 50.97 MB, 69.77 MB, 86.32 MB, 105.13 MB, 125.84 MB, 141.93 MB, 158.90 MB, 170.77 MB, 188.84 MB, 214.79 MB, 238.82 MB, 260.84 MB, 284.32 MB, 314.10 MB, 353.66 MB, 395.35 MB, 418.58 MB, 435.84 MB, 473.95 MB, 505.89 MB, 540.73 MB, 564.49 MB, 585.05 MB, 601.02 MB, 627.53 MB, 655.26 MB, 696.91 MB, 720.95 MB, 744.53 MB, 768.27 MB, 786.52 MB, 803.72 MB, 817.80 MB, 825.48 MB, 830.71 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 9.87
- BytesRead: 832.40 MB (872835506)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 832.40 MB (872835506)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 832.40 MB (872835506)
- DecompressionTime: 1s414ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 126 (126)
- PeakMemoryUsage: 82.43 MB (86433912)
- PerReadThreadRawHdfsThroughput: 53.74 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.52M (11522240)
- RowsReturned: 23 (23)
- RowsReturnedRate: 0
- ScanRangesComplete: 587 (587)
- ScannerThreadsInvoluntaryContextSwitches: 717 (717)
- ScannerThreadsTotalWallClockTime: 1h37m
- MaterializeTupleTime(*): 15s845ms
- ScannerThreadsSysTime: 560.10ms
- ScannerThreadsUserTime: 18s821ms
- ScannerThreadsVoluntaryContextSwitches: 45.81K (45807)
- TotalRawHdfsReadTime(*): 15s488ms
- TotalReadThroughput: 1.40 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016af (host=hadoopc1h19:28002):(Total: 9m54s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:41/148.49 MB 1:41/95.54 MB 2:44/150.84 MB 3:42/143.84 MB 4:47/126.37 MB 5:46/127.56 MB 6:51/153.69 MB 7:48/141.39 MB 8:55/198.41 MB 9:57/187.63 MB 10:41/116.34 MB 11:39/100.22 MB
MemoryUsage(16s000ms): 24.06 MB, 42.08 MB, 27.88 MB, 32.49 MB, 42.06 MB, 37.06 MB, 38.16 MB, 39.33 MB, 55.63 MB, 55.19 MB, 44.77 MB, 28.86 MB, 39.32 MB, 35.29 MB, 38.01 MB, 32.42 MB, 24.09 MB, 33.67 MB, 29.35 MB, 45.15 MB, 41.02 MB, 56.39 MB, 39.44 MB, 34.67 MB, 30.02 MB, 38.15 MB, 40.81 MB, 35.55 MB, 37.81 MB, 38.18 MB, 30.63 MB, 27.47 MB, 26.50 MB, 25.67 MB, 19.94 MB, 12.57 MB, 11.04 MB
ThreadUsage(16s000ms): 9, 10, 10, 10, 11, 11, 11, 10, 10, 9, 8, 8, 7, 7, 7, 9, 8, 8, 8, 9, 10, 10, 10, 8, 8, 8, 9, 9, 7, 7, 8, 8, 9, 11, 12, 7, 3
- AverageThreadTokens: 9.43
- PeakMemoryUsage: 84.76 MB (88877656)
- PerHostPeakMemUsage: 84.76 MB (88877656)
- PrepareTime: 196.527ms
- RowsProduced: 10 (10)
- TotalCpuTime: 1h23m
- TotalNetworkReceiveTime: 5s488ms
- TotalNetworkSendTime: 1.235us
- TotalStorageWaitTime: 1h23m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 651.151ms, non-child: 651.151ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 126.739ms
- LoadTime: 152.400us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 339.842ms
- PrepareTime: 182.901ms
DataStreamSender (dst_id=13):(Total: 214.759us, non-child: 214.759us, % non-child: 100.00%)
- BytesSent: 3.12 KB (3197)
- NetworkThroughput(*): 1.35 MB/sec
- OverallThroughput: 14.20 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 139.930us
- ThriftTransmitTime(*): 2.265ms
- UncompressedRowBatchSize: 4.54 KB (4644)
HASH_JOIN_NODE (id=6):(Total: 9m54s, non-child: 196.395ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 5.248us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.558us
- GetNewBlockTime: 61.868us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.06 MB (26276176)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 151.966us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 851ns
EXCHANGE_NODE (id=12):(Total: 2s617ms, non-child: 19.558us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.786us
- DeserializeRowBatchTimer: 14.381us
- FirstBatchArrivalWaitTime: 2s615ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m54s, non-child: 2s892ms, % non-child: 0.49%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.764us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 12.497us
- GetNewBlockTime: 54.226us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.84 MB (16612904)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 147.532us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 795ns
EXCHANGE_NODE (id=9):(Total: 2s871ms, non-child: 42.195us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.9us
- DeserializeRowBatchTimer: 14.901us
- FirstBatchArrivalWaitTime: 2s867ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m51s, non-child: 9m51s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:41/148.49 MB 1:41/95.54 MB 2:44/150.84 MB 3:42/143.84 MB 4:47/126.37 MB 5:46/127.56 MB 6:51/153.69 MB 7:48/141.39 MB 8:55/198.41 MB 9:57/187.63 MB 10:41/116.34 MB 11:39/100.22 MB
Hdfs Read Thread Concurrency Bucket: 0:97.73% 1:2.187% 2:0.0841% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:25944
ExecOption: Codegen enabled: 0 out of 552
BytesRead(16s000ms): 8.22 MB, 37.46 MB, 57.24 MB, 73.00 MB, 89.63 MB, 111.51 MB, 131.09 MB, 146.83 MB, 179.38 MB, 204.74 MB, 227.51 MB, 249.28 MB, 281.32 MB, 312.90 MB, 344.72 MB, 374.96 MB, 400.47 MB, 425.43 MB, 438.17 MB, 454.18 MB, 481.29 MB, 511.33 MB, 528.11 MB, 548.24 MB, 566.62 MB, 584.42 MB, 603.62 MB, 625.28 MB, 660.72 MB, 689.00 MB, 711.99 MB, 741.59 MB, 766.97 MB, 792.16 MB, 809.08 MB, 818.74 MB, 823.71 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 8.43
- BytesRead: 829.02 MB (869292869)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 829.02 MB (869292869)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 829.02 MB (869292869)
- DecompressionTime: 830.154ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 107 (107)
- PeakMemoryUsage: 73.64 MB (77212984)
- PerReadThreadRawHdfsThroughput: 73.68 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.59M (11587107)
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- ScanRangesComplete: 552 (552)
- ScannerThreadsInvoluntaryContextSwitches: 671 (671)
- ScannerThreadsTotalWallClockTime: 1h23m
- MaterializeTupleTime(*): 15s299ms
- ScannerThreadsSysTime: 444.6ms
- ScannerThreadsUserTime: 18s117ms
- ScannerThreadsVoluntaryContextSwitches: 48.10K (48100)
- TotalRawHdfsReadTime(*): 11s251ms
- TotalReadThroughput: 1.39 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016be (host=hadoopc1h9:28002):(Total: 9m46s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:45/117.60 MB 1:57/145.26 MB 2:46/114.71 MB 3:40/103.31 MB 4:46/109.39 MB 5:53/158.10 MB 6:43/123.51 MB 7:53/155.23 MB 8:51/112.58 MB 9:55/177.98 MB 10:57/190.81 MB 11:53/160.02 MB
MemoryUsage(16s000ms): 19.32 MB, 31.34 MB, 33.42 MB, 50.98 MB, 50.91 MB, 42.31 MB, 41.89 MB, 32.33 MB, 45.86 MB, 28.54 MB, 29.67 MB, 27.42 MB, 31.57 MB, 28.61 MB, 37.58 MB, 37.27 MB, 30.48 MB, 28.42 MB, 35.63 MB, 39.77 MB, 37.02 MB, 39.25 MB, 32.60 MB, 34.21 MB, 30.81 MB, 27.14 MB, 39.49 MB, 37.67 MB, 35.00 MB, 42.35 MB, 25.15 MB, 24.07 MB, 24.00 MB, 40.59 MB, 31.08 MB, 23.45 MB
ThreadUsage(16s000ms): 9, 10, 10, 11, 12, 12, 12, 10, 10, 8, 8, 7, 8, 9, 9, 8, 7, 7, 8, 8, 8, 8, 10, 8, 8, 8, 9, 9, 7, 7, 9, 9, 10, 13, 18, 9
- AverageThreadTokens: 9.79
- PeakMemoryUsage: 79.79 MB (83663176)
- PerHostPeakMemUsage: 79.79 MB (83663176)
- PrepareTime: 212.754ms
- RowsProduced: 18 (18)
- TotalCpuTime: 1h25m
- TotalNetworkReceiveTime: 5s418ms
- TotalNetworkSendTime: 1.636us
- TotalStorageWaitTime: 1h25m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 686.454ms, non-child: 686.454ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 133.279ms
- LoadTime: 156.882us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 354.157ms
- PrepareTime: 196.901ms
DataStreamSender (dst_id=13):(Total: 295.723us, non-child: 295.723us, % non-child: 100.00%)
- BytesSent: 4.87 KB (4985)
- NetworkThroughput(*): 1.29 MB/sec
- OverallThroughput: 16.08 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 204.811us
- ThriftTransmitTime(*): 3.681ms
- UncompressedRowBatchSize: 8.08 KB (8278)
HASH_JOIN_NODE (id=6):(Total: 9m46s, non-child: 213.182ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.94us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.605us
- GetNewBlockTime: 67.666us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.69 MB (26941664)
- PinTime: 0ns
- ProbeRows: 18 (18)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 200.805us
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 931ns
EXCHANGE_NODE (id=12):(Total: 2s581ms, non-child: 26.522us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.945us
- DeserializeRowBatchTimer: 43.169us
- FirstBatchArrivalWaitTime: 2s578ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m46s, non-child: 2s857ms, % non-child: 0.49%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.446us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.55us
- GetNewBlockTime: 54.43us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.26 MB (16002120)
- PinTime: 0ns
- ProbeRows: 18 (18)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 160.506us
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 945ns
EXCHANGE_NODE (id=9):(Total: 2s836ms, non-child: 20.616us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.447us
- DeserializeRowBatchTimer: 20.105us
- FirstBatchArrivalWaitTime: 2s829ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m43s, non-child: 9m43s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:45/117.60 MB 1:57/145.26 MB 2:46/114.71 MB 3:40/103.31 MB 4:46/109.39 MB 5:53/158.10 MB 6:43/123.51 MB 7:53/155.23 MB 8:51/112.58 MB 9:55/177.98 MB 10:57/190.81 MB 11:53/160.02 MB
Hdfs Read Thread Concurrency Bucket: 0:98.38% 1:1.62% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28153
ExecOption: Codegen enabled: 0 out of 599
BytesRead(16s000ms): 5.85 MB, 26.00 MB, 45.40 MB, 65.70 MB, 84.60 MB, 101.15 MB, 122.28 MB, 136.80 MB, 154.92 MB, 170.99 MB, 185.45 MB, 209.33 MB, 243.57 MB, 263.38 MB, 287.33 MB, 325.49 MB, 361.20 MB, 389.91 MB, 406.19 MB, 424.86 MB, 446.66 MB, 473.43 MB, 493.55 MB, 516.61 MB, 534.62 MB, 548.54 MB, 567.55 MB, 594.45 MB, 628.13 MB, 660.04 MB, 684.69 MB, 710.45 MB, 732.37 MB, 760.10 MB, 798.20 MB, 814.94 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 8.78
- BytesRead: 822.20 MB (862139399)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 822.20 MB (862139399)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 822.20 MB (862139399)
- DecompressionTime: 1s530ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 121 (121)
- PeakMemoryUsage: 67.96 MB (71258848)
- PerReadThreadRawHdfsThroughput: 79.17 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.43M (11429596)
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- ScanRangesComplete: 599 (599)
- ScannerThreadsInvoluntaryContextSwitches: 856 (856)
- ScannerThreadsTotalWallClockTime: 1h25m
- MaterializeTupleTime(*): 15s620ms
- ScannerThreadsSysTime: 472.5ms
- ScannerThreadsUserTime: 18s497ms
- ScannerThreadsVoluntaryContextSwitches: 53.66K (53664)
- TotalRawHdfsReadTime(*): 10s384ms
- TotalReadThroughput: 1.40 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016cb (host=hadoopc1h78:28002):(Total: 9m45s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:55/170.62 MB 1:46/127.72 MB 2:41/114.79 MB 3:50/162.48 MB 4:41/142.77 MB 5:41/130.27 MB 6:52/183.71 MB 7:47/128.47 MB 8:50/136.14 MB 9:43/91.77 MB 10:52/201.38 MB 11:44/106.78 MB
MemoryUsage(16s000ms): 17.01 MB, 40.76 MB, 57.97 MB, 52.43 MB, 51.71 MB, 57.36 MB, 40.46 MB, 47.30 MB, 51.61 MB, 43.53 MB, 36.97 MB, 33.13 MB, 25.53 MB, 30.97 MB, 27.41 MB, 33.12 MB, 28.20 MB, 45.11 MB, 35.19 MB, 36.03 MB, 49.52 MB, 46.05 MB, 48.46 MB, 41.59 MB, 40.75 MB, 35.49 MB, 31.96 MB, 39.61 MB, 31.83 MB, 27.13 MB, 26.00 MB, 43.12 MB, 34.12 MB, 36.51 MB, 27.06 MB, 19.44 MB
ThreadUsage(16s000ms): 11, 12, 12, 12, 13, 15, 15, 12, 13, 10, 10, 8, 8, 8, 8, 9, 8, 9, 10, 11, 11, 11, 11, 9, 10, 9, 10, 10, 8, 9, 9, 10, 9, 12, 14, 8
- AverageThreadTokens: 10.91
- PeakMemoryUsage: 84.12 MB (88205520)
- PerHostPeakMemUsage: 84.12 MB (88205520)
- PrepareTime: 201.963ms
- RowsProduced: 21 (21)
- TotalCpuTime: 1h36m
- TotalNetworkReceiveTime: 5s355ms
- TotalNetworkSendTime: 1.706us
- TotalStorageWaitTime: 1h36m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 718.621ms, non-child: 718.621ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 148.31ms
- LoadTime: 149.800us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 382.402ms
- PrepareTime: 186.294ms
DataStreamSender (dst_id=13):(Total: 403.17us, non-child: 403.17us, % non-child: 100.00%)
- BytesSent: 5.98 KB (6120)
- NetworkThroughput(*): 2.10 MB/sec
- OverallThroughput: 14.48 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 303.990us
- ThriftTransmitTime(*): 2.783ms
- UncompressedRowBatchSize: 9.44 KB (9666)
HASH_JOIN_NODE (id=6):(Total: 9m45s, non-child: 202.505ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 9.270us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.834us
- GetNewBlockTime: 75.583us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 20.63 MB (21633008)
- PinTime: 0ns
- ProbeRows: 21 (21)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 210.422us
- RowsReturned: 21 (21)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 951ns
EXCHANGE_NODE (id=12):(Total: 2s554ms, non-child: 34.558us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.984us
- DeserializeRowBatchTimer: 12.201us
- FirstBatchArrivalWaitTime: 2s545ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m45s, non-child: 2s819ms, % non-child: 0.48%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 10.289us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 6.590us
- GetNewBlockTime: 85.877us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 16.27 MB (17061056)
- PinTime: 0ns
- ProbeRows: 21 (21)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 222.91us
- RowsReturned: 21 (21)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.434us
EXCHANGE_NODE (id=9):(Total: 2s801ms, non-child: 32.486us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.704us
- DeserializeRowBatchTimer: 10.484us
- FirstBatchArrivalWaitTime: 2s797ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m42s, non-child: 9m42s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:55/170.62 MB 1:46/127.72 MB 2:41/114.79 MB 3:50/162.48 MB 4:41/142.77 MB 5:41/130.27 MB 6:52/183.71 MB 7:47/128.47 MB 8:50/136.14 MB 9:43/91.77 MB 10:52/201.38 MB 11:44/106.78 MB
Hdfs Read Thread Concurrency Bucket: 0:97.44% 1:2.477% 2:0.0854% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:26414
ExecOption: Codegen enabled: 0 out of 562
BytesRead(16s000ms): 4.93 MB, 26.96 MB, 53.67 MB, 73.07 MB, 90.67 MB, 115.83 MB, 134.59 MB, 151.29 MB, 173.16 MB, 188.88 MB, 202.39 MB, 223.11 MB, 243.98 MB, 265.84 MB, 289.58 MB, 313.66 MB, 347.01 MB, 378.83 MB, 393.49 MB, 406.20 MB, 432.58 MB, 456.73 MB, 479.30 MB, 507.13 MB, 536.32 MB, 552.23 MB, 566.19 MB, 590.53 MB, 625.94 MB, 644.56 MB, 662.52 MB, 692.98 MB, 728.26 MB, 762.63 MB, 799.66 MB, 818.80 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 9.92
- BytesRead: 831.74 MB (872143164)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 831.74 MB (872143164)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 831.74 MB (872143164)
- DecompressionTime: 420.771ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 136 (136)
- PeakMemoryUsage: 78.50 MB (82311856)
- PerReadThreadRawHdfsThroughput: 59.14 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.67M (11673990)
- RowsReturned: 21 (21)
- RowsReturnedRate: 0
- ScanRangesComplete: 562 (562)
- ScannerThreadsInvoluntaryContextSwitches: 859 (859)
- ScannerThreadsTotalWallClockTime: 1h36m
- MaterializeTupleTime(*): 17s026ms
- ScannerThreadsSysTime: 540.11ms
- ScannerThreadsUserTime: 19s861ms
- ScannerThreadsVoluntaryContextSwitches: 45.31K (45314)
- TotalRawHdfsReadTime(*): 14s064ms
- TotalReadThroughput: 1.42 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016ac (host=hadoopc1h31:28002):(Total: 9m39s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:44/130.22 MB 1:51/157.79 MB 2:47/148.77 MB 3:61/184.83 MB 4:36/120.61 MB 5:40/149.41 MB 6:44/128.26 MB 7:44/155.49 MB 8:37/97.94 MB 9:44/130.80 MB 10:53/195.30 MB 11:40/119.17 MB
MemoryUsage(16s000ms): 19.17 MB, 31.64 MB, 43.20 MB, 44.41 MB, 45.34 MB, 39.59 MB, 57.48 MB, 40.21 MB, 40.93 MB, 35.67 MB, 41.33 MB, 23.06 MB, 28.23 MB, 27.60 MB, 21.25 MB, 29.82 MB, 25.60 MB, 32.87 MB, 29.90 MB, 41.49 MB, 40.17 MB, 39.92 MB, 47.47 MB, 30.16 MB, 37.18 MB, 30.33 MB, 38.17 MB, 43.66 MB, 28.98 MB, 34.60 MB, 42.21 MB, 18.75 MB, 18.32 MB, 13.71 MB, 9.17 MB, 14.99 MB
ThreadUsage(16s000ms): 9, 9, 10, 10, 11, 12, 11, 10, 10, 8, 7, 7, 6, 6, 6, 7, 6, 7, 8, 9, 9, 9, 11, 9, 8, 8, 10, 9, 7, 7, 7, 8, 8, 10, 8, 5
- AverageThreadTokens: 9.06
- PeakMemoryUsage: 84.32 MB (88416024)
- PerHostPeakMemUsage: 84.32 MB (88416024)
- PrepareTime: 194.880ms
- RowsProduced: 19 (19)
- TotalCpuTime: 1h17m
- TotalNetworkReceiveTime: 5s481ms
- TotalNetworkSendTime: 1.655us
- TotalStorageWaitTime: 1h17m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 664.867ms, non-child: 664.867ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 130.758ms
- LoadTime: 129.950us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 351.801ms
- PrepareTime: 180.279ms
DataStreamSender (dst_id=13):(Total: 283.919us, non-child: 283.919us, % non-child: 100.00%)
- BytesSent: 5.43 KB (5561)
- NetworkThroughput(*): 1.67 MB/sec
- OverallThroughput: 18.68 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 184.432us
- ThriftTransmitTime(*): 3.181ms
- UncompressedRowBatchSize: 8.47 KB (8676)
HASH_JOIN_NODE (id=6):(Total: 9m39s, non-child: 195.89ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.2us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.544us
- GetNewBlockTime: 130.988us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 22.23 MB (23307344)
- PinTime: 0ns
- ProbeRows: 19 (19)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 186.706us
- RowsReturned: 19 (19)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 782ns
EXCHANGE_NODE (id=12):(Total: 2s609ms, non-child: 23.394us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.674us
- DeserializeRowBatchTimer: 7.785us
- FirstBatchArrivalWaitTime: 2s605ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m39s, non-child: 2s895ms, % non-child: 0.50%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.726us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.834us
- GetNewBlockTime: 131.203us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.26 MB (15999968)
- PinTime: 0ns
- ProbeRows: 19 (19)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 176.263us
- RowsReturned: 19 (19)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.255us
EXCHANGE_NODE (id=9):(Total: 2s871ms, non-child: 22.889us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.107us
- DeserializeRowBatchTimer: 8.160us
- FirstBatchArrivalWaitTime: 2s858ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m36s, non-child: 9m36s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:44/130.22 MB 1:51/157.79 MB 2:47/148.77 MB 3:61/184.83 MB 4:36/120.61 MB 5:40/149.41 MB 6:44/128.26 MB 7:44/155.49 MB 8:37/97.94 MB 9:44/130.80 MB 10:53/195.30 MB 11:40/119.17 MB
Hdfs Read Thread Concurrency Bucket: 0:98.28% 1:1.638% 2:0.08621% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:25427
ExecOption: Codegen enabled: 0 out of 541
BytesRead(16s000ms): 6.11 MB, 23.79 MB, 53.91 MB, 75.94 MB, 92.72 MB, 113.20 MB, 143.05 MB, 163.61 MB, 195.22 MB, 214.11 MB, 237.42 MB, 255.80 MB, 281.90 MB, 306.13 MB, 325.46 MB, 359.69 MB, 401.26 MB, 432.34 MB, 451.66 MB, 474.05 MB, 508.74 MB, 546.41 MB, 582.57 MB, 611.47 MB, 634.74 MB, 650.48 MB, 668.56 MB, 695.91 MB, 723.61 MB, 747.81 MB, 780.00 MB, 797.94 MB, 811.64 MB, 821.15 MB, 827.52 MB, 834.23 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 8.06
- BytesRead: 841.31 MB (882181084)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 841.31 MB (882181084)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 841.31 MB (882181084)
- DecompressionTime: 1s787ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 132 (132)
- PeakMemoryUsage: 83.77 MB (87838424)
- PerReadThreadRawHdfsThroughput: 70.97 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.80M (11798556)
- RowsReturned: 19 (19)
- RowsReturnedRate: 0
- ScanRangesComplete: 541 (541)
- ScannerThreadsInvoluntaryContextSwitches: 497 (497)
- ScannerThreadsTotalWallClockTime: 1h17m
- MaterializeTupleTime(*): 15s891ms
- ScannerThreadsSysTime: 488.7ms
- ScannerThreadsUserTime: 18s649ms
- ScannerThreadsVoluntaryContextSwitches: 47.86K (47859)
- TotalRawHdfsReadTime(*): 11s854ms
- TotalReadThroughput: 1.45 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016ad (host=hadoopc1h84:28002):(Total: 9m39s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:55/169.16 MB 1:60/142.38 MB 2:51/162.98 MB 3:55/158.96 MB 4:45/124.59 MB 5:48/130.78 MB 6:45/109.53 MB 7:49/107.32 MB 8:52/112.22 MB 9:54/134.84 MB 10:44/128.82 MB 11:50/127.87 MB
MemoryUsage(16s000ms): 34.47 MB, 31.67 MB, 42.29 MB, 41.74 MB, 40.06 MB, 42.23 MB, 64.18 MB, 44.37 MB, 49.87 MB, 66.26 MB, 52.13 MB, 43.27 MB, 48.40 MB, 42.60 MB, 38.94 MB, 46.63 MB, 41.21 MB, 41.88 MB, 40.51 MB, 43.33 MB, 37.58 MB, 44.41 MB, 55.49 MB, 49.69 MB, 44.65 MB, 53.15 MB, 57.43 MB, 42.54 MB, 38.68 MB, 34.21 MB, 39.18 MB, 37.27 MB, 33.27 MB, 31.66 MB, 13.85 MB, 11.32 MB
ThreadUsage(16s000ms): 14, 15, 14, 13, 14, 17, 16, 15, 17, 17, 17, 16, 13, 17, 17, 16, 15, 19, 20, 19, 17, 18, 20, 19, 17, 14, 18, 18, 14, 13, 13, 17, 18, 18, 8, 4
- AverageThreadTokens: 16.32
- PeakMemoryUsage: 83.81 MB (87878320)
- PerHostPeakMemUsage: 83.81 MB (87878320)
- PrepareTime: 193.787ms
- RowsProduced: 15 (15)
- TotalCpuTime: 2h28m
- TotalNetworkReceiveTime: 5s464ms
- TotalNetworkSendTime: 1.331us
- TotalStorageWaitTime: 2h27m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 674.201ms, non-child: 674.201ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 133.195ms
- LoadTime: 152.905us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 360.383ms
- PrepareTime: 178.703ms
DataStreamSender (dst_id=13):(Total: 248.358us, non-child: 248.358us, % non-child: 100.00%)
- BytesSent: 3.92 KB (4016)
- NetworkThroughput(*): 1.88 MB/sec
- OverallThroughput: 15.42 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 177.126us
- ThriftTransmitTime(*): 2.32ms
- UncompressedRowBatchSize: 6.74 KB (6899)
HASH_JOIN_NODE (id=6):(Total: 9m39s, non-child: 194.197ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.63us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.500us
- GetNewBlockTime: 59.740us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.89 MB (26097104)
- PinTime: 0ns
- ProbeRows: 15 (15)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 183.294us
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 734ns
EXCHANGE_NODE (id=12):(Total: 2s602ms, non-child: 22.924us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.536us
- DeserializeRowBatchTimer: 16.971us
- FirstBatchArrivalWaitTime: 2s597ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m39s, non-child: 2s881ms, % non-child: 0.50%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.1us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.420us
- GetNewBlockTime: 53.109us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 14.49 MB (15198920)
- PinTime: 0ns
- ProbeRows: 15 (15)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 197.241us
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 829ns
EXCHANGE_NODE (id=9):(Total: 2s862ms, non-child: 25.758us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.851us
- DeserializeRowBatchTimer: 17.387us
- FirstBatchArrivalWaitTime: 2s849ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m36s, non-child: 9m36s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:55/169.16 MB 1:60/142.38 MB 2:51/162.98 MB 3:55/158.96 MB 4:45/124.59 MB 5:48/130.78 MB 6:45/109.53 MB 7:49/107.32 MB 8:52/112.22 MB 9:54/134.84 MB 10:44/128.82 MB 11:50/127.87 MB
Hdfs Read Thread Concurrency Bucket: 0:98.02% 1:1.897% 2:0.08621% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28576
ExecOption: Codegen enabled: 0 out of 608
BytesRead(16s000ms): 9.55 MB, 28.35 MB, 46.50 MB, 60.27 MB, 72.57 MB, 88.26 MB, 112.43 MB, 127.45 MB, 145.63 MB, 167.49 MB, 183.42 MB, 203.26 MB, 235.81 MB, 258.02 MB, 275.52 MB, 307.06 MB, 342.65 MB, 380.18 MB, 395.70 MB, 412.72 MB, 442.14 MB, 470.22 MB, 499.07 MB, 530.35 MB, 552.03 MB, 570.30 MB, 590.11 MB, 613.78 MB, 643.51 MB, 665.11 MB, 693.32 MB, 721.33 MB, 745.93 MB, 770.85 MB, 787.71 MB, 790.95 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 15.32
- BytesRead: 796.83 MB (835535513)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 796.83 MB (835535513)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 796.83 MB (835535513)
- DecompressionTime: 589.355ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 150 (150)
- PeakMemoryUsage: 77.26 MB (81014864)
- PerReadThreadRawHdfsThroughput: 61.86 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 10.99M (10993161)
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- ScanRangesComplete: 608 (608)
- ScannerThreadsInvoluntaryContextSwitches: 478 (478)
- ScannerThreadsTotalWallClockTime: 2h28m
- MaterializeTupleTime(*): 14s620ms
- ScannerThreadsSysTime: 544.8ms
- ScannerThreadsUserTime: 17s337ms
- ScannerThreadsVoluntaryContextSwitches: 44.93K (44927)
- TotalRawHdfsReadTime(*): 12s880ms
- TotalReadThroughput: 1.37 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016a6 (host=hadoopc1h26:28002):(Total: 9m39s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:43/113.39 MB 1:41/128.23 MB 2:44/149.83 MB 3:47/130.25 MB 4:45/116.01 MB 5:58/179.69 MB 6:47/134.79 MB 7:46/146.69 MB 8:45/129.38 MB 9:48/143.32 MB 10:58/182.87 MB 11:37/122.71 MB
MemoryUsage(16s000ms): 21.93 MB, 32.78 MB, 35.12 MB, 44.34 MB, 35.86 MB, 41.14 MB, 45.30 MB, 51.38 MB, 29.57 MB, 44.83 MB, 35.45 MB, 29.99 MB, 33.34 MB, 21.80 MB, 25.03 MB, 35.13 MB, 29.79 MB, 24.88 MB, 33.30 MB, 43.50 MB, 38.31 MB, 35.21 MB, 42.96 MB, 25.03 MB, 30.85 MB, 45.00 MB, 41.13 MB, 34.50 MB, 29.12 MB, 30.04 MB, 27.59 MB, 36.53 MB, 27.02 MB, 13.54 MB, 15.71 MB, 14.73 MB
ThreadUsage(16s000ms): 9, 8, 11, 10, 10, 12, 12, 10, 10, 10, 7, 6, 6, 6, 6, 7, 7, 7, 8, 8, 9, 8, 10, 8, 8, 7, 9, 9, 7, 7, 7, 9, 8, 10, 12, 8
- AverageThreadTokens: 9.11
- PeakMemoryUsage: 71.41 MB (74876728)
- PerHostPeakMemUsage: 71.41 MB (74876728)
- PrepareTime: 192.556ms
- RowsProduced: 20 (20)
- TotalCpuTime: 1h18m
- TotalNetworkReceiveTime: 5s309ms
- TotalNetworkSendTime: 1.591us
- TotalStorageWaitTime: 1h17m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 749.80ms, non-child: 749.80ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 170.673ms
- LoadTime: 148.706us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 398.528ms
- PrepareTime: 178.194ms
DataStreamSender (dst_id=13):(Total: 271.770us, non-child: 271.770us, % non-child: 100.00%)
- BytesSent: 5.52 KB (5648)
- NetworkThroughput(*): 1.83 MB/sec
- OverallThroughput: 19.82 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 186.741us
- ThriftTransmitTime(*): 2.946ms
- UncompressedRowBatchSize: 8.88 KB (9090)
HASH_JOIN_NODE (id=6):(Total: 9m39s, non-child: 192.759ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.671us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.381us
- GetNewBlockTime: 60.726us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 23.86 MB (25016624)
- PinTime: 0ns
- ProbeRows: 20 (20)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 220.393us
- RowsReturned: 20 (20)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 759ns
EXCHANGE_NODE (id=12):(Total: 2s534ms, non-child: 38.297us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.292us
- DeserializeRowBatchTimer: 16.582us
- FirstBatchArrivalWaitTime: 2s521ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m39s, non-child: 2s793ms, % non-child: 0.48%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.979us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.250us
- GetNewBlockTime: 46.958us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.78 MB (16543304)
- PinTime: 0ns
- ProbeRows: 20 (20)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 186.718us
- RowsReturned: 20 (20)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 877ns
EXCHANGE_NODE (id=9):(Total: 2s775ms, non-child: 32.737us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.930us
- DeserializeRowBatchTimer: 14.218us
- FirstBatchArrivalWaitTime: 2s773ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m36s, non-child: 9m36s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:43/113.39 MB 1:41/128.23 MB 2:44/149.83 MB 3:47/130.25 MB 4:45/116.01 MB 5:58/179.69 MB 6:47/134.79 MB 7:46/146.69 MB 8:45/129.38 MB 9:48/143.32 MB 10:58/182.87 MB 11:37/122.71 MB
Hdfs Read Thread Concurrency Bucket: 0:98.7% 1:1.295% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:26273
ExecOption: Codegen enabled: 0 out of 559
BytesRead(16s000ms): 8.03 MB, 32.18 MB, 54.64 MB, 73.30 MB, 86.72 MB, 102.80 MB, 128.50 MB, 150.97 MB, 167.98 MB, 186.10 MB, 204.86 MB, 224.75 MB, 257.39 MB, 277.64 MB, 302.60 MB, 340.74 MB, 382.74 MB, 409.11 MB, 423.60 MB, 441.42 MB, 468.20 MB, 490.51 MB, 518.21 MB, 541.61 MB, 560.68 MB, 585.22 MB, 604.57 MB, 627.72 MB, 654.30 MB, 684.33 MB, 714.57 MB, 753.12 MB, 783.85 MB, 798.19 MB, 806.15 MB, 814.23 MB
- AverageHdfsReadThreadConcurrency: 0.01
- AverageScannerThreadConcurrency: 8.11
- BytesRead: 823.86 MB (863875554)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 823.86 MB (863875554)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 823.86 MB (863875554)
- DecompressionTime: 1s132ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 125 (125)
- PeakMemoryUsage: 69.74 MB (73125832)
- PerReadThreadRawHdfsThroughput: 96.28 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.52M (11516306)
- RowsReturned: 20 (20)
- RowsReturnedRate: 0
- ScanRangesComplete: 559 (559)
- ScannerThreadsInvoluntaryContextSwitches: 500 (500)
- ScannerThreadsTotalWallClockTime: 1h18m
- MaterializeTupleTime(*): 15s393ms
- ScannerThreadsSysTime: 444.4ms
- ScannerThreadsUserTime: 18s237ms
- ScannerThreadsVoluntaryContextSwitches: 49.69K (49692)
- TotalRawHdfsReadTime(*): 8s557ms
- TotalReadThroughput: 1.42 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016c6 (host=hadoopc1h143:28002):(Total: 9m35s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:41/124.83 MB 1:46/141.05 MB 2:53/161.56 MB 3:43/154.50 MB 4:46/150.90 MB 5:38/94.89 MB 6:54/175.37 MB 7:42/108.34 MB 8:48/149.38 MB 9:50/130.27 MB 10:55/154.99 MB 11:41/145.16 MB
MemoryUsage(16s000ms): 25.46 MB, 41.13 MB, 50.75 MB, 46.99 MB, 36.40 MB, 65.63 MB, 57.02 MB, 42.27 MB, 55.79 MB, 37.42 MB, 35.43 MB, 39.24 MB, 34.05 MB, 32.30 MB, 34.20 MB, 39.74 MB, 28.87 MB, 38.86 MB, 39.80 MB, 50.83 MB, 42.78 MB, 36.32 MB, 43.48 MB, 43.67 MB, 36.90 MB, 28.88 MB, 39.14 MB, 38.82 MB, 27.99 MB, 36.16 MB, 40.04 MB, 36.22 MB, 33.94 MB, 19.55 MB, 25.28 MB, 12.47 MB
ThreadUsage(16s000ms): 11, 11, 13, 12, 12, 15, 15, 12, 12, 10, 9, 8, 8, 8, 8, 9, 8, 8, 10, 9, 10, 9, 11, 10, 9, 9, 10, 10, 9, 9, 9, 11, 10, 10, 13, 5
- AverageThreadTokens: 10.85
- PeakMemoryUsage: 82.98 MB (87009840)
- PerHostPeakMemUsage: 82.98 MB (87009840)
- PrepareTime: 188.125ms
- RowsProduced: 14 (14)
- TotalCpuTime: 1h34m
- TotalNetworkReceiveTime: 5s360ms
- TotalNetworkSendTime: 1.326us
- TotalStorageWaitTime: 1h34m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 709.251ms, non-child: 709.251ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 143.284ms
- LoadTime: 143.663us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 390.820ms
- PrepareTime: 173.334ms
DataStreamSender (dst_id=13):(Total: 237.706us, non-child: 237.706us, % non-child: 100.00%)
- BytesSent: 4.25 KB (4356)
- NetworkThroughput(*): 1.64 MB/sec
- OverallThroughput: 17.48 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 156.817us
- ThriftTransmitTime(*): 2.535ms
- UncompressedRowBatchSize: 6.26 KB (6415)
HASH_JOIN_NODE (id=6):(Total: 9m35s, non-child: 188.461ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 12.160us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.601us
- GetNewBlockTime: 72.264us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 22.99 MB (24109080)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 170.814us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 787ns
EXCHANGE_NODE (id=12):(Total: 2s558ms, non-child: 32.989us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.279us
- DeserializeRowBatchTimer: 21.932us
- FirstBatchArrivalWaitTime: 2s545ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m35s, non-child: 2s818ms, % non-child: 0.49%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.376us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 6.833us
- GetNewBlockTime: 64.888us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 14.92 MB (15646640)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 175.172us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.326us
EXCHANGE_NODE (id=9):(Total: 2s801ms, non-child: 27.603us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.539us
- DeserializeRowBatchTimer: 18.953us
- FirstBatchArrivalWaitTime: 2s797ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m32s, non-child: 9m32s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:41/124.83 MB 1:46/141.05 MB 2:53/161.56 MB 3:43/154.50 MB 4:46/150.90 MB 5:38/94.89 MB 6:54/175.37 MB 7:42/108.34 MB 8:48/149.38 MB 9:50/130.27 MB 10:55/154.99 MB 11:41/145.16 MB
Hdfs Read Thread Concurrency Bucket: 0:97.57% 1:2.433% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:26179
ExecOption: Codegen enabled: 0 out of 557
BytesRead(16s000ms): 6.63 MB, 26.85 MB, 47.65 MB, 67.13 MB, 79.06 MB, 103.44 MB, 131.02 MB, 149.54 MB, 172.08 MB, 190.29 MB, 207.25 MB, 234.20 MB, 260.30 MB, 284.63 MB, 312.23 MB, 347.67 MB, 379.84 MB, 408.14 MB, 423.49 MB, 440.92 MB, 466.85 MB, 496.37 MB, 515.68 MB, 538.97 MB, 562.99 MB, 578.16 MB, 596.88 MB, 620.55 MB, 650.31 MB, 674.04 MB, 703.48 MB, 743.94 MB, 775.16 MB, 794.74 MB, 816.23 MB, 827.45 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 9.85
- BytesRead: 829.97 MB (870287544)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 829.97 MB (870287544)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 829.97 MB (870287544)
- DecompressionTime: 645.723ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 144 (144)
- PeakMemoryUsage: 76.95 MB (80690064)
- PerReadThreadRawHdfsThroughput: 57.31 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.60M (11602161)
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- ScanRangesComplete: 557 (557)
- ScannerThreadsInvoluntaryContextSwitches: 472 (472)
- ScannerThreadsTotalWallClockTime: 1h34m
- MaterializeTupleTime(*): 15s413ms
- ScannerThreadsSysTime: 604.11ms
- ScannerThreadsUserTime: 18s021ms
- ScannerThreadsVoluntaryContextSwitches: 41.98K (41984)
- TotalRawHdfsReadTime(*): 14s482ms
- TotalReadThroughput: 1.44 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016c5 (host=hadoopc1h82:28002):(Total: 9m32s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:52/125.77 MB 1:45/123.28 MB 2:52/137.71 MB 3:51/128.92 MB 4:55/152.90 MB 5:44/115.25 MB 6:50/141.26 MB 7:56/138.81 MB 8:62/135.54 MB 9:63/167.74 MB 10:48/162.50 MB 11:53/141.37 MB
MemoryUsage(16s000ms): 39.74 MB, 50.67 MB, 58.97 MB, 51.44 MB, 48.49 MB, 47.07 MB, 44.10 MB, 39.54 MB, 34.11 MB, 37.22 MB, 39.03 MB, 35.68 MB, 43.07 MB, 40.90 MB, 45.38 MB, 35.18 MB, 29.76 MB, 34.03 MB, 49.86 MB, 53.09 MB, 46.51 MB, 46.73 MB, 55.64 MB, 54.39 MB, 45.82 MB, 42.77 MB, 37.81 MB, 36.63 MB, 33.20 MB, 32.53 MB, 31.31 MB, 38.42 MB, 25.45 MB, 19.04 MB, 17.85 MB
ThreadUsage(16s000ms): 18, 19, 19, 19, 23, 20, 21, 16, 15, 17, 16, 13, 12, 12, 13, 13, 10, 13, 14, 16, 14, 13, 15, 13, 11, 11, 11, 12, 10, 10, 11, 13, 14, 15, 11
- AverageThreadTokens: 14.93
- PeakMemoryUsage: 90.40 MB (94793792)
- PerHostPeakMemUsage: 90.40 MB (94793792)
- PrepareTime: 197.532ms
- RowsProduced: 10 (10)
- TotalCpuTime: 2h13m
- TotalNetworkReceiveTime: 5s418ms
- TotalNetworkSendTime: 885ns
- TotalStorageWaitTime: 2h12m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 691.489ms, non-child: 691.489ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 138.219ms
- LoadTime: 149.211us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 369.290ms
- PrepareTime: 182.102ms
DataStreamSender (dst_id=13):(Total: 208.323us, non-child: 208.323us, % non-child: 100.00%)
- BytesSent: 3.24 KB (3319)
- NetworkThroughput(*): 1.97 MB/sec
- OverallThroughput: 15.19 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 149.510us
- ThriftTransmitTime(*): 1.603ms
- UncompressedRowBatchSize: 4.52 KB (4630)
HASH_JOIN_NODE (id=6):(Total: 9m32s, non-child: 197.482ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.626us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.378us
- GetNewBlockTime: 77.309us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 23.05 MB (24166672)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 172.442us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 730ns
EXCHANGE_NODE (id=12):(Total: 2s583ms, non-child: 25.296us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.860us
- DeserializeRowBatchTimer: 17.38us
- FirstBatchArrivalWaitTime: 2s572ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m32s, non-child: 2s852ms, % non-child: 0.50%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.899us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.370us
- GetNewBlockTime: 62.345us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.00 MB (15727800)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 160.422us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 707ns
EXCHANGE_NODE (id=9):(Total: 2s834ms, non-child: 28.45us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.428us
- DeserializeRowBatchTimer: 15.519us
- FirstBatchArrivalWaitTime: 2s824ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m29s, non-child: 9m29s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:52/125.77 MB 1:45/123.28 MB 2:52/137.71 MB 3:51/128.92 MB 4:55/152.90 MB 5:44/115.25 MB 6:50/141.26 MB 7:56/138.81 MB 8:62/135.54 MB 9:63/167.74 MB 10:48/162.50 MB 11:53/141.37 MB
Hdfs Read Thread Concurrency Bucket: 0:98.52% 1:1.485% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:29657
ExecOption: Codegen enabled: 0 out of 631
BytesRead(16s000ms): 11.81 MB, 39.01 MB, 69.12 MB, 86.17 MB, 98.66 MB, 112.87 MB, 131.35 MB, 144.71 MB, 157.70 MB, 171.16 MB, 183.90 MB, 203.55 MB, 229.72 MB, 251.62 MB, 276.48 MB, 317.58 MB, 359.33 MB, 399.00 MB, 418.49 MB, 442.92 MB, 474.04 MB, 507.87 MB, 537.15 MB, 571.05 MB, 597.72 MB, 615.85 MB, 630.69 MB, 651.84 MB, 682.26 MB, 707.32 MB, 731.56 MB, 773.09 MB, 796.76 MB, 808.95 MB, 819.77 MB
- AverageHdfsReadThreadConcurrency: 0.01
- AverageScannerThreadConcurrency: 13.94
- BytesRead: 827.35 MB (867537698)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 827.35 MB (867537698)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 827.35 MB (867537698)
- DecompressionTime: 529.176ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 167 (167)
- PeakMemoryUsage: 81.66 MB (85623184)
- PerReadThreadRawHdfsThroughput: 67.98 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.39M (11386544)
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- ScanRangesComplete: 631 (631)
- ScannerThreadsInvoluntaryContextSwitches: 470 (470)
- ScannerThreadsTotalWallClockTime: 2h12m
- MaterializeTupleTime(*): 15s234ms
- ScannerThreadsSysTime: 576.9ms
- ScannerThreadsUserTime: 18s025ms
- ScannerThreadsVoluntaryContextSwitches: 48.54K (48539)
- TotalRawHdfsReadTime(*): 12s170ms
- TotalReadThroughput: 1.44 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016b3 (host=hadoopc1h15:28002):(Total: 9m31s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:46/150.24 MB 1:41/143.39 MB 2:42/151.26 MB 3:38/113.32 MB 4:58/157.56 MB 5:48/160.32 MB 6:39/125.31 MB 7:44/143.77 MB 8:41/128.60 MB 9:44/131.82 MB 10:51/146.05 MB 11:53/150.19 MB
MemoryUsage(16s000ms): 23.89 MB, 31.24 MB, 42.82 MB, 44.80 MB, 68.93 MB, 53.49 MB, 39.15 MB, 39.01 MB, 38.38 MB, 28.37 MB, 31.28 MB, 34.25 MB, 38.93 MB, 25.91 MB, 21.79 MB, 30.10 MB, 28.79 MB, 29.23 MB, 39.57 MB, 45.29 MB, 43.05 MB, 32.06 MB, 48.26 MB, 33.73 MB, 33.08 MB, 35.04 MB, 37.80 MB, 37.62 MB, 25.12 MB, 24.53 MB, 21.81 MB, 26.34 MB, 18.87 MB, 16.60 MB, 13.27 MB
ThreadUsage(16s000ms): 9, 9, 10, 10, 10, 12, 11, 10, 10, 7, 7, 6, 6, 6, 6, 7, 7, 7, 8, 8, 9, 8, 10, 8, 8, 7, 8, 8, 7, 7, 7, 7, 8, 10, 7
- AverageThreadTokens: 8.75
- PeakMemoryUsage: 87.07 MB (91299040)
- PerHostPeakMemUsage: 87.07 MB (91299040)
- PrepareTime: 199.723ms
- RowsProduced: 15 (15)
- TotalCpuTime: 1h13m
- TotalNetworkReceiveTime: 5s298ms
- TotalNetworkSendTime: 2.228us
- TotalStorageWaitTime: 1h13m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 739.161ms, non-child: 739.161ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 147.474ms
- LoadTime: 158.665us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 404.711ms
- PrepareTime: 185.378ms
DataStreamSender (dst_id=13):(Total: 271.0us, non-child: 271.0us, % non-child: 100.00%)
- BytesSent: 4.75 KB (4865)
- NetworkThroughput(*): 1.18 MB/sec
- OverallThroughput: 17.12 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 179.164us
- ThriftTransmitTime(*): 3.918ms
- UncompressedRowBatchSize: 6.78 KB (6943)
HASH_JOIN_NODE (id=6):(Total: 9m31s, non-child: 200.9ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.859us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.535us
- GetNewBlockTime: 69.646us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.66 MB (25858760)
- PinTime: 0ns
- ProbeRows: 15 (15)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 187.963us
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.194us
EXCHANGE_NODE (id=12):(Total: 2s523ms, non-child: 29.897us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.343us
- DeserializeRowBatchTimer: 23.645us
- FirstBatchArrivalWaitTime: 2s521ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m31s, non-child: 2s795ms, % non-child: 0.49%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.404us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.936us
- GetNewBlockTime: 49.354us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 17.03 MB (17854888)
- PinTime: 0ns
- ProbeRows: 15 (15)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 179.861us
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 692ns
EXCHANGE_NODE (id=9):(Total: 2s774ms, non-child: 28.941us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.831us
- DeserializeRowBatchTimer: 21.837us
- FirstBatchArrivalWaitTime: 2s773ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m28s, non-child: 9m28s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:46/150.24 MB 1:41/143.39 MB 2:42/151.26 MB 3:38/113.32 MB 4:58/157.56 MB 5:48/160.32 MB 6:39/125.31 MB 7:44/143.77 MB 8:41/128.60 MB 9:44/131.82 MB 10:51/146.05 MB 11:53/150.19 MB
Hdfs Read Thread Concurrency Bucket: 0:97.81% 1:2.187% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:25615
ExecOption: Codegen enabled: 0 out of 545
BytesRead(16s000ms): 8.16 MB, 32.22 MB, 57.70 MB, 80.56 MB, 110.63 MB, 140.88 MB, 164.95 MB, 182.19 MB, 201.69 MB, 217.90 MB, 236.08 MB, 261.54 MB, 301.09 MB, 333.17 MB, 361.34 MB, 396.77 MB, 433.81 MB, 465.24 MB, 481.12 MB, 501.46 MB, 531.18 MB, 552.76 MB, 580.47 MB, 607.50 MB, 630.51 MB, 651.45 MB, 669.58 MB, 696.14 MB, 722.19 MB, 743.28 MB, 761.68 MB, 785.91 MB, 802.38 MB, 813.48 MB, 822.96 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 7.75
- BytesRead: 832.02 MB (872435564)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 832.02 MB (872435564)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 832.02 MB (872435564)
- DecompressionTime: 468.116ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 128 (128)
- PeakMemoryUsage: 77.86 MB (81640904)
- PerReadThreadRawHdfsThroughput: 86.63 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.73M (11726318)
- RowsReturned: 15 (15)
- RowsReturnedRate: 0
- ScanRangesComplete: 545 (545)
- ScannerThreadsInvoluntaryContextSwitches: 524 (524)
- ScannerThreadsTotalWallClockTime: 1h13m
- MaterializeTupleTime(*): 15s206ms
- ScannerThreadsSysTime: 488.11ms
- ScannerThreadsUserTime: 17s877ms
- ScannerThreadsVoluntaryContextSwitches: 47.37K (47368)
- TotalRawHdfsReadTime(*): 9s604ms
- TotalReadThroughput: 1.45 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016c3 (host=hadoopc1h14:28002):(Total: 9m31s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:35/119.63 MB 1:44/151.96 MB 2:48/128.27 MB 3:42/148.91 MB 4:52/147.81 MB 5:49/167.30 MB 6:45/158.49 MB 7:40/130.94 MB 8:34/98.76 MB 9:51/168.32 MB 10:43/138.66 MB 11:39/128.69 MB
MemoryUsage(16s000ms): 25.92 MB, 30.64 MB, 33.54 MB, 35.40 MB, 41.60 MB, 55.44 MB, 48.22 MB, 45.80 MB, 43.77 MB, 38.34 MB, 31.10 MB, 26.98 MB, 37.49 MB, 24.15 MB, 30.14 MB, 31.96 MB, 31.16 MB, 34.59 MB, 51.96 MB, 34.34 MB, 42.23 MB, 34.80 MB, 32.35 MB, 43.36 MB, 31.98 MB, 24.43 MB, 42.11 MB, 31.69 MB, 33.56 MB, 41.65 MB, 29.05 MB, 25.06 MB, 22.52 MB, 20.33 MB, 26.77 MB
ThreadUsage(16s000ms): 8, 8, 10, 10, 10, 12, 12, 10, 10, 8, 7, 6, 6, 6, 7, 7, 7, 7, 8, 8, 9, 8, 10, 8, 8, 7, 9, 8, 7, 7, 6, 7, 7, 8, 10
- AverageThreadTokens: 8.82
- PeakMemoryUsage: 75.93 MB (79619424)
- PerHostPeakMemUsage: 75.93 MB (79619424)
- PrepareTime: 185.727ms
- RowsProduced: 12 (12)
- TotalCpuTime: 1h14m
- TotalNetworkReceiveTime: 5s514ms
- TotalNetworkSendTime: 1.299us
- TotalStorageWaitTime: 1h14m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 651.801ms, non-child: 651.801ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 125.84ms
- LoadTime: 129.947us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 353.94ms
- PrepareTime: 171.993ms
DataStreamSender (dst_id=13):(Total: 184.215us, non-child: 184.215us, % non-child: 100.00%)
- BytesSent: 3.56 KB (3648)
- NetworkThroughput(*): 1.77 MB/sec
- OverallThroughput: 18.89 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 120.488us
- ThriftTransmitTime(*): 1.962ms
- UncompressedRowBatchSize: 5.42 KB (5550)
HASH_JOIN_NODE (id=6):(Total: 9m31s, non-child: 185.359ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 9.181us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.95us
- GetNewBlockTime: 63.716us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 22.41 MB (23497128)
- PinTime: 0ns
- ProbeRows: 12 (12)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 125.0us
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 712ns
EXCHANGE_NODE (id=12):(Total: 2s630ms, non-child: 62.365us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 3.977us
- DeserializeRowBatchTimer: 30.231us
- FirstBatchArrivalWaitTime: 2s621ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m30s, non-child: 2s905ms, % non-child: 0.51%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 5.971us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.636us
- GetNewBlockTime: 73.108us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.49 MB (16246832)
- PinTime: 0ns
- ProbeRows: 12 (12)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 139.310us
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.270us
EXCHANGE_NODE (id=9):(Total: 2s883ms, non-child: 36.247us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.613us
- DeserializeRowBatchTimer: 29.428us
- FirstBatchArrivalWaitTime: 2s873ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m27s, non-child: 9m27s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:35/119.63 MB 1:44/151.96 MB 2:48/128.27 MB 3:42/148.91 MB 4:52/147.81 MB 5:49/167.30 MB 6:45/158.49 MB 7:40/130.94 MB 8:34/98.76 MB 9:51/168.32 MB 10:43/138.66 MB 11:39/128.69 MB
Hdfs Read Thread Concurrency Bucket: 0:98.16% 1:1.753% 2:0.08764% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:24534
ExecOption: Codegen enabled: 0 out of 522
BytesRead(16s000ms): 9.52 MB, 34.13 MB, 57.00 MB, 75.71 MB, 93.13 MB, 124.24 MB, 148.25 MB, 168.32 MB, 195.88 MB, 216.77 MB, 236.53 MB, 259.54 MB, 297.17 MB, 330.82 MB, 357.25 MB, 399.57 MB, 434.52 MB, 470.04 MB, 490.38 MB, 512.21 MB, 536.27 MB, 563.05 MB, 579.40 MB, 601.28 MB, 618.35 MB, 628.86 MB, 644.17 MB, 663.15 MB, 687.08 MB, 710.70 MB, 732.74 MB, 752.63 MB, 769.50 MB, 783.11 MB, 808.50 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 7.82
- BytesRead: 825.77 MB (865877763)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 825.77 MB (865877763)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 825.77 MB (865877763)
- DecompressionTime: 1s035ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 115 (115)
- PeakMemoryUsage: 75.38 MB (79041824)
- PerReadThreadRawHdfsThroughput: 69.65 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.60M (11603794)
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- ScanRangesComplete: 522 (522)
- ScannerThreadsInvoluntaryContextSwitches: 804 (804)
- ScannerThreadsTotalWallClockTime: 1h14m
- MaterializeTupleTime(*): 15s558ms
- ScannerThreadsSysTime: 432.7ms
- ScannerThreadsUserTime: 18s305ms
- ScannerThreadsVoluntaryContextSwitches: 46.27K (46269)
- TotalRawHdfsReadTime(*): 11s856ms
- TotalReadThroughput: 1.45 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016a5 (host=hadoopc1h91:28002):(Total: 9m29s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:48/174.79 MB 1:40/123.10 MB 2:39/130.75 MB 3:36/91.23 MB 4:57/177.59 MB 5:37/127.14 MB 6:42/112.77 MB 7:35/128.09 MB 8:51/181.86 MB 9:46/145.81 MB 10:52/188.46 MB 11:44/118.78 MB
MemoryUsage(16s000ms): 32.71 MB, 34.93 MB, 62.21 MB, 57.93 MB, 48.28 MB, 56.85 MB, 52.75 MB, 51.66 MB, 54.62 MB, 36.89 MB, 33.08 MB, 32.96 MB, 43.25 MB, 36.55 MB, 38.38 MB, 44.86 MB, 37.41 MB, 43.73 MB, 34.68 MB, 41.20 MB, 40.01 MB, 40.44 MB, 39.76 MB, 27.68 MB, 40.84 MB, 34.43 MB, 32.51 MB, 37.87 MB, 33.91 MB, 39.01 MB, 22.20 MB, 26.59 MB, 26.89 MB, 19.28 MB, 16.65 MB
ThreadUsage(16s000ms): 11, 12, 13, 13, 14, 15, 16, 12, 12, 10, 8, 8, 8, 8, 9, 10, 8, 9, 10, 11, 11, 10, 12, 10, 10, 9, 11, 11, 10, 10, 9, 10, 10, 11, 8
- AverageThreadTokens: 11.03
- PeakMemoryUsage: 88.59 MB (92892944)
- PerHostPeakMemUsage: 88.59 MB (92892944)
- PrepareTime: 197.470ms
- RowsProduced: 9 (9)
- TotalCpuTime: 1h35m
- TotalNetworkReceiveTime: 5s432ms
- TotalNetworkSendTime: 794ns
- TotalStorageWaitTime: 1h34m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 694.463ms, non-child: 694.463ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 138.705ms
- LoadTime: 149.603us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 371.227ms
- PrepareTime: 182.656ms
DataStreamSender (dst_id=13):(Total: 160.356us, non-child: 160.356us, % non-child: 100.00%)
- BytesSent: 2.39 KB (2445)
- NetworkThroughput(*): 1.92 MB/sec
- OverallThroughput: 14.54 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 112.116us
- ThriftTransmitTime(*): 1.215ms
- UncompressedRowBatchSize: 4.06 KB (4160)
HASH_JOIN_NODE (id=6):(Total: 9m29s, non-child: 197.824ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.149us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.20us
- GetNewBlockTime: 66.130us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.54 MB (25733536)
- PinTime: 0ns
- ProbeRows: 9 (9)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 157.237us
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 974ns
EXCHANGE_NODE (id=12):(Total: 2s591ms, non-child: 26.347us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.256us
- DeserializeRowBatchTimer: 14.223us
- FirstBatchArrivalWaitTime: 2s578ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m29s, non-child: 3s442ms, % non-child: 0.60%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.106us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.947us
- GetNewBlockTime: 106.432us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 16.18 MB (16970896)
- PinTime: 0ns
- ProbeRows: 9 (9)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 170.778us
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 746ns
EXCHANGE_NODE (id=9):(Total: 2s841ms, non-child: 25.157us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.355us
- DeserializeRowBatchTimer: 15.727us
- FirstBatchArrivalWaitTime: 2s830ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m26s, non-child: 9m26s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:48/174.79 MB 1:40/123.10 MB 2:39/130.75 MB 3:36/91.23 MB 4:57/177.59 MB 5:37/127.14 MB 6:42/112.77 MB 7:35/128.09 MB 8:51/181.86 MB 9:46/145.81 MB 10:52/188.46 MB 11:44/118.78 MB
Hdfs Read Thread Concurrency Bucket: 0:97.98% 1:2.019% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:24769
ExecOption: Codegen enabled: 0 out of 527
BytesRead(16s000ms): 11.03 MB, 32.95 MB, 60.53 MB, 83.74 MB, 99.42 MB, 121.13 MB, 145.87 MB, 165.37 MB, 191.76 MB, 206.40 MB, 218.13 MB, 237.66 MB, 271.63 MB, 304.02 MB, 336.32 MB, 371.92 MB, 409.82 MB, 456.13 MB, 470.68 MB, 485.57 MB, 513.29 MB, 539.62 MB, 561.39 MB, 585.29 MB, 612.81 MB, 630.40 MB, 645.67 MB, 670.73 MB, 702.90 MB, 732.10 MB, 750.59 MB, 773.81 MB, 792.37 MB, 806.31 MB, 820.80 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 10.04
- BytesRead: 829.57 MB (869869798)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 829.57 MB (869869798)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 829.57 MB (869869798)
- DecompressionTime: 591.593ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 129 (129)
- PeakMemoryUsage: 80.86 MB (84789224)
- PerReadThreadRawHdfsThroughput: 63.53 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.73M (11725678)
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- ScanRangesComplete: 527 (527)
- ScannerThreadsInvoluntaryContextSwitches: 648 (648)
- ScannerThreadsTotalWallClockTime: 1h35m
- MaterializeTupleTime(*): 16s040ms
- ScannerThreadsSysTime: 500.5ms
- ScannerThreadsUserTime: 18s653ms
- ScannerThreadsVoluntaryContextSwitches: 39.36K (39361)
- TotalRawHdfsReadTime(*): 13s057ms
- TotalReadThroughput: 1.46 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016b2 (host=hadoopc1h65:28002):(Total: 9m29s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:45/151.66 MB 1:47/137.62 MB 2:51/148.10 MB 3:42/149.57 MB 4:53/143.65 MB 5:43/126.08 MB 6:33/81.40 MB 7:42/128.80 MB 8:49/145.99 MB 9:50/177.58 MB 10:52/192.37 MB 11:36/108.48 MB
MemoryUsage(16s000ms): 23.21 MB, 42.55 MB, 44.46 MB, 43.63 MB, 48.09 MB, 43.79 MB, 65.79 MB, 49.07 MB, 51.45 MB, 37.57 MB, 36.46 MB, 34.38 MB, 31.80 MB, 55.63 MB, 45.90 MB, 37.66 MB, 32.20 MB, 35.81 MB, 48.78 MB, 50.18 MB, 46.70 MB, 38.72 MB, 47.97 MB, 35.77 MB, 40.46 MB, 40.09 MB, 44.52 MB, 35.25 MB, 44.97 MB, 38.51 MB, 35.43 MB, 37.37 MB, 24.44 MB, 32.24 MB, 19.36 MB
ThreadUsage(16s000ms): 11, 12, 13, 12, 12, 15, 15, 13, 12, 10, 9, 8, 8, 8, 8, 9, 8, 9, 10, 9, 10, 10, 11, 9, 10, 9, 12, 11, 9, 9, 9, 10, 9, 11, 9
- AverageThreadTokens: 10.85
- PeakMemoryUsage: 93.12 MB (97644160)
- PerHostPeakMemUsage: 93.12 MB (97644160)
- PrepareTime: 215.300ms
- RowsProduced: 6 (6)
- TotalCpuTime: 1h33m
- TotalNetworkReceiveTime: 5s305ms
- TotalNetworkSendTime: 889ns
- TotalStorageWaitTime: 1h33m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 744.215ms, non-child: 744.215ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 152.930ms
- LoadTime: 164.410us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 389.381ms
- PrepareTime: 199.865ms
DataStreamSender (dst_id=13):(Total: 117.448us, non-child: 117.448us, % non-child: 100.00%)
- BytesSent: 1.75 KB (1791)
- NetworkThroughput(*): 1.62 MB/sec
- OverallThroughput: 14.54 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 77.566us
- ThriftTransmitTime(*): 1.56ms
- UncompressedRowBatchSize: 2.68 KB (2747)
HASH_JOIN_NODE (id=6):(Total: 9m29s, non-child: 215.450ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 15.565us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.608us
- GetNewBlockTime: 69.449us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 23.72 MB (24871616)
- PinTime: 0ns
- ProbeRows: 6 (6)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 162.288us
- RowsReturned: 6 (6)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 871ns
EXCHANGE_NODE (id=12):(Total: 2s531ms, non-child: 40.100us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.772us
- DeserializeRowBatchTimer: 24.858us
- FirstBatchArrivalWaitTime: 2s517ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m29s, non-child: 2s796ms, % non-child: 0.49%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.483us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 6.397us
- GetNewBlockTime: 60.21us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.26 MB (16001536)
- PinTime: 0ns
- ProbeRows: 6 (6)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 139.570us
- RowsReturned: 6 (6)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 864ns
EXCHANGE_NODE (id=9):(Total: 2s774ms, non-child: 28.808us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.561us
- DeserializeRowBatchTimer: 22.893us
- FirstBatchArrivalWaitTime: 2s769ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m26s, non-child: 9m26s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:45/151.66 MB 1:47/137.62 MB 2:51/148.10 MB 3:42/149.57 MB 4:53/143.65 MB 5:43/126.08 MB 6:33/81.40 MB 7:42/128.80 MB 8:49/145.99 MB 9:50/177.58 MB 10:52/192.37 MB 11:36/108.48 MB
Hdfs Read Thread Concurrency Bucket: 0:97.01% 1:2.9% 2:0.08787% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:25521
ExecOption: Codegen enabled: 0 out of 543
BytesRead(16s000ms): 6.68 MB, 35.24 MB, 62.91 MB, 82.80 MB, 102.71 MB, 120.87 MB, 147.07 MB, 167.42 MB, 190.82 MB, 205.39 MB, 219.99 MB, 241.57 MB, 266.18 MB, 300.14 MB, 330.74 MB, 357.27 MB, 383.67 MB, 414.32 MB, 431.85 MB, 449.79 MB, 479.05 MB, 506.46 MB, 530.70 MB, 552.94 MB, 576.13 MB, 592.17 MB, 611.15 MB, 632.14 MB, 661.02 MB, 686.70 MB, 707.29 MB, 745.22 MB, 765.38 MB, 790.86 MB, 821.68 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 9.85
- BytesRead: 828.89 MB (869159403)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 828.89 MB (869159403)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 828.89 MB (869159403)
- DecompressionTime: 1s324ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 144 (144)
- PeakMemoryUsage: 87.20 MB (91439264)
- PerReadThreadRawHdfsThroughput: 45.24 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.63M (11632364)
- RowsReturned: 6 (6)
- RowsReturnedRate: 0
- ScanRangesComplete: 543 (543)
- ScannerThreadsInvoluntaryContextSwitches: 1.10K (1105)
- ScannerThreadsTotalWallClockTime: 1h33m
- MaterializeTupleTime(*): 17s632ms
- ScannerThreadsSysTime: 660.16ms
- ScannerThreadsUserTime: 20s405ms
- ScannerThreadsVoluntaryContextSwitches: 46.33K (46327)
- TotalRawHdfsReadTime(*): 18s323ms
- TotalReadThroughput: 1.46 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016ae (host=hadoopc1h83:28002):(Total: 9m27s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:60/152.52 MB 1:47/135.31 MB 2:50/131.33 MB 3:49/144.73 MB 4:50/112.65 MB 5:45/143.68 MB 6:56/148.65 MB 7:43/118.11 MB 8:54/125.59 MB 9:52/169.31 MB 10:50/146.32 MB 11:43/130.78 MB
MemoryUsage(16s000ms): 41.90 MB, 53.81 MB, 51.46 MB, 49.92 MB, 42.15 MB, 53.57 MB, 51.66 MB, 56.72 MB, 55.02 MB, 56.27 MB, 62.65 MB, 50.54 MB, 46.05 MB, 39.57 MB, 39.47 MB, 37.75 MB, 37.54 MB, 40.24 MB, 32.73 MB, 54.19 MB, 44.38 MB, 46.45 MB, 42.18 MB, 47.21 MB, 33.80 MB, 41.75 MB, 41.99 MB, 34.85 MB, 30.48 MB, 29.83 MB, 29.48 MB, 29.27 MB, 21.52 MB, 13.87 MB, 5.88 MB
ThreadUsage(16s000ms): 15, 17, 17, 16, 16, 22, 21, 21, 20, 18, 18, 17, 12, 13, 14, 13, 11, 13, 14, 14, 14, 16, 16, 14, 10, 13, 14, 10, 9, 9, 9, 13, 14, 8, 4
- AverageThreadTokens: 14.71
- PeakMemoryUsage: 79.82 MB (83695832)
- PerHostPeakMemUsage: 79.82 MB (83695832)
- PrepareTime: 194.743ms
- RowsProduced: 10 (10)
- TotalCpuTime: 2h9m
- TotalNetworkReceiveTime: 5s389ms
- TotalNetworkSendTime: 1.62us
- TotalStorageWaitTime: 2h9m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 702.193ms, non-child: 702.193ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 137.962ms
- LoadTime: 157.203us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 382.554ms
- PrepareTime: 179.714ms
DataStreamSender (dst_id=13):(Total: 189.931us, non-child: 189.931us, % non-child: 100.00%)
- BytesSent: 2.96 KB (3035)
- NetworkThroughput(*): 1.68 MB/sec
- OverallThroughput: 15.24 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 129.978us
- ThriftTransmitTime(*): 1.721ms
- UncompressedRowBatchSize: 4.50 KB (4613)
HASH_JOIN_NODE (id=6):(Total: 9m27s, non-child: 194.994ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.612us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.836us
- GetNewBlockTime: 73.902us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 22.87 MB (23984200)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 170.251us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 734ns
EXCHANGE_NODE (id=12):(Total: 2s566ms, non-child: 23.686us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.900us
- DeserializeRowBatchTimer: 9.161us
- FirstBatchArrivalWaitTime: 2s565ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m27s, non-child: 2s841ms, % non-child: 0.50%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.711us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.108us
- GetNewBlockTime: 136.961us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 12.65 MB (13259832)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 167.523us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 732ns
EXCHANGE_NODE (id=9):(Total: 2s823ms, non-child: 26.918us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.983us
- DeserializeRowBatchTimer: 11.8us
- FirstBatchArrivalWaitTime: 2s817ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m24s, non-child: 9m24s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:60/152.52 MB 1:47/135.31 MB 2:50/131.33 MB 3:49/144.73 MB 4:50/112.65 MB 5:45/143.68 MB 6:56/148.65 MB 7:43/118.11 MB 8:54/125.59 MB 9:52/169.31 MB 10:50/146.32 MB 11:43/130.78 MB
Hdfs Read Thread Concurrency Bucket: 0:97.89% 1:2.115% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28153
ExecOption: Codegen enabled: 0 out of 599
BytesRead(16s000ms): 11.53 MB, 40.95 MB, 67.58 MB, 85.78 MB, 99.12 MB, 118.16 MB, 141.69 MB, 160.17 MB, 181.48 MB, 200.20 MB, 219.48 MB, 240.85 MB, 272.80 MB, 301.87 MB, 327.21 MB, 357.27 MB, 395.87 MB, 443.04 MB, 460.87 MB, 484.21 MB, 520.01 MB, 548.53 MB, 572.77 MB, 603.24 MB, 627.27 MB, 644.58 MB, 663.57 MB, 689.03 MB, 717.26 MB, 739.58 MB, 759.44 MB, 785.20 MB, 803.07 MB, 810.35 MB, 815.08 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 13.72
- BytesRead: 818.81 MB (858587759)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 818.81 MB (858587759)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 818.81 MB (858587759)
- DecompressionTime: 753.997ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 135 (135)
- PeakMemoryUsage: 79.27 MB (83118232)
- PerReadThreadRawHdfsThroughput: 59.20 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.31M (11314256)
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- ScanRangesComplete: 599 (599)
- ScannerThreadsInvoluntaryContextSwitches: 519 (519)
- ScannerThreadsTotalWallClockTime: 2h9m
- MaterializeTupleTime(*): 14s866ms
- ScannerThreadsSysTime: 544.10ms
- ScannerThreadsUserTime: 17s741ms
- ScannerThreadsVoluntaryContextSwitches: 44.07K (44071)
- TotalRawHdfsReadTime(*): 13s831ms
- TotalReadThroughput: 1.44 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016a7 (host=hadoopc1h88:28002):(Total: 9m25s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:43/116.91 MB 1:46/127.68 MB 2:47/115.16 MB 3:49/130.83 MB 4:51/189.50 MB 5:52/157.70 MB 6:41/104.11 MB 7:56/169.19 MB 8:51/155.30 MB 9:49/135.06 MB 10:44/146.67 MB 11:55/138.02 MB
MemoryUsage(16s000ms): 25.16 MB, 48.98 MB, 53.85 MB, 42.31 MB, 32.55 MB, 52.41 MB, 57.11 MB, 50.75 MB, 41.28 MB, 38.05 MB, 41.06 MB, 41.14 MB, 31.94 MB, 36.59 MB, 35.91 MB, 42.76 MB, 38.59 MB, 31.89 MB, 33.89 MB, 29.95 MB, 34.49 MB, 29.40 MB, 40.42 MB, 35.48 MB, 43.61 MB, 55.58 MB, 48.73 MB, 31.46 MB, 43.29 MB, 39.69 MB, 43.52 MB, 42.38 MB, 30.56 MB, 23.46 MB, 21.55 MB
ThreadUsage(16s000ms): 11, 13, 13, 13, 14, 16, 17, 15, 13, 10, 9, 9, 8, 9, 10, 11, 8, 9, 10, 9, 11, 11, 13, 10, 11, 13, 14, 11, 9, 10, 11, 11, 11, 13, 8
- AverageThreadTokens: 11.84
- PeakMemoryUsage: 76.26 MB (79960720)
- PerHostPeakMemUsage: 76.26 MB (79960720)
- PrepareTime: 191.791ms
- RowsProduced: 14 (14)
- TotalCpuTime: 1h42m
- TotalNetworkReceiveTime: 5s412ms
- TotalNetworkSendTime: 1.959us
- TotalStorageWaitTime: 1h41m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 691.185ms, non-child: 691.185ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 135.832ms
- LoadTime: 144.10us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 376.643ms
- PrepareTime: 176.949ms
DataStreamSender (dst_id=13):(Total: 267.504us, non-child: 267.504us, % non-child: 100.00%)
- BytesSent: 4.12 KB (4222)
- NetworkThroughput(*): 1.70 MB/sec
- OverallThroughput: 15.05 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 187.664us
- ThriftTransmitTime(*): 2.365ms
- UncompressedRowBatchSize: 6.30 KB (6448)
HASH_JOIN_NODE (id=6):(Total: 9m25s, non-child: 191.893ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.829us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.229us
- GetNewBlockTime: 76.47us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.21 MB (26438440)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 202.293us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 850ns
EXCHANGE_NODE (id=12):(Total: 2s580ms, non-child: 28.914us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.723us
- DeserializeRowBatchTimer: 15.975us
- FirstBatchArrivalWaitTime: 2s577ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m25s, non-child: 2s848ms, % non-child: 0.50%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.553us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.241us
- GetNewBlockTime: 75.546us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.43 MB (16181584)
- PinTime: 0ns
- ProbeRows: 14 (14)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 188.76us
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 883ns
EXCHANGE_NODE (id=9):(Total: 2s831ms, non-child: 27.587us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.614us
- DeserializeRowBatchTimer: 16.395us
- FirstBatchArrivalWaitTime: 2s829ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m22s, non-child: 9m22s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:43/116.91 MB 1:46/127.68 MB 2:47/115.16 MB 3:49/130.83 MB 4:51/189.50 MB 5:52/157.70 MB 6:41/104.11 MB 7:56/169.19 MB 8:51/155.30 MB 9:49/135.06 MB 10:44/146.67 MB 11:55/138.02 MB
Hdfs Read Thread Concurrency Bucket: 0:96.28% 1:3.628% 2:0.0885% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:27448
ExecOption: Codegen enabled: 0 out of 584
BytesRead(16s000ms): 8.23 MB, 35.42 MB, 63.19 MB, 80.78 MB, 91.50 MB, 109.63 MB, 131.36 MB, 149.05 MB, 166.30 MB, 179.96 MB, 195.11 MB, 217.21 MB, 243.06 MB, 268.19 MB, 293.75 MB, 326.67 MB, 370.54 MB, 400.98 MB, 414.26 MB, 428.94 MB, 453.01 MB, 476.41 MB, 498.33 MB, 527.50 MB, 553.53 MB, 579.50 MB, 597.24 MB, 614.20 MB, 648.10 MB, 678.91 MB, 709.61 MB, 748.42 MB, 781.15 MB, 802.20 MB, 819.57 MB
- AverageHdfsReadThreadConcurrency: 0.04
- AverageScannerThreadConcurrency: 10.85
- BytesRead: 830.10 MB (870418709)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 830.10 MB (870418709)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 830.10 MB (870418709)
- DecompressionTime: 554.459ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 129 (129)
- PeakMemoryUsage: 68.90 MB (72248528)
- PerReadThreadRawHdfsThroughput: 51.07 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.54M (11541787)
- RowsReturned: 14 (14)
- RowsReturnedRate: 0
- ScanRangesComplete: 584 (584)
- ScannerThreadsInvoluntaryContextSwitches: 481 (481)
- ScannerThreadsTotalWallClockTime: 1h42m
- MaterializeTupleTime(*): 15s753ms
- ScannerThreadsSysTime: 508.7ms
- ScannerThreadsUserTime: 18s565ms
- ScannerThreadsVoluntaryContextSwitches: 43.56K (43556)
- TotalRawHdfsReadTime(*): 16s253ms
- TotalReadThroughput: 1.47 MB/sec
Instance c14f7b9b17987c60:d5379fb954101696 (host=hadoopc1h21:28002):(Total: 9m23s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:50/115.54 MB 1:57/183.92 MB 2:47/126.29 MB 3:45/104.31 MB 4:58/187.04 MB 5:48/125.77 MB 6:51/167.93 MB 7:40/104.70 MB 8:48/152.52 MB 9:56/170.32 MB 10:41/109.82 MB 11:50/133.41 MB
MemoryUsage(16s000ms): 23.70 MB, 38.24 MB, 39.01 MB, 36.58 MB, 54.92 MB, 45.67 MB, 46.99 MB, 41.59 MB, 39.83 MB, 37.08 MB, 36.93 MB, 23.50 MB, 40.66 MB, 31.80 MB, 33.79 MB, 29.47 MB, 21.86 MB, 33.48 MB, 46.54 MB, 31.84 MB, 37.85 MB, 51.08 MB, 34.01 MB, 33.55 MB, 39.20 MB, 43.84 MB, 38.21 MB, 31.39 MB, 34.34 MB, 30.62 MB, 30.08 MB, 35.53 MB, 26.48 MB, 17.56 MB, 17.76 MB
ThreadUsage(16s000ms): 9, 10, 10, 12, 13, 12, 12, 10, 10, 8, 7, 7, 7, 6, 7, 9, 8, 8, 10, 10, 11, 11, 11, 8, 9, 8, 10, 9, 7, 7, 8, 9, 9, 12, 5
- AverageThreadTokens: 9.81
- PeakMemoryUsage: 74.02 MB (77616344)
- PerHostPeakMemUsage: 74.02 MB (77616344)
- PrepareTime: 196.892ms
- RowsProduced: 17 (17)
- TotalCpuTime: 1h22m
- TotalNetworkReceiveTime: 5s495ms
- TotalNetworkSendTime: 1.490us
- TotalStorageWaitTime: 1h22m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 667.24ms, non-child: 667.24ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 143.464ms
- LoadTime: 184.787us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 339.327ms
- PrepareTime: 182.458ms
DataStreamSender (dst_id=13):(Total: 224.881us, non-child: 224.881us, % non-child: 100.00%)
- BytesSent: 4.55 KB (4660)
- NetworkThroughput(*): 1.90 MB/sec
- OverallThroughput: 19.76 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 160.566us
- ThriftTransmitTime(*): 2.344ms
- UncompressedRowBatchSize: 7.59 KB (7772)
HASH_JOIN_NODE (id=6):(Total: 9m23s, non-child: 197.283ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 15.157us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.553us
- GetNewBlockTime: 126.540us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 22.65 MB (23747552)
- PinTime: 0ns
- ProbeRows: 17 (17)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 170.710us
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 924ns
EXCHANGE_NODE (id=12):(Total: 2s622ms, non-child: 37.976us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.325us
- DeserializeRowBatchTimer: 13.946us
- FirstBatchArrivalWaitTime: 2s609ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m23s, non-child: 2s894ms, % non-child: 0.51%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.863us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.257us
- GetNewBlockTime: 121.716us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 14.81 MB (15524992)
- PinTime: 0ns
- ProbeRows: 17 (17)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 183.301us
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.9us
EXCHANGE_NODE (id=9):(Total: 2s872ms, non-child: 24.347us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.967us
- DeserializeRowBatchTimer: 8.879us
- FirstBatchArrivalWaitTime: 2s861ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m20s, non-child: 9m20s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:50/115.54 MB 1:57/183.92 MB 2:47/126.29 MB 3:45/104.31 MB 4:58/187.04 MB 5:48/125.77 MB 6:51/167.93 MB 7:40/104.70 MB 8:48/152.52 MB 9:56/170.32 MB 10:41/109.82 MB 11:50/133.41 MB
Hdfs Read Thread Concurrency Bucket: 0:97.87% 1:2.13% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:27777
ExecOption: Codegen enabled: 0 out of 591
BytesRead(16s000ms): 8.70 MB, 33.56 MB, 61.89 MB, 78.42 MB, 97.16 MB, 115.84 MB, 138.45 MB, 155.89 MB, 178.47 MB, 192.59 MB, 208.62 MB, 223.09 MB, 253.94 MB, 280.80 MB, 302.26 MB, 333.41 MB, 362.76 MB, 397.49 MB, 423.10 MB, 442.51 MB, 468.93 MB, 508.92 MB, 534.32 MB, 557.53 MB, 582.58 MB, 602.35 MB, 620.87 MB, 643.34 MB, 677.13 MB, 701.93 MB, 726.39 MB, 763.78 MB, 789.69 MB, 804.97 MB, 821.92 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 8.81
- BytesRead: 827.63 MB (867827859)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 827.63 MB (867827859)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 827.63 MB (867827859)
- DecompressionTime: 327.854ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 120 (120)
- PeakMemoryUsage: 68.42 MB (71745968)
- PerReadThreadRawHdfsThroughput: 75.60 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.53M (11531461)
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- ScanRangesComplete: 591 (591)
- ScannerThreadsInvoluntaryContextSwitches: 456 (456)
- ScannerThreadsTotalWallClockTime: 1h22m
- MaterializeTupleTime(*): 15s176ms
- ScannerThreadsSysTime: 528.9ms
- ScannerThreadsUserTime: 18s033ms
- ScannerThreadsVoluntaryContextSwitches: 51.23K (51234)
- TotalRawHdfsReadTime(*): 10s947ms
- TotalReadThroughput: 1.47 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016bb (host=hadoopc1h5:28002):(Total: 9m17s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:61/166.79 MB 1:46/130.96 MB 2:52/151.23 MB 3:52/109.46 MB 4:49/132.16 MB 5:54/165.89 MB 6:40/119.96 MB 7:49/130.04 MB 8:60/176.86 MB 9:35/91.41 MB 10:50/142.81 MB 11:57/153.65 MB
MemoryUsage(16s000ms): 20.91 MB, 31.36 MB, 40.71 MB, 39.46 MB, 49.27 MB, 38.91 MB, 47.96 MB, 39.31 MB, 38.53 MB, 48.18 MB, 34.29 MB, 24.30 MB, 26.05 MB, 36.43 MB, 36.42 MB, 38.60 MB, 35.35 MB, 54.82 MB, 36.18 MB, 38.34 MB, 36.95 MB, 44.03 MB, 47.94 MB, 37.94 MB, 31.44 MB, 43.13 MB, 40.66 MB, 23.57 MB, 27.74 MB, 32.25 MB, 32.73 MB, 26.23 MB, 17.25 MB, 15.62 MB
ThreadUsage(16s000ms): 9, 9, 11, 12, 13, 11, 13, 11, 10, 9, 9, 9, 9, 11, 12, 13, 9, 10, 11, 10, 10, 11, 12, 11, 11, 11, 12, 9, 8, 9, 10, 10, 10, 8
- AverageThreadTokens: 10.86
- PeakMemoryUsage: 95.79 MB (100448264)
- PerHostPeakMemUsage: 95.79 MB (100448264)
- PrepareTime: 210.844ms
- RowsProduced: 9 (9)
- TotalCpuTime: 1h31m
- TotalNetworkReceiveTime: 5s206ms
- TotalNetworkSendTime: 2.30us
- TotalStorageWaitTime: 1h30m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 789.593ms, non-child: 789.593ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 167.593ms
- LoadTime: 176.174us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 425.829ms
- PrepareTime: 194.463ms
DataStreamSender (dst_id=13):(Total: 423.388us, non-child: 423.388us, % non-child: 100.00%)
- BytesSent: 2.76 KB (2828)
- NetworkThroughput(*): 645.99 KB/sec
- OverallThroughput: 6.37 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 359.720us
- ThriftTransmitTime(*): 4.275ms
- UncompressedRowBatchSize: 4.10 KB (4196)
HASH_JOIN_NODE (id=6):(Total: 9m17s, non-child: 211.462ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 12.594us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 6.36us
- GetNewBlockTime: 115.867us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 28.35 MB (29723184)
- PinTime: 0ns
- ProbeRows: 9 (9)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 188.918us
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.381us
EXCHANGE_NODE (id=12):(Total: 2s481ms, non-child: 39.731us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 3.181us
- DeserializeRowBatchTimer: 21.844us
- FirstBatchArrivalWaitTime: 2s469ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m16s, non-child: 2s745ms, % non-child: 0.49%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.225us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.877us
- GetNewBlockTime: 66.491us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 18.86 MB (19781200)
- PinTime: 0ns
- ProbeRows: 9 (9)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 169.874us
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 873ns
EXCHANGE_NODE (id=9):(Total: 2s725ms, non-child: 23.367us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.416us
- DeserializeRowBatchTimer: 27.28us
- FirstBatchArrivalWaitTime: 2s721ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m14s, non-child: 9m14s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:61/166.79 MB 1:46/130.96 MB 2:52/151.23 MB 3:52/109.46 MB 4:49/132.16 MB 5:54/165.89 MB 6:40/119.96 MB 7:49/130.04 MB 8:60/176.86 MB 9:35/91.41 MB 10:50/142.81 MB 11:57/153.65 MB
Hdfs Read Thread Concurrency Bucket: 0:97.39% 1:2.606% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28435
ExecOption: Codegen enabled: 0 out of 605
BytesRead(16s000ms): 8.02 MB, 30.31 MB, 54.07 MB, 69.61 MB, 86.36 MB, 105.18 MB, 128.48 MB, 147.06 MB, 168.08 MB, 190.23 MB, 206.59 MB, 224.49 MB, 249.99 MB, 277.39 MB, 307.86 MB, 353.28 MB, 402.16 MB, 461.45 MB, 488.09 MB, 508.34 MB, 536.25 MB, 572.04 MB, 601.34 MB, 632.40 MB, 650.24 MB, 669.46 MB, 691.11 MB, 705.91 MB, 726.30 MB, 747.31 MB, 774.20 MB, 795.27 MB, 805.96 MB, 816.34 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 9.87
- BytesRead: 825.79 MB (865907880)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 825.79 MB (865907880)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 825.79 MB (865907880)
- DecompressionTime: 424.982ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 114 (114)
- PeakMemoryUsage: 85.75 MB (89920488)
- PerReadThreadRawHdfsThroughput: 61.99 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.42M (11423478)
- RowsReturned: 9 (9)
- RowsReturnedRate: 0
- ScanRangesComplete: 605 (605)
- ScannerThreadsInvoluntaryContextSwitches: 2.62K (2615)
- ScannerThreadsTotalWallClockTime: 1h31m
- MaterializeTupleTime(*): 20s793ms
- ScannerThreadsSysTime: 716.16ms
- ScannerThreadsUserTime: 24s293ms
- ScannerThreadsVoluntaryContextSwitches: 56.86K (56858)
- TotalRawHdfsReadTime(*): 13s321ms
- TotalReadThroughput: 1.48 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016ca (host=hadoopc1h141:28002):(Total: 9m17s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:52/167.79 MB 1:51/146.20 MB 2:43/109.85 MB 3:44/123.53 MB 4:50/157.51 MB 5:51/166.52 MB 6:38/149.61 MB 7:48/138.77 MB 8:37/153.06 MB 9:40/109.23 MB 10:42/136.30 MB 11:42/146.36 MB
MemoryUsage(16s000ms): 18.97 MB, 44.58 MB, 52.12 MB, 48.15 MB, 46.68 MB, 42.32 MB, 27.57 MB, 29.96 MB, 53.09 MB, 35.53 MB, 48.40 MB, 37.73 MB, 36.79 MB, 40.88 MB, 39.76 MB, 37.36 MB, 37.52 MB, 43.94 MB, 43.16 MB, 51.41 MB, 37.57 MB, 34.06 MB, 39.33 MB, 42.76 MB, 43.58 MB, 43.60 MB, 44.19 MB, 46.54 MB, 40.59 MB, 41.95 MB, 31.20 MB, 37.01 MB, 20.22 MB, 25.36 MB
ThreadUsage(16s000ms): 11, 11, 13, 12, 14, 15, 15, 12, 12, 10, 9, 8, 8, 8, 8, 9, 8, 9, 10, 11, 11, 10, 11, 9, 10, 9, 12, 11, 9, 9, 9, 10, 10, 11
- AverageThreadTokens: 10.99
- PeakMemoryUsage: 85.72 MB (89880968)
- PerHostPeakMemUsage: 85.72 MB (89880968)
- PrepareTime: 199.712ms
- RowsProduced: 10 (10)
- TotalCpuTime: 1h32m
- TotalNetworkReceiveTime: 5s431ms
- TotalNetworkSendTime: 1.179us
- TotalStorageWaitTime: 1h32m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 678.635ms, non-child: 678.635ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 131.455ms
- LoadTime: 136.157us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 360.446ms
- PrepareTime: 184.976ms
DataStreamSender (dst_id=13):(Total: 201.769us, non-child: 201.769us, % non-child: 100.00%)
- BytesSent: 3.07 KB (3141)
- NetworkThroughput(*): 1.50 MB/sec
- OverallThroughput: 14.85 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 136.305us
- ThriftTransmitTime(*): 1.998ms
- UncompressedRowBatchSize: 4.50 KB (4603)
HASH_JOIN_NODE (id=6):(Total: 9m17s, non-child: 311.879ms, % non-child: 0.06%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.659us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.648us
- GetNewBlockTime: 72.628us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 22.56 MB (23660552)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 153.165us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 842ns
EXCHANGE_NODE (id=12):(Total: 2s591ms, non-child: 22.598us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.766us
- DeserializeRowBatchTimer: 22.447us
- FirstBatchArrivalWaitTime: 2s586ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m16s, non-child: 2s867ms, % non-child: 0.52%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.446us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.96us
- GetNewBlockTime: 53.225us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 17.09 MB (17915056)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 166.310us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.134us
EXCHANGE_NODE (id=9):(Total: 2s839ms, non-child: 25.920us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.122us
- DeserializeRowBatchTimer: 21.200us
- FirstBatchArrivalWaitTime: 2s838ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m13s, non-child: 9m13s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:52/167.79 MB 1:51/146.20 MB 2:43/109.85 MB 3:44/123.53 MB 4:50/157.51 MB 5:51/166.52 MB 6:38/149.61 MB 7:48/138.77 MB 8:37/153.06 MB 9:40/109.23 MB 10:42/136.30 MB 11:42/146.36 MB
Hdfs Read Thread Concurrency Bucket: 0:96.86% 1:2.962% 2:0.1795% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:25286
ExecOption: Codegen enabled: 0 out of 538
BytesRead(16s000ms): 5.18 MB, 32.87 MB, 63.03 MB, 84.90 MB, 101.66 MB, 119.93 MB, 134.26 MB, 145.13 MB, 168.53 MB, 186.14 MB, 206.59 MB, 237.05 MB, 271.90 MB, 302.50 MB, 335.87 MB, 370.63 MB, 408.14 MB, 444.54 MB, 462.56 MB, 483.76 MB, 506.54 MB, 526.17 MB, 545.35 MB, 566.34 MB, 595.67 MB, 615.88 MB, 635.44 MB, 658.75 MB, 689.15 MB, 715.61 MB, 746.43 MB, 777.15 MB, 796.29 MB, 814.85 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 9.99
- BytesRead: 834.85 MB (875407485)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 834.85 MB (875407485)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 834.85 MB (875407485)
- DecompressionTime: 430.560ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 139 (139)
- PeakMemoryUsage: 85.17 MB (89303368)
- PerReadThreadRawHdfsThroughput: 58.39 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.73M (11731427)
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- ScanRangesComplete: 538 (538)
- ScannerThreadsInvoluntaryContextSwitches: 559 (559)
- ScannerThreadsTotalWallClockTime: 1h32m
- MaterializeTupleTime(*): 15s815ms
- ScannerThreadsSysTime: 508.9ms
- ScannerThreadsUserTime: 18s437ms
- ScannerThreadsVoluntaryContextSwitches: 40.20K (40200)
- TotalRawHdfsReadTime(*): 14s298ms
- TotalReadThroughput: 1.50 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016b8 (host=hadoopc1h27:28002):(Total: 9m14s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:46/149.99 MB 1:50/175.75 MB 2:43/153.04 MB 3:40/102.34 MB 4:51/163.53 MB 5:43/127.75 MB 6:48/168.39 MB 7:42/150.91 MB 8:37/93.75 MB 9:43/127.37 MB 10:50/140.52 MB 11:37/125.99 MB
MemoryUsage(16s000ms): 21.47 MB, 34.35 MB, 50.46 MB, 39.08 MB, 46.87 MB, 30.51 MB, 38.77 MB, 38.80 MB, 43.26 MB, 33.90 MB, 31.74 MB, 31.08 MB, 34.15 MB, 35.35 MB, 29.65 MB, 27.74 MB, 30.76 MB, 26.44 MB, 26.87 MB, 34.81 MB, 37.43 MB, 48.16 MB, 37.38 MB, 28.66 MB, 30.21 MB, 30.15 MB, 28.26 MB, 42.55 MB, 32.22 MB, 36.05 MB, 32.68 MB, 33.09 MB, 29.13 MB, 27.01 MB
ThreadUsage(16s000ms): 9, 9, 10, 10, 10, 11, 11, 10, 10, 8, 7, 6, 6, 6, 6, 7, 6, 7, 8, 9, 9, 8, 9, 8, 8, 7, 8, 8, 7, 7, 6, 7, 7, 9
- AverageThreadTokens: 8.76
- PeakMemoryUsage: 83.99 MB (88071432)
- PerHostPeakMemUsage: 83.99 MB (88071432)
- PrepareTime: 223.998ms
- RowsProduced: 12 (12)
- TotalCpuTime: 1h11m
- TotalNetworkReceiveTime: 5s318ms
- TotalNetworkSendTime: 936ns
- TotalStorageWaitTime: 1h11m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 735.346ms, non-child: 735.346ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 139.336ms
- LoadTime: 117.542us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 384.466ms
- PrepareTime: 209.899ms
DataStreamSender (dst_id=13):(Total: 160.936us, non-child: 160.936us, % non-child: 100.00%)
- BytesSent: 3.12 KB (3195)
- NetworkThroughput(*): 2.24 MB/sec
- OverallThroughput: 18.93 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 112.654us
- ThriftTransmitTime(*): 1.358ms
- UncompressedRowBatchSize: 5.33 KB (5463)
HASH_JOIN_NODE (id=6):(Total: 9m14s, non-child: 224.51ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.760us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.924us
- GetNewBlockTime: 143.297us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.98 MB (27246720)
- PinTime: 0ns
- ProbeRows: 12 (12)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 137.600us
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 957ns
EXCHANGE_NODE (id=12):(Total: 2s532ms, non-child: 22.402us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.703us
- DeserializeRowBatchTimer: 11.953us
- FirstBatchArrivalWaitTime: 2s526ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m14s, non-child: 3s466ms, % non-child: 0.63%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.189us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.792us
- GetNewBlockTime: 85.358us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 16.70 MB (17507512)
- PinTime: 0ns
- ProbeRows: 12 (12)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 130.526us
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.152us
EXCHANGE_NODE (id=9):(Total: 2s785ms, non-child: 24.722us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.628us
- DeserializeRowBatchTimer: 10.846us
- FirstBatchArrivalWaitTime: 2s778ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m10s, non-child: 9m10s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:46/149.99 MB 1:50/175.75 MB 2:43/153.04 MB 3:40/102.34 MB 4:51/163.53 MB 5:43/127.75 MB 6:48/168.39 MB 7:42/150.91 MB 8:37/93.75 MB 9:43/127.37 MB 10:50/140.52 MB 11:37/125.99 MB
Hdfs Read Thread Concurrency Bucket: 0:98.1% 1:1.895% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:24910
ExecOption: Codegen enabled: 0 out of 530
BytesRead(16s000ms): 8.79 MB, 31.36 MB, 65.45 MB, 84.19 MB, 103.41 MB, 123.45 MB, 142.16 MB, 159.75 MB, 182.11 MB, 202.32 MB, 219.03 MB, 245.29 MB, 273.96 MB, 301.70 MB, 330.06 MB, 362.24 MB, 396.93 MB, 428.88 MB, 439.91 MB, 454.65 MB, 480.17 MB, 511.49 MB, 535.59 MB, 554.38 MB, 577.70 MB, 592.77 MB, 604.25 MB, 630.92 MB, 661.98 MB, 684.80 MB, 707.14 MB, 743.53 MB, 769.20 MB, 797.96 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 7.76
- BytesRead: 822.25 MB (862187617)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 822.25 MB (862187617)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 822.25 MB (862187617)
- DecompressionTime: 1s330ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 116 (116)
- PeakMemoryUsage: 83.44 MB (87493832)
- PerReadThreadRawHdfsThroughput: 72.11 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.55M (11545861)
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- ScanRangesComplete: 530 (530)
- ScannerThreadsInvoluntaryContextSwitches: 1.98K (1983)
- ScannerThreadsTotalWallClockTime: 1h11m
- MaterializeTupleTime(*): 17s388ms
- ScannerThreadsSysTime: 536.11ms
- ScannerThreadsUserTime: 20s265ms
- ScannerThreadsVoluntaryContextSwitches: 48.66K (48657)
- TotalRawHdfsReadTime(*): 11s402ms
- TotalReadThroughput: 1.48 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016cd (host=hadoopc1h10:28002):(Total: 9m8s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:48/181.20 MB 1:37/114.62 MB 2:42/127.72 MB 3:43/157.22 MB 4:47/147.42 MB 5:47/150.87 MB 6:38/118.01 MB 7:40/114.06 MB 8:42/130.43 MB 9:42/150.55 MB 10:50/148.80 MB 11:46/162.09 MB
MemoryUsage(16s000ms): 17.87 MB, 37.75 MB, 37.17 MB, 33.64 MB, 45.51 MB, 59.58 MB, 51.19 MB, 38.87 MB, 50.03 MB, 42.50 MB, 33.24 MB, 34.74 MB, 25.52 MB, 32.15 MB, 33.35 MB, 34.65 MB, 24.70 MB, 37.34 MB, 38.11 MB, 51.94 MB, 39.45 MB, 32.95 MB, 42.56 MB, 40.89 MB, 33.62 MB, 30.92 MB, 34.88 MB, 29.69 MB, 36.38 MB, 32.89 MB, 20.43 MB, 33.37 MB, 24.03 MB, 21.14 MB
ThreadUsage(16s000ms): 9, 9, 10, 10, 11, 13, 11, 10, 10, 8, 7, 6, 7, 6, 7, 7, 7, 8, 8, 9, 9, 8, 10, 8, 8, 7, 8, 8, 7, 7, 7, 8, 7, 8
- AverageThreadTokens: 8.89
- PeakMemoryUsage: 95.96 MB (100621968)
- PerHostPeakMemUsage: 95.96 MB (100621968)
- PrepareTime: 182.45ms
- RowsProduced: 11 (11)
- TotalCpuTime: 1h12m
- TotalNetworkReceiveTime: 5s443ms
- TotalNetworkSendTime: 1.52us
- TotalStorageWaitTime: 1h11m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 676.954ms, non-child: 676.954ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 146.128ms
- LoadTime: 143.136us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 360.526ms
- PrepareTime: 168.632ms
DataStreamSender (dst_id=13):(Total: 170.362us, non-child: 170.362us, % non-child: 100.00%)
- BytesSent: 3.02 KB (3095)
- NetworkThroughput(*): 597.02 KB/sec
- OverallThroughput: 17.33 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 112.739us
- ThriftTransmitTime(*): 5.62ms
- UncompressedRowBatchSize: 4.93 KB (5053)
HASH_JOIN_NODE (id=6):(Total: 9m8s, non-child: 182.648ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.413us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 14.74us
- GetNewBlockTime: 97.517us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 23.20 MB (24331296)
- PinTime: 0ns
- ProbeRows: 11 (11)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 167.347us
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.141us
EXCHANGE_NODE (id=12):(Total: 2s592ms, non-child: 23.622us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.7us
- DeserializeRowBatchTimer: 13.791us
- FirstBatchArrivalWaitTime: 2s591ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m8s, non-child: 2s867ms, % non-child: 0.52%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 16.54us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 12.391us
- GetNewBlockTime: 57.881us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 16.72 MB (17536360)
- PinTime: 0ns
- ProbeRows: 11 (11)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 160.209us
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 807ns
EXCHANGE_NODE (id=9):(Total: 2s850ms, non-child: 38.433us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.485us
- DeserializeRowBatchTimer: 23.936us
- FirstBatchArrivalWaitTime: 2s843ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m5s, non-child: 9m5s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:48/181.20 MB 1:37/114.62 MB 2:42/127.72 MB 3:43/157.22 MB 4:47/147.42 MB 5:47/150.87 MB 6:38/118.01 MB 7:40/114.06 MB 8:42/130.43 MB 9:42/150.55 MB 10:50/148.80 MB 11:46/162.09 MB
Hdfs Read Thread Concurrency Bucket: 0:98.91% 1:1.093% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:24534
ExecOption: Codegen enabled: 0 out of 522
BytesRead(16s000ms): 5.49 MB, 30.27 MB, 52.39 MB, 67.64 MB, 83.22 MB, 110.33 MB, 141.90 MB, 160.60 MB, 186.95 MB, 209.01 MB, 227.67 MB, 255.74 MB, 283.18 MB, 305.44 MB, 332.12 MB, 372.76 MB, 407.82 MB, 439.11 MB, 459.96 MB, 485.71 MB, 510.36 MB, 536.34 MB, 557.19 MB, 582.58 MB, 607.27 MB, 619.95 MB, 634.42 MB, 651.84 MB, 684.70 MB, 711.41 MB, 732.02 MB, 758.70 MB, 792.16 MB, 817.25 MB
- AverageHdfsReadThreadConcurrency: 0.01
- AverageScannerThreadConcurrency: 7.88
- BytesRead: 831.42 MB (871802600)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 831.42 MB (871802600)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 831.42 MB (871802600)
- DecompressionTime: 983.901ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 109 (109)
- PeakMemoryUsage: 84.86 MB (88982512)
- PerReadThreadRawHdfsThroughput: 83.56 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.74M (11742520)
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- ScanRangesComplete: 522 (522)
- ScannerThreadsInvoluntaryContextSwitches: 462 (462)
- ScannerThreadsTotalWallClockTime: 1h12m
- MaterializeTupleTime(*): 15s494ms
- ScannerThreadsSysTime: 480.6ms
- ScannerThreadsUserTime: 18s133ms
- ScannerThreadsVoluntaryContextSwitches: 45.19K (45194)
- TotalRawHdfsReadTime(*): 9s949ms
- TotalReadThroughput: 1.51 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016c1 (host=hadoopc1h140:28002):(Total: 9m7s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:43/102.31 MB 1:53/180.25 MB 2:43/144.55 MB 3:40/108.97 MB 4:44/180.70 MB 5:45/141.59 MB 6:43/120.52 MB 7:40/128.61 MB 8:46/161.39 MB 9:49/179.13 MB 10:42/125.49 MB 11:40/128.13 MB
MemoryUsage(16s000ms): 19.45 MB, 53.80 MB, 48.38 MB, 38.89 MB, 54.41 MB, 58.86 MB, 62.12 MB, 45.65 MB, 42.32 MB, 35.85 MB, 38.88 MB, 40.62 MB, 39.09 MB, 32.58 MB, 35.69 MB, 36.68 MB, 34.17 MB, 52.37 MB, 30.77 MB, 41.96 MB, 41.15 MB, 40.40 MB, 41.27 MB, 33.97 MB, 44.25 MB, 41.84 MB, 37.80 MB, 36.32 MB, 37.01 MB, 45.30 MB, 36.01 MB, 30.13 MB, 29.86 MB, 15.78 MB
ThreadUsage(16s000ms): 11, 12, 13, 13, 13, 14, 15, 12, 12, 11, 9, 8, 8, 7, 8, 9, 8, 9, 10, 10, 11, 10, 13, 9, 10, 11, 12, 11, 9, 9, 9, 10, 9, 6
- AverageThreadTokens: 11.02
- PeakMemoryUsage: 77.19 MB (80944696)
- PerHostPeakMemUsage: 77.19 MB (80944696)
- PrepareTime: 196.138ms
- RowsProduced: 11 (11)
- TotalCpuTime: 1h31m
- TotalNetworkReceiveTime: 5s440ms
- TotalNetworkSendTime: 1.527us
- TotalStorageWaitTime: 1h31m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 678.242ms, non-child: 678.242ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 138.697ms
- LoadTime: 151.377us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 356.408ms
- PrepareTime: 181.267ms
DataStreamSender (dst_id=13):(Total: 230.361us, non-child: 230.361us, % non-child: 100.00%)
- BytesSent: 3.55 KB (3634)
- NetworkThroughput(*): 1.65 MB/sec
- OverallThroughput: 15.04 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 161.746us
- ThriftTransmitTime(*): 2.96ms
- UncompressedRowBatchSize: 4.97 KB (5085)
HASH_JOIN_NODE (id=6):(Total: 9m7s, non-child: 196.417ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.76us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.432us
- GetNewBlockTime: 53.137us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 27.05 MB (28363824)
- PinTime: 0ns
- ProbeRows: 11 (11)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 197.864us
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 790ns
EXCHANGE_NODE (id=12):(Total: 2s593ms, non-child: 25.479us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.8us
- DeserializeRowBatchTimer: 11.371us
- FirstBatchArrivalWaitTime: 2s587ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 9m7s, non-child: 2s865ms, % non-child: 0.52%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.724us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.508us
- GetNewBlockTime: 84.694us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 16.18 MB (16965416)
- PinTime: 0ns
- ProbeRows: 11 (11)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 198.598us
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.373us
EXCHANGE_NODE (id=9):(Total: 2s847ms, non-child: 27.553us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.931us
- DeserializeRowBatchTimer: 12.746us
- FirstBatchArrivalWaitTime: 2s839ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 9m4s, non-child: 9m4s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:43/102.31 MB 1:53/180.25 MB 2:43/144.55 MB 3:40/108.97 MB 4:44/180.70 MB 5:45/141.59 MB 6:43/120.52 MB 7:40/128.61 MB 8:46/161.39 MB 9:49/179.13 MB 10:42/125.49 MB 11:40/128.13 MB
Hdfs Read Thread Concurrency Bucket: 0:97.81% 1:2.1% 2:0.09132% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:24816
ExecOption: Codegen enabled: 0 out of 528
BytesRead(16s000ms): 5.90 MB, 36.63 MB, 66.74 MB, 82.02 MB, 99.75 MB, 126.23 MB, 153.58 MB, 171.98 MB, 193.58 MB, 208.08 MB, 225.52 MB, 251.54 MB, 280.10 MB, 303.78 MB, 326.93 MB, 359.19 MB, 392.87 MB, 428.63 MB, 446.72 MB, 468.37 MB, 502.56 MB, 534.13 MB, 559.14 MB, 580.91 MB, 604.87 MB, 626.31 MB, 639.39 MB, 660.38 MB, 692.84 MB, 722.00 MB, 755.86 MB, 783.26 MB, 810.31 MB, 825.96 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 10.02
- BytesRead: 831.88 MB (872287309)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 831.88 MB (872287309)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 831.88 MB (872287309)
- DecompressionTime: 764.44ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 134 (134)
- PeakMemoryUsage: 74.83 MB (78467960)
- PerReadThreadRawHdfsThroughput: 59.57 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.71M (11712465)
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- ScanRangesComplete: 528 (528)
- ScannerThreadsInvoluntaryContextSwitches: 521 (521)
- ScannerThreadsTotalWallClockTime: 1h31m
- MaterializeTupleTime(*): 15s896ms
- ScannerThreadsSysTime: 532.8ms
- ScannerThreadsUserTime: 18s529ms
- ScannerThreadsVoluntaryContextSwitches: 40.73K (40735)
- TotalRawHdfsReadTime(*): 13s965ms
- TotalReadThroughput: 1.52 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016c2 (host=hadoopc1h30:28002):(Total: 8m57s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:46/95.45 MB 1:45/143.48 MB 2:53/162.26 MB 3:36/111.81 MB 4:47/168.81 MB 5:54/199.09 MB 6:41/157.96 MB 7:53/184.61 MB 8:36/103.46 MB 9:46/141.20 MB 10:40/89.62 MB 11:45/145.11 MB
MemoryUsage(16s000ms): 26.11 MB, 27.31 MB, 33.64 MB, 45.91 MB, 36.90 MB, 46.86 MB, 52.58 MB, 45.54 MB, 50.96 MB, 29.42 MB, 23.38 MB, 33.12 MB, 28.76 MB, 26.87 MB, 29.32 MB, 31.81 MB, 34.34 MB, 38.90 MB, 36.23 MB, 30.26 MB, 41.92 MB, 48.16 MB, 59.29 MB, 25.53 MB, 34.04 MB, 39.74 MB, 44.22 MB, 39.29 MB, 26.05 MB, 31.68 MB, 20.21 MB, 22.60 MB, 20.12 MB
ThreadUsage(16s000ms): 9, 8, 10, 11, 11, 12, 12, 10, 10, 8, 7, 6, 7, 6, 7, 7, 7, 8, 9, 8, 10, 9, 11, 8, 8, 8, 9, 8, 7, 8, 9, 10, 7
- AverageThreadTokens: 9.21
- PeakMemoryUsage: 81.52 MB (85476656)
- PerHostPeakMemUsage: 81.52 MB (85476656)
- PrepareTime: 186.805ms
- RowsProduced: 11 (11)
- TotalCpuTime: 1h13m
- TotalNetworkReceiveTime: 5s529ms
- TotalNetworkSendTime: 1.475us
- TotalStorageWaitTime: 1h13m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 640.516ms, non-child: 640.516ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 123.374ms
- LoadTime: 179.7us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 343.159ms
- PrepareTime: 172.274ms
DataStreamSender (dst_id=13):(Total: 205.642us, non-child: 205.642us, % non-child: 100.00%)
- BytesSent: 3.20 KB (3278)
- NetworkThroughput(*): 1.58 MB/sec
- OverallThroughput: 15.20 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 145.21us
- ThriftTransmitTime(*): 1.981ms
- UncompressedRowBatchSize: 4.95 KB (5071)
HASH_JOIN_NODE (id=6):(Total: 8m57s, non-child: 187.163ms, % non-child: 0.03%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.543us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.185us
- GetNewBlockTime: 63.175us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.20 MB (25373720)
- PinTime: 0ns
- ProbeRows: 11 (11)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 180.418us
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 819ns
EXCHANGE_NODE (id=12):(Total: 2s638ms, non-child: 25.392us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.79us
- DeserializeRowBatchTimer: 16.20us
- FirstBatchArrivalWaitTime: 2s626ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 8m57s, non-child: 2s911ms, % non-child: 0.54%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.868us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.3us
- GetNewBlockTime: 63.147us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.02 MB (15752768)
- PinTime: 0ns
- ProbeRows: 11 (11)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 158.877us
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 955ns
EXCHANGE_NODE (id=9):(Total: 2s890ms, non-child: 25.816us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.987us
- DeserializeRowBatchTimer: 35.557us
- FirstBatchArrivalWaitTime: 2s878ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 8m54s, non-child: 8m54s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:46/95.45 MB 1:45/143.48 MB 2:53/162.26 MB 3:36/111.81 MB 4:47/168.81 MB 5:54/199.09 MB 6:41/157.96 MB 7:53/184.61 MB 8:36/103.46 MB 9:46/141.20 MB 10:40/89.62 MB 11:45/145.11 MB
Hdfs Read Thread Concurrency Bucket: 0:98.51% 1:1.395% 2:0.09302% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:25474
ExecOption: Codegen enabled: 0 out of 542
BytesRead(16s000ms): 9.67 MB, 32.92 MB, 51.58 MB, 74.24 MB, 88.28 MB, 107.24 MB, 133.63 MB, 153.56 MB, 180.30 MB, 200.95 MB, 209.91 MB, 231.63 MB, 262.63 MB, 286.67 MB, 309.88 MB, 351.08 MB, 403.58 MB, 454.24 MB, 481.42 MB, 496.90 MB, 530.05 MB, 566.90 MB, 605.39 MB, 629.11 MB, 647.85 MB, 667.91 MB, 686.24 MB, 713.04 MB, 741.59 MB, 768.76 MB, 789.20 MB, 805.80 MB, 823.31 MB
- AverageHdfsReadThreadConcurrency: 0.01
- AverageScannerThreadConcurrency: 8.21
- BytesRead: 833.29 MB (873763871)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 833.29 MB (873763871)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 833.29 MB (873763871)
- DecompressionTime: 1s034ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 120 (120)
- PeakMemoryUsage: 74.36 MB (77968696)
- PerReadThreadRawHdfsThroughput: 76.64 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.71M (11709107)
- RowsReturned: 11 (11)
- RowsReturnedRate: 0
- ScanRangesComplete: 542 (542)
- ScannerThreadsInvoluntaryContextSwitches: 1.48K (1476)
- ScannerThreadsTotalWallClockTime: 1h13m
- MaterializeTupleTime(*): 16s994ms
- ScannerThreadsSysTime: 580.7ms
- ScannerThreadsUserTime: 19s877ms
- ScannerThreadsVoluntaryContextSwitches: 48.18K (48178)
- TotalRawHdfsReadTime(*): 10s872ms
- TotalReadThroughput: 1.55 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016b0 (host=hadoopc1h68:28002):(Total: 8m57s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:46/149.74 MB 1:42/133.44 MB 2:37/114.87 MB 3:39/110.23 MB 4:50/167.99 MB 5:45/173.62 MB 6:46/152.53 MB 7:48/183.81 MB 8:35/109.70 MB 9:44/112.03 MB 10:41/155.81 MB 11:42/170.51 MB
MemoryUsage(16s000ms): 26.75 MB, 44.11 MB, 56.43 MB, 52.64 MB, 45.40 MB, 52.56 MB, 57.54 MB, 50.09 MB, 46.79 MB, 50.70 MB, 40.75 MB, 31.66 MB, 37.38 MB, 26.76 MB, 40.65 MB, 37.84 MB, 30.29 MB, 34.43 MB, 43.44 MB, 51.56 MB, 40.77 MB, 35.72 MB, 48.59 MB, 40.97 MB, 41.70 MB, 40.36 MB, 37.58 MB, 48.62 MB, 39.39 MB, 32.85 MB, 36.29 MB, 33.97 MB, 21.12 MB
ThreadUsage(16s000ms): 11, 11, 13, 12, 13, 12, 15, 12, 12, 10, 9, 8, 8, 7, 8, 10, 8, 9, 10, 10, 10, 9, 11, 9, 9, 9, 10, 10, 9, 9, 9, 10, 9
- AverageThreadTokens: 10.76
- PeakMemoryUsage: 86.55 MB (90759136)
- PerHostPeakMemUsage: 86.55 MB (90759136)
- PrepareTime: 206.904ms
- RowsProduced: 8 (8)
- TotalCpuTime: 1h27m
- TotalNetworkReceiveTime: 5s395ms
- TotalNetworkSendTime: 1.198us
- TotalStorageWaitTime: 1h27m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 709.148ms, non-child: 709.148ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 139.711ms
- LoadTime: 131.172us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 375.476ms
- PrepareTime: 191.969ms
DataStreamSender (dst_id=13):(Total: 213.179us, non-child: 213.179us, % non-child: 100.00%)
- BytesSent: 2.77 KB (2832)
- NetworkThroughput(*): 1.59 MB/sec
- OverallThroughput: 12.67 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 150.209us
- ThriftTransmitTime(*): 1.694ms
- UncompressedRowBatchSize: 3.63 KB (3717)
HASH_JOIN_NODE (id=6):(Total: 8m57s, non-child: 207.142ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.37us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 5.578us
- GetNewBlockTime: 84.484us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.85 MB (27104896)
- PinTime: 0ns
- ProbeRows: 8 (8)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 181.436us
- RowsReturned: 8 (8)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.202us
EXCHANGE_NODE (id=12):(Total: 2s568ms, non-child: 32.5us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.406us
- DeserializeRowBatchTimer: 18.446us
- FirstBatchArrivalWaitTime: 2s564ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 8m56s, non-child: 2s843ms, % non-child: 0.53%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.973us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.853us
- GetNewBlockTime: 58.417us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 17.00 MB (17823064)
- PinTime: 0ns
- ProbeRows: 8 (8)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 160.728us
- RowsReturned: 8 (8)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 778ns
EXCHANGE_NODE (id=9):(Total: 2s826ms, non-child: 27.483us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.112us
- DeserializeRowBatchTimer: 14.791us
- FirstBatchArrivalWaitTime: 2s816ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 8m54s, non-child: 8m54s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:46/149.74 MB 1:42/133.44 MB 2:37/114.87 MB 3:39/110.23 MB 4:50/167.99 MB 5:45/173.62 MB 6:46/152.53 MB 7:48/183.81 MB 8:35/109.70 MB 9:44/112.03 MB 10:41/155.81 MB 11:42/170.51 MB
Hdfs Read Thread Concurrency Bucket: 0:97.21% 1:2.7% 2:0.09311% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:24205
ExecOption: Codegen enabled: 0 out of 515
BytesRead(16s000ms): 7.91 MB, 33.06 MB, 60.52 MB, 82.15 MB, 99.11 MB, 121.22 MB, 148.01 MB, 167.61 MB, 187.86 MB, 212.15 MB, 229.44 MB, 253.07 MB, 285.46 MB, 309.87 MB, 341.05 MB, 375.81 MB, 413.34 MB, 446.19 MB, 463.88 MB, 482.19 MB, 509.95 MB, 534.79 MB, 561.46 MB, 589.57 MB, 613.28 MB, 632.76 MB, 648.53 MB, 678.38 MB, 723.12 MB, 748.13 MB, 777.70 MB, 813.01 MB, 833.53 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 9.76
- BytesRead: 846.11 MB (887211488)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 846.11 MB (887211488)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 846.11 MB (887211488)
- DecompressionTime: 1s250ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 133 (133)
- PeakMemoryUsage: 78.37 MB (82175608)
- PerReadThreadRawHdfsThroughput: 48.96 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.94M (11939481)
- RowsReturned: 8 (8)
- RowsReturnedRate: 0
- ScanRangesComplete: 515 (515)
- ScannerThreadsInvoluntaryContextSwitches: 800 (800)
- ScannerThreadsTotalWallClockTime: 1h27m
- MaterializeTupleTime(*): 16s526ms
- ScannerThreadsSysTime: 540.10ms
- ScannerThreadsUserTime: 19s213ms
- ScannerThreadsVoluntaryContextSwitches: 40.28K (40280)
- TotalRawHdfsReadTime(*): 17s281ms
- TotalReadThroughput: 1.57 MB/sec
Instance c14f7b9b17987c60:d5379fb95410169e (host=hadoopc1h24:28002):(Total: 8m56s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:62/169.52 MB 1:39/105.10 MB 2:57/148.21 MB 3:60/171.55 MB 4:53/125.21 MB 5:48/137.77 MB 6:43/113.20 MB 7:53/152.73 MB 8:54/134.62 MB 9:58/159.73 MB 10:35/88.96 MB 11:55/147.81 MB
MemoryUsage(16s000ms): 18.98 MB, 31.47 MB, 42.79 MB, 32.46 MB, 35.95 MB, 44.96 MB, 27.23 MB, 27.26 MB, 40.93 MB, 26.00 MB, 34.70 MB, 47.75 MB, 42.05 MB, 46.48 MB, 43.67 MB, 40.64 MB, 39.63 MB, 42.54 MB, 52.85 MB, 46.43 MB, 40.16 MB, 43.79 MB, 40.74 MB, 32.34 MB, 31.55 MB, 42.18 MB, 35.92 MB, 30.07 MB, 25.11 MB, 35.40 MB, 22.89 MB, 25.53 MB, 13.68 MB
ThreadUsage(16s000ms): 9, 10, 10, 11, 11, 12, 13, 12, 13, 11, 10, 11, 13, 12, 11, 11, 11, 11, 12, 12, 11, 12, 12, 14, 12, 13, 13, 12, 11, 9, 11, 13, 7
- AverageThreadTokens: 11.91
- PeakMemoryUsage: 72.33 MB (75846272)
- PerHostPeakMemUsage: 72.33 MB (75846272)
- PrepareTime: 192.250ms
- RowsProduced: 19 (19)
- TotalCpuTime: 1h37m
- TotalNetworkReceiveTime: 5s475ms
- TotalNetworkSendTime: 7.554us
- TotalStorageWaitTime: 1h37m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 663.677ms, non-child: 663.677ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 127.129ms
- LoadTime: 177.260us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 356.962ms
- PrepareTime: 177.774ms
DataStreamSender (dst_id=13):(Total: 314.251us, non-child: 314.251us, % non-child: 100.00%)
- BytesSent: 5.24 KB (5362)
- NetworkThroughput(*): 1.01 MB/sec
- OverallThroughput: 16.27 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 205.244us
- ThriftTransmitTime(*): 5.59ms
- UncompressedRowBatchSize: 8.44 KB (8643)
HASH_JOIN_NODE (id=6):(Total: 8m56s, non-child: 193.249ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 17.336us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.607us
- GetNewBlockTime: 68.492us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.38 MB (25569336)
- PinTime: 0ns
- ProbeRows: 19 (19)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 148.760us
- RowsReturned: 19 (19)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 813ns
EXCHANGE_NODE (id=12):(Total: 2s611ms, non-child: 31.11us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.192us
- DeserializeRowBatchTimer: 13.134us
- FirstBatchArrivalWaitTime: 2s604ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 8m56s, non-child: 2s884ms, % non-child: 0.54%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.710us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.604us
- GetNewBlockTime: 80.119us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.92 MB (16688496)
- PinTime: 0ns
- ProbeRows: 19 (19)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 173.582us
- RowsReturned: 19 (19)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.776us
EXCHANGE_NODE (id=9):(Total: 2s864ms, non-child: 22.684us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.577us
- DeserializeRowBatchTimer: 18.412us
- FirstBatchArrivalWaitTime: 2s856ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 8m53s, non-child: 8m53s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:62/169.52 MB 1:39/105.10 MB 2:57/148.21 MB 3:60/171.55 MB 4:53/125.21 MB 5:48/137.77 MB 6:43/113.20 MB 7:53/152.73 MB 8:54/134.62 MB 9:58/159.73 MB 10:35/88.96 MB 11:55/147.81 MB
Hdfs Read Thread Concurrency Bucket: 0:98.42% 1:1.584% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28999
ExecOption: Codegen enabled: 0 out of 617
BytesRead(16s000ms): 7.28 MB, 31.26 MB, 59.99 MB, 78.12 MB, 90.36 MB, 113.14 MB, 133.38 MB, 146.69 MB, 166.28 MB, 179.41 MB, 193.88 MB, 227.83 MB, 267.98 MB, 301.55 MB, 343.69 MB, 392.78 MB, 446.84 MB, 487.55 MB, 512.98 MB, 533.12 MB, 565.92 MB, 600.70 MB, 627.78 MB, 649.22 MB, 671.00 MB, 689.17 MB, 703.21 MB, 723.59 MB, 743.33 MB, 763.78 MB, 783.91 MB, 801.94 MB, 814.65 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 10.91
- BytesRead: 819.25 MB (859045381)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 819.25 MB (859045381)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 819.25 MB (859045381)
- DecompressionTime: 335.932ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 109 (109)
- PeakMemoryUsage: 65.05 MB (68209840)
- PerReadThreadRawHdfsThroughput: 75.22 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.28M (11282737)
- RowsReturned: 19 (19)
- RowsReturnedRate: 0
- ScanRangesComplete: 617 (617)
- ScannerThreadsInvoluntaryContextSwitches: 670 (670)
- ScannerThreadsTotalWallClockTime: 1h37m
- MaterializeTupleTime(*): 15s118ms
- ScannerThreadsSysTime: 560.11ms
- ScannerThreadsUserTime: 18s137ms
- ScannerThreadsVoluntaryContextSwitches: 54.64K (54640)
- TotalRawHdfsReadTime(*): 10s891ms
- TotalReadThroughput: 1.53 MB/sec
Instance c14f7b9b17987c60:d5379fb95410169b (host=hadoopc1h4:28002):(Total: 8m54s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:53/132.83 MB 1:45/112.50 MB 2:53/140.13 MB 3:50/136.36 MB 4:57/126.02 MB 5:49/114.07 MB 6:38/101.94 MB 7:60/151.67 MB 8:39/115.87 MB 9:53/159.99 MB 10:59/176.46 MB 11:58/164.79 MB
MemoryUsage(16s000ms): 22.31 MB, 30.77 MB, 30.32 MB, 36.19 MB, 40.33 MB, 49.56 MB, 42.99 MB, 51.55 MB, 60.67 MB, 45.56 MB, 31.39 MB, 45.34 MB, 31.84 MB, 33.73 MB, 39.40 MB, 43.79 MB, 48.66 MB, 41.87 MB, 51.37 MB, 49.83 MB, 35.40 MB, 48.04 MB, 30.71 MB, 30.57 MB, 33.58 MB, 29.42 MB, 24.98 MB, 25.74 MB, 21.91 MB, 27.01 MB, 22.22 MB, 13.81 MB, 10.95 MB
ThreadUsage(16s000ms): 11, 12, 11, 12, 14, 14, 15, 14, 15, 14, 12, 12, 9, 9, 10, 13, 11, 12, 14, 14, 13, 13, 13, 12, 10, 10, 10, 10, 8, 9, 10, 8, 5
- AverageThreadTokens: 12.13
- PeakMemoryUsage: 82.84 MB (86868136)
- PerHostPeakMemUsage: 82.84 MB (86868136)
- PrepareTime: 207.89ms
- RowsProduced: 12 (12)
- TotalCpuTime: 1h39m
- TotalNetworkReceiveTime: 5s325ms
- TotalNetworkSendTime: 1.806us
- TotalStorageWaitTime: 1h38m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 737.808ms, non-child: 737.808ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 146.477ms
- LoadTime: 165.602us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 398.505ms
- PrepareTime: 191.99ms
DataStreamSender (dst_id=13):(Total: 251.985us, non-child: 251.985us, % non-child: 100.00%)
- BytesSent: 3.76 KB (3850)
- NetworkThroughput(*): 1.23 MB/sec
- OverallThroughput: 14.57 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 177.161us
- ThriftTransmitTime(*): 2.975ms
- UncompressedRowBatchSize: 5.40 KB (5530)
HASH_JOIN_NODE (id=6):(Total: 8m54s, non-child: 365.679ms, % non-child: 0.07%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.700us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.658us
- GetNewBlockTime: 59.290us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 24.18 MB (25352184)
- PinTime: 0ns
- ProbeRows: 12 (12)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 194.666us
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 865ns
EXCHANGE_NODE (id=12):(Total: 2s531ms, non-child: 26.838us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.97us
- DeserializeRowBatchTimer: 17.946us
- FirstBatchArrivalWaitTime: 2s529ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 8m53s, non-child: 2s817ms, % non-child: 0.53%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.669us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.936us
- GetNewBlockTime: 65.987us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.87 MB (16641280)
- PinTime: 0ns
- ProbeRows: 12 (12)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 203.170us
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 898ns
EXCHANGE_NODE (id=9):(Total: 2s793ms, non-child: 25.326us, % non-child: 0.00%)
BytesReceived(16s000ms): 89.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.564us
- DeserializeRowBatchTimer: 21.925us
- FirstBatchArrivalWaitTime: 2s781ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 8m51s, non-child: 8m51s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:53/132.83 MB 1:45/112.50 MB 2:53/140.13 MB 3:50/136.36 MB 4:57/126.02 MB 5:49/114.07 MB 6:38/101.94 MB 7:60/151.67 MB 8:39/115.87 MB 9:53/159.99 MB 10:59/176.46 MB 11:58/164.79 MB
Hdfs Read Thread Concurrency Bucket: 0:96.63% 1:3.274% 2:0.09355% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28858
ExecOption: Codegen enabled: 0 out of 614
BytesRead(16s000ms): 6.31 MB, 26.42 MB, 44.71 MB, 63.04 MB, 77.82 MB, 101.55 MB, 126.38 MB, 147.23 MB, 177.32 MB, 201.65 MB, 218.79 MB, 247.23 MB, 286.65 MB, 321.92 MB, 361.55 MB, 414.45 MB, 471.65 MB, 515.87 MB, 538.38 MB, 560.08 MB, 583.27 MB, 617.93 MB, 637.25 MB, 656.93 MB, 677.54 MB, 695.46 MB, 708.30 MB, 727.30 MB, 746.97 MB, 767.80 MB, 785.98 MB, 803.10 MB, 806.12 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 11.13
- BytesRead: 808.29 MB (847554075)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 808.29 MB (847554075)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 808.29 MB (847554075)
- DecompressionTime: 720.344ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 127 (127)
- PeakMemoryUsage: 74.68 MB (78303416)
- PerReadThreadRawHdfsThroughput: 67.76 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.15M (11147056)
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- ScanRangesComplete: 614 (614)
- ScannerThreadsInvoluntaryContextSwitches: 569 (569)
- ScannerThreadsTotalWallClockTime: 1h39m
- MaterializeTupleTime(*): 16s602ms
- ScannerThreadsSysTime: 664.11ms
- ScannerThreadsUserTime: 19s569ms
- ScannerThreadsVoluntaryContextSwitches: 50.50K (50500)
- TotalRawHdfsReadTime(*): 11s929ms
- TotalReadThroughput: 1.51 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016a8 (host=hadoopc1h69:28002):(Total: 8m50s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:41/160.28 MB 1:38/98.73 MB 2:49/149.43 MB 3:38/151.42 MB 4:41/171.49 MB 5:44/151.83 MB 6:43/152.30 MB 7:43/132.90 MB 8:34/82.09 MB 9:45/126.21 MB 10:47/160.03 MB 11:45/158.91 MB
MemoryUsage(16s000ms): 34.18 MB, 36.07 MB, 61.46 MB, 44.38 MB, 42.57 MB, 53.94 MB, 60.14 MB, 52.84 MB, 48.79 MB, 46.23 MB, 41.57 MB, 36.45 MB, 31.53 MB, 35.94 MB, 41.73 MB, 38.37 MB, 41.95 MB, 37.62 MB, 45.76 MB, 31.14 MB, 58.51 MB, 42.94 MB, 41.89 MB, 41.07 MB, 51.08 MB, 49.14 MB, 48.88 MB, 41.33 MB, 41.25 MB, 34.46 MB, 37.57 MB, 18.67 MB, 9.46 MB
ThreadUsage(16s000ms): 11, 12, 13, 13, 13, 15, 15, 12, 12, 11, 9, 9, 8, 7, 9, 9, 8, 9, 10, 10, 11, 10, 12, 10, 10, 10, 11, 11, 9, 9, 9, 9, 4
- AverageThreadTokens: 11.00
- PeakMemoryUsage: 83.41 MB (87465344)
- PerHostPeakMemUsage: 83.41 MB (87465344)
- PrepareTime: 191.829ms
- RowsProduced: 18 (18)
- TotalCpuTime: 1h28m
- TotalNetworkReceiveTime: 5s426ms
- TotalNetworkSendTime: 3.340us
- TotalStorageWaitTime: 1h28m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 691.35ms, non-child: 691.35ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 146.148ms
- LoadTime: 138.757us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 366.90ms
- PrepareTime: 177.37ms
DataStreamSender (dst_id=13):(Total: 338.329us, non-child: 338.329us, % non-child: 100.00%)
- BytesSent: 5.40 KB (5527)
- NetworkThroughput(*): 1.86 MB/sec
- OverallThroughput: 15.58 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 240.11us
- ThriftTransmitTime(*): 2.834ms
- UncompressedRowBatchSize: 8.04 KB (8237)
HASH_JOIN_NODE (id=6):(Total: 8m50s, non-child: 192.218ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.930us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.851us
- GetNewBlockTime: 177.70us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 20.33 MB (21321584)
- PinTime: 0ns
- ProbeRows: 18 (18)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 184.78us
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.253us
EXCHANGE_NODE (id=12):(Total: 2s587ms, non-child: 23.624us, % non-child: 0.00%)
BytesReceived(16s000ms): 97.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.841us
- DeserializeRowBatchTimer: 13.125us
- FirstBatchArrivalWaitTime: 2s583ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 8m50s, non-child: 2s858ms, % non-child: 0.54%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.292us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.569us
- GetNewBlockTime: 153.818us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 14.49 MB (15189128)
- PinTime: 0ns
- ProbeRows: 18 (18)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 203.685us
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.203us
EXCHANGE_NODE (id=9):(Total: 2s838ms, non-child: 21.891us, % non-child: 0.00%)
BytesReceived(16s000ms): 93.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.944us
- DeserializeRowBatchTimer: 9.994us
- FirstBatchArrivalWaitTime: 2s835ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 8m47s, non-child: 8m47s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:41/160.28 MB 1:38/98.73 MB 2:49/149.43 MB 3:38/151.42 MB 4:41/171.49 MB 5:44/151.83 MB 6:43/152.30 MB 7:43/132.90 MB 8:34/82.09 MB 9:45/126.21 MB 10:47/160.03 MB 11:45/158.91 MB
Hdfs Read Thread Concurrency Bucket: 0:97.83% 1:2.074% 2:0.09425% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:23876
ExecOption: Codegen enabled: 0 out of 508
BytesRead(16s000ms): 10.22 MB, 35.95 MB, 66.36 MB, 83.09 MB, 96.70 MB, 115.65 MB, 141.79 MB, 161.60 MB, 183.45 MB, 200.42 MB, 216.82 MB, 240.85 MB, 266.91 MB, 297.88 MB, 328.91 MB, 363.38 MB, 403.65 MB, 434.77 MB, 449.10 MB, 465.53 MB, 498.20 MB, 531.37 MB, 558.64 MB, 583.39 MB, 617.09 MB, 638.39 MB, 660.16 MB, 689.90 MB, 729.52 MB, 764.90 MB, 792.35 MB, 815.72 MB, 823.84 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 10.01
- BytesRead: 826.47 MB (866619266)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 826.47 MB (866619266)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 826.47 MB (866619266)
- DecompressionTime: 1s140ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 139 (139)
- PeakMemoryUsage: 81.14 MB (85079416)
- PerReadThreadRawHdfsThroughput: 58.75 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.70M (11696425)
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- ScanRangesComplete: 508 (508)
- ScannerThreadsInvoluntaryContextSwitches: 461 (461)
- ScannerThreadsTotalWallClockTime: 1h28m
- MaterializeTupleTime(*): 15s625ms
- ScannerThreadsSysTime: 544.10ms
- ScannerThreadsUserTime: 18s173ms
- ScannerThreadsVoluntaryContextSwitches: 38.37K (38367)
- TotalRawHdfsReadTime(*): 14s068ms
- TotalReadThroughput: 1.56 MB/sec
Instance c14f7b9b17987c60:d5379fb95410169c (host=hadoopc1h25:28002):(Total: 8m19s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:56/120.79 MB 1:40/147.03 MB 2:39/105.04 MB 3:46/128.50 MB 4:53/154.62 MB 5:51/160.39 MB 6:53/139.39 MB 7:51/136.86 MB 8:44/130.97 MB 9:56/161.19 MB 10:48/123.87 MB 11:52/150.84 MB
MemoryUsage(8s000ms): 11.99 MB, 38.93 MB, 42.27 MB, 44.40 MB, 49.01 MB, 61.83 MB, 41.24 MB, 43.79 MB, 36.99 MB, 41.57 MB, 36.50 MB, 57.64 MB, 45.32 MB, 43.22 MB, 48.18 MB, 32.24 MB, 44.64 MB, 51.21 MB, 41.18 MB, 48.48 MB, 43.88 MB, 49.86 MB, 35.71 MB, 39.53 MB, 33.60 MB, 52.06 MB, 36.17 MB, 44.33 MB, 50.20 MB, 43.11 MB, 34.01 MB, 59.57 MB, 40.85 MB, 45.19 MB, 43.62 MB, 44.68 MB, 47.39 MB, 53.07 MB, 50.68 MB, 48.52 MB, 44.72 MB, 42.59 MB, 40.22 MB, 45.22 MB, 39.75 MB, 40.19 MB, 28.47 MB, 40.12 MB, 51.90 MB, 43.36 MB, 38.50 MB, 35.53 MB, 46.59 MB, 34.11 MB, 40.43 MB, 35.30 MB, 40.91 MB, 23.80 MB, 20.60 MB, 19.30 MB, 18.51 MB, 14.60 MB
ThreadUsage(8s000ms): 10, 15, 14, 12, 13, 13, 13, 14, 14, 13, 14, 16, 16, 15, 15, 14, 14, 16, 13, 10, 11, 10, 11, 10, 10, 11, 11, 11, 11, 10, 11, 12, 11, 12, 14, 14, 14, 14, 14, 13, 14, 13, 13, 13, 15, 14, 10, 11, 14, 14, 14, 15, 14, 15, 13, 12, 12, 11, 13, 10, 9, 6
- AverageThreadTokens: 13.02
- PeakMemoryUsage: 79.28 MB (83129144)
- PerHostPeakMemUsage: 79.28 MB (83129144)
- PrepareTime: 186.904ms
- RowsProduced: 18 (18)
- TotalCpuTime: 1h40m
- TotalNetworkReceiveTime: 5s548ms
- TotalNetworkSendTime: 1.741us
- TotalStorageWaitTime: 1h39m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 634.340ms, non-child: 634.340ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 125.896ms
- LoadTime: 153.949us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 333.716ms
- PrepareTime: 172.848ms
DataStreamSender (dst_id=13):(Total: 303.354us, non-child: 303.354us, % non-child: 100.00%)
- BytesSent: 5.26 KB (5383)
- NetworkThroughput(*): 1.16 MB/sec
- OverallThroughput: 16.92 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 217.849us
- ThriftTransmitTime(*): 4.415ms
- UncompressedRowBatchSize: 8.07 KB (8264)
HASH_JOIN_NODE (id=6):(Total: 8m19s, non-child: 187.910ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 6.349us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.532us
- GetNewBlockTime: 67.970us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 23.52 MB (24664496)
- PinTime: 0ns
- ProbeRows: 18 (18)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 202.405us
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.28us
EXCHANGE_NODE (id=12):(Total: 2s647ms, non-child: 29.702us, % non-child: 0.00%)
BytesReceived(8s000ms): 75.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.310us
- DeserializeRowBatchTimer: 19.335us
- FirstBatchArrivalWaitTime: 2s641ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 8m19s, non-child: 3s316ms, % non-child: 0.66%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.829us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 6.297us
- GetNewBlockTime: 68.796us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.31 MB (16052680)
- PinTime: 0ns
- ProbeRows: 18 (18)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 189.126us
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 850ns
EXCHANGE_NODE (id=9):(Total: 2s900ms, non-child: 23.78us, % non-child: 0.00%)
BytesReceived(8s000ms): 64.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.336us
- DeserializeRowBatchTimer: 19.190us
- FirstBatchArrivalWaitTime: 2s893ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 8m16s, non-child: 8m16s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:56/120.79 MB 1:40/147.03 MB 2:39/105.04 MB 3:46/128.50 MB 4:53/154.62 MB 5:51/160.39 MB 6:53/139.39 MB 7:51/136.86 MB 8:44/130.97 MB 9:56/161.19 MB 10:48/123.87 MB 11:52/150.84 MB
Hdfs Read Thread Concurrency Bucket: 0:97.7% 1:2.202% 2:0.1001% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:27683
ExecOption: Codegen enabled: 0 out of 589
BytesRead(8s000ms): 2.96 MB, 12.43 MB, 24.59 MB, 40.26 MB, 55.02 MB, 70.08 MB, 75.99 MB, 85.91 MB, 89.52 MB, 97.91 MB, 104.69 MB, 118.12 MB, 130.44 MB, 137.66 MB, 148.32 MB, 154.19 MB, 165.17 MB, 177.50 MB, 187.38 MB, 199.27 MB, 211.83 MB, 223.75 MB, 235.10 MB, 250.35 MB, 267.62 MB, 291.40 MB, 310.66 MB, 331.55 MB, 353.17 MB, 372.17 MB, 391.55 MB, 424.31 MB, 452.92 MB, 479.02 MB, 503.53 MB, 519.33 MB, 528.66 MB, 536.25 MB, 548.57 MB, 558.43 MB, 572.96 MB, 590.84 MB, 604.47 MB, 617.33 MB, 627.93 MB, 638.15 MB, 648.33 MB, 660.66 MB, 676.15 MB, 687.38 MB, 696.11 MB, 702.32 MB, 713.69 MB, 721.52 MB, 736.16 MB, 757.02 MB, 772.95 MB, 786.46 MB, 791.87 MB, 800.88 MB, 808.48 MB, 811.15 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 12.02
- BytesRead: 817.15 MB (856845538)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 817.15 MB (856845538)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 817.15 MB (856845538)
- DecompressionTime: 567.817ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 112 (112)
- PeakMemoryUsage: 70.37 MB (73791032)
- PerReadThreadRawHdfsThroughput: 64.02 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.39M (11388468)
- RowsReturned: 18 (18)
- RowsReturnedRate: 0
- ScanRangesComplete: 589 (589)
- ScannerThreadsInvoluntaryContextSwitches: 1.97K (1971)
- ScannerThreadsTotalWallClockTime: 1h40m
- MaterializeTupleTime(*): 17s984ms
- ScannerThreadsSysTime: 708.20ms
- ScannerThreadsUserTime: 21s105ms
- ScannerThreadsVoluntaryContextSwitches: 54.66K (54661)
- TotalRawHdfsReadTime(*): 12s764ms
- TotalReadThroughput: 1.63 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016c8 (host=hadoopc1h29:28002):(Total: 8m17s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:56/153.79 MB 1:46/122.57 MB 2:39/153.63 MB 3:44/149.70 MB 4:48/178.66 MB 5:36/94.64 MB 6:55/177.08 MB 7:36/99.72 MB 8:42/120.33 MB 9:37/109.75 MB 10:48/158.65 MB 11:54/182.38 MB
MemoryUsage(8s000ms): 11.48 MB, 41.48 MB, 57.62 MB, 28.20 MB, 25.31 MB, 53.04 MB, 39.39 MB, 43.68 MB, 28.41 MB, 33.25 MB, 31.56 MB, 41.29 MB, 47.70 MB, 30.70 MB, 32.49 MB, 35.13 MB, 28.12 MB, 40.44 MB, 26.28 MB, 17.73 MB, 23.35 MB, 18.01 MB, 27.95 MB, 29.97 MB, 38.55 MB, 27.07 MB, 33.11 MB, 43.16 MB, 44.76 MB, 35.28 MB, 35.79 MB, 41.05 MB, 33.64 MB, 29.96 MB, 31.89 MB, 24.23 MB, 28.44 MB, 44.75 MB, 34.93 MB, 30.60 MB, 34.42 MB, 32.68 MB, 38.00 MB, 42.97 MB, 28.90 MB, 39.55 MB, 28.29 MB, 44.34 MB, 37.66 MB, 44.56 MB, 29.55 MB, 25.18 MB, 35.16 MB, 39.51 MB, 20.29 MB, 19.74 MB, 25.93 MB, 15.64 MB, 13.53 MB, 18.13 MB, 11.65 MB, 8.22 MB
ThreadUsage(8s000ms): 7, 11, 8, 9, 10, 10, 10, 10, 10, 10, 11, 12, 13, 11, 10, 10, 10, 11, 8, 7, 8, 7, 7, 6, 7, 7, 7, 6, 7, 7, 8, 7, 7, 7, 8, 8, 9, 10, 9, 9, 9, 9, 8, 9, 11, 10, 9, 9, 9, 8, 8, 9, 9, 11, 10, 10, 8, 8, 9, 8, 6, 3
- AverageThreadTokens: 9.14
- PeakMemoryUsage: 72.19 MB (75699784)
- PerHostPeakMemUsage: 72.19 MB (75699784)
- PrepareTime: 193.199ms
- RowsProduced: 10 (10)
- TotalCpuTime: 1h7m
- TotalNetworkReceiveTime: 5s454ms
- TotalNetworkSendTime: 2.243us
- TotalStorageWaitTime: 1h7m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 120.31 GB (129184563200)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 678.724ms, non-child: 678.724ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 142.848ms
- LoadTime: 118.970us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 355.86ms
- PrepareTime: 179.117ms
DataStreamSender (dst_id=13):(Total: 234.174us, non-child: 234.174us, % non-child: 100.00%)
- BytesSent: 3.09 KB (3161)
- NetworkThroughput(*): 1.39 MB/sec
- OverallThroughput: 12.87 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 173.228us
- ThriftTransmitTime(*): 2.171ms
- UncompressedRowBatchSize: 4.51 KB (4619)
HASH_JOIN_NODE (id=6):(Total: 8m17s, non-child: 393.679ms, % non-child: 0.08%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 9.194us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.250us
- GetNewBlockTime: 66.627us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 29.40 MB (30831952)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 190.658us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 888ns
EXCHANGE_NODE (id=12):(Total: 2s599ms, non-child: 25.466us, % non-child: 0.00%)
BytesReceived(8s000ms): 67.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.823us
- DeserializeRowBatchTimer: 15.486us
- FirstBatchArrivalWaitTime: 2s592ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 8m17s, non-child: 2s875ms, % non-child: 0.58%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.330us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.463us
- GetNewBlockTime: 57.602us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 16.97 MB (17790952)
- PinTime: 0ns
- ProbeRows: 10 (10)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 181.85us
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 930ns
EXCHANGE_NODE (id=9):(Total: 2s855ms, non-child: 26.373us, % non-child: 0.00%)
BytesReceived(8s000ms): 64.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.10us
- DeserializeRowBatchTimer: 14.818us
- FirstBatchArrivalWaitTime: 2s844ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 8m14s, non-child: 8m14s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:56/153.79 MB 1:46/122.57 MB 2:39/153.63 MB 3:44/149.70 MB 4:48/178.66 MB 5:36/94.64 MB 6:55/177.08 MB 7:36/99.72 MB 8:42/120.33 MB 9:37/109.75 MB 10:48/158.65 MB 11:54/182.38 MB
Hdfs Read Thread Concurrency Bucket: 0:97.19% 1:2.814% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:25427
ExecOption: Codegen enabled: 0 out of 541
BytesRead(8s000ms): 2.99 MB, 16.53 MB, 36.19 MB, 49.76 MB, 59.07 MB, 73.62 MB, 81.10 MB, 93.75 MB, 99.20 MB, 107.62 MB, 116.76 MB, 128.23 MB, 148.90 MB, 159.96 MB, 169.75 MB, 180.01 MB, 192.58 MB, 211.35 MB, 219.52 MB, 223.33 MB, 233.49 MB, 238.35 MB, 250.16 MB, 267.90 MB, 293.12 MB, 312.07 MB, 326.58 MB, 348.30 MB, 372.31 MB, 392.53 MB, 418.38 MB, 454.94 MB, 487.39 MB, 518.31 MB, 543.06 MB, 556.99 MB, 563.80 MB, 573.55 MB, 583.66 MB, 593.84 MB, 608.06 MB, 625.86 MB, 644.54 MB, 661.04 MB, 672.99 MB, 686.14 MB, 699.91 MB, 719.04 MB, 734.10 MB, 746.73 MB, 753.28 MB, 759.70 MB, 769.83 MB, 780.43 MB, 787.40 MB, 793.90 MB, 806.47 MB, 815.19 MB, 818.16 MB, 824.00 MB, 829.26 MB, 830.86 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 8.14
- BytesRead: 832.87 MB (873327965)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 832.87 MB (873327965)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 832.87 MB (873327965)
- DecompressionTime: 1s055ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 114 (114)
- PeakMemoryUsage: 71.64 MB (75122184)
- PerReadThreadRawHdfsThroughput: 64.71 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.70M (11703986)
- RowsReturned: 10 (10)
- RowsReturnedRate: 0
- ScanRangesComplete: 541 (541)
- ScannerThreadsInvoluntaryContextSwitches: 3.80K (3803)
- ScannerThreadsTotalWallClockTime: 1h7m
- MaterializeTupleTime(*): 19s749ms
- ScannerThreadsSysTime: 744.22ms
- ScannerThreadsUserTime: 22s637ms
- ScannerThreadsVoluntaryContextSwitches: 51.00K (51000)
- TotalRawHdfsReadTime(*): 12s870ms
- TotalReadThroughput: 1.67 MB/sec
Instance c14f7b9b17987c60:d5379fb954101697 (host=hadoopc1h66:28002):(Total: 8m16s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:40/115.59 MB 1:34/145.99 MB 2:37/138.66 MB 3:40/139.57 MB 4:46/153.09 MB 5:44/174.27 MB 6:27/98.19 MB 7:44/173.07 MB 8:40/150.09 MB 9:33/129.82 MB 10:41/135.43 MB 11:42/155.62 MB
MemoryUsage(8s000ms): 11.41 MB, 52.60 MB, 74.65 MB, 46.60 MB, 68.89 MB, 54.05 MB, 50.44 MB, 62.66 MB, 64.77 MB, 59.00 MB, 54.62 MB, 49.79 MB, 53.33 MB, 61.41 MB, 49.16 MB, 49.17 MB, 46.61 MB, 52.16 MB, 69.51 MB, 44.83 MB, 38.96 MB, 41.01 MB, 48.00 MB, 46.37 MB, 42.51 MB, 49.45 MB, 38.84 MB, 33.08 MB, 30.84 MB, 38.51 MB, 48.35 MB, 35.07 MB, 47.02 MB, 36.76 MB, 37.53 MB, 51.13 MB, 49.26 MB, 56.59 MB, 47.27 MB, 45.09 MB, 41.47 MB, 36.87 MB, 33.91 MB, 28.94 MB, 32.66 MB, 47.29 MB, 44.22 MB, 38.08 MB, 46.68 MB, 44.74 MB, 27.26 MB, 45.54 MB, 44.78 MB, 47.04 MB, 58.65 MB, 37.55 MB, 30.13 MB, 45.83 MB, 38.35 MB, 19.82 MB, 16.29 MB, 11.16 MB
ThreadUsage(8s000ms): 8, 14, 14, 11, 13, 13, 13, 13, 13, 14, 14, 15, 17, 14, 13, 12, 12, 14, 12, 10, 9, 9, 8, 8, 8, 9, 8, 8, 8, 8, 10, 8, 9, 8, 9, 9, 10, 11, 11, 11, 11, 11, 10, 10, 11, 11, 9, 9, 10, 9, 9, 10, 11, 12, 12, 11, 10, 9, 10, 9, 5, 3
- AverageThreadTokens: 10.96
- PeakMemoryUsage: 106.20 MB (111363192)
- PerHostPeakMemUsage: 106.20 MB (111363192)
- PrepareTime: 201.549ms
- RowsProduced: 5 (5)
- TotalCpuTime: 1h22m
- TotalNetworkReceiveTime: 5s369ms
- TotalNetworkSendTime: 986ns
- TotalStorageWaitTime: 1h22m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 192 (192)
- BlocksRecycled: 68 (68)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 512.00 KB (524288)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 726.767ms, non-child: 726.767ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 158.100ms
- LoadTime: 152.866us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 379.868ms
- PrepareTime: 186.990ms
DataStreamSender (dst_id=13):(Total: 116.506us, non-child: 116.506us, % non-child: 100.00%)
- BytesSent: 1.61 KB (1649)
- NetworkThroughput(*): 1.45 MB/sec
- OverallThroughput: 13.50 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 75.96us
- ThriftTransmitTime(*): 1.81ms
- UncompressedRowBatchSize: 2.24 KB (2290)
HASH_JOIN_NODE (id=6):(Total: 8m16s, non-child: 201.838ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.847us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.604us
- GetNewBlockTime: 38.848us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 25.70 MB (26947408)
- PinTime: 0ns
- ProbeRows: 5 (5)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 143.876us
- RowsReturned: 5 (5)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 843ns
EXCHANGE_NODE (id=12):(Total: 2s553ms, non-child: 23.934us, % non-child: 0.00%)
BytesReceived(8s000ms): 75.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.723us
- DeserializeRowBatchTimer: 10.584us
- FirstBatchArrivalWaitTime: 2s549ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 8m16s, non-child: 2s831ms, % non-child: 0.57%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.917us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 3.903us
- GetNewBlockTime: 45.881us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 16.45 MB (17251968)
- PinTime: 0ns
- ProbeRows: 5 (5)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 150.683us
- RowsReturned: 5 (5)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 986ns
EXCHANGE_NODE (id=9):(Total: 2s815ms, non-child: 21.772us, % non-child: 0.00%)
BytesReceived(8s000ms): 64.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.801us
- DeserializeRowBatchTimer: 9.994us
- FirstBatchArrivalWaitTime: 2s801ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 8m13s, non-child: 8m13s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:40/115.59 MB 1:34/145.99 MB 2:37/138.66 MB 3:40/139.57 MB 4:46/153.09 MB 5:44/174.27 MB 6:27/98.19 MB 7:44/173.07 MB 8:40/150.09 MB 9:33/129.82 MB 10:41/135.43 MB 11:42/155.62 MB
Hdfs Read Thread Concurrency Bucket: 0:97.38% 1:2.518% 2:0.1007% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:21996
ExecOption: Codegen enabled: 0 out of 468
BytesRead(8s000ms): 3.64 MB, 17.03 MB, 40.42 MB, 59.70 MB, 79.11 MB, 88.40 MB, 98.26 MB, 111.63 MB, 118.90 MB, 130.92 MB, 141.18 MB, 152.68 MB, 166.90 MB, 179.39 MB, 190.45 MB, 199.74 MB, 210.62 MB, 226.47 MB, 237.87 MB, 246.58 MB, 257.47 MB, 266.44 MB, 279.94 MB, 295.85 MB, 315.09 MB, 334.10 MB, 347.96 MB, 358.23 MB, 370.69 MB, 383.17 MB, 404.74 MB, 427.42 MB, 453.93 MB, 477.92 MB, 497.13 MB, 511.53 MB, 519.10 MB, 528.20 MB, 539.16 MB, 550.50 MB, 566.14 MB, 584.41 MB, 599.18 MB, 609.96 MB, 622.02 MB, 634.66 MB, 650.19 MB, 662.23 MB, 675.54 MB, 688.89 MB, 697.60 MB, 705.96 MB, 717.44 MB, 729.52 MB, 748.19 MB, 763.38 MB, 777.98 MB, 798.91 MB, 807.21 MB, 817.09 MB, 825.50 MB, 826.63 MB
- AverageHdfsReadThreadConcurrency: 0.03
- AverageScannerThreadConcurrency: 9.97
- BytesRead: 828.22 MB (868453865)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 828.22 MB (868453865)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 828.22 MB (868453865)
- DecompressionTime: 571.537ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 121 (121)
- PeakMemoryUsage: 98.55 MB (103332992)
- PerReadThreadRawHdfsThroughput: 67.23 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.84M (11839894)
- RowsReturned: 5 (5)
- RowsReturnedRate: 0
- ScanRangesComplete: 468 (468)
- ScannerThreadsInvoluntaryContextSwitches: 543 (543)
- ScannerThreadsTotalWallClockTime: 1h22m
- MaterializeTupleTime(*): 15s991ms
- ScannerThreadsSysTime: 504.7ms
- ScannerThreadsUserTime: 18s557ms
- ScannerThreadsVoluntaryContextSwitches: 36.00K (35999)
- TotalRawHdfsReadTime(*): 12s319ms
- TotalReadThroughput: 1.67 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016bf (host=hadoopc1h67:28002):(Total: 8m16s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:41/136.75 MB 1:42/116.16 MB 2:35/99.68 MB 3:33/98.32 MB 4:52/184.85 MB 5:35/101.62 MB 6:44/127.58 MB 7:44/147.97 MB 8:46/179.72 MB 9:47/168.69 MB 10:41/147.71 MB 11:47/196.52 MB
MemoryUsage(8s000ms): 10.18 MB, 48.19 MB, 42.45 MB, 45.68 MB, 52.53 MB, 49.10 MB, 50.00 MB, 49.51 MB, 44.64 MB, 48.69 MB, 50.47 MB, 53.41 MB, 61.39 MB, 72.58 MB, 46.42 MB, 45.69 MB, 50.24 MB, 72.00 MB, 50.13 MB, 32.04 MB, 39.67 MB, 29.00 MB, 44.91 MB, 30.43 MB, 36.38 MB, 35.96 MB, 29.23 MB, 27.26 MB, 32.90 MB, 36.39 MB, 42.12 MB, 58.51 MB, 43.21 MB, 44.19 MB, 37.40 MB, 31.52 MB, 36.78 MB, 40.57 MB, 52.38 MB, 52.75 MB, 42.26 MB, 59.62 MB, 37.99 MB, 46.65 MB, 41.25 MB, 66.02 MB, 37.52 MB, 38.19 MB, 29.06 MB, 42.46 MB, 44.45 MB, 34.50 MB, 35.28 MB, 37.70 MB, 43.07 MB, 25.61 MB, 28.27 MB, 22.45 MB, 11.14 MB, 16.30 MB, 6.90 MB, 11.51 MB
ThreadUsage(8s000ms): 8, 14, 12, 11, 13, 13, 13, 12, 12, 13, 14, 16, 17, 14, 12, 12, 11, 14, 11, 9, 9, 8, 8, 8, 8, 9, 9, 7, 7, 8, 10, 10, 9, 9, 9, 10, 11, 11, 10, 11, 12, 12, 11, 11, 12, 13, 10, 11, 11, 10, 11, 11, 12, 12, 12, 12, 11, 8, 7, 5, 3, 2
- AverageThreadTokens: 11.06
- PeakMemoryUsage: 88.58 MB (92882784)
- PerHostPeakMemUsage: 88.58 MB (92882784)
- PrepareTime: 200.819ms
- RowsProduced: 24 (24)
- TotalCpuTime: 1h23m
- TotalNetworkReceiveTime: 5s403ms
- TotalNetworkSendTime: 2.196us
- TotalStorageWaitTime: 1h22m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 703.634ms, non-child: 703.634ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 142.617ms
- LoadTime: 150.566us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 373.89ms
- PrepareTime: 185.976ms
DataStreamSender (dst_id=13):(Total: 412.381us, non-child: 412.381us, % non-child: 100.00%)
- BytesSent: 6.97 KB (7140)
- NetworkThroughput(*): 2.13 MB/sec
- OverallThroughput: 16.51 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 312.617us
- ThriftTransmitTime(*): 3.190ms
- UncompressedRowBatchSize: 10.74 KB (10994)
HASH_JOIN_NODE (id=6):(Total: 8m16s, non-child: 201.204ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 11.643us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.621us
- GetNewBlockTime: 65.857us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 26.16 MB (27433168)
- PinTime: 0ns
- ProbeRows: 24 (24)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 176.244us
- RowsReturned: 24 (24)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 899ns
EXCHANGE_NODE (id=12):(Total: 2s569ms, non-child: 26.773us, % non-child: 0.00%)
BytesReceived(8s000ms): 75.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 1.963us
- DeserializeRowBatchTimer: 20.413us
- FirstBatchArrivalWaitTime: 2s567ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 8m16s, non-child: 2s850ms, % non-child: 0.57%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.781us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.520us
- GetNewBlockTime: 55.493us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.70 MB (16458400)
- PinTime: 0ns
- ProbeRows: 24 (24)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 208.240us
- RowsReturned: 24 (24)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 939ns
EXCHANGE_NODE (id=9):(Total: 2s833ms, non-child: 26.75us, % non-child: 0.00%)
BytesReceived(8s000ms): 64.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.3us
- DeserializeRowBatchTimer: 53.851us
- FirstBatchArrivalWaitTime: 2s819ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 8m13s, non-child: 8m13s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:41/136.75 MB 1:42/116.16 MB 2:35/99.68 MB 3:33/98.32 MB 4:52/184.85 MB 5:35/101.62 MB 6:44/127.58 MB 7:44/147.97 MB 8:46/179.72 MB 9:47/168.69 MB 10:41/147.71 MB 11:47/196.52 MB
Hdfs Read Thread Concurrency Bucket: 0:98.09% 1:1.913% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:23829
ExecOption: Codegen enabled: 0 out of 507
BytesRead(8s000ms): 3.25 MB, 16.27 MB, 30.61 MB, 44.95 MB, 58.37 MB, 68.38 MB, 76.95 MB, 87.33 MB, 93.24 MB, 102.58 MB, 112.46 MB, 123.78 MB, 140.59 MB, 153.29 MB, 162.89 MB, 171.08 MB, 185.70 MB, 203.92 MB, 212.63 MB, 217.40 MB, 227.22 MB, 232.13 MB, 247.32 MB, 262.16 MB, 278.00 MB, 295.75 MB, 304.40 MB, 313.36 MB, 326.66 MB, 342.81 MB, 364.16 MB, 402.60 MB, 433.68 MB, 466.84 MB, 496.81 MB, 512.23 MB, 520.22 MB, 529.07 MB, 544.15 MB, 561.82 MB, 580.43 MB, 605.44 MB, 620.48 MB, 638.53 MB, 653.51 MB, 671.51 MB, 685.48 MB, 702.42 MB, 712.41 MB, 723.51 MB, 737.26 MB, 744.09 MB, 752.65 MB, 764.60 MB, 785.47 MB, 799.06 MB, 811.61 MB, 820.73 MB, 823.25 MB, 826.39 MB, 828.54 MB, 830.64 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 10.07
- BytesRead: 832.05 MB (872463264)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 832.05 MB (872463264)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 832.05 MB (872463264)
- DecompressionTime: 1s085ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 145 (145)
- PeakMemoryUsage: 81.15 MB (85087424)
- PerReadThreadRawHdfsThroughput: 64.25 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.75M (11745970)
- RowsReturned: 24 (24)
- RowsReturnedRate: 0
- ScanRangesComplete: 507 (507)
- ScannerThreadsInvoluntaryContextSwitches: 498 (498)
- ScannerThreadsTotalWallClockTime: 1h23m
- MaterializeTupleTime(*): 15s877ms
- ScannerThreadsSysTime: 596.7ms
- ScannerThreadsUserTime: 18s409ms
- ScannerThreadsVoluntaryContextSwitches: 38.91K (38908)
- TotalRawHdfsReadTime(*): 12s950ms
- TotalReadThroughput: 1.67 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016a4 (host=hadoopc1h6:28002):(Total: 8m2s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:38/91.38 MB 1:52/125.66 MB 2:57/149.56 MB 3:59/162.70 MB 4:58/152.44 MB 5:43/117.47 MB 6:43/126.42 MB 7:47/136.07 MB 8:50/169.69 MB 9:61/155.92 MB 10:43/130.67 MB 11:52/147.82 MB
MemoryUsage(8s000ms): 10.50 MB, 42.16 MB, 37.32 MB, 39.10 MB, 39.08 MB, 45.79 MB, 47.58 MB, 65.10 MB, 41.95 MB, 39.58 MB, 31.66 MB, 56.65 MB, 48.64 MB, 40.51 MB, 45.23 MB, 41.86 MB, 37.49 MB, 40.18 MB, 46.22 MB, 41.13 MB, 48.36 MB, 31.63 MB, 39.27 MB, 28.50 MB, 31.12 MB, 29.06 MB, 47.29 MB, 29.68 MB, 48.19 MB, 45.84 MB, 37.29 MB, 35.99 MB, 44.05 MB, 37.92 MB, 36.32 MB, 43.16 MB, 50.84 MB, 57.37 MB, 42.67 MB, 46.22 MB, 39.78 MB, 43.80 MB, 48.48 MB, 41.17 MB, 41.88 MB, 39.25 MB, 24.27 MB, 26.04 MB, 32.40 MB, 35.85 MB, 30.63 MB, 25.79 MB, 28.20 MB, 30.61 MB, 18.86 MB, 18.70 MB, 24.85 MB, 17.39 MB, 23.25 MB, 18.95 MB
ThreadUsage(8s000ms): 7, 11, 10, 11, 10, 11, 12, 14, 14, 13, 12, 13, 13, 11, 11, 11, 11, 12, 13, 10, 10, 10, 10, 8, 9, 11, 12, 10, 10, 11, 10, 9, 8, 9, 10, 12, 13, 14, 12, 14, 14, 13, 15, 14, 15, 13, 13, 12, 10, 10, 10, 11, 12, 12, 11, 10, 10, 9, 10, 5
- AverageThreadTokens: 11.52
- PeakMemoryUsage: 78.39 MB (82199736)
- PerHostPeakMemUsage: 78.39 MB (82199736)
- PrepareTime: 216.32ms
- RowsProduced: 17 (17)
- TotalCpuTime: 1h24m
- TotalNetworkReceiveTime: 5s234ms
- TotalNetworkSendTime: 2.188us
- TotalStorageWaitTime: 1h24m
BlockMgr:
- BlockWritesOutstanding: 0 (0)
- BlocksCreated: 36 (36)
- BlocksRecycled: 32 (32)
- BufferedPins: 0 (0)
- BytesWritten: 0
- MaxBlockSize: 8.00 MB (8388608)
- MemoryLimit: 163.83 GB (175909109760)
- PeakMemoryUsage: 64.00 B (64)
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
CodeGen:(Total: 778.760ms, non-child: 778.760ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 151.590ms
- LoadTime: 161.144us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 425.534ms
- PrepareTime: 199.658ms
DataStreamSender (dst_id=13):(Total: 344.204us, non-child: 344.204us, % non-child: 100.00%)
- BytesSent: 5.54 KB (5671)
- NetworkThroughput(*): 1.27 MB/sec
- OverallThroughput: 15.71 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 239.201us
- ThriftTransmitTime(*): 4.248ms
- UncompressedRowBatchSize: 7.65 KB (7835)
HASH_JOIN_NODE (id=6):(Total: 8m2s, non-child: 215.772ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.269us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.895us
- GetNewBlockTime: 70.302us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 21.99 MB (23063104)
- PinTime: 0ns
- ProbeRows: 17 (17)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 212.739us
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 854ns
EXCHANGE_NODE (id=12):(Total: 2s495ms, non-child: 30.121us, % non-child: 0.00%)
BytesReceived(8s000ms): 75.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.722us
- DeserializeRowBatchTimer: 22.128us
- FirstBatchArrivalWaitTime: 2s481ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 8m1s, non-child: 2s756ms, % non-child: 0.57%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 7.424us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.719us
- GetNewBlockTime: 58.358us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 13.70 MB (14361592)
- PinTime: 0ns
- ProbeRows: 17 (17)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 218.440us
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 942ns
EXCHANGE_NODE (id=9):(Total: 2s738ms, non-child: 25.914us, % non-child: 0.00%)
BytesReceived(8s000ms): 64.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 1.766us
- DeserializeRowBatchTimer: 18.336us
- FirstBatchArrivalWaitTime: 2s733ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 7m59s, non-child: 7m59s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:38/91.38 MB 1:52/125.66 MB 2:57/149.56 MB 3:59/162.70 MB 4:58/152.44 MB 5:43/117.47 MB 6:43/126.42 MB 7:47/136.07 MB 8:50/169.69 MB 9:61/155.92 MB 10:43/130.67 MB 11:52/147.82 MB
Hdfs Read Thread Concurrency Bucket: 0:98.03% 1:1.973% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:28341
ExecOption: Codegen enabled: 0 out of 603
BytesRead(8s000ms): 2.92 MB, 14.91 MB, 28.45 MB, 42.41 MB, 54.59 MB, 66.30 MB, 78.79 MB, 92.42 MB, 99.09 MB, 105.66 MB, 114.69 MB, 131.48 MB, 152.96 MB, 165.55 MB, 178.45 MB, 193.12 MB, 208.19 MB, 224.13 MB, 236.42 MB, 246.14 MB, 260.38 MB, 270.07 MB, 285.63 MB, 303.74 MB, 320.67 MB, 337.11 MB, 354.29 MB, 365.51 MB, 387.27 MB, 408.48 MB, 432.62 MB, 462.07 MB, 492.68 MB, 518.34 MB, 540.34 MB, 559.57 MB, 573.86 MB, 582.58 MB, 591.82 MB, 606.15 MB, 622.80 MB, 640.66 MB, 660.97 MB, 678.67 MB, 692.38 MB, 703.94 MB, 715.02 MB, 725.27 MB, 735.41 MB, 746.00 MB, 752.99 MB, 758.63 MB, 769.95 MB, 776.72 MB, 782.09 MB, 787.16 MB, 795.96 MB, 802.46 MB, 808.54 MB, 818.03 MB
- AverageHdfsReadThreadConcurrency: 0.02
- AverageScannerThreadConcurrency: 10.53
- BytesRead: 821.97 MB (861900481)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 821.97 MB (861900481)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 821.97 MB (861900481)
- DecompressionTime: 1s420ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 113 (113)
- PeakMemoryUsage: 74.28 MB (77883936)
- PerReadThreadRawHdfsThroughput: 78.21 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.39M (11389721)
- RowsReturned: 17 (17)
- RowsReturnedRate: 0
- ScanRangesComplete: 603 (603)
- ScannerThreadsInvoluntaryContextSwitches: 627 (627)
- ScannerThreadsTotalWallClockTime: 1h24m
- MaterializeTupleTime(*): 16s810ms
- ScannerThreadsSysTime: 612.13ms
- ScannerThreadsUserTime: 19s901ms
- ScannerThreadsVoluntaryContextSwitches: 50.17K (50174)
- TotalRawHdfsReadTime(*): 10s510ms
- TotalReadThroughput: 1.70 MB/sec
Instance c14f7b9b17987c60:d5379fb9541016c9 (host=hadoopc1h70:28002):(Total: 7m37s, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:49/167.75 MB 1:42/115.01 MB 2:50/164.27 MB 3:47/142.94 MB 4:45/164.82 MB 5:40/140.71 MB 6:59/181.77 MB 7:55/158.98 MB 8:45/175.41 MB 9:45/147.99 MB 10:43/132.05 MB 11:5/16.97 MB
MemoryUsage(8s000ms): 16.28 MB, 38.60 MB, 33.44 MB, 32.73 MB, 51.98 MB, 46.15 MB, 30.39 MB, 26.62 MB, 51.05 MB, 39.64 MB, 45.37 MB, 46.47 MB, 46.05 MB, 37.64 MB, 39.27 MB, 23.90 MB, 35.57 MB, 39.00 MB, 38.53 MB, 29.59 MB, 30.60 MB, 31.36 MB, 28.34 MB, 22.62 MB, 26.54 MB, 29.22 MB, 23.05 MB, 37.72 MB, 29.10 MB, 29.85 MB, 44.09 MB, 35.94 MB, 41.90 MB, 28.12 MB, 47.71 MB, 35.85 MB, 32.10 MB, 39.75 MB, 48.13 MB, 46.45 MB, 50.54 MB, 38.02 MB, 32.49 MB, 27.25 MB, 29.58 MB, 35.11 MB, 19.43 MB, 18.02 MB, 12.88 MB, 18.56 MB, 12.84 MB, 11.10 MB, 12.82 MB, 15.33 MB, 14.57 MB, 5.03 MB, 7.65 MB
ThreadUsage(8s000ms): 8, 12, 10, 10, 11, 11, 11, 12, 14, 13, 12, 14, 14, 12, 12, 10, 10, 12, 10, 9, 8, 9, 8, 8, 8, 8, 8, 7, 8, 8, 10, 8, 7, 8, 8, 9, 10, 10, 10, 10, 10, 10, 9, 10, 11, 12, 9, 8, 10, 10, 9, 9, 8, 7, 6, 5, 3
- AverageThreadTokens: 9.96
- PeakMemoryUsage: 72.35 MB (75861768)
- PerHostPeakMemUsage: 72.44 MB (75960072)
- PrepareTime: 195.638ms
- RowsProduced: 12 (12)
- TotalCpuTime: 1h8m
- TotalNetworkReceiveTime: 5s432ms
- TotalNetworkSendTime: 952ns
- TotalStorageWaitTime: 1h7m
CodeGen:(Total: 687.280ms, non-child: 687.280ms, % non-child: 100.00%)
- CodegenTime: 0ns
- CompileTime: 139.269ms
- LoadTime: 111.571us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 365.83ms
- PrepareTime: 181.83ms
DataStreamSender (dst_id=13):(Total: 228.577us, non-child: 228.577us, % non-child: 100.00%)
- BytesSent: 3.49 KB (3577)
- NetworkThroughput(*): 2.03 MB/sec
- OverallThroughput: 14.92 MB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 166.19us
- ThriftTransmitTime(*): 1.682ms
- UncompressedRowBatchSize: 5.35 KB (5483)
HASH_JOIN_NODE (id=6):(Total: 7m37s, non-child: 195.310ms, % non-child: 0.04%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 8.420us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.15us
- GetNewBlockTime: 67.451us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 22.65 MB (23750344)
- PinTime: 0ns
- ProbeRows: 12 (12)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 167.154us
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 844ns
EXCHANGE_NODE (id=12):(Total: 2s592ms, non-child: 28.468us, % non-child: 0.00%)
BytesReceived(8s000ms): 75.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B, 120.00 B
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.723us
- DeserializeRowBatchTimer: 18.231us
- FirstBatchArrivalWaitTime: 2s579ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HASH_JOIN_NODE (id=5):(Total: 7m37s, non-child: 2s865ms, % non-child: 0.63%)
ExecOption: Build Side Codegen Enabled, Probe Side Codegen Enabled, Join Build-Side Prepared Asynchronously
- BuildPartitionTime: 14.370us
- BuildRows: 2 (2)
- BuildRowsPartitioned: 2 (2)
- BuildTime: 4.524us
- GetNewBlockTime: 55.105us
- HashBuckets: 4 (4)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 15.90 MB (16671968)
- PinTime: 0ns
- ProbeRows: 12 (12)
- ProbeRowsPartitioned: 0 (0)
- ProbeTime: 146.254us
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 827ns
EXCHANGE_NODE (id=9):(Total: 2s840ms, non-child: 34.618us, % non-child: 0.00%)
BytesReceived(8s000ms): 64.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B, 115.00 B
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.559us
- DeserializeRowBatchTimer: 22.128us
- FirstBatchArrivalWaitTime: 2s831ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
HDFS_SCAN_NODE (id=0):(Total: 7m34s, non-child: 7m34s, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:49/167.75 MB 1:42/115.01 MB 2:50/164.27 MB 3:47/142.94 MB 4:45/164.82 MB 5:40/140.71 MB 6:59/181.77 MB 7:55/158.98 MB 8:45/175.41 MB 9:45/147.99 MB 10:43/132.05 MB 11:5/16.97 MB
Hdfs Read Thread Concurrency Bucket: 0:99.34% 1:0.6565% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:24675
ExecOption: Codegen enabled: 0 out of 525
BytesRead(8s000ms): 4.32 MB, 26.32 MB, 46.97 MB, 66.76 MB, 91.56 MB, 111.55 MB, 123.00 MB, 132.30 MB, 145.63 MB, 158.58 MB, 177.06 MB, 198.40 MB, 225.65 MB, 239.84 MB, 251.22 MB, 258.55 MB, 271.57 MB, 292.81 MB, 306.54 MB, 316.71 MB, 329.18 MB, 341.29 MB, 356.43 MB, 373.69 MB, 393.82 MB, 415.06 MB, 427.54 MB, 445.89 MB, 464.40 MB, 483.11 MB, 511.35 MB, 541.18 MB, 576.54 MB, 604.86 MB, 634.12 MB, 652.15 MB, 660.27 MB, 667.88 MB, 680.97 MB, 692.39 MB, 713.96 MB, 734.85 MB, 750.00 MB, 764.11 MB, 778.89 MB, 797.40 MB, 807.50 MB, 815.34 MB, 818.62 MB, 821.57 MB, 823.18 MB, 824.34 MB, 825.20 MB, 826.22 MB, 829.15 MB, 830.08 MB, 831.33 MB
- AverageHdfsReadThreadConcurrency: 0.01
- AverageScannerThreadConcurrency: 8.97
- BytesRead: 833.75 MB (874251583)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 833.75 MB (874251583)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 833.75 MB (874251583)
- DecompressionTime: 892.665ms
- MaxCompressedTextFileLength: 0
- NumColumns: 47 (47)
- NumDisksAccessed: 12 (12)
- NumScannerThreadsStarted: 99 (99)
- PeakMemoryUsage: 68.52 MB (71843448)
- PerReadThreadRawHdfsThroughput: 251.38 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 11.80M (11799078)
- RowsReturned: 12 (12)
- RowsReturnedRate: 0
- ScanRangesComplete: 525 (525)
- ScannerThreadsInvoluntaryContextSwitches: 485 (485)
- ScannerThreadsTotalWallClockTime: 1h8m
- MaterializeTupleTime(*): 15s911ms
- ScannerThreadsSysTime: 588.15ms
- ScannerThreadsUserTime: 18s697ms
- ScannerThreadsVoluntaryContextSwitches: 39.79K (39794)
- TotalRawHdfsReadTime(*): 3s316ms
- TotalReadThroughput: 1.82 MB/sec
Fragment F04:
Instance c14f7b9b17987c60:d5379fb9541016ce (host=hadoopc1h66:28002):(Total: 2s713ms, non-child: 0ns, % non-child: 0.00%)
MemoryUsage(500.0ms): 3.36 MB, 3.36 MB, 3.36 MB, 3.36 MB, 3.36 MB, 3.36 MB
ThreadUsage(500.0ms): 1, 1, 1, 1, 1, 1
- AverageThreadTokens: 1.00
- PeakMemoryUsage: 3.38 MB (3538944)
- PerHostPeakMemUsage: 13.19 MB (13835264)
- PrepareTime: 195.945ms
- RowsProduced: 2 (2)
- TotalCpuTime: 515.965ms
- TotalNetworkReceiveTime: 2s510ms
- TotalNetworkSendTime: 391.855us
- TotalStorageWaitTime: 0ns
CodeGen:(Total: 490.990ms, non-child: 490.990ms, % non-child: 100.00%)
- CodegenTime: 2.886ms
- CompileTime: 45.696ms
- LoadTime: 133.49us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 252.637ms
- PrepareTime: 190.687ms
DataStreamSender (dst_id=12):(Total: 877.949us, non-child: 877.949us, % non-child: 100.00%)
- BytesSent: 6.68 KB (6840)
- NetworkThroughput(*): 247.41 KB/sec
- OverallThroughput: 7.43 MB/sec
- PeakMemoryUsage: 228.00 KB (233472)
- SerializeBatchTime: 14.832us
- ThriftTransmitTime(*): 26.998ms
- UncompressedRowBatchSize: 8.68 KB (8892)
AGGREGATION_NODE (id=11):(Total: 2s706ms, non-child: 196.363ms, % non-child: 7.26%)
ExecOption: Codegen Enabled
- BuildTime: 9.582us
- GetNewBlockTime: 129.749us
- GetResultsTime: 4.355us
- HashBuckets: 2.05K (2048)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 3.14 MB (3293184)
- PinTime: 0ns
- RowsRepartitioned: 0 (0)
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 784ns
EXCHANGE_NODE (id=10):(Total: 2s510ms, non-child: 19.428us, % non-child: 0.00%)
BytesReceived(500.0ms): 0, 0, 0, 0, 0, 0
- BytesReceived: 120.00 B (120)
- ConvertRowBatchTime: 2.22us
- DeserializeRowBatchTimer: 14.66us
- FirstBatchArrivalWaitTime: 2s509ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
Fragment F03:
Instance c14f7b9b17987c60:d5379fb9541016cf (host=hadoopc1h66:28002):(Total: 2s431ms, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 5:1/703.00 B
MemoryUsage(500.0ms): 0, 3.14 MB, 3.20 MB, 3.20 MB, 3.21 MB, 3.21 MB
ThreadUsage(500.0ms): 1, 1, 2, 2, 2, 2
- AverageThreadTokens: 1.67
- PeakMemoryUsage: 3.22 MB (3380224)
- PerHostPeakMemUsage: 13.19 MB (13835264)
- PrepareTime: 193.971ms
- RowsProduced: 2 (2)
- TotalCpuTime: 3s049ms
- TotalNetworkReceiveTime: 0ns
- TotalNetworkSendTime: 298.890us
- TotalStorageWaitTime: 1s782ms
CodeGen:(Total: 560.333ms, non-child: 560.333ms, % non-child: 100.00%)
- CodegenTime: 3.495ms
- CompileTime: 63.686ms
- LoadTime: 95.195us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 307.18ms
- PrepareTime: 187.870ms
DataStreamSender (dst_id=10):(Total: 240.99us, non-child: 240.99us, % non-child: 100.00%)
- BytesSent: 120.00 B (120)
- NetworkThroughput(*): 414.71 KB/sec
- OverallThroughput: 488.08 KB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 16.461us
- ThriftTransmitTime(*): 282.576us
- UncompressedRowBatchSize: 156.00 B (156)
AGGREGATION_NODE (id=4):(Total: 2s430ms, non-child: 401.535ms, % non-child: 16.52%)
ExecOption: Codegen Enabled
- BuildTime: 31.444us
- GetNewBlockTime: 48.844us
- GetResultsTime: 1.821us
- HashBuckets: 2.05K (2048)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 3.21 MB (3367936)
- PinTime: 0ns
- RowsRepartitioned: 0 (0)
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 1.185us
HDFS_SCAN_NODE (id=3):(Total: 2s029ms, non-child: 2s029ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 5:1/703.00 B
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:3
ExecOption: Codegen enabled: 0 out of 1
BytesRead(500.0ms): 0, 0, 0, 754.00 B, 817.00 B
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 892.00 B (892)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 892.00 B (892)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 892.00 B (892)
- DecompressionTime: 7.57us
- MaxCompressedTextFileLength: 0
- NumColumns: 3 (3)
- NumDisksAccessed: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 76.00 KB (77824)
- PerReadThreadRawHdfsThroughput: 31.37 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 2 (2)
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 2s028ms
- MaterializeTupleTime(*): 7.593us
- ScannerThreadsSysTime: 0ns
- ScannerThreadsUserTime: 0ns
- ScannerThreadsVoluntaryContextSwitches: 9 (9)
- TotalRawHdfsReadTime(*): 27.120us
- TotalReadThroughput: 326.00 B/sec
Fragment F02:
Instance c14f7b9b17987c60:d5379fb9541016d0 (host=hadoopc1h66:28002):(Total: 2s561ms, non-child: 0ns, % non-child: 0.00%)
MemoryUsage(500.0ms): 3.36 MB, 3.36 MB, 3.36 MB, 3.36 MB, 3.36 MB, 3.36 MB
ThreadUsage(500.0ms): 1, 1, 1, 1, 1, 1
- AverageThreadTokens: 1.00
- PeakMemoryUsage: 3.38 MB (3538944)
- PerHostPeakMemUsage: 13.19 MB (13835264)
- PrepareTime: 199.748ms
- RowsProduced: 2 (2)
- TotalCpuTime: 515.267ms
- TotalNetworkReceiveTime: 2s352ms
- TotalNetworkSendTime: 366.750us
- TotalStorageWaitTime: 0ns
CodeGen:(Total: 488.184ms, non-child: 488.184ms, % non-child: 100.00%)
- CodegenTime: 2.562ms
- CompileTime: 43.837ms
- LoadTime: 114.82us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 247.491ms
- PrepareTime: 195.74ms
DataStreamSender (dst_id=9):(Total: 1.139ms, non-child: 1.139ms, % non-child: 100.00%)
- BytesSent: 6.40 KB (6555)
- NetworkThroughput(*): 239.69 KB/sec
- OverallThroughput: 5.49 MB/sec
- PeakMemoryUsage: 228.00 KB (233472)
- SerializeBatchTime: 15.35us
- ThriftTransmitTime(*): 26.706ms
- UncompressedRowBatchSize: 9.69 KB (9918)
AGGREGATION_NODE (id=8):(Total: 2s552ms, non-child: 200.119ms, % non-child: 7.84%)
ExecOption: Codegen Enabled
- BuildTime: 7.646us
- GetNewBlockTime: 127.195us
- GetResultsTime: 4.457us
- HashBuckets: 2.05K (2048)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 3.14 MB (3293184)
- PinTime: 0ns
- RowsRepartitioned: 0 (0)
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 603ns
EXCHANGE_NODE (id=7):(Total: 2s352ms, non-child: 24.890us, % non-child: 0.00%)
BytesReceived(500.0ms): 0, 0, 0, 0, 0, 0
- BytesReceived: 115.00 B (115)
- ConvertRowBatchTime: 2.52us
- DeserializeRowBatchTimer: 13.114us
- FirstBatchArrivalWaitTime: 2s351ms
- PeakMemoryUsage: 0
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SendersBlockedTimer: 0ns
- SendersBlockedTotalTimer(*): 0ns
Fragment F01:
Instance c14f7b9b17987c60:d5379fb9541016d1 (host=hadoopc1h66:28002):(Total: 2s278ms, non-child: 0ns, % non-child: 0.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 5:1/703.00 B
MemoryUsage(500.0ms): 0, 3.14 MB, 3.14 MB, 3.20 MB, 3.21 MB, 3.15 MB
ThreadUsage(500.0ms): 1, 2, 2, 2, 2, 1
- AverageThreadTokens: 1.67
- PeakMemoryUsage: 3.22 MB (3380224)
- PerHostPeakMemUsage: 13.19 MB (13835264)
- PrepareTime: 216.361ms
- RowsProduced: 2 (2)
- TotalCpuTime: 3s238ms
- TotalNetworkReceiveTime: 0ns
- TotalNetworkSendTime: 263.511us
- TotalStorageWaitTime: 1s335ms
CodeGen:(Total: 571.640ms, non-child: 571.640ms, % non-child: 100.00%)
- CodegenTime: 3.236ms
- CompileTime: 60.668ms
- LoadTime: 102.841us
- ModuleFileSize: 2.30 MB (2411072)
- OptimizationTime: 298.651ms
- PrepareTime: 210.222ms
DataStreamSender (dst_id=7):(Total: 167.531us, non-child: 167.531us, % non-child: 100.00%)
- BytesSent: 115.00 B (115)
- NetworkThroughput(*): 438.26 KB/sec
- OverallThroughput: 670.35 KB/sec
- PeakMemoryUsage: 4.00 KB (4096)
- SerializeBatchTime: 9.922us
- ThriftTransmitTime(*): 256.252us
- UncompressedRowBatchSize: 174.00 B (174)
AGGREGATION_NODE (id=2):(Total: 2s277ms, non-child: 341.394ms, % non-child: 14.99%)
ExecOption: Codegen Enabled
- BuildTime: 93.963us
- GetNewBlockTime: 38.841us
- GetResultsTime: 3.265us
- HashBuckets: 2.05K (2048)
- LargestPartitionPercent: 50 (50)
- MaxPartitionLevel: 0 (0)
- NumRepartitions: 0 (0)
- PartitionsCreated: 16 (16)
- PeakMemoryUsage: 3.21 MB (3367936)
- PinTime: 0ns
- RowsRepartitioned: 0 (0)
- RowsReturned: 2 (2)
- RowsReturnedRate: 0
- SpilledPartitions: 0 (0)
- UnpinTime: 756ns
HDFS_SCAN_NODE (id=1):(Total: 1s935ms, non-child: 1s935ms, % non-child: 100.00%)
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 5:1/703.00 B
Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0%
File Formats: PARQUET/SNAPPY:3
ExecOption: Codegen enabled: 0 out of 1
BytesRead(500.0ms): 0, 0, 0, 754.00 B
- AverageHdfsReadThreadConcurrency: 0.00
- AverageScannerThreadConcurrency: 1.00
- BytesRead: 892.00 B (892)
- BytesReadDataNodeCache: 0
- BytesReadLocal: 892.00 B (892)
- BytesReadRemoteUnexpected: 0
- BytesReadShortCircuit: 892.00 B (892)
- DecompressionTime: 8.123us
- MaxCompressedTextFileLength: 0
- NumColumns: 3 (3)
- NumDisksAccessed: 1 (1)
- NumScannerThreadsStarted: 1 (1)
- PeakMemoryUsage: 76.00 KB (77824)
- PerReadThreadRawHdfsThroughput: 30.61 MB/sec
- RemoteScanRanges: 0 (0)
- RowsRead: 2 (2)
- RowsReturned: 2 (2)
- RowsReturnedRate: 1.00 /sec
- ScanRangesComplete: 1 (1)
- ScannerThreadsInvoluntaryContextSwitches: 0 (0)
- ScannerThreadsTotalWallClockTime: 1s935ms
- MaterializeTupleTime(*): 8.595us
- ScannerThreadsSysTime: 0ns
- ScannerThreadsUserTime: 0ns
- ScannerThreadsVoluntaryContextSwitches: 7 (7)
- TotalRawHdfsReadTime(*): 27.789us
- TotalReadThroughput: 377.00 B/sec
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)