You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "lipf19860104 (via GitHub)" <gi...@apache.org> on 2023/06/28 06:44:52 UTC
[GitHub] [doris] lipf19860104 opened a new issue, #21278: [Bug] be crash when use a condition query on 100G table
lipf19860104 opened a new issue, #21278:
URL: https://github.com/apache/doris/issues/21278
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
doris-1.2.0-rc04-Unknown
### What's Wrong?
<img width="948" alt="微信图片_20230628100925" src="https://github.com/apache/doris/assets/13567711/485627c4-5de9-446b-a2ac-3b16c51368dd">
I found nothing about OOM message when execute dmesg -T in be node.
### What You Expected?
query ok
### How to Reproduce?
I hava a 104G table wiht 144978726 row count.
table schema:
```
CREATE TABLE `file_info_his_init_update2` (
`id` int(11) NULL,
`fileId` varchar(300) NULL ,
`fileName` varchar(500) NULL ,
`fileExt` varchar(50) NULL ,
`fileStatus` varchar(10) NULL ,
`fileSize` varchar(100) NULL ,
`storageType` varchar(10) NULL ,
`storageId` varchar(100) NULL,
`storagePath` varchar(500) NULL ,
`storageSubPath` varchar(500) NULL ,
`bizNo` varchar(100) NULL ,
`bizCode` varchar(20) NULL ,
`sysCode` varchar(50) NULL ,
`md5Sign` varchar(100) NULL ,
`detailJson` varchar(500) NULL ,
`remark1` varchar(1000) NULL,
`remark2` varchar(1000) NULL ,
`createDatetime` varchar(1000) NULL ,
`updateDatetime` varchar(1000) NULL ,
`reserveStorageId` varchar(100) NULL ,
`reserveStorageType` char(10) NULL ,
`reserveStoragePath` varchar(500) NULL ,
`reserveStorageSubPath` varchar(500) NULL ,
`pathStatus` varchar(10) NULL ,
`opType` varchar(20) NULL,
`fileNameId` varchar(500) NULL )
ENGINE=OLAP UNIQUE KEY(`id`)
PARTITION BY RANGE(`id`) (
PARTITION p1 VALUES [("0"), ("10000000"))
...
PARTITION p20 VALUES [("190000000"), ("200000000"))
) DISTRIBUTED BY HASH(`id`)
BUCKETS 10
PROPERTIES (
"replication_allocation" = "tag.location.default: 3",
"in_memory" = "false", "storage_format" = "V2",
"disable_auto_compaction" = "false" );
```
all be crash when I execute query like follow:
```
select * from file_info_his_init_update2 where fileId = 'c7e556560ff2eb33672e22e6be0a5df7' limit 100
```
the query profile:
```
Query:
Summary:
- Start Time: 2023-06-27 16:18:48
- End Time: N/A
- Total: 4m43s
- Query State: RUNNING
- Query ID: ce91e8a1d43f404c-a1e26c1c56e6c7c2
- Query Type: Query
- Doris Version: doris-1.2.0-rc04
- User: admin
- Default Db: default_cluster:d_oss
- Sql Statement: select * from file_info_his_init_update2 where fileId = 'c7e556560ff2eb33672e22e6be0a5df7' limit 100
- Is Cached: No
- Total Instances Num: 8
- Instances Num Per BE: {11.6.2.50:8060=1, 11.6.2.51:8060=1, 11.6.2.52:8060=1, 11.6.2.53:8060=1, 11.6.2.54:8060=1, 11.6.2.55:8060=2, 11.6.2.56:8060=1}
- Trace ID:
Execution Summary:
- Analysis Time: 172.656us
- Plan Time: 828.49us
- Schedule Time: 49.56ms
- Wait and Fetch Result Time: N/A
Execution Profile ce91e8a1d43f404c-a1e26c1c56e6c7c2:(Active: 4m43s, % non-child: 100.00%)
Fragment 0:
Instance ce91e8a1d43f404c-a1e26c1c56e6c7ca (host=TNetworkAddress(hostname:11.6.2.55, port:9060)):(Active: 57.465ms, % non-child: 0.00%)
- FragmentCpuTime: 350.801us
- RowsProduced: 1
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
VDataBufferSender (dst_fragment_instance_id=-316e175e2bc0bfb4--5e1d93e3a9193836):
- AppendBatchTime: 268.111us
- ResultRendTime: 161.755us
- TupleConvertTime: 22.540us
- NumSentRows: 0
VEXCHANGE_NODE (id=1):(Active: 57.160ms, % non-child: 0.02%)
- BytesReceived: 1.00 KB
- DataArrivalWaitTime: 57.140ms
- DecompressBytes: 965.00 B
- DecompressTime: 2.511us
- DeserializeRowBatchTimer: 65.779us
- FirstBatchArrivalWaitTime: 57.140ms
- PeakMemoryUsage: 24.50 KB
- ProjectionTime: 0ns
- RowsReturned: 1
- RowsReturnedRate: 17
- SendersBlockedTotalTimer(*): 0ns
Fragment 1:
Instance ce91e8a1d43f404c-a1e26c1c56e6c7c5 (host=TNetworkAddress(hostname:11.6.2.50, port:9060)):(Active: 1s597ms, % non-child: 0.00%)
- FragmentCpuTime: 1.862ms
- RowsProduced: 0
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
VDataStreamSender (dst_id=1, dst_fragments=[{"ce91e8a1d43f404c-a1e26c1c56e6c7ca"}]):(Active: 24.621us, % non-child: 0.00%)
- BrpcSendTime: 147.839us
- BrpcSendTime.Wait: 928.574us
- BytesSent: 0.00
- CompressTime: 0ns
- IgnoreRows: 0
- LocalBytesSent: 0.00
- LocalSentRows: 0
- OverallThroughput: 0.0 /sec
- PeakMemoryUsage: 0.00
- SerializeBatchTime: 0ns
- UncompressedRowBatchSize: 0.00
VNewOlapScanNode(file_info_his_init_update2) (id=0):(Active: 1s596ms, % non-child: 0.56%)
- PushDownPredicates: [{fileId IN [c7e556560ff2eb33672e22e6be0a5df7]}, {__DORIS_DELETE_SIGN__ IN [0]}]
- KeyRanges: ScanKeys:ScanKey=[null(-2147483648) : 2147483647]
- TabletIds: [1290015, 1290019, 1289983, 1290215, 1290219, 1290243, 1290139, 1290163, 1290167, 1290467, 1290483, 1290487, 1290291, 1290315, 1290267, 1290643, 1290647, 1290507, 1290543, 1290551, 1290559]
- RemainedDownPredicates: CompoundPredicate (and){
VectorizedFn[eq({"fileId", "String"})]{
SlotRef(slot_id=1 type=VARCHAR codegen=false),
VLiteral (name = String, type = String, value = (c7e556560ff2eb33672e22e6be0a5df7))},
VectorizedFn[eq({"__DORIS_DELETE_SIGN__", "Int8"})]{
SlotRef(slot_id=26 type=TINYINT codegen=false),
VLiteral (name = Int8, type = Int8, value = (00))}}
- AcuireRuntimeFilterTime: 5.513us
- GetNextTime: 1s594ms
- MaxScannerThreadNum: 12
- NewlyCreateFreeBlocksNum: 0
- NumDiskAccess: 1
- NumScanners: 21
- PeakMemoryUsage: 17.90 KB
- PreAllocFreeBlocksNum: 12
- ProcessConjunctTime: 120.415us
- ProjectionTime: 0ns
- RowsRead: 3.411206M (3411206)
- RowsReturned: 0
- RowsReturnedRate: 0
- ScannerBatchWaitTime: 1s594ms
- ScannerCtxSchedCount: 22
- ScannerSchedCount: 21
- ScannerWorkerWaitTime: 8.192ms
- TabletNum: 21
- TotalReadThroughput: 0
VScanner:
- ColumnPredicates:
ComparisonPredicateBase(STRING, EQ), column_id=1, opposite=false
ComparisonPredicateBase(TINYINT, EQ), column_id=26, opposite=false
- BlockConvertTime: 0ns
- BlockFetchTime: 8s350ms
- ReaderInitTime: 0ns
- RowsDelFiltered: 0
- ScannerConvertBlockTime: 0ns
- ScannerFilterTime: 545.809ms
- ScannerGetBlockTime: 7s862ms
- ScannerInitTime: 1.23ms
- ScannerPrefilterTime: 0ns
SegmentIterator:
- BitmapIndexFilterTimer: 52.368us
- BlockConditionsFilteredTime: 3.254ms
- BlockInitSeekCount: 775
- BlockInitSeekTime: 318.369ms
- BlockInitTime: 2s709ms
- BlockLoadTime: 8s344ms
- BlocksLoad: 83.315K (83315)
- CachedPagesNum: 1.254K (1254)
- CompressedBytesRead: 1.07 GB
- DecompressorTimer: 805.397ms
- FirstReadSeekCount: 373
- FirstReadSeekTime: 1s97ms
- FirstReadTime: 4s601ms
- IOTimer: 4s540ms
- LazyReadSeekCount: 0
- LazyReadSeekTime: 0ns
- LazyReadTime: 34.543ms
- NumSegmentFiltered: 0
- NumSegmentTotal: 99
- OutputColumnTime: 25.678ms
- RawRowsRead: 11.636307M (11636307)
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 8.362536M (8362536)
- RowsKeyRangeFiltered: 0
- RowsStatsFiltered: 8.362536M (8362536)
- RowsVectorPredFiltered: 8.225101M (8225101)
- ShortPredEvalTime: 452.835ms
- TotalPagesNum: 24.928K (24928)
- UncompressedBytesRead: 1.27 GB
- VectorPredEvalTime: 10.720ms
Instance ce91e8a1d43f404c-a1e26c1c56e6c7c3 (host=TNetworkAddress(hostname:11.6.2.55, port:9060)):(Active: 1s533ms, % non-child: 0.00%)
- FragmentCpuTime: 2.138ms
- RowsProduced: 0
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
VDataStreamSender (dst_id=1, dst_fragments=[{"ce91e8a1d43f404c-a1e26c1c56e6c7ca"}]):(Active: 22.915us, % non-child: 0.00%)
- BrpcSendTime: 102.616us
- BrpcSendTime.Wait: 203.341us
- BytesSent: 0.00
- CompressTime: 0ns
- IgnoreRows: 0
- LocalBytesSent: 0.00
- LocalSentRows: 0
- OverallThroughput: 0.0 /sec
- PeakMemoryUsage: 0.00
- SerializeBatchTime: 0ns
- UncompressedRowBatchSize: 0.00
VNewOlapScanNode(file_info_his_init_update2) (id=0):(Active: 1s532ms, % non-child: 0.54%)
- PushDownPredicates: [{fileId IN [c7e556560ff2eb33672e22e6be0a5df7]}, {__DORIS_DELETE_SIGN__ IN [0]}]
- KeyRanges: ScanKeys:ScanKey=[null(-2147483648) : 2147483647]
- TabletIds: [1290067, 1290075, 1290043, 1290047, 1290003, 1290235, 1290175, 1290207, 1290151, 1290119, 1290463, 1290475, 1290339, 1290299, 1290255, 1290271, 1290619, 1290495, 1290531, 1290539, 1290431, 1290439]
- RemainedDownPredicates: CompoundPredicate (and){
VectorizedFn[eq({"fileId", "String"})]{
SlotRef(slot_id=1 type=VARCHAR codegen=false),
VLiteral (name = String, type = String, value = (c7e556560ff2eb33672e22e6be0a5df7))},
VectorizedFn[eq({"__DORIS_DELETE_SIGN__", "Int8"})]{
SlotRef(slot_id=26 type=TINYINT codegen=false),
VLiteral (name = Int8, type = Int8, value = (00))}}
- AcuireRuntimeFilterTime: 5.64us
- GetNextTime: 1s530ms
- MaxScannerThreadNum: 12
- NewlyCreateFreeBlocksNum: 0
- NumDiskAccess: 1
- NumScanners: 22
- PeakMemoryUsage: 17.90 KB
- PreAllocFreeBlocksNum: 12
- ProcessConjunctTime: 111.683us
- ProjectionTime: 0ns
- RowsRead: 2.954296M (2954296)
- RowsReturned: 0
- RowsReturnedRate: 0
- ScannerBatchWaitTime: 1s530ms
- ScannerCtxSchedCount: 22
- ScannerSchedCount: 22
- ScannerWorkerWaitTime: 8.116ms
- TabletNum: 22
- TotalReadThroughput: 0
VScanner:
- ColumnPredicates:
ComparisonPredicateBase(STRING, EQ), column_id=1, opposite=false
ComparisonPredicateBase(TINYINT, EQ), column_id=26, opposite=false
- BlockConvertTime: 0ns
- BlockFetchTime: 7s999ms
- ReaderInitTime: 0ns
- RowsDelFiltered: 0
- ScannerConvertBlockTime: 0ns
- ScannerFilterTime: 481.845ms
- ScannerGetBlockTime: 8s108ms
- ScannerInitTime: 1.12ms
- ScannerPrefilterTime: 0ns
SegmentIterator:
- BitmapIndexFilterTimer: 58.297us
- BlockConditionsFilteredTime: 5.631ms
- BlockInitSeekCount: 619
- BlockInitSeekTime: 95.789ms
- BlockInitTime: 2s337ms
- BlockLoadTime: 7s990ms
- BlocksLoad: 92.063K (92063)
- CachedPagesNum: 802
- CompressedBytesRead: 1.03 GB
- DecompressorTimer: 727.144ms
- FirstReadSeekCount: 368
- FirstReadSeekTime: 1s100ms
- FirstReadTime: 4s350ms
- IOTimer: 4s161ms
- LazyReadSeekCount: 0
- LazyReadSeekTime: 0ns
- LazyReadTime: 43.24ms
- NumSegmentFiltered: 0
- NumSegmentTotal: 78
- OutputColumnTime: 31.653ms
- RawRowsRead: 12.071214M (12071214)
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 9.426797M (9426797)
- RowsKeyRangeFiltered: 0
- RowsStatsFiltered: 9.426797M (9426797)
- RowsVectorPredFiltered: 9.116918M (9116918)
- ShortPredEvalTime: 632.552ms
- TotalPagesNum: 22.625K (22625)
- UncompressedBytesRead: 1.21 GB
- VectorPredEvalTime: 13.653ms
Instance ce91e8a1d43f404c-a1e26c1c56e6c7c7 (host=TNetworkAddress(hostname:11.6.2.52, port:9060)):(Active: 1s409ms, % non-child: 0.00%)
- FragmentCpuTime: 2.221ms
- RowsProduced: 0
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
VDataStreamSender (dst_id=1, dst_fragments=[{"ce91e8a1d43f404c-a1e26c1c56e6c7ca"}]):(Active: 23.881us, % non-child: 0.00%)
- BrpcSendTime: 134.146us
- BrpcSendTime.Wait: 429.573us
- BytesSent: 0.00
- CompressTime: 0ns
- IgnoreRows: 0
- LocalBytesSent: 0.00
- LocalSentRows: 0
- OverallThroughput: 0.0 /sec
- PeakMemoryUsage: 0.00
- SerializeBatchTime: 0ns
- UncompressedRowBatchSize: 0.00
VNewOlapScanNode(file_info_his_init_update2) (id=0):(Active: 1s408ms, % non-child: 0.50%)
- PushDownPredicates: [{fileId IN [c7e556560ff2eb33672e22e6be0a5df7]}, {__DORIS_DELETE_SIGN__ IN [0]}]
- KeyRanges: ScanKeys:ScanKey=[null(-2147483648) : 2147483647]
- TabletIds: [1290083, 1290023, 1290035, 1289975, 1289999, 1290247, 1290191, 1290131, 1290155, 1290103, 1290455, 1290479, 1290303, 1290319, 1290627, 1290635, 1290639, 1290503, 1290547, 1290567, 1290427]
- RemainedDownPredicates: CompoundPredicate (and){
VectorizedFn[eq({"fileId", "String"})]{
SlotRef(slot_id=1 type=VARCHAR codegen=false),
VLiteral (name = String, type = String, value = (c7e556560ff2eb33672e22e6be0a5df7))},
VectorizedFn[eq({"__DORIS_DELETE_SIGN__", "Int8"})]{
SlotRef(slot_id=26 type=TINYINT codegen=false),
VLiteral (name = Int8, type = Int8, value = (-460898627072))}}
- AcuireRuntimeFilterTime: 786ns
- GetNextTime: 1s406ms
- MaxScannerThreadNum: 12
- NewlyCreateFreeBlocksNum: 0
- NumDiskAccess: 1
- NumScanners: 21
- PeakMemoryUsage: 17.90 KB
- PreAllocFreeBlocksNum: 12
- ProcessConjunctTime: 147.45us
- ProjectionTime: 0ns
- RowsRead: 3.20913M (3209130)
- RowsReturned: 0
- RowsReturnedRate: 0
- ScannerBatchWaitTime: 1s406ms
- ScannerCtxSchedCount: 22
- ScannerSchedCount: 21
- ScannerWorkerWaitTime: 9.453ms
- TabletNum: 21
- TotalReadThroughput: 0
VScanner:
- ColumnPredicates:
ComparisonPredicateBase(STRING, EQ), column_id=1, opposite=false
ComparisonPredicateBase(TINYINT, EQ), column_id=26, opposite=false
- BlockConvertTime: 0ns
- BlockFetchTime: 7s623ms
- ReaderInitTime: 0ns
- RowsDelFiltered: 0
- ScannerConvertBlockTime: 0ns
- ScannerFilterTime: 547.98ms
- ScannerGetBlockTime: 7s594ms
- ScannerInitTime: 1.102ms
- ScannerPrefilterTime: 0ns
SegmentIterator:
- BitmapIndexFilterTimer: 51.530us
- BlockConditionsFilteredTime: 3.737ms
- BlockInitSeekCount: 733
- BlockInitSeekTime: 229.267ms
- BlockInitTime: 2s434ms
- BlockLoadTime: 7s616ms
- BlocksLoad: 81.586K (81586)
- CachedPagesNum: 952
- CompressedBytesRead: 1.04 GB
- DecompressorTimer: 821.687ms
- FirstReadSeekCount: 351
- FirstReadSeekTime: 1s45ms
- FirstReadTime: 3s991ms
- IOTimer: 3s798ms
- LazyReadSeekCount: 0
- LazyReadSeekTime: 0ns
- LazyReadTime: 34.472ms
- NumSegmentFiltered: 0
- NumSegmentTotal: 91
- OutputColumnTime: 28.780ms
- RawRowsRead: 11.26826M (11268260)
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 8.223989M (8223989)
- RowsKeyRangeFiltered: 0
- RowsStatsFiltered: 8.223989M (8223989)
- RowsVectorPredFiltered: 8.05913M (8059130)
- ShortPredEvalTime: 571.517ms
- TotalPagesNum: 23.827K (23827)
- UncompressedBytesRead: 1.23 GB
- VectorPredEvalTime: 10.961ms
Instance ce91e8a1d43f404c-a1e26c1c56e6c7c4 (host=TNetworkAddress(hostname:11.6.2.54, port:9060)):(Active: 1s408ms, % non-child: 0.00%)
- FragmentCpuTime: 2.235ms
- RowsProduced: 0
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
VDataStreamSender (dst_id=1, dst_fragments=[{"ce91e8a1d43f404c-a1e26c1c56e6c7ca"}]):(Active: 23.189us, % non-child: 0.00%)
- BrpcSendTime: 146.780us
- BrpcSendTime.Wait: 569.690us
- BytesSent: 0.00
- CompressTime: 0ns
- IgnoreRows: 0
- LocalBytesSent: 0.00
- LocalSentRows: 0
- OverallThroughput: 0.0 /sec
- PeakMemoryUsage: 0.00
- SerializeBatchTime: 0ns
- UncompressedRowBatchSize: 0.00
VNewOlapScanNode(file_info_his_init_update2) (id=0):(Active: 1s407ms, % non-child: 0.50%)
- PushDownPredicates: [{fileId IN [c7e556560ff2eb33672e22e6be0a5df7]}, {__DORIS_DELETE_SIGN__ IN [0]}]
- KeyRanges: ScanKeys:ScanKey=[null(-2147483648) : 2147483647]
- TabletIds: [1290051, 1290027, 1290039, 1289991, 1290211, 1290171, 1290183, 1290195, 1290159, 1290115, 1290459, 1290335, 1290347, 1290295, 1290327, 1290263, 1290287, 1290623, 1290515, 1290411, 1290423, 1290447]
- RemainedDownPredicates: CompoundPredicate (and){
VectorizedFn[eq({"fileId", "String"})]{
SlotRef(slot_id=1 type=VARCHAR codegen=false),
VLiteral (name = String, type = String, value = (c7e556560ff2eb33672e22e6be0a5df7))},
VectorizedFn[eq({"__DORIS_DELETE_SIGN__", "Int8"})]{
SlotRef(slot_id=26 type=TINYINT codegen=false),
VLiteral (name = Int8, type = Int8, value = (-46081884876288))}}
- AcuireRuntimeFilterTime: 614ns
- GetNextTime: 1s405ms
- MaxScannerThreadNum: 12
- NewlyCreateFreeBlocksNum: 0
- NumDiskAccess: 1
- NumScanners: 22
- PeakMemoryUsage: 17.90 KB
- PreAllocFreeBlocksNum: 12
- ProcessConjunctTime: 123.451us
- ProjectionTime: 0ns
- RowsRead: 4.06005M (4060050)
- RowsReturned: 0
- RowsReturnedRate: 0
- ScannerBatchWaitTime: 1s405ms
- ScannerCtxSchedCount: 22
- ScannerSchedCount: 22
- ScannerWorkerWaitTime: 15.497ms
- TabletNum: 22
- TotalReadThroughput: 0
VScanner:
- ColumnPredicates:
ComparisonPredicateBase(STRING, EQ), column_id=1, opposite=false
ComparisonPredicateBase(TINYINT, EQ), column_id=26, opposite=false
- BlockConvertTime: 0ns
- BlockFetchTime: 6s202ms
- ReaderInitTime: 0ns
- RowsDelFiltered: 0
- ScannerConvertBlockTime: 0ns
- ScannerFilterTime: 801.327ms
- ScannerGetBlockTime: 9s192ms
- ScannerInitTime: 1.198ms
- ScannerPrefilterTime: 0ns
SegmentIterator:
- BitmapIndexFilterTimer: 60.208us
- BlockConditionsFilteredTime: 13.657ms
- BlockInitSeekCount: 780
- BlockInitSeekTime: 23.26ms
- BlockInitTime: 790.655ms
- BlockLoadTime: 6s194ms
- BlocksLoad: 89.714K (89714)
- CachedPagesNum: 968
- CompressedBytesRead: 1.27 GB
- DecompressorTimer: 1s93ms
- FirstReadSeekCount: 388
- FirstReadSeekTime: 629.747ms
- FirstReadTime: 3s968ms
- IOTimer: 1s440ms
- LazyReadSeekCount: 0
- LazyReadSeekTime: 0ns
- LazyReadTime: 44.824ms
- NumSegmentFiltered: 0
- NumSegmentTotal: 98
- OutputColumnTime: 28.957ms
- RawRowsRead: 12.909393M (12909393)
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 8.588034M (8588034)
- RowsKeyRangeFiltered: 0
- RowsStatsFiltered: 8.588034M (8588034)
- RowsVectorPredFiltered: 8.849343M (8849343)
- ShortPredEvalTime: 682.748ms
- TotalPagesNum: 28.471K (28471)
- UncompressedBytesRead: 1.51 GB
- VectorPredEvalTime: 13.896ms
Instance ce91e8a1d43f404c-a1e26c1c56e6c7c9 (host=TNetworkAddress(hostname:11.6.2.56, port:9060)):(Active: 1s343ms, % non-child: 0.00%)
- FragmentCpuTime: 3.744ms
- RowsProduced: 1
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
VDataStreamSender (dst_id=1, dst_fragments=[{"ce91e8a1d43f404c-a1e26c1c56e6c7ca"}]):(Active: 176.606us, % non-child: 0.00%)
- BrpcSendTime: 154.451us
- BrpcSendTime.Wait: 413.30us
- BytesSent: 518.00 B
- CompressTime: 20.982us
- IgnoreRows: 0
- LocalBytesSent: 0.00
- LocalSentRows: 0
- OverallThroughput: 2.7972049713134766 MB/sec
- PeakMemoryUsage: 7.30 KB
- SerializeBatchTime: 63.781us
- UncompressedRowBatchSize: 965.00 B
VNewOlapScanNode(file_info_his_init_update2) (id=0):(Active: 1s342ms, % non-child: 0.47%)
- PushDownPredicates: [{fileId IN [c7e556560ff2eb33672e22e6be0a5df7]}, {__DORIS_DELETE_SIGN__ IN [0]}]
- KeyRanges: ScanKeys:ScanKey=[null(-2147483648) : 2147483647]
- TabletIds: [1290055, 1290071, 1289971, 1289979, 1290007, 1290231, 1290187, 1290147, 1290099, 1290111, 1290451, 1290331, 1290367, 1290307, 1290251, 1290279, 1290283, 1290631, 1290527, 1290535, 1290419, 1290443]
- RemainedDownPredicates: CompoundPredicate (and){
VectorizedFn[eq({"fileId", "String"})]{
SlotRef(slot_id=1 type=VARCHAR codegen=false),
VLiteral (name = String, type = String, value = (c7e556560ff2eb33672e22e6be0a5df7))},
VectorizedFn[eq({"__DORIS_DELETE_SIGN__", "Int8"})]{
SlotRef(slot_id=26 type=TINYINT codegen=false),
VLiteral (name = Int8, type = Int8, value = (00))}}
- AcuireRuntimeFilterTime: 584ns
- GetNextTime: 1s339ms
- MaxScannerThreadNum: 12
- NewlyCreateFreeBlocksNum: 0
- NumDiskAccess: 1
- NumScanners: 22
- PeakMemoryUsage: 18.05 KB
- PreAllocFreeBlocksNum: 12
- ProcessConjunctTime: 119.496us
- ProjectionTime: 0ns
- RowsRead: 4.2054M (4205400)
- RowsReturned: 1
- RowsReturnedRate: 0
- ScannerBatchWaitTime: 1s339ms
- ScannerCtxSchedCount: 22
- ScannerSchedCount: 22
- ScannerWorkerWaitTime: 6.711ms
- TabletNum: 22
- TotalReadThroughput: 0
VScanner:
- ColumnPredicates:
ComparisonPredicateBase(STRING, EQ), column_id=1, opposite=false
ComparisonPredicateBase(TINYINT, EQ), column_id=26, opposite=false
- BlockConvertTime: 0ns
- BlockFetchTime: 5s468ms
- ReaderInitTime: 0ns
- RowsDelFiltered: 0
- ScannerConvertBlockTime: 0ns
- ScannerFilterTime: 844.772ms
- ScannerGetBlockTime: 9s862ms
- ScannerInitTime: 2.181ms
- ScannerPrefilterTime: 0ns
SegmentIterator:
- BitmapIndexFilterTimer: 54.695us
- BlockConditionsFilteredTime: 3.362ms
- BlockInitSeekCount: 840
- BlockInitSeekTime: 7.888ms
- BlockInitTime: 479.400ms
- BlockLoadTime: 5s462ms
- BlocksLoad: 80.127K (80127)
- CachedPagesNum: 809
- CompressedBytesRead: 1.30 GB
- DecompressorTimer: 1s110ms
- FirstReadSeekCount: 397
- FirstReadSeekTime: 546.819ms
- FirstReadTime: 3s833ms
- IOTimer: 1s143ms
- LazyReadSeekCount: 0
- LazyReadSeekTime: 0ns
- LazyReadTime: 50.569ms
- NumSegmentFiltered: 0
- NumSegmentTotal: 106
- OutputColumnTime: 25.77ms
- RawRowsRead: 12.493383M (12493383)
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 9.004544M (9004544)
- RowsKeyRangeFiltered: 0
- RowsStatsFiltered: 9.004544M (9004544)
- RowsVectorPredFiltered: 8.287983M (8287983)
- ShortPredEvalTime: 499.223ms
- TotalPagesNum: 29.754K (29754)
- UncompressedBytesRead: 1.56 GB
- VectorPredEvalTime: 12.977ms
Instance ce91e8a1d43f404c-a1e26c1c56e6c7c6 (host=TNetworkAddress(hostname:11.6.2.51, port:9060)):(Active: 1s95ms, % non-child: 0.00%)
- FragmentCpuTime: 20.173ms
- RowsProduced: 0
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
VDataStreamSender (dst_id=1, dst_fragments=[{"ce91e8a1d43f404c-a1e26c1c56e6c7ca"}]):(Active: 33.966us, % non-child: 0.00%)
- BrpcSendTime: 67.378us
- BrpcSendTime.Wait: 441.42us
- BytesSent: 0.00
- CompressTime: 0ns
- IgnoreRows: 0
- LocalBytesSent: 0.00
- LocalSentRows: 0
- OverallThroughput: 0.0 /sec
- PeakMemoryUsage: 0.00
- SerializeBatchTime: 0ns
- UncompressedRowBatchSize: 0.00
VNewOlapScanNode(file_info_his_init_update2) (id=0):(Active: 1s95ms, % non-child: 0.39%)
- PushDownPredicates: [{fileId IN [c7e556560ff2eb33672e22e6be0a5df7]}, {__DORIS_DELETE_SIGN__ IN [0]}]
- KeyRanges: ScanKeys:ScanKey=[null(-2147483648) : 2147483647]
- TabletIds: [1290063, 1290087, 1290011, 1289995, 1290223, 1290239, 1290203, 1290135, 1290091, 1290107, 1290127, 1290343, 1290351, 1290363, 1290323, 1290275, 1290611, 1290491, 1290523, 1290563, 1290415, 1290435]
- RemainedDownPredicates: CompoundPredicate (and){
VectorizedFn[eq({"fileId", "String"})]{
SlotRef(slot_id=1 type=VARCHAR codegen=false),
VLiteral (name = String, type = String, value = (c7e556560ff2eb33672e22e6be0a5df7))},
VectorizedFn[eq({"__DORIS_DELETE_SIGN__", "Int8"})]{
SlotRef(slot_id=26 type=TINYINT codegen=false),
VLiteral (name = Int8, type = Int8, value = (00))}}
- AcuireRuntimeFilterTime: 409ns
- GetNextTime: 1s75ms
- MaxScannerThreadNum: 12
- NewlyCreateFreeBlocksNum: 0
- NumDiskAccess: 1
- NumScanners: 22
- PeakMemoryUsage: 18.05 KB
- PreAllocFreeBlocksNum: 12
- ProcessConjunctTime: 101.569us
- ProjectionTime: 0ns
- RowsRead: 3.386157M (3386157)
- RowsReturned: 0
- RowsReturnedRate: 0
- ScannerBatchWaitTime: 1s75ms
- ScannerCtxSchedCount: 22
- ScannerSchedCount: 22
- ScannerWorkerWaitTime: 4.129ms
- TabletNum: 22
- TotalReadThroughput: 0
VScanner:
- ColumnPredicates:
ComparisonPredicateBase(STRING, EQ), column_id=1, opposite=false
ComparisonPredicateBase(TINYINT, EQ), column_id=26, opposite=false
- BlockConvertTime: 0ns
- BlockFetchTime: 4s591ms
- ReaderInitTime: 0ns
- RowsDelFiltered: 0
- ScannerConvertBlockTime: 0ns
- ScannerFilterTime: 590.111ms
- ScannerGetBlockTime: 7s273ms
- ScannerInitTime: 19.272ms
- ScannerPrefilterTime: 0ns
SegmentIterator:
- BitmapIndexFilterTimer: 49.11us
- BlockConditionsFilteredTime: 3.435ms
- BlockInitSeekCount: 844
- BlockInitSeekTime: 6.167ms
- BlockInitTime: 380.419ms
- BlockLoadTime: 4s585ms
- BlocksLoad: 85.488K (85488)
- CachedPagesNum: 1.179K (1179)
- CompressedBytesRead: 1.09 GB
- DecompressorTimer: 801.621ms
- FirstReadSeekCount: 398
- FirstReadSeekTime: 413.737ms
- FirstReadTime: 3s60ms
- IOTimer: 1s100ms
- LazyReadSeekCount: 0
- LazyReadSeekTime: 0ns
- LazyReadTime: 35.555ms
- NumSegmentFiltered: 0
- NumSegmentTotal: 106
- OutputColumnTime: 25.865ms
- RawRowsRead: 11.827082M (11827082)
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 9.670191M (9670191)
- RowsKeyRangeFiltered: 0
- RowsStatsFiltered: 9.670191M (9670191)
- RowsVectorPredFiltered: 8.440925M (8440925)
- ShortPredEvalTime: 526.901ms
- TotalPagesNum: 25.591K (25591)
- UncompressedBytesRead: 1.29 GB
- VectorPredEvalTime: 10.972ms
Instance ce91e8a1d43f404c-a1e26c1c56e6c7c8 (host=TNetworkAddress(hostname:11.6.2.53, port:9060)):
```
I don't know what cause be crashed. what should I to do. tell me pls.
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] lipf19860104 commented on issue #21278: [Bug] be crash when use a condition query on 100G table
Posted by "lipf19860104 (via GitHub)" <gi...@apache.org>.
lipf19860104 commented on issue #21278:
URL: https://github.com/apache/doris/issues/21278#issuecomment-1619390013
ok. I'll try to upgrade and reply whether it work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] lipf19860104 closed issue #21278: [Bug] be crash when use a condition query on 100G table
Posted by "lipf19860104 (via GitHub)" <gi...@apache.org>.
lipf19860104 closed issue #21278: [Bug] be crash when use a condition query on 100G table
URL: https://github.com/apache/doris/issues/21278
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] lipf19860104 commented on issue #21278: [Bug] be crash when use a condition query on 100G table
Posted by "lipf19860104 (via GitHub)" <gi...@apache.org>.
lipf19860104 commented on issue #21278:
URL: https://github.com/apache/doris/issues/21278#issuecomment-1619629704
I have upgraded to 1.2.4.1 from 1.2.0. it have solved my problem and have a higher performanced than 1.2.0. thx.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yuanyuan8983 commented on issue #21278: [Bug] be crash when use a condition query on 100G table
Posted by "yuanyuan8983 (via GitHub)" <gi...@apache.org>.
yuanyuan8983 commented on issue #21278:
URL: https://github.com/apache/doris/issues/21278#issuecomment-1617474731
It is recommended to upgrade to version 1.2.4.1 for testing, the problem of 1.2.0 should have been fixed, you can talk about it privately, my wechat is : cyllyy810222
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] lipf19860104 commented on issue #21278: [Bug] be crash when use a condition query on 100G table
Posted by "lipf19860104 (via GitHub)" <gi...@apache.org>.
lipf19860104 commented on issue #21278:
URL: https://github.com/apache/doris/issues/21278#issuecomment-1612363957
I find a core file. and gdb it show follow. but I don't know how to fixed it.
<img width="938" alt="be_core" src="https://github.com/apache/doris/assets/13567711/f67b73ce-10c1-443e-a637-5b54aeb2bc35">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org