You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2019/11/14 07:09:39 UTC
[GitHub] [incubator-doris] blackfox1983 opened a new issue #2201: stream_load设置了timeout。但结果显示cancel
blackfox1983 opened a new issue #2201: stream_load设置了timeout。但结果显示cancel
URL: https://github.com/apache/incubator-doris/issues/2201
**Describe the bug**
流式导入时的timeout可能没生效
**To Reproduce**
**Expected behavior**
1、version：11-6日 master
2、源文件：8.1G 约9500w+行
3、半个小时超时。curl --location-trusted -u root:root -H "label:${label}" -H "timeout:1800" -H "max_filter_ratio:0.0001" -T ${file} http://ip:port/api/dataware/${table}/_stream_load
4、返回：{
"TxnId": 3025,
"Label": "speaker_attend_online_data_20191114",
"Status": "Fail",
"Message": "cancelled",
"NumberTotalRows": 0,
"NumberLoadedRows": 0,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 5327780904,
"LoadTimeMs": 624943
}
5、查看be日志
be.WARNING：
W1114 12:41:32.477231 202636 broker_scan_node.cpp:414] Scanner[0] prcess failed. status=Cancelled
W1114 12:41:32.489692 8414 stream_load_executor.cpp:83] fragment execute failed, query_id=b2478944e297cab3-782bbf958dc12981, errmsg=Cancelledid=b2478944e297cab3-782bbf958dc12981, job id=-1, txn id=3025, label=speaker_attend_online_data_20191114
W1114 12:41:32.494271 8570 stream_load.cpp:272] append body content failed. errmsg=cancelledid=b2478944e297cab3-782bbf958dc12981, job id=-1, txn id=3025, label=speaker_attend_online_data_20191114
be.INFO
I1114 12:41:32.476306 8378 fragment_mgr.cpp:516] FragmentMgr cancel worker going to cancel fragment b2478944e297cab3-782bbf958dc12982
I1114 12:41:32.476336 8378 plan_fragment_executor.cpp:530] cancel(): fragment_instance_id=b2478944e297cab3-782bbf958dc12982
W1114 12:41:32.477231 202636 broker_scan_node.cpp:414] Scanner[0] prcess failed. status=Cancelled
I1114 12:41:32.489665 8414 plan_fragment_executor.cpp:581] Fragment b2478944e297cab3-782bbf958dc12982:(Active: 10m1s, % non-child: 0.00%)
- AverageThreadTokens: 1.00
- PeakReservation: 0
- PeakUsedReservation: 0
- RowsProduced: 57.00M
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0
- MaxBlockSize: 8.00 MB
- MemoryLimit: 2.00 GB
- TotalBufferWaitTime: 0.000ns
- TotalEncryptionTime: 0.000ns
- TotalIntegrityCheckTime: 0.000ns
- TotalReadBlockTime: 0.000ns
OlapTableSink:(Active: 10m, % non-child: 0.00%)
- CloseTime: 0.000ns
- ConvertBatchTime: 0.000ns
- OpenTime: 1.151ms
- RowsFiltered: 0
- RowsRead: 0
- RowsReturned: 0
- SendDataTime: 0.000ns
- SerializeBatchTime: 0.000ns
- ValidateDataTime: 0.000ns
- WaitInFlightPacketTime: 0.000ns
BROKER_SCAN_NODE (id=0):(Active: 398.968ms, % non-child: 0.00%)
- BytesDecompressed: 0
- BytesRead: 4.96 GB
- DecompressTime: 0.000ns
- FileReadTime: 22s919ms
- MemoryUsed: 0
- NumThread: 0
- PerReadThreadRawHdfsThroughput: 22.41 MB/sec
- RowsRead: 57.07M
- RowsReturned: 57.00M
- RowsReturnedRate: 142.88 M/sec
- ScanRangesComplete: 0
- ScannerThreadsInvoluntaryContextSwitches: 0
- ScannerThreadsTotalWallClockTime: 0.000ns
- MaterializeTupleTime(*): 3m7s
- ScannerThreadsSysTime: 0.000ns
- ScannerThreadsUserTime: 0.000ns
- ScannerThreadsVoluntaryContextSwitches: 0
- TotalRawReadTime(*): 3m46s
- TotalReadThroughput: 8.45 MB/sec
- WaitScannerTime: 150.973ms
W1114 12:41:32.489692 8414 stream_load_executor.cpp:83] fragment execute failed, query_id=b2478944e297cab3-782bbf958dc12981, errmsg=Cancelledid=b2478944e297cab3-782bbf958dc12981, job id=-1, txn id=3025, label=speaker_attend_online_data_20191114
W1114 12:41:32.494271 8570 stream_load.cpp:272] append body content failed. errmsg=cancelledid=b2478944e297cab3-782bbf958dc12981, job id=-1, txn id=3025, label=speaker_attend_online_data_20191114
I1114 12:41:32.501806 8568 load_channel_mgr.cpp:179] load channel has been cancelled: b2478944e297cab3-782bbf958dc12981
I1114 12:41:32.501834 8568 load_channel.cpp:36] load channel mem peak usage: 2149740544, info: limit: 2147483648; consumption: 2144481280; label: b2478944e297cab3-782bbf958dc12981; all tracker size: 2; limit trackers size: 2; parent is null: false; , load id: b2478944e297cab3-782bbf958dc12981
I1114 12:41:32.501863 8568 txn_manager.cpp:320] rollback transaction from engine successfully. partition_id: 22526, transaction_id: 3025, tablet: 23009.794985093.9141f533740c81ab-d614d2539afa75b9
I1114 12:41:32.501943 8568 segment_group.cpp:218] delete index file. path=/home/doris/doris/be/palo/data/188/23009/794985093/020000000000028a2549928d52f5605645db56265dd12984_0_0.idx
I1114 12:41:32.502055 8568 segment_group.cpp:230] delete data file. path=/home/doris/doris/be/palo/data/188/23009/794985093/020000000000028a2549928d52f5605645db56265dd12984_0_0.dat
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org