You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/04/16 05:17:43 UTC

[GitHub] [incubator-doris] hf200012 opened a new issue #5663: 0.14.7 stream load

hf200012 opened a new issue #5663:
URL: https://github.com/apache/incubator-doris/issues/5663


   After the doris-0.14.7 stream load task runs for two or three days, there is a large backlog of tasks, which can’t be ended.
   Some users also encountered this problem in 0.13
   
   The current phenomenon is that it is stuck in close_wait, and there is a problem with the timeout mechanism.
   The timeout time of tablet_add_batch rpc each time is the timeout time of stream load, that is, the timeout time of each batch is the timeout time of stream load, which may be very large when added together.
   Since libevent's http is not good for multi-threading support, an http request is stuck, and the http request of this be, if it hits the same libevent http event_base, it will also be stuck
   
   I have this problem three times


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] hf200012 commented on issue #5663: 0.14.7 stream load Transactions in the committed state cannot be terminated

Posted by GitBox <gi...@apache.org>.
hf200012 commented on issue #5663:
URL: https://github.com/apache/incubator-doris/issues/5663#issuecomment-828116870


   The solution to this problem is to change label_keep_max_second from three days to 12 hours by default, or 6 hours less


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] hf200012 commented on issue #5663: 0.14.7 stream load

Posted by GitBox <gi...@apache.org>.
hf200012 commented on issue #5663:
URL: https://github.com/apache/incubator-doris/issues/5663#issuecomment-821061922


   W0416 11:58:26.940357 69885 data_stream_sender.cpp:155] failed to send brpc batch, error=Host is down, error_text=[E110]Fail to connect Socket{id=513 addr=172.22.197.248:8060} (0x0x1f93a200): Connection timed out [R1][E112]Not connected to 172.22.197.248:8060 yet, server_id=513 [R2][E112]Not connected to 172.22.197.248:8060 yet, server_id=513 [R3][E112]Not connected to 172.22.197.248:8060 yet, server_id=513, client: 172.22.197.243
   I0416 11:58:26.949580 69885 plan_fragment_executor.cpp:518] cancel(): fragment_instance_id=e1fbe79253c04603-8d67b18a55efc2da
   W0416 11:58:26.949627 69885 fragment_mgr.cpp:230] Got error while opening fragment e1fbe79253c04603-8d67b18a55efc2da: Thrift rpc error: failed to send brpc batch, error=Host is down, error_text=[E110]Fail to connect Socket{id=513 addr=172.22.197.248:8060} (0x0x1f93a200): Connection timed out [R1][E112]Not connected to 172.22.197.248:8060 yet, server_id=513 [R2][E112]Not connected to 172.22.197.248:8060 yet, server_id=513 [R3][E112]Not connected to 172.22.197.248:8060 yet, server_id=513, client: 172.22.197.243
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] hf200012 commented on issue #5663: 0.14.7 stream load

Posted by GitBox <gi...@apache.org>.
hf200012 commented on issue #5663:
URL: https://github.com/apache/incubator-doris/issues/5663#issuecomment-821060656


   BE LOG:
   W0416 11:14:06.426590 69908 thrift_rpc_helper.cpp:79] call frontend service failed, address=TNetworkAddress(hostname=172.22.197.240, port=9020), reason=THRIFT_EAGAIN (timed out)
   W0416 11:14:08.427516 69908 stream_load_executor.cpp:261] transaction rollback failed. errmsg=failed to call frontend serviceid=a122e6ca46294429-a41256c5d668a113, job_id=15703530, txn_id=18082359, label=POS_ETL_PRO_DISH_LIST_DETAIL_TASK-15703530-a122e6ca46294429-a41256c5d668a113-18082359
   I0416 11:14:08.427676 69908 routine_load_task_executor.cpp:176] finished routine load task id=a122e6ca46294429-a41256c5d668a113, job_id=15703530, txn_id=18082359, label=POS_ETL_PRO_DISH_LIST_DETAIL_TASK-15703530-a122e6ca46294429-a41256c5d668a113-18082359, status: failed to call frontend service, current tasks num: 0
   I0416 11:14:09.471413 70019 task_worker_pool.cpp:1549] finish report TASK. master host: 172.22.197.240, port: 9020
   
   W0416 11:15:35.579210 70020 task_worker_pool.cpp:1545] finish report DISK failed. status:6, error msg:failed to put report task to queue. queue size: 121,err: the report queue size exceeds the limit: 120. current: 121
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] hf200012 commented on issue #5663: 0.14.7 stream load

Posted by GitBox <gi...@apache.org>.
hf200012 commented on issue #5663:
URL: https://github.com/apache/incubator-doris/issues/5663#issuecomment-821041766


   2021-04-16 11:52:44,170 INFO (txnCleaner|86) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17885294] is expired, remove it from transaction manager
   2021-04-16 11:52:44,178 INFO (leaderCheckpointer|89) [Catalog.saveImage():1960] start save image to /hdd_data01/doris-meta/image/image.ckpt. is ckpt: true
   2021-04-16 11:52:44,181 WARN (doris-mysql-nio-pool-11|188) [ProfileManager.createElement():125] failed to build profile tree
   org.apache.doris.common.UserException: errCode = 2, detailMessage = Invalid profile. Expected Execution Profile
   	at org.apache.doris.common.profile.MultiProfileTreeBuilder.unwrapProfile(MultiProfileTreeBuilder.java:82) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.profile.MultiProfileTreeBuilder.build(MultiProfileTreeBuilder.java:66) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.util.ProfileManager.createElement(ProfileManager.java:122) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.util.ProfileManager.pushProfile(ProfileManager.java:137) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.writeProfile(StmtExecutor.java:452) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:339) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:284) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:344) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:545) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_261]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_261]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
   2021-04-16 11:52:44,199 INFO (PUBLISH_VERSION|32) [DatabaseTransactionMgr.finishTransaction():840] finish transaction TransactionState. transaction id: 18085062, label: audit_20210416_114300_172_22_197_240_9010, db id: 11810, table id list: 11811, callback id: -1, coordinator: BE: 172.22.197.249, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1618544580798, commit time: 1618544580829, finish time: 1618545164142, reason:  successfully
   2021-04-16 11:52:44,228 INFO (txnCleaner|86) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17885229] is expired, remove it from transaction manager
   2021-04-16 11:52:44,247 WARN (doris-mysql-nio-pool-12|189) [ProfileManager.createElement():125] failed to build profile tree
   org.apache.doris.common.UserException: errCode = 2, detailMessage = Invalid profile. Expected Execution Profile
   	at org.apache.doris.common.profile.MultiProfileTreeBuilder.unwrapProfile(MultiProfileTreeBuilder.java:82) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.profile.MultiProfileTreeBuilder.build(MultiProfileTreeBuilder.java:66) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.util.ProfileManager.createElement(ProfileManager.java:122) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.util.ProfileManager.pushProfile(ProfileManager.java:137) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.writeProfile(StmtExecutor.java:452) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:339) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:284) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:344) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:545) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_261]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_261]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
   2021-04-16 11:52:44,256 INFO (PUBLISH_VERSION|32) [DatabaseTransactionMgr.finishTransaction():840] finish transaction TransactionState. transaction id: 18086043, label: audit_20210416_114746_172_22_197_240_9010, db id: 11810, table id list: 11811, callback id: -1, coordinator: BE: 172.22.197.243, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1618544866685, commit time: 1618544866731, finish time: 1618545164199, reason:  successfully
   2021-04-16 11:52:44,258 INFO (Thread-50|115) [ReportHandler.handleSetTabletInMemory():925] find [0] tablets need set in memory meta
   2021-04-16 11:52:44,258 INFO (Thread-50|115) [ReportHandler.tabletReport():315] tablet report from backend[10007] cost: 907 ms
   2021-04-16 11:52:44,259 INFO (Thread-50|115) [ReportHandler.tabletReport():241] backend[10004] reports 1263 tablet(s). report version: 16181986760484
   2021-04-16 11:52:44,285 INFO (txnCleaner|86) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17885300] is expired, remove it from transaction manager
   2021-04-16 11:52:44,313 INFO (PUBLISH_VERSION|32) [DatabaseTransactionMgr.finishTransaction():840] finish transaction TransactionState. transaction id: 18084042, label: insert_482dc3c21bcc4978-9749e97b5880f208, db id: 14688493, table id list: 15673238, callback id: -1, coordinator: FE: 172.22.197.240, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1618544101029, commit time: 1618544102214, finish time: 1618545164257, reason:  successfully
   2021-04-16 11:52:44,314 WARN (doris-mysql-nio-pool-13|190) [ProfileManager.createElement():125] failed to build profile tree
   org.apache.doris.common.UserException: errCode = 2, detailMessage = Invalid profile. Expected Execution Profile
   	at org.apache.doris.common.profile.MultiProfileTreeBuilder.unwrapProfile(MultiProfileTreeBuilder.java:82) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.profile.MultiProfileTreeBuilder.build(MultiProfileTreeBuilder.java:66) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.util.ProfileManager.createElement(ProfileManager.java:122) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.util.ProfileManager.pushProfile(ProfileManager.java:137) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.writeProfile(StmtExecutor.java:452) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:339) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:284) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:344) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:545) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_261]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_261]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
   2021-04-16 11:52:44,317 INFO (Thread-50|115) [TabletInvertedIndex.tabletReport():134] begin to do tablet diff with backend[10004]. num: 1263
   2021-04-16 11:52:44,348 INFO (txnCleaner|86) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17885301] is expired, remove it from transaction manager
   2021-04-16 11:52:44,372 WARN (doris-mysql-nio-pool-4|145) [ProfileManager.createElement():125] failed to build profile tree
   org.apache.doris.common.UserException: errCode = 2, detailMessage = Invalid profile. Expected Execution Profile
   	at org.apache.doris.common.profile.MultiProfileTreeBuilder.unwrapProfile(MultiProfileTreeBuilder.java:82) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.profile.MultiProfileTreeBuilder.build(MultiProfileTreeBuilder.java:66) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.util.ProfileManager.createElement(ProfileManager.java:122) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.util.ProfileManager.pushProfile(ProfileManager.java:137) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.writeProfile(StmtExecutor.java:452) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:339) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:284) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:344) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:545) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_261]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_261]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
   2021-04-16 11:52:44,376 INFO (PUBLISH_VERSION|32) [DatabaseTransactionMgr.finishTransaction():840] finish transaction TransactionState. transaction id: 18084043, label: insert_d3b62d2ea3824e40-8ab5176f49c3a14e, db id: 14688493, table id list: 15673238, callback id: -1, coordinator: FE: 172.22.197.240, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1618544115065, commit time: 1618544116798, finish time: 1618545164314, reason:  successfully
   2021-04-16 11:52:44,381 INFO (tablet scheduler|39) [TabletScheduler.updateWorkingSlots():169] not all backends have path info
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] hf200012 commented on issue #5663: 0.14.7 stream load

Posted by GitBox <gi...@apache.org>.
hf200012 commented on issue #5663:
URL: https://github.com/apache/incubator-doris/issues/5663#issuecomment-821033872


   2021-04-16 12:29:31,968 WARN (doris-mysql-nio-pool-3|128) [StmtExecutor.handleInsertStmt():1238] handle insert stmt fail: insert_9a1ec36317724140-b77de4319e4d670b
   org.apache.doris.common.DdlException: errCode = 2, detailMessage = failed to send brpc batch, error=Host is down, error_text=[E110]Fail to connect Socket{id=132 addr=172.22.197.244:8060} (0x0x6902800): Connection timed out [R1][E112]Not connected to 172.22.197.244:8060 yet, server_id=132 [R2][E112]Not connected to 172.22.197.244:8060 yet, server_id=132 [R3][E112]Not connected to 172.22.197.244:8060 yet, server_id=132, client: 172.22.197.250
   	at org.apache.doris.common.ErrorReport.reportDdlException(ErrorReport.java:67) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.handleInsertStmt(StmtExecutor.java:1191) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:367) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:284) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:344) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:545) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_261]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_261]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] hf200012 closed issue #5663: 0.14.7 stream load Transactions in the committed state cannot be terminated

Posted by GitBox <gi...@apache.org>.
hf200012 closed issue #5663:
URL: https://github.com/apache/incubator-doris/issues/5663


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] hf200012 commented on issue #5663: 0.14.7 stream load

Posted by GitBox <gi...@apache.org>.
hf200012 commented on issue #5663:
URL: https://github.com/apache/incubator-doris/issues/5663#issuecomment-821061169


   W0416 11:53:43.734288 70089 tablet_sink.h:124] failed to send brpc batch, error=Host is down, error_text=[E111]Fail to connect Socket{id=513 addr=172.22.197.248:8060} (0x0x1f93a200): Connection refused [R1][E112]Not connected to 172.22.197.248:8060 yet, server_id=513 [R2][E112]Not connected to 172.22.197.248:8060 yet, server_id=513 [R3][E112]Not connected to 172.22.197.248:8060 yet, server_id=513
   W0416 11:53:43.735491 70089 tablet_sink.cpp:149] NodeChannel[15399245-10009] add batch req rpc failed, load_id=a0467655e2734208-b7190ab365dd69f3, txn_id=18087045, node=172.22.197.248:8060
   I0416 11:53:43.748622 70042 tablets_channel.cpp:141] close tablets channel: (id=a0467655e2734208-b7190ab365dd69f3,index_id=15399245), sender id: 0
   I0416 11:53:43.750087 254369 tablet_sink.cpp:979] all node channels are stopped(maybe finished/offending/cancelled), consumer thread exit.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


[GitHub] [incubator-doris] hf200012 commented on issue #5663: 0.14.7 stream load

Posted by GitBox <gi...@apache.org>.
hf200012 commented on issue #5663:
URL: https://github.com/apache/incubator-doris/issues/5663#issuecomment-821035580


   
   2021-04-16 12:29:32,080 WARN (doris-mysql-nio-pool-3|128) [ProfileManager.createElement():125] failed to build profile tree
   org.apache.doris.common.UserException: errCode = 2, detailMessage = Invalid instance profile, without sender or exec node: Instance 9a1ec36317724140-b77de4319e4d6712 (host=TNetworkAddress(hostname:172.22.197.244, port:9060)):
   
   	at org.apache.doris.common.profile.ProfileTreeBuilder.buildSingleInstanceTree(ProfileTreeBuilder.java:207) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.profile.ProfileTreeBuilder.analyzeAndBuildFragmentTree(ProfileTreeBuilder.java:174) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.profile.ProfileTreeBuilder.analyzeAndBuildFragmentTrees(ProfileTreeBuilder.java:139) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.profile.ProfileTreeBuilder.build(ProfileTreeBuilder.java:118) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.profile.MultiProfileTreeBuilder.buildTrees(MultiProfileTreeBuilder.java:98) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.profile.MultiProfileTreeBuilder.build(MultiProfileTreeBuilder.java:67) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.util.ProfileManager.createElement(ProfileManager.java:122) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.common.util.ProfileManager.pushProfile(ProfileManager.java:137) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.writeProfile(StmtExecutor.java:452) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:369) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:284) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:344) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:545) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_261]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_261]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
   2021-04-16 12:29:32,080 INFO (doris-mysql-nio-pool-3|128) [QeProcessorImpl.unregisterQuery():78] deregister query id 9a1ec36317724140-b77de4319e4d670b
   2021-04-16 12:29:32,081 WARN (doris-mysql-nio-pool-3|128) [StmtExecutor.execute():421] errors when abort txn
   org.apache.doris.transaction.TransactionNotFoundException: errCode = 2, detailMessage = transaction not found
   	at org.apache.doris.transaction.DatabaseTransactionMgr.abortTransaction(DatabaseTransactionMgr.java:963) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.transaction.GlobalTransactionMgr.abortTransaction(GlobalTransactionMgr.java:221) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.transaction.GlobalTransactionMgr.abortTransaction(GlobalTransactionMgr.java:216) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:417) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:284) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:344) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:545) ~[palo-fe.jar:3.4.0]
   	at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_261]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_261]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_261]
   	
   	
   2021-04-16 12:29:54,084 WARN (thrift-server-pool-15|152) [FrontendServiceImpl.loadTxnRollback():885] failed to rollback txn 18094052: errCode = 2, detailMessage = transaction's state is already COMMITTED, could not abort
   2021-04-16 12:29:54,084 INFO (audit loader thread|50) [DorisStreamLoader.loadBatch():141] AuditLoader plugin load with label: audit_20210416_122931_172_22_197_240_9010, response code: 200, msg: OK, content: {    "TxnId": 18094052,    "Label": "audit_20210416_122931_172_22_197_240_9010",    "Status": "Fail",    "Message": "failed to call frontend service",    "NumberTotalRows": 196,    "NumberLoadedRows": 196,    "NumberFilteredRows": 0,    "NumberUnselectedRows": 0,    "LoadBytes": 41781,    "LoadTimeMs": 23042,    "BeginTxnTimeMs": 1,    "StreamLoadPutTimeMs": 18,    "ReadDataTimeMs": 0,    "WriteDataTimeMs": 19,    "CommitAndPublishTimeMs": 0}
   2021-04-16 12:29:54,116 WARN (thrift-server-pool-38|211) [ReportHandler.putToQueue():182] the report queue size exceeds the limit: 120. current: 121


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org