You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/04/12 07:14:14 UTC
[GitHub] [incubator-doris] hf200012 opened a new issue #5635: Doris 0.14.7 version, there is a backlog of stream load tasks
hf200012 opened a new issue #5635:
URL: https://github.com/apache/incubator-doris/issues/5635
This problem did not occur in version 0.13.15
The specific phenomenon is as follows ：
Stream load task error, report: failed to call frontend service
The number of txns quickly reached the upper limit：errCode = 2, detailMessage = current running txns on db 11001 is 150, larger than limit 150
View backends heartbeat loss
[https://github.com/hf200012/images/blob/master/doris/20210412145426.png](url)
Restart BE, the log shows that the heartbeat of FE has not been received
To check transactions, it shows that many tasks are not over ：
[https://github.com/hf200012/images/blob/master/doris/20210412145444.png](url)
**Finally, only restart FE, and then return to normal**
Fe master keeps refreshing the following logs ：
2021-04-12 11:45:43,053 WARN (thrift-server-pool-297|673) [ReportHandler.putToQueue():182] the report queue size exceeds the limit: 120. current: 121
2021-04-12 11:45:43,060 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():105] get number of low load paths: 10, with medium: HDD
2021-04-12 11:45:43,061 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():186] select alternative tablets for cluster: default_cluster, medium: HDD, num: 2, detail: [5092902, 5092902]
2021-04-12 11:45:43,061 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():84] cluster is balance: default_cluster with medium: SSD. skip
2021-04-12 11:45:43,119 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605972] is expired, remove it from transaction manager
2021-04-12 11:45:43,165 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605973] is expired, remove it from transaction manager
2021-04-12 11:45:43,211 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605974] is expired, remove it from transaction manager
2021-04-12 11:45:43,257 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605979] is expired, remove it from transaction manager
2021-04-12 11:45:43,302 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605981] is expired, remove it from transaction manager
2021-04-12 11:45:43,348 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605984] is expired, remove it from transaction manager
2021-04-12 11:45:43,400 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605985] is expired, remove it from transaction manager
2021-04-12 11:45:43,451 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605986] is expired, remove it from transaction manager
2021-04-12 11:45:43,502 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606000] is expired, remove it from transaction manager
2021-04-12 11:45:43,554 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606001] is expired, remove it from transaction manager
2021-04-12 11:45:43,605 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606007] is expired, remove it from transaction manager
2021-04-12 11:45:43,663 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606008] is expired, remove it from transaction manager
2021-04-12 11:45:43,720 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606009] is expired, remove it from transaction manager
2021-04-12 11:45:43,771 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606017] is expired, remove it from transaction manager
2021-04-12 11:45:43,823 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606018] is expired, remove it from transaction manager
2021-04-12 11:45:43,874 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606031] is expired, remove it from transaction manager
2021-04-12 11:45:43,926 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606033] is expired, remove it from transaction manager
2021-04-12 11:45:43,977 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606034] is expired, remove it from transaction manager
2021-04-12 11:45:44,029 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606040] is expired, remove it from transaction manager
2021-04-12 11:45:44,061 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():105] get number of low load paths: 10, with medium: HDD
2021-04-12 11:45:44,062 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():186] select alternative tablets for cluster: default_cluster, medium: HDD, num: 2, detail: [5092902, 5092902]
2021-04-12 11:45:44,062 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():84] cluster is balance: default_cluster with medium: SSD. skip
2021-04-12 11:45:44,080 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606041] is expired, remove it from transaction manager
2021-04-12 11:45:44,132 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606045] is expired, remove it from transaction manager
2021-04-12 11:45:44,183 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606046] is expired, remove it from transaction manager
2021-04-12 11:45:44,235 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606047] is expired, remove it from transaction manager
2021-04-12 11:45:44,286 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606074] is expired, remove it from transaction manager
2021-04-12 11:45:44,338 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606075] is expired, remove it from transaction manager
2021-04-12 11:45:44,389 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606081] is expired, remove it from transaction manager
2021-04-12 11:45:44,441 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606082] is expired, remove it from transaction manager
2021-04-12 11:45:44,492 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606083] is expired, remove it from transaction manager
2021-04-12 11:45:44,544 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606089] is expired, remove it from transaction manager
2021-04-12 11:45:44,595 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606092] is expired, remove it from transaction manager
2021-04-12 11:45:44,647 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606107] is expired, remove it from transaction manager
2021-04-12 11:45:44,698 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606109] is expired, remove it from transaction manager
2021-04-12 11:45:44,750 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606111] is expired, remove it from transaction manager
2021-04-12 11:45:44,801 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606119] is expired, remove it from transaction manager
2021-04-12 11:45:44,853 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606120] is expired, remove it from transaction manager
2021-04-12 11:45:44,904 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606124] is expired, remove it from transaction manager
2021-04-12 11:45:44,956 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606125] is expired, remove it from transaction manager
2021-04-12 11:45:45,007 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606126] is expired, remove it from transaction manager
2021-04-12 11:45:45,022 WARN (thrift-server-pool-852|14940) [ReportHandler.putToQueue():182] the report queue size exceeds the limit: 120. current: 121
2021-04-12 11:45:45,062 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():105] get number of low load paths: 10, with medium: HDD
2021-04-12 11:45:45,063 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():186] select alternative tablets for cluster: default_cluster, medium: HDD, num: 2, detail: [5092902, 5092902]
2021-04-12 11:45:45,063 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():84] cluster is balance: default_cluster with medium: SSD. skip
2021-04-12 11:45:45,066 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606135] is expired, remove it from transaction manager
2021-04-12 11:45:45,123 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606136] is expired, remove it from transaction manager
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] hf200012 commented on issue #5635: Doris 0.14.7 version, there is a backlog of stream load tasks
Posted by GitBox <gi...@apache.org>.
hf200012 commented on issue #5635:
URL: https://github.com/apache/incubator-doris/issues/5635#issuecomment-820309749
This is caused by my stream load task
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] hf200012 closed issue #5635: Doris 0.14.7 version, there is a backlog of stream load tasks
Posted by GitBox <gi...@apache.org>.
hf200012 closed issue #5635:
URL: https://github.com/apache/incubator-doris/issues/5635
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org