You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/03/25 02:59:49 UTC
[GitHub] [dolphinscheduler] DHBin opened a new issue #9180: [Bug] [Master] when worker node down,save alert message to database causes mysql deadlock
DHBin opened a new issue #9180:
URL: https://github.com/apache/dolphinscheduler/issues/9180
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
environment: 3 worker nodes, 3 master nodes
1. close one worker node
2. multiple master node will invoke `WorkerDataListener#notify` method save alert message to db
log:
```
[ERROR] 2022-03-25 09:33:44.604 org.apache.dolphinscheduler.server.master.registry.ServerNodeManager:[256] - WorkerGroupListener capture data change and get data failed
org.springframework.dao.DeadlockLoserDataAccessException:
### Error updating database. Cause: com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
### The error may exist in class path resource [org/apache/dolphinscheduler/dao/mapper/AlertMapper.xml]
### The error may involve defaultParameterMap
### The error occurred while setting parameters
### SQL: insert into t_ds_alert(title, content, alert_status, log, alertgroup_id, create_time, update_time) SELECT ?, ?, ?, ?, ?, ?, ? from t_ds_alert where content = ? and alert_status = ? having count(*) = 0
### Cause: com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
; Deadlock found when trying to get lock; try restarting transaction; nested exception is com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:271)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:70)
at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:74)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:440)
at com.sun.proxy.$Proxy86.insert(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.insert(SqlSessionTemplate.java:271)
at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:58)
at com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:61)
at com.sun.proxy.$Proxy120.insertAlertWhenServerCrash(Unknown Source)
at org.apache.dolphinscheduler.dao.AlertDao.sendServerStopedAlert(AlertDao.java:107)
at org.apache.dolphinscheduler.server.master.registry.ServerNodeManager$WorkerDataListener.notify(ServerNodeManager.java:243)
at org.apache.dolphinscheduler.plugin.registry.zookeeper.ZookeeperRegistry.lambda$subscribe$1(ZookeeperRegistry.java:127)
at org.apache.curator.framework.recipes.cache.TreeCache$2.apply(TreeCache.java:760)
at org.apache.curator.framework.recipes.cache.TreeCache$2.apply(TreeCache.java:754)
at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:100)
at org.apache.curator.shaded.com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:92)
at org.apache.curator.framework.recipes.cache.TreeCache.callListeners(TreeCache.java:753)
at org.apache.curator.framework.recipes.cache.TreeCache.access$1900(TreeCache.java:75)
at org.apache.curator.framework.recipes.cache.TreeCache$4.run(TreeCache.java:865)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:123)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:955)
at com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:372)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:47)
at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:63)
at com.sun.proxy.$Proxy126.update(Unknown Source)
at com.baomidou.mybatisplus.core.executor.MybatisSimpleExecutor.doUpdate(MybatisSimpleExecutor.java:54)
at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:197)
at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:184)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:426)
... 21 common frames omitted
```
mysql deadlock log:
```
=====================================
2022-03-25 09:46:14 0x7fe2332a7700 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 33 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 109072 srv_active, 0 srv_shutdown, 238221 srv_idle
srv_master_thread log flush and writes: 0
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 34979
OS WAIT ARRAY INFO: signal count 29032
RW-shared spins 294, rounds 339, OS waits 54
RW-excl spins 319, rounds 8006, OS waits 210
RW-sx spins 23, rounds 501, OS waits 9
Spin rounds per wait: 1.15 RW-shared, 25.10 RW-excl, 21.78 RW-sx
------------------------
LATEST DETECTED DEADLOCK
------------------------
2022-03-25 09:33:44 0x7fe3247e9700
*** (1) TRANSACTION:
TRANSACTION 5327196, ACTIVE 0 sec inserting
mysql tables in use 2, locked 2
LOCK WAIT 852 lock struct(s), heap size 90320, 8225 row lock(s)
MySQL thread id 14550, OS thread handle 140609501624064, query id 3817331 10.246.0.122 itb Sending data
insert into t_ds_alert(title, content, alert_status, log, alertgroup_id, create_time, update_time)
SELECT 'Fault tolerance warning', '[{"type":"WORKER","host":"/nodes/worker/default/dolphinscheduler-worker-1.dolphinscheduler-worker-headless:1234","event":"SERVER_DOWN","warningLevel":"SERIOUS"}]', 0, null, 1,
'2022-03-24 20:33:44.305', '2022-03-24 20:33:44.305'
from t_ds_alert
where content = '[{"type":"WORKER","host":"/nodes/worker/default/dolphinscheduler-worker-1.dolphinscheduler-worker-headless:1234","event":"SERVER_DOWN","warningLevel":"SERIOUS"}]' and alert_status = 0
having count(*) = 0
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 829 page no 962 n bits 80 index PRIMARY of table `dolphinscheduler`.`t_ds_alert` trx id 5327196 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** (2) TRANSACTION:
TRANSACTION 5327197, ACTIVE 0 sec inserting
mysql tables in use 2, locked 2
852 lock struct(s), heap size 90320, 8225 row lock(s)
MySQL thread id 14768, OS thread handle 140613546579712, query id 3817333 10.246.0.124 itb Sending data
insert into t_ds_alert(title, content, alert_status, log, alertgroup_id, create_time, update_time)
SELECT 'Fault tolerance warning', '[{"type":"WORKER","host":"/nodes/worker/default/dolphinscheduler-worker-1.dolphinscheduler-worker-headless:1234","event":"SERVER_DOWN","warningLevel":"SERIOUS"}]', 0, null, 1,
'2022-03-24 20:33:44.301', '2022-03-24 20:33:44.301'
from t_ds_alert
where content = '[{"type":"WORKER","host":"/nodes/worker/default/dolphinscheduler-worker-1.dolphinscheduler-worker-headless:1234","event":"SERVER_DOWN","warningLevel":"SERIOUS"}]' and alert_status = 0
having count(*) = 0
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 829 page no 962 n bits 80 index PRIMARY of table `dolphinscheduler`.`t_ds_alert` trx id 5327197 lock mode S
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
Record lock, heap no 2 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001cc5; asc ;;
1: len 6; hex 00000050109b; asc P ;;
2: len 7; hex 0200000180296a; asc )j;;
3: len 23; hex 4661756c7420746f6c6572616e6365207761726e696e67; asc Fault tolerance warning;;
4: len 30; hex 5b7b2274797065223a22574f524b4552222c22686f7374223a222f6e6f64; asc [{"type":"WORKER","host":"/nod; (total 153 bytes);
5: len 1; hex 82; asc ;;
6: len 30; hex 41204a534f4e4f626a6563742074657874206d75737420626567696e2077; asc A JSONObject text must begin w; (total 63 bytes);
7: len 4; hex 80000001; asc ;;
8: len 5; hex 99ac703af5; asc p: ;;
9: len 5; hex 99ac703af6; asc p: ;;
Record lock, heap no 3 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001cc6; asc ;;
1: len 6; hex 0000005010c7; asc P ;;
2: len 7; hex 020000018b1620; asc ;;
3: len 23; hex 4661756c7420746f6c6572616e6365207761726e696e67; asc Fault tolerance warning;;
4: len 30; hex 5b7b2274797065223a224d4153544552222c22686f7374223a222f6e6f64; asc [{"type":"MASTER","host":"/nod; (total 145 bytes);
5: len 1; hex 82; asc ;;
6: len 30; hex 41204a534f4e4f626a6563742074657874206d75737420626567696e2077; asc A JSONObject text must begin w; (total 63 bytes);
7: len 4; hex 80000001; asc ;;
8: len 5; hex 99ac703b17; asc p; ;;
9: len 5; hex 99ac703b18; asc p; ;;
Record lock, heap no 4 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001cc7; asc ;;
1: len 6; hex 0000005010e8; asc P ;;
2: len 7; hex 02000000d324c0; asc $ ;;
3: len 23; hex 4661756c7420746f6c6572616e6365207761726e696e67; asc Fault tolerance warning;;
4: len 30; hex 5b7b2274797065223a22574f524b4552222c22686f7374223a222f6e6f64; asc [{"type":"WORKER","host":"/nod; (total 153 bytes);
5: len 1; hex 82; asc ;;
6: len 30; hex 41204a534f4e4f626a6563742074657874206d75737420626567696e2077; asc A JSONObject text must begin w; (total 63 bytes);
7: len 4; hex 80000001; asc ;;
8: len 5; hex 99ac703b30; asc p;0;;
9: len 5; hex 99ac703b31; asc p;1;;
Record lock, heap no 5 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001cc8; asc ;;
1: len 6; hex 000000514885; asc QH ;;
2: len 7; hex 02000001e614cf; asc ;;
3: len 6; hex e59b9ee8b083; asc ;;
4: len 30; hex 7b2274797065223a225441534b5f43414c4c4241434b222c2270726f6365; asc {"type":"TASK_CALLBACK","proce; (total 1904 bytes);
5: len 1; hex 82; asc ;;
6: len 17; hex 2d2d63622061726720697320626c616e6b; asc --cb arg is blank;;
7: len 4; hex 80000002; asc ;;
8: len 5; hex 99ac714802; asc qH ;;
9: len 5; hex 99ac714804; asc qH ;;
Record lock, heap no 6 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001cc9; asc ;;
1: len 6; hex 0000005148b8; asc QH ;;
2: len 7; hex 02000000fc1e06; asc ;;
3: len 6; hex e59b9ee8b083; asc ;;
4: len 30; hex 7b2274797065223a225441534b5f43414c4c4241434b222c2270726f6365; asc {"type":"TASK_CALLBACK","proce; (total 2063 bytes);
5: len 1; hex 82; asc ;;
6: len 17; hex 2d2d63622061726720697320626c616e6b; asc --cb arg is blank;;
7: len 4; hex 80000002; asc ;;
8: len 5; hex 99ac714817; asc qH ;;
9: len 5; hex 99ac714818; asc qH ;;
Record lock, heap no 7 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001cca; asc ;;
1: len 6; hex 0000005148ba; asc QH ;;
2: len 7; hex 01000002151a13; asc ;;
3: len 6; hex e59b9ee8b083; asc ;;
4: len 30; hex 7b2274797065223a225441534b5f43414c4c4241434b222c2270726f6365; asc {"type":"TASK_CALLBACK","proce; (total 1903 bytes);
5: len 1; hex 82; asc ;;
6: len 17; hex 2d2d63622061726720697320626c616e6b; asc --cb arg is blank;;
7: len 4; hex 80000002; asc ;;
8: len 5; hex 99ac714818; asc qH ;;
9: len 5; hex 99ac714818; asc qH ;;
Record lock, heap no 8 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001ccb; asc ;;
1: len 6; hex 0000005148ee; asc QH ;;
2: len 7; hex 02000000a91b64; asc d;;
3: len 23; hex 4661756c7420746f6c6572616e6365207761726e696e67; asc Fault tolerance warning;;
4: len 30; hex 5b7b2274797065223a22574f524b4552222c22686f7374223a222f6e6f64; asc [{"type":"WORKER","host":"/nod; (total 153 bytes);
5: len 1; hex 82; asc ;;
6: len 30; hex 41204a534f4e4f626a6563742074657874206d75737420626567696e2077; asc A JSONObject text must begin w; (total 63 bytes);
7: len 4; hex 80000001; asc ;;
8: len 5; hex 99ac71482f; asc qH/;;
9: len 5; hex 99ac714831; asc qH1;;
Record lock, heap no 9 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001ccc; asc ;;
1: len 6; hex 0000005148f0; asc QH ;;
2: len 7; hex 010000016a273d; asc j'=;;
3: len 17; hex 7363686564756c65722073756363657373; asc scheduler success;;
4: len 30; hex 5b7b2270726f6a6563744964223a312c2270726f6a6563744e616d65223a; asc [{"projectId":1,"projectName":; (total 396 bytes);
5: len 1; hex 82; asc ;;
6: len 23; hex 6e6f2062696e6420706c7567696e20696e7374616e6365; asc no bind plugin instance;;
7: len 4; hex 80000000; asc ;;
8: len 5; hex 99ac71482f; asc qH/;;
9: len 5; hex 99ac714831; asc qH1;;
Record lock, heap no 10 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001ccd; asc ;;
1: len 6; hex 0000005148f2; asc QH ;;
2: len 7; hex 020000016a273d; asc j'=;;
3: len 6; hex e59b9ee8b083; asc ;;
4: len 30; hex 7b2274797065223a225441534b5f43414c4c4241434b222c2270726f6365; asc {"type":"TASK_CALLBACK","proce; (total 2062 bytes);
5: len 1; hex 82; asc ;;
6: len 17; hex 2d2d63622061726720697320626c616e6b; asc --cb arg is blank;;
7: len 4; hex 80000002; asc ;;
8: len 5; hex 99ac71482f; asc qH/;;
9: len 5; hex 99ac714831; asc qH1;;
Record lock, heap no 12 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001cce; asc ;;
1: len 6; hex 0000005148f9; asc QH ;;
2: len 7; hex 02000000fd20a8; asc ;;
3: len 23; hex 4661756c7420746f6c6572616e6365207761726e696e67; asc Fault tolerance warning;;
4: len 30; hex 5b7b2274797065223a224d4153544552222c22686f7374223a222f6e6f64; asc [{"type":"MASTER","host":"/nod; (total 145 bytes);
5: len 1; hex 82; asc ;;
6: len 30; hex 41204a534f4e4f626a6563742074657874206d75737420626567696e2077; asc A JSONObject text must begin w; (total 63 bytes);
7: len 4; hex 80000001; asc ;;
8: len 5; hex 99ac714835; asc qH5;;
9: len 5; hex 99ac714836; asc qH6;;
Record lock, heap no 14 PHYSICAL RECORD: n_fields 10; compact format; info bits 0
0: len 4; hex 80001ccf; asc ;;
1: len 6; hex 000000514923; asc QI#;;
2: len 7; hex 02000000c821f9; asc ! ;;
3: len 6; hex e59b9ee8b083; asc ;;
4: len 30; hex 7b2274797065223a225441534b5f43414c4c4241434b222c2270726f6365; asc {"type":"TASK_CALLBACK","proce; (total 1527 bytes);
5: len 1; hex 82; asc ;;
6: len 17; hex 2d2d63622061726720697320626c616e6b; asc --cb arg is blank;;
7: len 4; hex 80000002; asc ;;
8: len 5; hex 99ac714847; asc qHG;;
9: len 5; hex 99ac714849; asc qHI;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 829 page no 962 n bits 80 index PRIMARY of table `dolphinscheduler`.`t_ds_alert` trx id 5327197 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex 73757072656d756d; asc supremum;;
*** WE ROLL BACK TRANSACTION (2)
------------
TRANSACTIONS
------------
Trx id counter 5328497
Purge done for trx's n:o < 5328496 undo n:o < 0 state: running but idle
History list length 36
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 422090267924104, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267923184, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267974704, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267973784, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267972864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267971944, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267965504, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267963664, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267971024, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267970104, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267955384, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267905704, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267969184, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267968264, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267912144, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267967344, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267966424, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267862464, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267938824, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267964584, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267960904, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267962744, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267879944, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267961824, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267959984, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267959064, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267958144, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267944344, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267957224, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267956304, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267954464, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267953544, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267952624, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267951704, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267950784, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267926864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267949864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267948944, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267948024, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267947104, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267946184, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267945264, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267927784, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267918584, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267943424, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267942504, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267941584, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267940664, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267939744, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267937904, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267936984, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267936064, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267935144, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267934224, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267880864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267869824, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267888224, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267933304, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267902024, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267932384, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267931464, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267930544, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267929624, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267928704, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267861544, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267925944, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267879024, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267925024, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267922264, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267921344, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267920424, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267919504, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267917664, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267863384, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267916744, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267915824, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267914904, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267913984, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267913064, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267911224, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267909384, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267874424, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267908464, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267907544, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267906624, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267903864, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267896504, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267895584, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267873504, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267876264, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267891904, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267904784, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267900184, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267897424, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267871664, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267867984, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267877184, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267875344, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267865224, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267878104, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267860624, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267901104, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267893744, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267899264, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267898344, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267892824, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267894664, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267859704, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267887304, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267872584, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267889144, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267866144, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267858784, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267882704, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267885464, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267870744, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267884544, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267883624, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267864304, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267910304, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267890984, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267890064, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267886384, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267902944, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267867064, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267881784, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 422090267868904, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 0
2513 OS file reads, 1917970 OS file writes, 1237416 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 7.09 writes/s, 5.30 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 0, seg size 2, 7 merges
merged operations:
insert 9, delete mark 0, delete 0
discarded operations:
insert 0, delete mark 0, delete 0
Hash table size 276707, node heap has 2 buffer(s)
Hash table size 276707, node heap has 5 buffer(s)
Hash table size 276707, node heap has 12 buffer(s)
Hash table size 276707, node heap has 2 buffer(s)
Hash table size 276707, node heap has 4 buffer(s)
Hash table size 276707, node heap has 3 buffer(s)
Hash table size 276707, node heap has 1 buffer(s)
Hash table size 276707, node heap has 6 buffer(s)
334.17 hash searches/s, 28.33 non-hash searches/s
---
LOG
---
Log sequence number 1765696060
Log buffer assigned up to 1765696060
Log buffer completed up to 1765696060
Log written up to 1765696060
Log flushed up to 1765696060
Added dirty pages up to 1765696060
Pages flushed up to 1765696060
Last checkpoint at 1765696060
629691 log i/o's done, 2.64 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 1099431936
Dictionary memory allocated 1436117
Buffer pool size 65536
Free buffers 61597
Database pages 3904
Old database pages 638
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 2125, created 1779, written 931453
0.00 reads/s, 0.00 creates/s, 2.94 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 3904, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
----------------------
INDIVIDUAL BUFFER POOL INFO
----------------------
---BUFFER POOL 0
Buffer pool size 8192
Free buffers 7610
Database pages 578
Old database pages 211
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 318, created 260, written 250612
0.00 reads/s, 0.00 creates/s, 0.70 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 578, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 1
Buffer pool size 8192
Free buffers 7780
Database pages 407
Old database pages 0
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 222, created 185, written 54076
0.00 reads/s, 0.00 creates/s, 0.18 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 407, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 2
Buffer pool size 8192
Free buffers 7791
Database pages 396
Old database pages 0
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 228, created 168, written 59686
0.00 reads/s, 0.00 creates/s, 0.15 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 396, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 3
Buffer pool size 8192
Free buffers 7738
Database pages 449
Old database pages 0
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 271, created 178, written 100531
0.00 reads/s, 0.00 creates/s, 0.18 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 449, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 4
Buffer pool size 8192
Free buffers 7695
Database pages 493
Old database pages 0
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 265, created 228, written 96871
0.00 reads/s, 0.00 creates/s, 0.21 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 493, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 5
Buffer pool size 8192
Free buffers 7671
Database pages 516
Old database pages 210
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 230, created 286, written 76394
0.00 reads/s, 0.00 creates/s, 0.39 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 516, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 6
Buffer pool size 8192
Free buffers 7622
Database pages 567
Old database pages 217
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 263, created 304, written 92917
0.00 reads/s, 0.00 creates/s, 0.55 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 567, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
---BUFFER POOL 7
Buffer pool size 8192
Free buffers 7690
Database pages 498
Old database pages 0
Modified db pages 0
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 0, not young 0
0.00 youngs/s, 0.00 non-youngs/s
Pages read 328, created 170, written 200366
0.00 reads/s, 0.00 creates/s, 0.58 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 498, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
0 read views open inside InnoDB
Process ID=1, Main thread ID=140613693191936 , state=sleeping
Number of rows inserted 17255, updated 157617, deleted 3344, read 179206087
0.06 inserts/s, 0.76 updates/s, 0.00 deletes/s, 1966.15 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================
```
### What you expected to happen
-
### How to reproduce
-
### Anything else
_No response_
### Version
2.0.5
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #9180: [Bug] [Master] when worker node down,save alert message to database causes mysql deadlock
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #9180:
URL: https://github.com/apache/dolphinscheduler/issues/9180#issuecomment-1078605492
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org