You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@rocketmq.apache.org by "yukon (JIRA)" <ji...@apache.org> on 2017/04/20 06:40:04 UTC
[jira] [Closed] (ROCKETMQ-152) lockForPutMessage CAS may cause
system load very high and putMessage cost more than 500ms if the
sendMessageThread num is very much
[ https://issues.apache.org/jira/browse/ROCKETMQ-152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
yukon closed ROCKETMQ-152.
--------------------------
Resolution: Won't Fix
> lockForPutMessage CAS may cause system load very high and putMessage cost more than 500ms if the sendMessageThread num is very much
> --------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ROCKETMQ-152
> URL: https://issues.apache.org/jira/browse/ROCKETMQ-152
> Project: Apache RocketMQ
> Issue Type: Improvement
> Components: rocketmq-store
> Affects Versions: 4.0.0-incubating
> Environment: sendMessageThreadPoolNums=128 and
> JDK jdk1.7.0_80 linux 2.6.32-504.el6.x86_64 and have more than 50 g free mem
> Reporter: zhaoziyan
> Assignee: yukon
> Attachments: QQ截图20170322152327.png
>
>
> broker store sendMessageThreadPoolNums=128
> 16 thread 1000bytes benchmark test ， System load to 100
> put message acquire the lock may spin，cause System load high and putMessage cost more than 500ms
> /**
> * Spin util acquired the lock.
> */
> private void lockForPutMessage() {
> if (this.defaultMessageStore.getMessageStoreConfig().isUseReentrantLockWhenPutMessage()) {
> putMessageNormalLock.lock();
> } else {
> boolean flag;
> do {
> flag = this.putMessageSpinLock.compareAndSet(true, false);
> }
> while (!flag);
> }
> }
> 2017-03-22 14:40:16 WARN SendMessageThread_101 - putMessage not in lock eclipse time(ms)=1030, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_34 - putMessage not in lock eclipse time(ms)=1932, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_3 - putMessage not in lock eclipse time(ms)=581, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_57 - putMessage not in lock eclipse time(ms)=583, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_123 - putMessage not in lock eclipse time(ms)=2225, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_1 - putMessage not in lock eclipse time(ms)=1642, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_58 - putMessage not in lock eclipse time(ms)=587, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_19 - putMessage not in lock eclipse time(ms)=1369, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_32 - putMessage not in lock eclipse time(ms)=1896, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_2 - putMessage not in lock eclipse time(ms)=1018, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_122 - putMessage not in lock eclipse time(ms)=657, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_120 - putMessage not in lock eclipse time(ms)=592, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_88 - putMessage not in lock eclipse time(ms)=559, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_44 - putMessage not in lock eclipse time(ms)=892, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_79 - putMessage not in lock eclipse time(ms)=699, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_84 - putMessage not in lock eclipse time(ms)=616, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_112 - putMessage not in lock eclipse time(ms)=515, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_59 - putMessage not in lock eclipse time(ms)=1301, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_96 - putMessage not in lock eclipse time(ms)=635, bodyLength=10013
> 2017-03-22 14:40:16 WARN SendMessageThread_90 - ********lockForPutMessage cost:317
> 2017-03-22 14:40:16 WARN SendMessageThread_10 - ********lockForPutMessage cost:447
> 2017-03-22 14:40:16 WARN SendMessageThread_62 - ********lockForPutMessage cost:450
> 2017-03-22 14:40:16 WARN SendMessageThread_93 - ********lockForPutMessage cost:131
> 2017-03-22 14:40:16 WARN SendMessageThread_8 - ********lockForPutMessage cost:171
> 2017-03-22 14:40:16 WARN SendMessageThread_93 - ********lockForPutMessage cost:4
> 2017-03-22 14:40:16 WARN SendMessageThread_105 - ********lockForPutMessage cost:666
> 2017-03-22 14:40:16 WARN SendMessageThread_127 - ********lockForPutMessage cost:689
> 2017-03-22 14:40:16 WARN SendMessageThread_30 - ********lockForPutMessage cost:341
> 2017-03-22 14:40:16 WARN SendMessageThread_70 - ********lockForPutMessage cost:317
> 2017-03-22 14:40:16 WARN SendMessageThread_92 - ********lockForPutMessage cost:58
> 2017-03-22 14:40:16 WARN SendMessageThread_63 - ********lockForPutMessage cost:261
> 2017-03-22 14:40:16 WARN SendMessageThread_98 - ********lockForPutMessage cost:261
> 2017-03-22 14:40:16 WARN SendMessageThread_27 - ********lockForPutMessage cost:546
> 2017-03-22 14:40:16 WARN SendMessageThread_53 - ********lockForPutMessage cost:257
> 2017-03-22 14:40:16 WARN SendMessageThread_115 - ********lockForPutMessage cost:1047
> 2017-03-22 14:40:16 WARN SendMessageThread_82 - ********lockForPutMessage cost:1891
> 2017-03-22 14:40:16 WARN SendMessageThread_57 - ********lockForPutMessage cost:559
> 2017-03-22 14:40:16 WARN SendMessageThread_40 - ********lockForPutMessage cost:235
> 2017-03-22 14:40:16 WARN SendMessageThread_77 - ********lockForPutMessage cost:265
> 2017-03-22 14:40:16 WARN SendMessageThread_35 - ********lockForPutMessage cost:909
> 2017-03-22 14:40:16 WARN SendMessageThread_113 - ********lockForPutMessage cost:244
> 2017-03-22 14:40:16 WARN SendMessageThread_97 - ********lockForPutMessage cost:922
> 2017-03-22 14:40:16 WARN SendMessageThread_114 - ********lockForPutMessage cost:40
> 2017-03-22 14:40:16 WARN SendMessageThread_43 - ********lockForPutMessage cost:351
> 2017-03-22 14:40:16 WARN SendMessageThread_126 - ********lockForPutMessage cost:346
> 2017-03-22 14:40:16 WARN SendMessageThread_110 - ********lockForPutMessage cost:887
> 2017-03-22 14:40:16 WARN SendMessageThread_15 - ********lockForPutMessage cost:320
> 2017-03-22 14:40:16 WARN SendMessageThread_3 - ********lockForPutMessage cost:566
> 2017-03-22 14:40:16 WARN SendMessageThread_68 - ********lockForPutMessage cost:748
> 2017-03-22 14:40:16 WARN SendMessageThread_22 - ********lockForPutMessage cost:308
> 2017-03-22 14:40:16 WARN SendMessageThread_34 - ********lockForPutMessage cost:1926
> 2017-03-22 14:40:16 WARN SendMessageThread_91 - ********lockForPutMessage cost:358
> 2017-03-22 14:40:16 WARN SendMessageThread_25 - ********lockForPutMessage cost:107
> 2017-03-22 14:40:16 WARN SendMessageThread_101 - ********lockForPutMessage cost:1024
> 2017-03-22 14:40:16 WARN SendMessageThread_89 - ********lockForPutMessage cost:326
> 2017-03-22 14:40:16 WARN SendMessageThread_43 - ********lockForPutMessage cost:5
> 2017-03-22 14:40:16 WARN SendMessageThread_30 - ********lockForPutMessage cost:1
> 2017-03-22 14:40:16 WARN SendMessageThread_42 - ********lockForPutMessage cost:326
> 2017-03-22 14:40:16 WARN SendMessageThread_95 - ********lockForPutMessage cost:58
> 2017-03-22 14:40:16 WARN SendMessageThread_19 - ********lockForPutMessage cost:1363
> 2017-03-22 14:40:16 WARN SendMessageThread_86 - ********lockForPutMessage cost:92
> 2017-03-22 14:40:16 WARN SendMessageThread_1 - ********lockForPutMessage cost:1640
> 2017-03-22 14:40:16 WARN SendMessageThread_82 - ********lockForPutMessage cost:30
> 2017-03-22 14:40:16 WARN SendMessageThread_115 - ********lockForPutMessage cost:33
> 2017-03-22 14:40:16 WARN SendMessageThread_58 - ********lockForPutMessage cost:582
> 2017-03-22 14:40:16 WARN SendMessageThread_53 - ********lockForPutMessage cost:34
> 2017-03-22 14:40:16 WARN SendMessageThread_123 - ********lockForPutMessage cost:2224
> 2017-03-22 14:40:16 WARN SendMessageThread_78 - ********lockForPutMessage cost:286
> 2017-03-22 14:40:16 WARN SendMessageThread_63 - ********lockForPutMessage cost:27
> 2017-03-22 14:40:16 WARN SendMessageThread_32 - ********lockForPutMessage cost:1874
> 2017-03-22 14:40:16 WARN SendMessageThread_87 - ********lockForPutMessage cost:312
> 2017-03-22 14:40:16 WARN SendMessageThread_53 - ********lockForPutMessage cost:14
> 2017-03-22 14:40:16 WARN SendMessageThread_123 - ********lockForPutMessage cost:21
> 2017-03-22 14:40:16 WARN SendMessageThread_45 - ********lockForPutMessage cost:382
> 2017-03-22 14:40:16 WARN SendMessageThread_89 - ********lockForPutMessage cost:37
> 2017-03-22 14:40:16 WARN SendMessageThread_15 - ********lockForPutMessage cost:51
> 2017-03-22 14:40:16 WARN SendMessageThread_2 - ********lockForPutMessage cost:1015
> 2017-03-22 14:40:16 WARN SendMessageThread_68 - ********lockForPutMessage cost:50
> 2017-03-22 14:40:16 WARN SendMessageThread_22 - ********lockForPutMessage cost:51
> 2017-03-22 14:40:16 WARN SendMessageThread_84 - ********lockForPutMessage cost:606
> 2017-03-22 14:40:16 WARN SendMessageThread_3 - ********lockForPutMessage cost:42
> 2017-03-22 14:40:16 WARN SendMessageThread_122 - ********lockForPutMessage cost:652
> 2017-03-22 14:40:16 WARN SendMessageThread_115 - ********lockForPutMessage cost:19
> 2017-03-22 14:40:16 WARN SendMessageThread_126 - ********lockForPutMessage cost:24
> 2017-03-22 14:40:16 WARN SendMessageThread_57 - ********lockForPutMessage cost:3
> 2017-03-22 14:40:16 WARN SendMessageThread_110 - ********lockForPutMessage cost:3
> 2017-03-22 14:40:16 WARN SendMessageThread_70 - ********lockForPutMessage cost:40
> 2017-03-22 14:40:16 WARN SendMessageThread_118 - ********lockForPutMessage cost:279
> 2017-03-22 14:40:16 WARN SendMessageThread_103 - ********lockForPutMessage cost:301
> 2017-03-22 14:40:16 WARN SendMessageThread_36 - ********lockForPutMessage cost:384
> 2017-03-22 14:40:16 WARN SendMessageThread_120 - ********lockForPutMessage cost:587
> 2017-03-22 14:40:16 WARN SendMessageThread_88 - ********lockForPutMessage cost:554
> 2017-03-22 14:40:16 WARN SendMessageThread_124 - ********lockForPutMessage cost:324
> 2017-03-22 14:40:16 WARN SendMessageThread_44 - ********lockForPutMessage cost:888
> 2017-03-22 14:40:16 WARN SendMessageThread_49 - ********lockForPutMessage cost:311
> 2017-03-22 14:40:16 WARN SendMessageThread_41 - ********lockForPutMessage cost:312
> 2017-03-22 14:40:16 WARN SendMessageThread_5 - ********lockForPutMessage cost:115
> 2017-03-22 14:40:16 WARN SendMessageThread_4 - ********lockForPutMessage cost:151
> 2017-03-22 14:40:16 WARN SendMessageThread_119 - ********lockForPutMessage cost:79
> 2017-03-22 14:40:16 WARN SendMessageThread_65 - ********lockForPutMessage cost:107
> 2017-03-22 14:40:16 WARN SendMessageThread_128 - ********lockForPutMessage cost:112
> 2017-03-22 14:40:16 WARN SendMessageThread_96 - ********lockForPutMessage cost:609
> 2017-03-22 14:40:16 WARN SendMessageThread_79 - ********lockForPutMessage cost:697
> 2017-03-22 14:40:16 WARN SendMessageThread_7 - ********lockForPutMessage cost:417
> 2017-03-22 14:40:16 WARN SendMessageThread_94 - ********lockForPutMessage cost:419
> 2017-03-22 14:40:16 WARN SendMessageThread_72 - ********lockForPutMessage cost:208
> 2017-03-22 14:40:16 WARN SendMessageThread_8 - ********lockForPutMessage cost:42
> 2017-03-22 14:40:16 WARN SendMessageThread_99 - ********lockForPutMessage cost:237
> 2017-03-22 14:40:16 WARN SendMessageThread_16 - ********lockForPutMessage cost:123
> 2017-03-22 14:40:16 WARN SendMessageThread_85 - ********lockForPutMessage cost:133
> 2017-03-22 14:40:16 WARN SendMessageThread_111 - ********lockForPutMessage cost:139
> 2017-03-22 14:40:16 WARN SendMessageThread_112 - ********lockForPutMessage cost:510
> 2017-03-22 14:40:16 WARN SendMessageThread_45 - ********lockForPutMessage cost:2
> 2017-03-22 14:40:16 WARN SendMessageThread_2 - ********lockForPutMessage cost:3
> 2017-03-22 14:40:16 WARN SendMessageThread_59 - ********lockForPutMessage cost:1291
> 2017-03-22 14:40:16 WARN SendMessageThread_64 - ********lockForPutMessage cost:105
> 2017-03-22 14:40:16 WARN SendMessageThread_59 - ********lockForPutMessage cost:48
> 2017-03-22 14:40:16 WARN SendMessageThread_46 - ********lockForPutMessage cost:255
> 2017-03-22 14:40:16 WARN SendMessageThread_49 - ********lockForPutMessage cost:95
> 2017-03-22 14:40:16 WARN SendMessageThread_44 - ********lockForPutMessage cost:7
> 2017-03-22 14:40:16 WARN SendMessageThread_112 - ********lockForPutMessage cost:37
> 2017-03-22 14:40:16 WARN SendMessageThread_7 - ********lockForPutMessage cost:75
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)