You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/12/18 10:50:05 UTC
[GitHub] [rocketmq-client-cpp] fengxiao2019 opened a new issue #392: 收到重复消息
fengxiao2019 opened a new issue #392:
URL: https://github.com/apache/rocketmq-client-cpp/issues/392
消息ID： 0A64F892048A1B9871B45B0C6972005C
[2021-Dec-18 16:19:00.037053](info):processResponseCommand, code:11, opaque:1028, maxRetryTimes:1, retrySendTimes:1[processResponseCommand:520]
[2021-Dec-18 16:19:00.037081](info):cancelTimerCallback: opaque:1028[cancelTimerCallback:625]
[2021-Dec-18 16:19:00.037153](info):handleAsyncRequestTimeout aborted opaque:1028, e_code:125, msg:操作已取消[handleAsyncRequestTimeout:538]
[2021-Dec-18 16:19:00.037315](info):incConsumeRT before Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:13608, Count: 1[incConsumeRT:103]
[2021-Dec-18 16:19:00.037352](info):incConsumeRT After Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:13608, Count: 1[incConsumeRT:113]
[2021-Dec-18 16:19:00.037385](info):PullRequest: putMessage m_queueOffsetMax:3025 [putMessage:66]
[2021-Dec-18 16:19:00.037470](info):FOUND:MessageQueue [topic=zhujiuyin_async_topic, brokerName=RaftNode04, queueId=4] with size:1, nextBeginOffset:3026[onSuccess:88]
[2021-Dec-18 16:19:00.037159](info):code:0, language:JAVA, version:393, opaque:1027, flag:1, remark:FOUND, headLen:219, bodyLen:338 [Decode:177]
[2021-Dec-18 16:19:00.037514](info):=====Receive Messages,Topic[zhujiuyin_async_topic], MsgId[0A64F892048A1B9871B45B0C6972005C],Body[{"app_name": "server", "script_name": "crontab_check_rocketmq_lag", "param": null, "log_tag": null}],RetryTimes[0][ConsumeRequest:179]
[2021-Dec-18 16:19:00.037572](info):find_response opaque:1027[ProcessData:507]
[2021-Dec-18 16:19:00.037610](info):processResponseCommand, code:11, opaque:1027, maxRetryTimes:1, retrySendTimes:1[processResponseCommand:520]
[2021-Dec-18 16:19:00.037631](info):cancelTimerCallback: opaque:1027[cancelTimerCallback:625]
[2021-Dec-18 16:19:00.037685](info):code:0, language:JAVA, version:393, opaque:1029, flag:1, remark:FOUND, headLen:219, bodyLen:338 [Decode:177]
[2021-Dec-18 16:19:00.037717](info):find_response opaque:1029[ProcessData:507]
[2021-Dec-18 16:19:00.037739](info):processResponseCommand, code:11, opaque:1029, maxRetryTimes:1, retrySendTimes:1[processResponseCommand:520]
[2021-Dec-18 16:19:00.037757](info):cancelTimerCallback: opaque:1029[cancelTimerCallback:625]
[2021-Dec-18 16:19:00.037862](info):incConsumeRT before Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:13608, Count: 1[incConsumeRT:103]
[2021-Dec-18 16:19:00.037897](info):incConsumeRT After Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:27216, Count: 2[incConsumeRT:113]
[2021-Dec-18 16:19:00.037931](info):PullRequest: putMessage m_queueOffsetMax:3025 [putMessage:66]
[2021-Dec-18 16:19:00.037984](info):FOUND:MessageQueue [topic=zhujiuyin_async_topic, brokerName=RaftNode04, queueId=4] with size:1, nextBeginOffset:3026[onSuccess:88]
[2021-Dec-18 16:19:00.038067](info):incConsumeRT before Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:13609, Count: 1[incConsumeRT:103]
[2021-Dec-18 16:19:00.038102](info):incConsumeRT After Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:40825, Count: 3[incConsumeRT:113]
[2021-Dec-18 16:19:00.038126](info):PullRequest: putMessage m_queueOffsetMax:3025 [putMessage:66]
[2021-Dec-18 16:19:00.038163](info):FOUND:MessageQueue [topic=zhujiuyin_async_topic, brokerName=RaftNode04, queueId=4] with size:1, nextBeginOffset:3026[onSuccess:88]
[2021-Dec-18 16:19:00.038242](info):handleAsyncRequestTimeout aborted opaque:1027, e_code:125, msg:操作已取消[handleAsyncRequestTimeout:538]
[2021-Dec-18 16:19:00.038274](info):handleAsyncRequestTimeout aborted opaque:1029, e_code:125, msg:操作已取消[handleAsyncRequestTimeout:538]
[2021-Dec-18 16:19:00.038413](info):before insert declared filed,MAP SIZE is:2[doBeforeRequest:43]
[2021-Dec-18 16:19:00.038454](info):after insert declared filed, MAP SIZE is:12[doBeforeRequest:47]
[2021-Dec-18 16:19:00.038473](info):total msg info are:qadeptRMQ3025zhujiuyin_async_consumer32430261639814925133*150007zhujiuyin_async_topic, size is:75[doBeforeRequest:63]
[2021-Dec-18 16:19:00.038643](info):invokeAsync success, addr:10.100.27.18:50911, code:11, opaque:1037[invokeAsync:265]
[2021-Dec-18 16:19:00.038715](info):before insert declared filed,MAP SIZE is:2[doBeforeRequest:43]
[2021-Dec-18 16:19:00.038748](info):after insert declared filed, MAP SIZE is:12[doBeforeRequest:47]
[2021-Dec-18 16:19:00.038768](info):total msg info are:qadeptRMQ3025zhujiuyin_async_consumer32430261639814925133*150007zhujiuyin_async_topic, size is:75[doBeforeRequest:63]
[2021-Dec-18 16:19:00.038870](info):invokeAsync success, addr:10.100.27.18:50911, code:11, opaque:1038[invokeAsync:265]
[2021-Dec-18 16:19:00.038934](info):before insert declared filed,MAP SIZE is:2[doBeforeRequest:43]
[2021-Dec-18 16:19:00.038978](info):after insert declared filed, MAP SIZE is:12[doBeforeRequest:47]
[2021-Dec-18 16:19:00.039006](info):total msg info are:qadeptRMQ3025zhujiuyin_async_consumer32430261639814925133*150007zhujiuyin_async_topic, size is:75[doBeforeRequest:63]
[2021-Dec-18 16:19:00.039151](info):invokeAsync success, addr:10.100.27.18:50911, code:11, opaque:1039[invokeAsync:265]
[2021-Dec-18 16:19:00.079686](info):incConsumeRT before Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:42, Count: 1[incConsumeRT:103]
[2021-Dec-18 16:19:00.079877](info):incConsumeRT After Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:40867, Count: 4[incConsumeRT:113]
[2021-Dec-18 16:19:00.079912](info):incConsumeOKTPS Before Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, Count: 1[incConsumeOKTPS:117]
[2021-Dec-18 16:19:00.079931](info):incConsumeOKTPS After Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, Count: 1[incConsumeOKTPS:124]
[2021-Dec-18 16:19:00.079950](info):m_queueOffsetMax is:3025[removeMessage:111]
[2021-Dec-18 16:19:00.079979](info): offset result is:3026, m_queueOffsetMax is:3025, msgs size:1[removeMessage:115]
[2021-Dec-18 16:19:00.079998](info):remove these msg from m_msgTreeMap, its offset:3025[removeMessage:118]
[2021-Dec-18 16:19:00.080093](info):=====Receive Messages,Topic[zhujiuyin_async_topic], MsgId[0A64F892048A1B9871B45B0C6972005C],Body[{"app_name": "server", "script_name": "crontab_check_rocketmq_lag", "param": null, "log_tag": null}],RetryTimes[0][ConsumeRequest:179]
[2021-Dec-18 16:19:00.082514](info):incConsumeRT before Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:2, Count: 1[incConsumeRT:103]
[2021-Dec-18 16:19:00.082592](info):incConsumeRT After Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:40869, Count: 5[incConsumeRT:113]
[2021-Dec-18 16:19:00.082613](info):incConsumeOKTPS Before Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, Count: 1[incConsumeOKTPS:117]
[2021-Dec-18 16:19:00.082630](info):incConsumeOKTPS After Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, Count: 2[incConsumeOKTPS:124]
[2021-Dec-18 16:19:00.082647](info):m_queueOffsetMax is:3025[removeMessage:111]
[2021-Dec-18 16:19:00.082668](warning):Note: Get local offset for mq:MessageQueue [topic=zhujiuyin_async_topic, brokerName=RaftNode04, queueId=4] failed, may be it is updated before. skip..[ConsumeRequest:316]
[2021-Dec-18 16:19:00.082717](info):=====Receive Messages,Topic[zhujiuyin_async_topic], MsgId[0A64F892048A1B9871B45B0C6972005C],Body[{"app_name": "server", "script_name": "crontab_check_rocketmq_lag", "param": null, "log_tag": null}],RetryTimes[0][ConsumeRequest:179]
[2021-Dec-18 16:19:00.084563](info):incConsumeRT before Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:2, Count: 1[incConsumeRT:103]
[2021-Dec-18 16:19:00.084614](info):incConsumeRT After Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, RT:40871, Count: 6[incConsumeRT:113]
[2021-Dec-18 16:19:00.084635](info):incConsumeOKTPS Before Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, Count: 1[incConsumeOKTPS:117]
[2021-Dec-18 16:19:00.084651](info):incConsumeOKTPS After Key:zhujiuyin_async_topic@zhujiuyin_async_consumer, Count: 3[incConsumeOKTPS:124]
[2021-Dec-18 16:19:00.084668](info):m_queueOffsetMax is:3025[removeMessage:111]
[2021-Dec-18 16:19:00.084686](warning):Note: Get local offset for mq:MessageQueue [topic=zhujiuyin_async_topic, brokerName=RaftNode04, queueId=4] failed, may be it is updated before. skip..[ConsumeRequest:316]
[2021-Dec-18 16:19:00.201338](info):Client factory start persistAllConsumerOffset[persistAllConsumerOffset:811]
[2021-Dec-18 16:19:00.201573](info):oneway updateConsumeOffsetToBroker of mq:MessageQueue [topic=zhujiuyin_async_topic, brokerName=RaftNode04, queueId=4], its offset is:3026[updateConsumeOffsetToBroker:310]
[2021-Dec-18 16:19:00.201633](info):before insert declared filed,MAP SIZE is:2[doBeforeRequest:43]
[2021-Dec-18 16:19:00.201668](info):after insert declared filed, MAP SIZE is:6[doBeforeRequest:47]
[2021-Dec-18 16:19:00.201694](info):total msg info are:qadeptRMQ3026zhujiuyin_async_consumer4zhujiuyin_async_topic, size is:49[doBeforeRequest:63]
[2021-Dec-18 16:19:00.201919](info):invokeOneway success. addr:10.100.27.18:50911, code:15[invokeOneway:281]
[2021-Dec-18 16:19:00.202328](info):had received all data. msgLen:117, from:45, recvLen:121[readNextMessageIntCallback:262]
[2021-Dec-18 16:19:00.202429](info):too little data received with sum = 0[readNextMessageIntCallback:254]
[2021-Dec-18 16:19:00.202484](info):code:0, language:JAVA, version:393, opaque:1040, flag:1, remark:, headLen:113, bodyLen:0 [Decode:177]
[2021-Dec-18 16:19:00.202543](info):responseFuture was deleted by timeout of opaque:1040[ProcessData:502]
[2021-Dec-18 16:19:03.164889](info):had received all data. msgLen:291, from:24, recvLen:295[readNextMessageIntCallback:262]
[2021-Dec-18 16:19:03.165150](info):too little data received with sum = 0[readNextMessageIntCallback:254]
[2021-Dec-18 16:19:03.165265](info):code:307, language:JAVA, version:393, opaque:-591779724, flag:0, remark:, headLen:287, bodyLen:0 [Decode:177]
[2021-Dec-18 16:19:03.165430](info):request Command received:processRequest, addr:10.100.22.245:30911, code:307[processRequest:31]
[2021-Dec-18 16:19:03.165469](info):consumerGroup:zhujiuyin_async_consumer, clientId:10.100.65.22@1406#DEFAULT, jstackEnable:0[Decode:624]
[2021-Dec-18 16:19:03.165488](info):getConsumerRunningInfo:zhujiuyin_async_consumer[getConsumerRunningInfo:111]
[2021-Dec-18 16:19:03.165627](info):getConsumeStats Topic:%RETRY%zhujiuyin_async_consumer, Group:zhujiuyin_async_consumer[getConsumeStats:74]
[2021-Dec-18 16:19:03.165650](info):getConsumeStats Key:%RETRY%zhujiuyin_async_consumer@zhujiuyin_async_consumer[getConsumeStats:77]
[2021-Dec-18 16:19:03.165677](info):getConsumeStats Topic:zhujiuyin_async_topic, Group:zhujiuyin_async_consumer[getConsumeStats:74]
[2021-Dec-18 16:19:03.165693](info):getConsumeStats Key:zhujiuyin_async_topic@zhujiuyin_async_consumer[getConsumeStats:77]
[2021-Dec-18 16:19:03.165899](info):before insert declared filed,MAP SIZE is:2[doBeforeRequest:43]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-cpp] fengxiao2019 commented on issue #392: received duplicate messages, branch: master
Posted by GitBox <gi...@apache.org>.
fengxiao2019 commented on issue #392:
URL: https://github.com/apache/rocketmq-client-cpp/issues/392#issuecomment-997595511
I compiled the version with the master branch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-cpp] fengxiao2019 commented on issue #392: received duplicate messages, branch: master
Posted by GitBox <gi...@apache.org>.
fengxiao2019 commented on issue #392:
URL: https://github.com/apache/rocketmq-client-cpp/issues/392#issuecomment-997595071
Multiple threads with the same process will receive messages with the same message_id at the same time (or at many intervals, usually at the millisecond level).
Single process with a single thread: it will receive the same message_id several times in a row after the current message_id has been processed.
This scenario is still quite frequent.
同一个进程多个线程，会在同一时刻（或者间隔很多的时间内，通常是在毫秒级别）收到相同message_id的消息。
单个进程单个线程：会在当前message_id处理完之后，连续收到多次同一个message_id的消息。
这种场景还是挺频繁的。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-cpp] ifplusor commented on issue #392: received duplicate messages, branch: master
Posted by GitBox <gi...@apache.org>.
ifplusor commented on issue #392:
URL: https://github.com/apache/rocketmq-client-cpp/issues/392#issuecomment-997593339
@fengxiao2019 Could you describe your case in simple sentences?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-cpp] ifplusor commented on issue #392: received duplicate messages, branch: master
Posted by GitBox <gi...@apache.org>.
ifplusor commented on issue #392:
URL: https://github.com/apache/rocketmq-client-cpp/issues/392#issuecomment-997597209
@fengxiao2019 Do you operate consumer cluster before receive duplicated messages?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-client-cpp] fengxiao2019 commented on issue #392: received duplicate messages, branch: master
Posted by GitBox <gi...@apache.org>.
fengxiao2019 commented on issue #392:
URL: https://github.com/apache/rocketmq-client-cpp/issues/392#issuecomment-997599313
no, I didn't.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org