[jira] [Comment Edited] (ROCKETMQ-332) MappedFileQueue is not thread safe, which will cause message loss.

["Jas0n918 (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ROCKETMQ-332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299508#comment-16299508 ] 

Jas0n918 edited comment on ROCKETMQ-332 at 12/21/17 3:53 AM:
-------------------------------------------------------------

Please check rocketmq.log (including related logs in broker.log store.log  storeerror.log client.log).



was (Author: jas0n918):
including:
broker.log
store.log
storeerror.log
client.log


> MappedFileQueue is not thread safe, which will cause message loss.
> ------------------------------------------------------------------
>
>                 Key: ROCKETMQ-332
>                 URL: https://issues.apache.org/jira/browse/ROCKETMQ-332
>             Project: Apache RocketMQ
>          Issue Type: Bug
>          Components: rocketmq-store
>    Affects Versions: 4.0.0-incubating, 4.1.0-incubating
>            Reporter: Jas0n918
>            Assignee: yukon
>         Attachments: rocketmq.log
>
>
> In RocketMQ V3.5.8, there is a readWriteLock in com.alibaba.rocketmq.store.MapedFileQueue, which guarantee thread safety. But in the new org.apache.rocketmq.store.MappedFileQueue, there is not any concurrent control mechanism. 
> when consumer is fetching message(no large lag), broker calls
> org.apache.rocketmq.broker.processor.PullMessageProcessor#processRequest ==>
> org.apache.rocketmq.store.DefaultMessageStore#getMessage  ==>
> org.apache.rocketmq.store.ConsumeQueue#getIndexBuffer ==>
> org.apache.rocketmq.store.MappedFileQueue#findMappedFileByOffset
> but findMappedFileByOffset is not thread safe, as
> org.apache.rocketmq.store.MappedFileQueue#deleteExpiredFile maybe running concurrently(  the size of mappedFiles maybe change) , which will results in ConsumeQueue#getIndexBuffer returns null, causing 
> _nextBeginOffset  = nextOffsetCorrection(offset, consumeQueue.rollNextFile(offset));_+
> which will skip the whole consumeQueue file, any messages left in this ConsumeQueue will not be consumed by client.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)