[jira] [Comment Edited] (ARTEMIS-1755) Delete message from duplicate cache on message consume

["Rini Bollarapu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARTEMIS-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406771#comment-16406771 ] 

Rini Bollarapu edited comment on ARTEMIS-1755 at 3/20/18 6:14 PM:
------------------------------------------------------------------

This is what's causing your problem, and in my opinion is a misuse/misunderstanding of the duplicate ID. You're only supposed to use the same duplicate ID on messages which are *exactly* the same so that you avoid duplication of the same data. In your case the messages are different so you can use a different duplicate ID on each.

-I want to use the same ID because they are exactly the same message. The only problem is the first message was consumed already, and the second message came in after sometime and it's no longer a duplicate because it's not in the queue anymore but still detected as duplicate because it's in the cache.

I recommend you use duplicate detection as it was designed to be used, and I don't think you'll have this problem. At this point, I don't see a reason to change how the current duplication detection functionality is implemented.

-I agree that duplicate detection design doesn't support this use case. But wouldn't being able to configure it to delete ID from the cache on delete/consume from the queue make it more flexible? Because an ID being present in duplicate cache makes an incoming message with same ID be detected as duplicate with respect to the cache, but that message may be consumed from the queue already. 

If this feature is not going to be supported due to design issues, maybe I can make some changes to the source code and use that.

 


was (Author: rbollarapu):
This is what's causing your problem, and in my opinion is a misuse/misunderstanding of the duplicate ID. You're only supposed to use the same duplicate ID on messages which are *exactly* the same so that you avoid duplication of the same data. In your case the messages are different so you can use a different duplicate ID on each.

-I want to use the same ID because they are exactly the same message. The only problem is the first message was consumed already, and the second message came in after sometime and it's no longer a duplicate because it's not in the queue anymore but still detected as duplicate because it's in the cache.

I recommend you use duplicate detection as it was designed to be used, and I don't think you'll have this problem. At this point, I don't see a reason to change how the current duplication detection functionality is implemented.

-I agree that duplicate detection design doesn't support this use case. But wouldn't being able to configure it to delete ID from the cache on delete/consume from the queue make it more flexible? Because an ID being present in duplicate cache makes an incoming message with same ID be detected as duplicate with respect to the cache, but that message may be consumed from the queue already. 

If this feature is not going to be supported due to design issues, I can make some changes to the source code and use that.

 

> Delete message from duplicate cache on message consume
> ------------------------------------------------------
>
>                 Key: ARTEMIS-1755
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-1755
>             Project: ActiveMQ Artemis
>          Issue Type: Improvement
>          Components: Broker
>    Affects Versions: 2.4.0
>            Reporter: Rini Bollarapu
>            Priority: Minor
>              Labels: newbie
>
> Hi, duplicate cache still contains the message even if the message was consumed from the queue or deleted from the queue. From the documentation, it looks like there's no configuration to handle something like this. 
> Example:
> A message containing a person's information is sent to a queue and consumed. After sometime, the person's address is updated and the update message is sent to queue. This updated message is ignored as a duplicate because the original message was not removed from the duplicate cache when it was consumed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)