You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/02/24 15:02:00 UTC
[GitHub] [pulsar] MonicaMagoniCom opened a new issue #9699: Key_Shared subcription not working correctly
MonicaMagoniCom opened a new issue #9699:
URL: https://github.com/apache/pulsar/issues/9699
We are using Pulsar in our cloud environment (deployed with Kubernetes).
Pulsar is configured in **key_shared** mode subscription and it happens that when the traffic is higher than usual (in our case, Pulsar suddendly receives 300 messages), messages which have the same key are mapped to a different consumers.
This lasts for the duration of the traffic peak, than the situation is restored and the messages with the same key are normally deliver to their right consumer. In our environment this causes an Optimistic Lock Exception, since 2 messages with the same key arrives at 2 different consumers, and they both try to update the same document in MongoDB.
**To Reproduce**
Pulsar 2.6.2
Key_shared mode
Sudden peak of traffic
**Expected behavior**
I expect that messages with the same key are delivered to the same consumer, also when traffic increases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ilpanich commented on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
ilpanich commented on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-789026320
Hi @codelipenghui.
Is your suggestion to enable producer batching mandatory in order to consume batch?
We are only using batch consumers, our producers still work in standard mode.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] MonicaMagoniCom closed issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
MonicaMagoniCom closed issue #9699:
URL: https://github.com/apache/pulsar/issues/9699
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ilpanich edited a comment on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
ilpanich edited a comment on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-789026320
Hi @codelipenghui.
Is your suggestion to enable producer batching mandatory in order to consume batch?
We are only using batch consumers, our producers still work in standard mode.
As I stated in one of my previous message, please consider also that we have 3 pulsar producer receving messages from our sources, and we can't guarantee that the same source will always send its messages to the same pulsar producer (i.e.: both producer1 and producer2 may receive a message from source1 in a short time range).
In our case, using pulsar batch producer can be quite tricky.
Also, we have not found using prducer batching as a requiremnt for consumer batching in the docs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ilpanich commented on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
ilpanich commented on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-828273987
Hi @codelipenghui, we have checked and resolved as you suggested.
You can close this issue, there are no further actions needed.
Thank you for your support..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-789781833
@ilpanich The producer enabled the batching by default. You can print the message ID that the consumer received to check if the message is a batch message. If it is a batch message, you will see the batch index, if not, the batch index will always -1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ilpanich edited a comment on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
ilpanich edited a comment on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-789026320
Hi @codelipenghui.
Is your suggestion to enable producer batching mandatory in order to consume batch?
We are only using batch consumers, our producers still work in standard mode.
As I stated in one of my previous message, please consider also that we have 3 pulsar producer receving messages from our sources, and we can't guarantee that the same source will always send its messages to the same pulsar producer (i.e.: both producer1 and producer2 may receive a message from source1 in a short time range)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-789022113
@ilpanich If you want to enable producer batching and want to consume by the key_shared subscription, you can set the `batcherBuilder(BatcherBuilder.KEY_BASED)`. So that the producer will combine messages of the same key in a batch to send.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ilpanich commented on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
ilpanich commented on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-789532421
@codelipenghui There is a misunderstanding.
As I stated above, we use batching only on consumers, there is no batching enabled on the producers side.
So I suppose we already use option 1.
The consumers still receive messages from different keys in the batch only during the peak.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] MarvinCai commented on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
MarvinCai commented on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-785472992
hi, what to confirm is any consumer added during the peak?
Cause if new consumer is added default behavior will cause rehash and message with same key can then be sent to different consumer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] asadsalman commented on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
asadsalman commented on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-788760568
Ah okay, thanks for the additional details. I only had one producer and I didn't add batching. I'll try with that.
Moreover, were the messages persistent or non-persistent? I don't know if it makes a difference, just trying to match your setup as closely as possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] MonicaMagoniCom edited a comment on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
MonicaMagoniCom edited a comment on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-785686109
@MarvinCai Hi, actually not, we are not creating any new consumer during the peak. That's why I do not expect any rehashing process to be performed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-789415865
@ilpanich The producer enabled the default batch policy based on the time(1ms for a batch), so this will lead to the messages with different keys add to one batch and send to the broker. So after the batch dispatched to the consumer, the consumer might receive the different keys of the batch.
There are 2 options to get it to work
1. Disable the message batching on the producer side
2. Use the KEY_BASE batching on the producer side, since the batch message of the KEY_BASED batcher only contains the messages with the same key.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] MonicaMagoniCom commented on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
MonicaMagoniCom commented on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-785686109
@MarvinCai Hi, actually not, we are not creating any new consumer during the peak. That's why I do not expect any reshashing process to be performed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ilpanich commented on issue #9699: Key_Shared subcription not working correctly
Posted by GitBox <gi...@apache.org>.
ilpanich commented on issue #9699:
URL: https://github.com/apache/pulsar/issues/9699#issuecomment-788735453
Hi @MarvinCai.
Any news about this issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org