You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/04/25 23:37:52 UTC

[GitHub] [pulsar] DR9885 opened a new issue, #15317: Key_Shared Nack causes events to be played out of order for a specific key.

DR9885 opened a new issue, #15317:
URL: https://github.com/apache/pulsar/issues/15317

   **Describe the bug**
   If events are consumed as key_shared, and the first event was nacked it changes the order of the events for that key. It has that event pushed to the end.
   
   This is an issue, when building a materialized view based off a key/id. If the events are played out of order it could change the resulting object.
   
   **To Reproduce**
   C# test example
   ``` 
   public class ExampleTest
   {
       [Fact]
       public async Task Test()
       {
           const int size = 10;
           const string topic = "persistent://public/default/topic";
   
           var client = await new PulsarClientBuilder()
               .ServiceUrl("pulsar://localhost:6650")
               .BuildAsync();
   
           var producer = await client.NewProducer()
               .Topic(topic)
               .BatchBuilder(BatchBuilder.KeyBased)
               .CreateAsync();
   
           for (var i = 0; i < size; i++)
           {
               var payload = Encoding.UTF8.GetBytes($"message-{i}");
               var msg = producer.NewMessage(payload, $"key-{i}");
               await producer.SendAsync(msg);
           }
   
           var subscribe = await client.NewConsumer()
               .Topic(topic)
               .SubscriptionName("test-group")
               .SubscriptionType(SubscriptionType.KeyShared)
               .SubscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
               .SubscribeAsync();
   
           var batch = await subscribe.ReceiveAsync();
           await subscribe.NegativeAcknowledge(batch.MessageId);
   
   
           for (var i = 0; i < size; i++)
           {
               var item = await subscribe.ReceiveAsync();
               Assert.Equal($"key-{i}", item.Key);
           }
       }
   }
   
   ```
   
   **Expected behavior**
   It should wait to further events from that key
   
   **Screenshots**
   If applicable, add screenshots to help explain your problem.
   
   **Desktop (please complete the following information):**
    - OS: Windows 11
   
   **Additional context**
   Note: if retry limit is hit, the messages would never play. so maybe there should also be a way to reset retry counts to try again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] github-actions[bot] commented on issue #15317: Key_Shared Nack causes events to be played out of order for a specific key.

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #15317:
URL: https://github.com/apache/pulsar/issues/15317#issuecomment-1166399272

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] DR9885 closed issue #15317: Key_Shared Nack causes events to be played out of order for a specific key.

Posted by "DR9885 (via GitHub)" <gi...@apache.org>.
DR9885 closed issue #15317: Key_Shared Nack causes events to be played out of order for a specific key.
URL: https://github.com/apache/pulsar/issues/15317


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] DR9885 commented on issue #15317: Key_Shared Nack causes events to be played out of order for a specific key.

Posted by GitBox <gi...@apache.org>.
DR9885 commented on issue #15317:
URL: https://github.com/apache/pulsar/issues/15317#issuecomment-1109507247

   @codelipenghui, but lets say a specific key is failing again and again. If we did this locally, we could block the queue from processing and the subscription could timeout... It would be better if we could use a re-queue mechanism to backlog all subsequent keys... Otherwise I'm just going to throw them in a queue to be reprocessed later... but this should be a pulsar mechanism.
   
   Isn't event order important for event sourcing? and isn't this a stream processing system to support event sourcing?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] github-actions[bot] commented on issue #15317: Key_Shared Nack causes events to be played out of order for a specific key.

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #15317:
URL: https://github.com/apache/pulsar/issues/15317#issuecomment-1139213131

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] codelipenghui commented on issue #15317: Key_Shared Nack causes events to be played out of order for a specific key.

Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #15317:
URL: https://github.com/apache/pulsar/issues/15317#issuecomment-1109299646

   @DR9885 For the Key_Shared subscription, the redelivery operation will cause the message to redeliver to the same consumer again, you can retry the message locally, don't need to call the redelivery method, the redelivery operation will break the message reading order.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org