[GitHub] [pulsar-client-node] Matt-Esch opened a new issue #192: Producer send order with async workers

[GitBox <gi...@apache.org>]

Matt-Esch opened a new issue #192:
URL: https://github.com/apache/pulsar-client-node/issues/192


   The client implementation breaks the expected node semantics due to the internal use of async workers.
   
   Essentially, sending 3 messages followed by awaiting flush should flush all 3 messages in order.
   
   ```
   producer.send(...message1);
   producer.send(...message2);
   producer.send(...message3);
   await producer.flush();
   ```
   
   At the moment the send commands invoke in no specific order as they are all executed in an async worker. This breaks the order and what gets flushed in the flush call is non-deterministic.
   
   The obvious workaround would be to await each send before sending the next, but this is clearly problematic for client throughput and it only works when being invoked inside an async function.  I am sending messages where I can't wait for the send to execute before sending the next message, so it needs to synchronously queue up the message in order.
   
   There is an existing ticket to move to the async function API https://github.com/apache/pulsar-client-node/issues/14 but I think that this probably needs to be fixed in the context of the async worker pattern as I'm assuming that overhauling this away from async workers is too much work.
   
   I may have to implement a workaround by wrapping the producer in a js function that does the ordering under the hood but it would be good have sensible defaults in the client implementation.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org