[GitHub] [pulsar-client-go] cckellogg commented on issue #164: Discusses the logic of the current connection pool implementation

[GitBox <gi...@apache.org>]

cckellogg commented on issue #164: Discusses the logic of the current connection pool implementation
URL: https://github.com/apache/pulsar-client-go/issues/164#issuecomment-572198622
 
 
   @wolfstudy 
   
   The design of the go client is a little different from the java client. The job of the dispatcher is to ensure the receiver queue channel is never blocked so we can reuse client connections.
   The dispatcher can ensure this because it will only ask the broker for more messages (send permits) when there is room on the queue channel. This should work because 
   the queue channel accepts a slice of messages. In java the receiver queue is a list of single messages and that’s the reason the java client has to expand its receiver queue.
   
   For example:
   
   Let’s say `receiveQueueSize=10`
   
   Now we can create a consumer and we will ask for 10 messages from the broker. Let’s say the 10 messages we get back from the broker are all single messages.
   So our receiver queue channel will look like this:
   
   ```
   m1 | m2 | m3 | m4 | m5 | m6 | m7 | m8 | m9 | m10
   ```
   
   Let’s receive/process 5 message so the queue channel will look like this now:
   
   ```
   m6 | m7 | m8 | m9 | m10 | empty | empty | empty | empty | empty
   ```
   
   The dispatcher will ask for 5 new messages. Let’s say the next message is a batch of 10 messages so the broker will send that batch message to the consumer. The client will 
   have 15 message in the receiver queue after getting that batch message from the broker. After receiving the batch the queue channel will look like this:
   
   ```
   m6 | m7 | m8 | m9 | m10 | m11-m20 | empty | empty | empty | empty
   ```
   
   Since the queue channel is a slice of messages all of the messages will fit in the queue channel and we didn’t need to expand it. That batch of 10 messages will only occupy one slot in the queue channel. The dispatcher will have to process 10 messages (The current flow control asks for messages when we fall below receiveQueueSize/2)  before will will ask for more messages from the broker and at that point there will still be space in the queue.
   
   The queue channel at that point will be:
   
   ```
   m16-m20 | empty | empty | empty | empty | empty | empty | empty | empty | empty
   ```
   
   
   In java this is different because the queue stores single messages so it would have to expand its queue to fit all the messages. 
   
   Does this design make sense?  Thoughts?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services