You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/04/14 13:43:45 UTC
[GitHub] [pulsar] huangydyn opened a new issue, #15176: [Doc] Question on Flow Control
huangydyn opened a new issue, #15176:
URL: https://github.com/apache/pulsar/issues/15176
### What issue do you find in Pulsar docs?
It's said that consumer send Flow command to ask give permission to the broker to push messages.
My Question is that how consumer send flow command to broker? Is this a long polling or short polling?
refer doc [Pulsar binary protocol specification](https://pulsar.apache.org/docs/en/develop-binary-protocol/#consumer)
### What is your suggestion?
As we know, both kafka and rocketmq use long polling to pull message, and pulsar use push and pull.
Could this be explained in the doc why pulsar use push and pull and why it is better then other implementation?
My understand is that
- Flow command is not long polling
- Kafka and RocketMq use long polling which caused more network consumption, and Repeat consumption might be introduced
- Push/Pull is used in Pulsar to relieve network consumption and client consuming pressure
### Do you have any references?
https://pulsar.apache.org/docs/en/develop-binary-protocol/#consumer
### Would you like to fix this issue?
yes, i'm will to ^^
### Note
- [X] I have researched my question.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on issue #15176: [Doc] Question on Flow Control
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on issue #15176:
URL: https://github.com/apache/pulsar/issues/15176#issuecomment-1099837243
# At case `message receive`, Pulsar is not same as RoccketMQ
Pulsar implements like this:
- Client has a message queue( default max-len is 1000 ) cache at local memory, like this:
`[ ] [ ] [ ] [ ] [ ]`
- After consumer registry, broker will push message to consumer, then consumer will cache this message.
`[ * ] [ * ] [ * ] [ * ] [ * ]`
- When execute `consumer.receive()`, client will pop message from local queue
This is completely asynchronous communication.
If this solves your problem, please close this issue. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on issue #15176: [Doc] Question on Flow Control
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on issue #15176:
URL: https://github.com/apache/pulsar/issues/15176#issuecomment-1099839259
> Is this a long polling or short polling?
I think it should not be referred as "polling". In Pulsar, there is a separated channel that receives messages from broker. The client only sends the FLOW command to indicate that it needs more messages.
For example, after a consumer whose receiver queue size is 1000 is created, it will send a FLOW request whose permits is 500, i.e. "I can cache 500 messages". Then how will broker dispatch messages is determined by the implementation of the dispatcher. Another common case is that when a message is received, the client will check the internal available permits and send a FLOW request whose permits is 1 if necessary, i.e. "my receiver queue has over 500 available spots, please send messages".
> Repeat consumption might be introduced
I think repeated consumption is not related to polling. Let's talk about Kafka, when a consumer sends multiple FETCH requests, the broker can process them in order so that FETCH responses won't return overlapped messages.
The repeated consumption happens only when an offset is not committed correctly while the consumer has already processed it. For example,
1. Consumer fetched "msg-0" and processed it.
2. Consumer committed offset 0.
3. There was a network issue and consumer restarted
4. Consumer restarted:
- If the offset has been committed successfully, the next message will be "msg-1".
- Otherwise, the next message will be "msg-0".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] huangydyn commented on issue #15176: [Doc] Question on Flow Control
Posted by GitBox <gi...@apache.org>.
huangydyn commented on issue #15176:
URL: https://github.com/apache/pulsar/issues/15176#issuecomment-1099846671
> BTW, I've written a [Chinese blog](https://bewaremypower.github.io/2021/09/02/Pulsar-%E6%BA%90%E7%A0%81%E9%98%85%E8%AF%BB-%E6%B6%88%E8%B4%B9%E8%80%85%E7%9A%84%E8%AE%A2%E9%98%85/) that might be related to this issue. You can take a look.
Learned a lot. Many Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] huangydyn closed issue #15176: [Doc] Question on Flow Control
Posted by GitBox <gi...@apache.org>.
huangydyn closed issue #15176: [Doc] Question on Flow Control
URL: https://github.com/apache/pulsar/issues/15176
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on issue #15176: [Doc] Question on Flow Control
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on issue #15176:
URL: https://github.com/apache/pulsar/issues/15176#issuecomment-1099840058
BTW, I've written a [Chinese blog](https://bewaremypower.github.io/2021/09/02/Pulsar-%E6%BA%90%E7%A0%81%E9%98%85%E8%AF%BB-%E6%B6%88%E8%B4%B9%E8%80%85%E7%9A%84%E8%AE%A2%E9%98%85/) that might be related to this issue. You can take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org