You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/06/05 06:41:54 UTC
[GitHub] [pulsar-client-go] LvBay opened a new issue #267: Does consumer's throughput have anything to do with ConsumerOptions.ReceiverQueueSize and ConsumerOptions.MessageChannel and the time required to process each message?
LvBay opened a new issue #267:
URL: https://github.com/apache/pulsar-client-go/issues/267
For example, if the topic has 10 partitions, I need the consumer to achieve a throughput of 1000msg/s, and it takes 10ms to process each message. How can I set the value of ConsumerOptions?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] wolfstudy edited a comment on issue #267: Is there any relationship between consumer throughput and ConsumerOptions.ReceiverQueueSize and ConsumerOptions.MessageChannel and the time required to process each message?
Posted by GitBox <gi...@apache.org>.
wolfstudy edited a comment on issue #267:
URL: https://github.com/apache/pulsar-client-go/issues/267#issuecomment-639340257
- The `MessageChannel` is used to receive messages, all messages received by consumers will be filled into MessageChannel, the code example as follows:
```
channel := make(chan pulsar.ConsumerMessage, 100)
options := pulsar.ConsumerOptions{
Topic: "topic-1",
SubscriptionName: "my-subscription",
Type: pulsar.Shared,
}
options.MessageChannel = channel
consumer, err := client.Subscribe(options)
if err != nil {
log.Fatal(err)
}
defer consumer.Close()
// Receive messages from channel. The channel returns a struct which contains message and the consumer from where
// the message was received. It's not necessary here since we have 1 single consumer, but the channel could be
// shared across multiple consumers as well
for cm := range channel {
msg := cm.Message
fmt.Printf("Received message msgId: %v -- content: '%s'\n",
msg.ID(), string(msg.Payload()))
consumer.Ack(msg)
}
```
- The `ReceiverQueueSize` sets the size of the consumer to receive queue. It is a buffer of a channel in go language. Default value is 1000
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] LvBay closed issue #267: Is there any relationship between consumer throughput and ConsumerOptions.ReceiverQueueSize and ConsumerOptions.MessageChannel and the time required to process each message?
Posted by GitBox <gi...@apache.org>.
LvBay closed issue #267:
URL: https://github.com/apache/pulsar-client-go/issues/267
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] LvBay commented on issue #267: Is there any relationship between consumer throughput and ConsumerOptions.ReceiverQueueSize and ConsumerOptions.MessageChannel and the time required to process each message?
Posted by GitBox <gi...@apache.org>.
LvBay commented on issue #267:
URL: https://github.com/apache/pulsar-client-go/issues/267#issuecomment-640329793
> @LvBay
>
> If the consumer takes 10ms to process each message, it means that for a single thread you can process 100 messages/second. So you can have 10 threads to process messages to achieve 1000 msgs/second.
Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] LvBay commented on issue #267: Is there any relationship between consumer throughput and ConsumerOptions.ReceiverQueueSize and ConsumerOptions.MessageChannel and the time required to process each message?
Posted by GitBox <gi...@apache.org>.
LvBay commented on issue #267:
URL: https://github.com/apache/pulsar-client-go/issues/267#issuecomment-639959004
> For example, if the topic has 10 partitions, I need the consumer to achieve a throughput of 1000msg/s, and it takes 10ms to process each message. How can I set the value of ConsumerOptions?
Are there any best practices regarding this situation?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] sijie commented on issue #267: Is there any relationship between consumer throughput and ConsumerOptions.ReceiverQueueSize and ConsumerOptions.MessageChannel and the time required to process each message?
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #267:
URL: https://github.com/apache/pulsar-client-go/issues/267#issuecomment-640315757
@LvBay
If the consumer takes 10ms to process each message, it means that for a single thread you can process 100 messages/second. So you can have 10 threads to process messages to achieve 1000 msgs/second.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-client-go] wolfstudy commented on issue #267: Is there any relationship between consumer throughput and ConsumerOptions.ReceiverQueueSize and ConsumerOptions.MessageChannel and the time required to process each message?
Posted by GitBox <gi...@apache.org>.
wolfstudy commented on issue #267:
URL: https://github.com/apache/pulsar-client-go/issues/267#issuecomment-639340257
The `MessageChannel` is used to receive messages, all messages received by consumers will be filled into MessageChannel, the code example as follows:
```
channel := make(chan pulsar.ConsumerMessage, 100)
options := pulsar.ConsumerOptions{
Topic: "topic-1",
SubscriptionName: "my-subscription",
Type: pulsar.Shared,
}
options.MessageChannel = channel
consumer, err := client.Subscribe(options)
if err != nil {
log.Fatal(err)
}
defer consumer.Close()
// Receive messages from channel. The channel returns a struct which contains message and the consumer from where
// the message was received. It's not necessary here since we have 1 single consumer, but the channel could be
// shared across multiple consumers as well
for cm := range channel {
msg := cm.Message
fmt.Printf("Received message msgId: %v -- content: '%s'\n",
msg.ID(), string(msg.Payload()))
consumer.Ack(msg)
}
```
The `ReceiverQueueSize` sets the size of the consumer to receive queue. It is a buffer of a channel in go language. Default value is 1000
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org