You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "ragaur-tibco (via GitHub)" <gi...@apache.org> on 2024/04/12 10:53:31 UTC
[D] Sending the message size greater then 5 MB from producer throwing error in pulsar [pulsar]
GitHub user ragaur-tibco created a discussion: Sending the message size greater then 5 MB from producer throwing error in pulsar
While sending the massages greater then 5 MB from producer and enabling the batching
```
ProducerBuilder producerBuilderForStringBatching = client.newProducer(Schema.STRING)
.accessMode("Shared").topic("topicName")
.compressionType("NONE")
.enableBatching(true)
.blockIfQueueFull(true)
.batchingMaxMessages(4)
.batchingMaxBytes(131072)
.batchingMaxPublishDelay(60000, TimeUnit.MILLISECONDS);
ProducerBase<String> producerBuilderForStringBatching .sendTimeout(Integer.valueOf(String.valueOf(sendTimeout)), TimeUnit.SECONDS).create()
```
We are trying one use-case where we have payload size which is greater than the maxMessageSize parameter at the Broker config i.e. Max 5mb. When we are enabling the Batching at Producer side and trying to send the message, we are getting an error which is the "**Message size is bigger than 5242880 bytes**". So, we wanted to know that the result of this case is working as expected or it should be worked somewhat w.r.t Batching fields that we will be providing like for. eg **batchingMaxBytes** which(We are assuming should batch the message in the size that we are specifying). Please let us know if our thinking is correct or not and what should be the expected outcome?
@lhotari
GitHub link: https://github.com/apache/pulsar/discussions/22492
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org
Re: [D] Sending the message size greater then 5 MB from producer throwing error in pulsar [pulsar]
Posted by "slawrencemd (via GitHub)" <gi...@apache.org>.
GitHub user slawrencemd added a comment to the discussion: Sending the message size greater then 5 MB from producer throwing error in pulsar
Note: I'm pretty new to Pulsar myself, so this is just from my basic understanding.
That error makes sense to me. Batching's purpose seems to be to be for increasing throughput gains - instead of making multiple TCP calls to the broker, use 1 network call with a batch of messages. It amortizes the network costs.
I think you might be looking for [chunking ](https://pulsar.apache.org/docs/next/concepts-messaging/#chunking)instead. But note that chunking cannot be enabled when batching is on.
GitHub link: https://github.com/apache/pulsar/discussions/22492#discussioncomment-9096348
----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org