[GitHub] [pulsar] jackson-chris opened a new issue, #20934: [Bug] Dead Letter Policy in Consumer configuration doesn't account for chunking or batching configuration.

["jackson-chris (via GitHub)" <gi...@apache.org>]

jackson-chris opened a new issue, #20934:
URL: https://github.com/apache/pulsar/issues/20934

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
   
   
   ### Version
   
   2.11.1
   
   ### Minimal reproduce step
   
   - Produce a message to a topic that exceeds the `maxMessageSize` default value of `5MB`. 
   - Create a consumer that consumes from that topic. This consumer should be configured with a `deadLetterPolicy` that includes retry letter and dead letter topics. 
   - Simulate a failure in the consumer code by forcing a call to `reconsumeLater`
   
   ### What did you expect to see?
   
   I would expect that a message will be produced to the retry letter topic that was configured in the deadLetterPolicy for that consumer.
   
   ### What did you see instead?
   
   The following error is observed:
   ```
   org.apache.pulsar.client.api.PulsarClientException$InvalidMessageException: The producer mdl-pulsar-0-12 of the topic persistent://metadatalake/metadatalake/part-rlt-v1 sends a  message with 7094331 bytes that exceeds 5242880 bytes
   ```
   
   ### Anything else?
   
   The root cause is expected to be that the consumer implementation with respect to creating the producers for whatever is provided in `deadLetterPolicy` does not have any configuration parameters regarding if batching or chunking should be enabled. Problematic code:
   - https://github.com/apache/pulsar/blob/ecd16d68e29700cc9521e6d77d124a403c3486c9/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java#L2092-L2097
   - https://github.com/apache/pulsar/blob/ecd16d68e29700cc9521e6d77d124a403c3486c9/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java#L613-L617
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] codelipenghui closed issue #20934: [Bug] Dead Letter Policy in Consumer configuration doesn't account for chunking or batching configuration.

["codelipenghui (via GitHub)" <gi...@apache.org>]

codelipenghui closed issue #20934: [Bug] Dead Letter Policy in Consumer configuration doesn't account for chunking or batching configuration.
URL: https://github.com/apache/pulsar/issues/20934


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org