You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/01/29 07:38:03 UTC
[GitHub] [pulsar] momo-jun opened a new pull request #14037: [Doc] Add sections about how to enable and configure chunking for java client producers/consumers
momo-jun opened a new pull request #14037:
URL: https://github.com/apache/pulsar/pull/14037
### Motivation
To enhance the [Pulsar Feature Matrix](https://docs.google.com/spreadsheets/d/1YHYTkIXR8-Ql103u-IMI18TXLlGStK8uJjDsOOA0T20/edit#gid=1784579914), add the required document for chunking in the java client category.
### Modifications
1. Add a section about how to enable message chunking for java client producers.
2. Add a section about how to configure message chunking for java client consumers, and add three parameters.
### Documentation
- [ ] `doc`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #14037: [Doc] Add sections about how to enable and configure chunking for java client producers/consumers
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14037:
URL: https://github.com/apache/pulsar/pull/14037#issuecomment-1024857705
@momo-jun:Thanks for providing doc info!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on pull request #14037: [Doc] Add sections about how to enable and configure chunking for java client producers/consumers
Posted by GitBox <gi...@apache.org>.
momo-jun commented on pull request #14037:
URL: https://github.com/apache/pulsar/pull/14037#issuecomment-1024857709
Hi @RobertIndie @Anonymitaet , can you please review this doc PR? Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a change in pull request #14037: [Doc] Add sections about how to enable and configure chunking for java client producers/consumers
Posted by GitBox <gi...@apache.org>.
momo-jun commented on a change in pull request #14037:
URL: https://github.com/apache/pulsar/pull/14037#discussion_r795150277
##########
File path: site2/docs/client-libraries-java.md
##########
@@ -404,6 +423,22 @@ consumer.acknowledge(messages)
> .build();
> ```
+### Configure chunking
+
+You can limit the maximum number of chunked messages a consumer maintains concurrently by configuring the `maxPendingChunkedMessage` and `autoAckOldestChunkedMessageOnQueueFull` parameter. When the threshold is reached, the consumer drops pending messages by silently acknowledging them or asking the broker to redeliver them later. The `expireTimeOfIncompleteChunkedMessage` parameter decides the time interval to expire incomplete chunks if the consumer fails to receive all chunks of a message within the specified time period.
Review comment:
Sharp eyes! Thanks for pointing this out. I've corrected it. PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #14037: [Doc] Add sections about how to enable and configure chunking for java client producers/consumers
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14037:
URL: https://github.com/apache/pulsar/pull/14037#issuecomment-1024857683
@momo-jun:Thanks for your contribution. For this PR, do we need to update docs?
(The [PR template contains info about doc](https://github.com/apache/pulsar/blob/master/.github/PULL_REQUEST_TEMPLATE.md#documentation), which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] RobertIndie commented on a change in pull request #14037: [Doc] Add sections about how to enable and configure chunking for java client producers/consumers
Posted by GitBox <gi...@apache.org>.
RobertIndie commented on a change in pull request #14037:
URL: https://github.com/apache/pulsar/pull/14037#discussion_r795036040
##########
File path: site2/docs/client-libraries-java.md
##########
@@ -338,6 +354,9 @@ When you create a consumer, you can use the `loadConf` configuration. The follow
`replicateSubscriptionState`|boolean|If `replicateSubscriptionState` is enabled, a subscription state is replicated to geo-replicated clusters.|false
`negativeAckRedeliveryBackoff`|RedeliveryBackoff|Interface for custom message is negativeAcked policy. You can specify `RedeliveryBackoff` for a consumer.| `MultiplierRedeliveryBackoff`
`ackTimeoutRedeliveryBackoff`|RedeliveryBackoff|Interface for custom message is ackTimeout policy. You can specify `RedeliveryBackoff` for a consumer.| `MultiplierRedeliveryBackoff`
+`autoAckOldestChunkedMessageOnQueueFull`|boolean|Whether to automatically acknowledge pending chunked messages when the threashold of `maxPendingChunkedMessage` is reached. If set to `false`, these messages will be redelivered by their broker. |true
+`maxPendingChunkedMessage`|int| The maximum size of a queue holding pending chunked messages. When the threshold is reached, the consumer drops pending messages to optimize memory utilization.|10
+`expireTimeOfIncompleteChunkedMessageMillis`|long|The time interval to expire incomplete chunks if a consumer fails to receive all the chunks in the specified time period. The default value is 1 minute. | 1
Review comment:
```suggestion
`expireTimeOfIncompleteChunkedMessageMillis`|long|The time interval to expire incomplete chunks if a consumer fails to receive all the chunks in the specified time period. The default value is 1 minute. | 600000
```
##########
File path: site2/docs/client-libraries-java.md
##########
@@ -338,6 +354,9 @@ When you create a consumer, you can use the `loadConf` configuration. The follow
`replicateSubscriptionState`|boolean|If `replicateSubscriptionState` is enabled, a subscription state is replicated to geo-replicated clusters.|false
`negativeAckRedeliveryBackoff`|RedeliveryBackoff|Interface for custom message is negativeAcked policy. You can specify `RedeliveryBackoff` for a consumer.| `MultiplierRedeliveryBackoff`
`ackTimeoutRedeliveryBackoff`|RedeliveryBackoff|Interface for custom message is ackTimeout policy. You can specify `RedeliveryBackoff` for a consumer.| `MultiplierRedeliveryBackoff`
+`autoAckOldestChunkedMessageOnQueueFull`|boolean|Whether to automatically acknowledge pending chunked messages when the threashold of `maxPendingChunkedMessage` is reached. If set to `false`, these messages will be redelivered by their broker. |true
+`maxPendingChunkedMessage`|int| The maximum size of a queue holding pending chunked messages. When the threshold is reached, the consumer drops pending messages to optimize memory utilization.|10
+`expireTimeOfIncompleteChunkedMessageMillis`|long|The time interval to expire incomplete chunks if a consumer fails to receive all the chunks in the specified time period. The default value is 1 minute. | 1
Review comment:
`expireTimeOfIncompleteChunkedMessageMillis ` here is in milliseconds unit. The default value is 60000(which is 1 min).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a change in pull request #14037: [Doc] Add sections about how to enable and configure chunking for java client producers/consumers
Posted by GitBox <gi...@apache.org>.
momo-jun commented on a change in pull request #14037:
URL: https://github.com/apache/pulsar/pull/14037#discussion_r795114530
##########
File path: site2/docs/client-libraries-java.md
##########
@@ -338,6 +354,9 @@ When you create a consumer, you can use the `loadConf` configuration. The follow
`replicateSubscriptionState`|boolean|If `replicateSubscriptionState` is enabled, a subscription state is replicated to geo-replicated clusters.|false
`negativeAckRedeliveryBackoff`|RedeliveryBackoff|Interface for custom message is negativeAcked policy. You can specify `RedeliveryBackoff` for a consumer.| `MultiplierRedeliveryBackoff`
`ackTimeoutRedeliveryBackoff`|RedeliveryBackoff|Interface for custom message is ackTimeout policy. You can specify `RedeliveryBackoff` for a consumer.| `MultiplierRedeliveryBackoff`
+`autoAckOldestChunkedMessageOnQueueFull`|boolean|Whether to automatically acknowledge pending chunked messages when the threashold of `maxPendingChunkedMessage` is reached. If set to `false`, these messages will be redelivered by their broker. |true
+`maxPendingChunkedMessage`|int| The maximum size of a queue holding pending chunked messages. When the threshold is reached, the consumer drops pending messages to optimize memory utilization.|10
+`expireTimeOfIncompleteChunkedMessageMillis`|long|The time interval to expire incomplete chunks if a consumer fails to receive all the chunks in the specified time period. The default value is 1 minute. | 1
Review comment:
Thanks for pointing this out. I've corrected it. PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on a change in pull request #14037: [Doc] Add sections about how to enable and configure chunking for java client producers/consumers
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on a change in pull request #14037:
URL: https://github.com/apache/pulsar/pull/14037#discussion_r795149425
##########
File path: site2/docs/client-libraries-java.md
##########
@@ -404,6 +423,22 @@ consumer.acknowledge(messages)
> .build();
> ```
+### Configure chunking
+
+You can limit the maximum number of chunked messages a consumer maintains concurrently by configuring the `maxPendingChunkedMessage` and `autoAckOldestChunkedMessageOnQueueFull` parameter. When the threshold is reached, the consumer drops pending messages by silently acknowledging them or asking the broker to redeliver them later. The `expireTimeOfIncompleteChunkedMessage` parameter decides the time interval to expire incomplete chunks if the consumer fails to receive all chunks of a message within the specified time period.
Review comment:
```suggestion
You can limit the maximum number of chunked messages a consumer maintains concurrently by configuring the `maxPendingChunkedMessage` and `autoAckOldestChunkedMessageOnQueueFull` parameters. When the threshold is reached, the consumer drops pending messages by silently acknowledging them or asking the broker to redeliver them later. The `expireTimeOfIncompleteChunkedMessage` parameter decides the time interval to expire incomplete chunks if the consumer fails to receive all chunks of a message within the specified time period.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet merged pull request #14037: [Doc] Add sections about how to enable and configure chunking for java client producers/consumers
Posted by GitBox <gi...@apache.org>.
Anonymitaet merged pull request #14037:
URL: https://github.com/apache/pulsar/pull/14037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org