You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/04/08 06:39:43 UTC
[GitHub] [pulsar] baomingyu opened a new pull request #10168: Delay msg early
baomingyu opened a new pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168
<!--
### Contribution Checklist
- Name the pull request in the form "[Issue XYZ][component] Title of the pull request", where *XYZ* should be replaced by the actual issue number.
Skip *Issue XYZ* if there is no associated github issue for this pull request.
Skip *component* if you are unsure about which is the best component. E.g. `[docs] Fix typo in produce method`.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
*(If this PR fixes a github issue, please add `Fixes #<xyz>`.)*
Fixes #<xyz>
*(or if this PR is one task of a github issue, please add `Master Issue: #<xyz>` to link to the master issue.)*
Master Issue: #<xyz>
### Motivation
*Explain here the context, and why you're making that change. What is the problem you're trying to solve.*
### Modifications
*Describe the modifications you've done.*
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads (10MB)*
- *Extended integration test for recovery after broker failure*
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API: (yes / no)
- The schema: (yes / no / don't know)
- The default values of configurations: (yes / no)
- The wire protocol: (yes / no)
- The rest endpoints: (yes / no)
- The admin cli options: (yes / no)
- Anything that affects deployment: (yes / no / don't know)
### Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
- If a feature is not applicable for documentation, explain why?
- If a feature is not documented yet in this PR, please create a followup issue for adding the documentation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] baomingyu commented on a change in pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
baomingyu commented on a change in pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#discussion_r611281636
##########
File path: pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##########
@@ -238,6 +238,9 @@
+ " affecting the accuracy of the delivery time compared to the scheduled time. Default is 1 second.")
private long delayedDeliveryTickTimeMillis = 1000;
+ @FieldContext(category = CATEGORY_SERVER, doc = "HashedWheelTimer duration tick time for when retrying on delayed delivery")
+ private long delayedDeliveryTickDurationMillis = 1000;
Review comment:
Now delayedDeliveryTickTimeMillis is used both delay message and wheeltimer , so i can not change config only one,
For decrease delayedDeliveryTickTimeMillis to 5ms or lower, will affect wheeltimer .
Sometimes, delay of several milliseconds is acceptable but not milliseconds ahead，so this configuration ‘delayedDeliveryTickTimeMillis’ can't fully meet the requirements.
I add this delayedDeliveryTickDurationMillis config to separate the two configurations，after adding delayedDeliveryTickDurationMillis ,i can config delayedDeliveryTickTimeMillis with 0, and delay message will not be consumer any millisecond level ahead.
Befor changing:
> Befor changing:
public void initialize(ServiceConfiguration config) {
this.timer = new HashedWheelTimer(new DefaultThreadFactory("pulsar-delayed-delivery"),
config.getDelayedDeliveryTickTimeMillis(), TimeUnit.MILLISECONDS);
this.tickTimeMillis = config.getDelayedDeliveryTickTimeMillis();
}
> after changing:
public void initialize(ServiceConfiguration config) {
this.timer = new HashedWheelTimer(new DefaultThreadFactory("pulsar-delayed-delivery"),
config.getDelayedDeliveryTickDurationMillis(), TimeUnit.MILLISECONDS);
this.tickTimeMillis = config.getDelayedDeliveryTickTimeMillis();
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on a change in pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on a change in pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#discussion_r611290115
##########
File path: pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##########
@@ -238,6 +238,9 @@
+ " affecting the accuracy of the delivery time compared to the scheduled time. Default is 1 second.")
private long delayedDeliveryTickTimeMillis = 1000;
+ @FieldContext(category = CATEGORY_SERVER, doc = "HashedWheelTimer duration tick time for when retrying on delayed delivery")
+ private long delayedDeliveryTickDurationMillis = 1000;
Review comment:
I know this PR is separating the configuration. I mean the tick time of the HashedWheelTimer also affects accuracy right? If the `delayedDeliveryTickTimeMillis` is 5ms and the `delayedDeliveryTickTimeMillis` is 1000ms, what kind of accuracy guarantee can we get?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#issuecomment-1058891829
@baomingyu:Thanks for your contribution. For this PR, do we need to update docs?
(The [PR template contains info about doc](https://github.com/apache/pulsar/blob/master/.github/PULL_REQUEST_TEMPLATE.md#documentation), which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] baomingyu commented on a change in pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
baomingyu commented on a change in pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#discussion_r611436118
##########
File path: pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##########
@@ -238,6 +238,9 @@
+ " affecting the accuracy of the delivery time compared to the scheduled time. Default is 1 second.")
private long delayedDeliveryTickTimeMillis = 1000;
+ @FieldContext(category = CATEGORY_SERVER, doc = "HashedWheelTimer duration tick time for when retrying on delayed delivery")
+ private long delayedDeliveryTickDurationMillis = 1000;
Review comment:
HashedWheelTimer use delayedDeliveryTickTimeMillis for initial , and InMemoryDelayedDeliveryTracker use delayedDeliveryTickTimeMillis value to check do or not for delay message in current tick period. when delayedDeliveryTickTimeMillis is 1000ms, delay message will be consumer serval millisecond ahead.
> if (deliveryAt < (now + tickTimeMillis)) {
// It's already about time to deliver this message. We add the buffer of
// `tickTimeMillis` because messages can be extracted from the tracker
// slightly before the expiration time. We don't want the messages to
// go back into the delay tracker (for a brief amount of time) when we're
// trying to dispatch to the consumer.
return false;
}
And Introduce 2 configurations to control the tick time of the delayed message and config delayedDeliveryTickTimeMillis with 0 ,will avoid delay message consumed millisecond ahead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#issuecomment-1058891627
The pr had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hoswey commented on a change in pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
hoswey commented on a change in pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#discussion_r619808752
##########
File path: pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##########
@@ -238,6 +238,9 @@
+ " affecting the accuracy of the delivery time compared to the scheduled time. Default is 1 second.")
private long delayedDeliveryTickTimeMillis = 1000;
+ @FieldContext(category = CATEGORY_SERVER, doc = "HashedWheelTimer duration tick time for when retrying on delayed delivery")
+ private long delayedDeliveryTickDurationMillis = 1000;
Review comment:
In this moment, the delayedDeliveryTickTimeMillis is used for the HashedWheelTimer, reduce to 5ms will increase the cpu usage. @codelipenghui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#issuecomment-815608716
Could you add a unit test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on a change in pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on a change in pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#discussion_r611453706
##########
File path: pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##########
@@ -238,6 +238,9 @@
+ " affecting the accuracy of the delivery time compared to the scheduled time. Default is 1 second.")
private long delayedDeliveryTickTimeMillis = 1000;
+ @FieldContext(category = CATEGORY_SERVER, doc = "HashedWheelTimer duration tick time for when retrying on delayed delivery")
+ private long delayedDeliveryTickDurationMillis = 1000;
Review comment:
@baomingyu Do you have a test for setting delayedDeliveryTickTimeMillis=5 and the delayedDeliveryTickDurationMillis=1000 to check how the delayed message performed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] baomingyu commented on a change in pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
baomingyu commented on a change in pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#discussion_r611282101
##########
File path: pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##########
@@ -1510,6 +1513,8 @@
private String managedLedgerDataReadPriority = OffloadPolicies.OffloadedReadPriority.TIERED_STORAGE_FIRST
.getValue();
+ private long stickConsumerBackOffDelayTimeMs = 500;
Review comment:
I add this for config backoff time for pr 10096.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] baomingyu commented on a change in pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
baomingyu commented on a change in pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#discussion_r611436118
##########
File path: pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##########
@@ -238,6 +238,9 @@
+ " affecting the accuracy of the delivery time compared to the scheduled time. Default is 1 second.")
private long delayedDeliveryTickTimeMillis = 1000;
+ @FieldContext(category = CATEGORY_SERVER, doc = "HashedWheelTimer duration tick time for when retrying on delayed delivery")
+ private long delayedDeliveryTickDurationMillis = 1000;
Review comment:
HashedWheelTimer use delayedDeliveryTickTimeMillis for initial , and InMemoryDelayedDeliveryTracker use delayedDeliveryTickTimeMillis value to check do or not for delay message in current tick period. when delayedDeliveryTickTimeMillis is 1000ms, delay message will be consumer serval millisecond ahead.
> if (deliveryAt < (now + tickTimeMillis)) {
// It's already about time to deliver this message. We add the buffer of
// `tickTimeMillis` because messages can be extracted from the tracker
// slightly before the expiration time. We don't want the messages to
// go back into the delay tracker (for a brief amount of time) when we're
// trying to dispatch to the consumer.
return false;
}
And Introduce 2 configurations to control the tick time of the delayed message and config delayedDeliveryTickTimeMillis with 0 ,will avoid delay message consumed millisecond ahead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on a change in pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on a change in pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#discussion_r611187892
##########
File path: pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##########
@@ -238,6 +238,9 @@
+ " affecting the accuracy of the delivery time compared to the scheduled time. Default is 1 second.")
private long delayedDeliveryTickTimeMillis = 1000;
+ @FieldContext(category = CATEGORY_SERVER, doc = "HashedWheelTimer duration tick time for when retrying on delayed delivery")
+ private long delayedDeliveryTickDurationMillis = 1000;
Review comment:
@baomingyu I don't think we need this config there. For most cases and the current implementation of Pulsar delay message, accuracy in the millisecond level is the expected behavior, If the default value 1000 is not work for you, I think you can decrease it to 5ms or lower. Introduce 2 configurations to control the tick time of the delayed message tracker also will introduce complexity to users to tune the delayed message behavior.
##########
File path: pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##########
@@ -1510,6 +1513,8 @@
private String managedLedgerDataReadPriority = OffloadPolicies.OffloadedReadPriority.TIERED_STORAGE_FIRST
.getValue();
+ private long stickConsumerBackOffDelayTimeMs = 500;
Review comment:
Seems not related to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hoswey commented on a change in pull request #10168: fix 10169 Delayed messages may be executed several milliseconds in advance
Posted by GitBox <gi...@apache.org>.
hoswey commented on a change in pull request #10168:
URL: https://github.com/apache/pulsar/pull/10168#discussion_r619808752
##########
File path: pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java
##########
@@ -238,6 +238,9 @@
+ " affecting the accuracy of the delivery time compared to the scheduled time. Default is 1 second.")
private long delayedDeliveryTickTimeMillis = 1000;
+ @FieldContext(category = CATEGORY_SERVER, doc = "HashedWheelTimer duration tick time for when retrying on delayed delivery")
+ private long delayedDeliveryTickDurationMillis = 1000;
Review comment:
In this moment, the delayedDeliveryTickTimeMillis is used for the HashedWheelTimer, reduce to 5ms will increase the cpu usage. @codelipenghui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org