You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/29 13:03:12 UTC
[GitHub] [pulsar] freeznet opened a new pull request #11831: [pulsar-functions] Pass `SubscriptionPosition` from `FunctionDetails` to `FunctionConfig` / `SinkConfig`
freeznet opened a new pull request #11831:
URL: https://github.com/apache/pulsar/pull/11831
### Motivation
FunctionDetails defined `SubscriptionPosition`, but the value is not converted back to `FunctionConfig` / `SinkConfig` and will cause `SubscriptionPosition` been reset to the default value.
### Modifications
Pass `SubscriptionPosition` from `FunctionDetails` to `FunctionConfig` / `SinkConfig`
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads (10MB)*
- *Extended integration test for recovery after broker failure*
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API: (yes / no)
- The schema: (yes / no / don't know)
- The default values of configurations: (yes / no)
- The wire protocol: (yes / no)
- The rest endpoints: (yes / no)
- The admin cli options: (yes / no)
- Anything that affects deployment: (yes / no / don't know)
### Documentation
Check the box below and label this PR (if you have committer privilege).
Need to update docs?
- [ ] doc-required
(If you need help on updating docs, create a doc issue)
- [ x] no-need-doc
(Please explain why)
internal bug fix
- [ ] doc
(If this PR contains doc changes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] freeznet commented on a change in pull request #11831: [pulsar-functions] Pass `SubscriptionPosition` from `FunctionDetails` to `FunctionConfig` / `SinkConfig`
Posted by GitBox <gi...@apache.org>.
freeznet commented on a change in pull request #11831:
URL: https://github.com/apache/pulsar/pull/11831#discussion_r698935643
##########
File path: pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java
##########
@@ -373,6 +373,14 @@ public static FunctionConfig convertFromDetails(FunctionDetails functionDetails)
functionConfig.setCleanupSubscription(functionDetails.getSource().getCleanupSubscription());
functionConfig.setAutoAck(functionDetails.getAutoAck());
+
+ // Set subscription position
+ if (Function.SubscriptionPosition.EARLIEST.equals(functionDetails.getSource().getSubscriptionPosition())) {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Earliest);
+ } else {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Latest);
+ }
Review comment:
We cannot directly set FunctionConfig's `SubscriptionPosition` to proto's `SubscriptionPosition`, needs to have a type conversion. So we can have a conversion util function and reuse the code, PTAL at the latest commit, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] freeznet commented on a change in pull request #11831: [pulsar-functions] Pass `SubscriptionPosition` from `FunctionDetails` to `FunctionConfig` / `SinkConfig`
Posted by GitBox <gi...@apache.org>.
freeznet commented on a change in pull request #11831:
URL: https://github.com/apache/pulsar/pull/11831#discussion_r698935643
##########
File path: pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java
##########
@@ -373,6 +373,14 @@ public static FunctionConfig convertFromDetails(FunctionDetails functionDetails)
functionConfig.setCleanupSubscription(functionDetails.getSource().getCleanupSubscription());
functionConfig.setAutoAck(functionDetails.getAutoAck());
+
+ // Set subscription position
+ if (Function.SubscriptionPosition.EARLIEST.equals(functionDetails.getSource().getSubscriptionPosition())) {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Earliest);
+ } else {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Latest);
+ }
Review comment:
We cannot directly set FunctionConfig's `SubscriptionPosition` to proto's `SubscriptionPosition`, needs to have a type conversion. So we can have a conversion util function and reuse the code, PTAL at the latest commit, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a change in pull request #11831: [pulsar-functions] Pass `SubscriptionPosition` from `FunctionDetails` to `FunctionConfig` / `SinkConfig`
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on a change in pull request #11831:
URL: https://github.com/apache/pulsar/pull/11831#discussion_r698938642
##########
File path: pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java
##########
@@ -373,6 +373,14 @@ public static FunctionConfig convertFromDetails(FunctionDetails functionDetails)
functionConfig.setCleanupSubscription(functionDetails.getSource().getCleanupSubscription());
functionConfig.setAutoAck(functionDetails.getAutoAck());
+
+ // Set subscription position
+ if (Function.SubscriptionPosition.EARLIEST.equals(functionDetails.getSource().getSubscriptionPosition())) {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Earliest);
+ } else {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Latest);
+ }
Review comment:
Thank you for the context. I hadn't noticed that nuance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a change in pull request #11831: [pulsar-functions] Pass `SubscriptionPosition` from `FunctionDetails` to `FunctionConfig` / `SinkConfig`
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on a change in pull request #11831:
URL: https://github.com/apache/pulsar/pull/11831#discussion_r698646549
##########
File path: pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java
##########
@@ -373,6 +373,14 @@ public static FunctionConfig convertFromDetails(FunctionDetails functionDetails)
functionConfig.setCleanupSubscription(functionDetails.getSource().getCleanupSubscription());
functionConfig.setAutoAck(functionDetails.getAutoAck());
+
+ // Set subscription position
+ if (Function.SubscriptionPosition.EARLIEST.equals(functionDetails.getSource().getSubscriptionPosition())) {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Earliest);
+ } else {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Latest);
+ }
Review comment:
What is the reason for this conditional? Is it because `getSubscriptionPosition` can return `null`? If we ensure that the `getSubscriptionPosition` defaults to `Latest`, we could skip this conditional and just have:
```java
functionConfig.setSubscriptionPosition(functionDetails.getSource().getSubscriptionPosition());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaoran10 commented on a change in pull request #11831: [pulsar-functions] Pass `SubscriptionPosition` from `FunctionDetails` to `FunctionConfig` / `SinkConfig`
Posted by GitBox <gi...@apache.org>.
gaoran10 commented on a change in pull request #11831:
URL: https://github.com/apache/pulsar/pull/11831#discussion_r698113364
##########
File path: pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java
##########
@@ -373,6 +373,15 @@ public static FunctionConfig convertFromDetails(FunctionDetails functionDetails)
functionConfig.setCleanupSubscription(functionDetails.getSource().getCleanupSubscription());
functionConfig.setAutoAck(functionDetails.getAutoAck());
+
+ // Set subscription position
+ Function.SubscriptionPosition subPosition = functionDetails.getSource().getSubscriptionPosition();
+ if (subPosition == Function.SubscriptionPosition.EARLIEST) {
Review comment:
Could you make the `Function.SubscriptionPosition.EARLIEST` before the equal symbol? Or use the equal method.
```
Function.SubscriptionPosition.EARLIEST == subPosition
```
or
```
Function.SubscriptionPosition.EARLIEST.equals(subPosition)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a change in pull request #11831: [pulsar-functions] Pass `SubscriptionPosition` from `FunctionDetails` to `FunctionConfig` / `SinkConfig`
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on a change in pull request #11831:
URL: https://github.com/apache/pulsar/pull/11831#discussion_r698646549
##########
File path: pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java
##########
@@ -373,6 +373,14 @@ public static FunctionConfig convertFromDetails(FunctionDetails functionDetails)
functionConfig.setCleanupSubscription(functionDetails.getSource().getCleanupSubscription());
functionConfig.setAutoAck(functionDetails.getAutoAck());
+
+ // Set subscription position
+ if (Function.SubscriptionPosition.EARLIEST.equals(functionDetails.getSource().getSubscriptionPosition())) {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Earliest);
+ } else {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Latest);
+ }
Review comment:
What is the reason for this conditional? Is it because `getSubscriptionPosition` can return `null`? If we ensure that the `getSubscriptionPosition` defaults to `Latest`, we could skip this conditional and just have:
```java
functionConfig.setSubscriptionPosition(functionDetails.getSource().getSubscriptionPosition());
```
##########
File path: pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java
##########
@@ -373,6 +373,14 @@ public static FunctionConfig convertFromDetails(FunctionDetails functionDetails)
functionConfig.setCleanupSubscription(functionDetails.getSource().getCleanupSubscription());
functionConfig.setAutoAck(functionDetails.getAutoAck());
+
+ // Set subscription position
+ if (Function.SubscriptionPosition.EARLIEST.equals(functionDetails.getSource().getSubscriptionPosition())) {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Earliest);
+ } else {
+ functionConfig.setSubscriptionPosition(SubscriptionInitialPosition.Latest);
+ }
Review comment:
Thank you for the context. I hadn't noticed that nuance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] freeznet merged pull request #11831: [pulsar-functions] Pass `SubscriptionPosition` from `FunctionDetails` to `FunctionConfig` / `SinkConfig`
Posted by GitBox <gi...@apache.org>.
freeznet merged pull request #11831:
URL: https://github.com/apache/pulsar/pull/11831
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] freeznet commented on a change in pull request #11831: [pulsar-functions] Pass `SubscriptionPosition` from `FunctionDetails` to `FunctionConfig` / `SinkConfig`
Posted by GitBox <gi...@apache.org>.
freeznet commented on a change in pull request #11831:
URL: https://github.com/apache/pulsar/pull/11831#discussion_r698116499
##########
File path: pulsar-functions/utils/src/main/java/org/apache/pulsar/functions/utils/FunctionConfigUtils.java
##########
@@ -373,6 +373,15 @@ public static FunctionConfig convertFromDetails(FunctionDetails functionDetails)
functionConfig.setCleanupSubscription(functionDetails.getSource().getCleanupSubscription());
functionConfig.setAutoAck(functionDetails.getAutoAck());
+
+ // Set subscription position
+ Function.SubscriptionPosition subPosition = functionDetails.getSource().getSubscriptionPosition();
+ if (subPosition == Function.SubscriptionPosition.EARLIEST) {
Review comment:
thanks for the suggestion, applied with the latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org