You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/18 19:26:53 UTC
[GitHub] [pulsar] lhotari opened a new pull request #11706: [Functions] Support KEY_BASED batch builder for Java based functions and sources
lhotari opened a new pull request #11706:
URL: https://github.com/apache/pulsar/pull/11706
### Motivation
#8523 added support for configuring KEY_BASED batch builder for functions and sources. However, the implemention missed some parts for Java based functions and sources. Support for Go was added in #8561 and for Python in #8540 .
### Modifications
Set `producerBuidler.batcherBuilder(BatcherBuilder.KEY_BASED)` when batch-builder is configured as `KEY_BASED`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #11706: [Functions] Support KEY_BASED batch builder for Java based functions and sources
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #11706:
URL: https://github.com/apache/pulsar/pull/11706#issuecomment-903713195
@codelipenghui @nlu90 @wolfstudy Please review the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari edited a comment on pull request #11706: [Functions] Support KEY_BASED batch builder for Java based functions and sources
Posted by GitBox <gi...@apache.org>.
lhotari edited a comment on pull request #11706:
URL: https://github.com/apache/pulsar/pull/11706#issuecomment-901675669
I have created a separate PR for backporting the required changes to branch-2.7: #11710 . The code in branch-2.7 is slightly different since #8523 changes haven't been reverted in branch-2.7 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #11706: [Functions] Support KEY_BASED batch builder for Java based functions and sources
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #11706:
URL: https://github.com/apache/pulsar/pull/11706#issuecomment-901665713
> There's a remaining gotcha with sources. For sources, the --batch-builder KEY_BASED argument isn't used. It is required to use --producer-config "{\"batchBuilder\": \"KEY_BASED\"}".
I also fixed this problem as part of this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #11706: [Functions] Support KEY_BASED batch builder for Java based functions and sources
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #11706:
URL: https://github.com/apache/pulsar/pull/11706#issuecomment-901648992
There's a remaining gotcha with sources. For sources, the `--batch-builder KEY_BASED` argument isn't used. It is required to use `--producer-config "{\"batchBuilder\": \"KEY_BASED\"}"`. for example:
```
./bin/pulsar-admin sources create --tenant public --namespace default --name datagen --destination-topic-name datagen -a builtin://data-generator --source-config "{\"sleepBetweenMessages\": 1000}" --producer-config "{\"batchBuilder\": \"KEY_BASED\"}"
```
I wonder if this is a bug or a feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #11706: [Functions] Support KEY_BASED batch builder for Java based functions and sources
Posted by GitBox <gi...@apache.org>.
sijie commented on pull request #11706:
URL: https://github.com/apache/pulsar/pull/11706#issuecomment-902104130
@nlu90 Can you review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #11706: [Functions] Support KEY_BASED batch builder for Java based functions and sources
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #11706:
URL: https://github.com/apache/pulsar/pull/11706#issuecomment-901665295
> The change makes sense to me.
> As usual we should add some test case that will save us from future regressions.
>
> I am sorry but we need a end to end integration test for this
I completely agree. The changes in the PR now fix the various issues and it's time to think about the test coverage to prevent regressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari commented on pull request #11706: [Functions] Support KEY_BASED batch builder for Java based functions and sources
Posted by GitBox <gi...@apache.org>.
lhotari commented on pull request #11706:
URL: https://github.com/apache/pulsar/pull/11706#issuecomment-901675669
I have created a separate PR for backporting the required changes to branch-2.7: #11710 . The code in branch-2.7 is slightly different since #8523 haven't been reverted in branch-2.7 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] lhotari merged pull request #11706: [Functions] Support KEY_BASED batch builder for Java based functions and sources
Posted by GitBox <gi...@apache.org>.
lhotari merged pull request #11706:
URL: https://github.com/apache/pulsar/pull/11706
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org