You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/11/11 09:27:18 UTC

[GitHub] [pulsar] wolfstudy opened a new pull request #8523: Support key_based batch builder for functions and sources

wolfstudy opened a new pull request #8523:
URL: https://github.com/apache/pulsar/pull/8523


   Signed-off-by: xiaolong.ran <rx...@apache.org>
   
   ### Motivation
   
   Currently, we support the Key_Shared subscription mode in Pulsar Function. In order to ensure that when batch is turned on, we can also ensure that messages are distributed to different consumers in the correct order, so we need to support the batch builder of `KEY_BASED` in Pulsar Functions.
   
   ### Modifications
   
   - Add `--batch-builder` for Pulsar Functions
   - Add `--batch-builder` for Pulsar Sources
   - Add test case
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] wolfstudy commented on pull request #8523: Support key_based batch builder for functions and sources

Posted by GitBox <gi...@apache.org>.
wolfstudy commented on pull request #8523:
URL: https://github.com/apache/pulsar/pull/8523#issuecomment-725840501


   /pulsarbot run-failure-checks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] wolfstudy commented on pull request #8523: Support key_based batch builder for functions and sources

Posted by GitBox <gi...@apache.org>.
wolfstudy commented on pull request #8523:
URL: https://github.com/apache/pulsar/pull/8523#issuecomment-725963517


   > Overall looks good. However, I didn't see any test case. Can you add one?
   
   The current unit tests can be covered. The change of this pull request is to add a **batchBuilder** field to `ProducerConfig`. The current unit test framework covers the Pojo object `ProducerConfig`, so we only need to add this field to the existing unit test framework. Refer to `pulsar-functions/utils/src/test/java/org/apache/pulsar/functions/utils/SourceConfigUtilsTest.java` and `pulsar-functions/utils/src/test/java/org/apache/pulsar/functions/utils/FunctionConfigUtilsTest.java`.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] sijie merged pull request #8523: Support key_based batch builder for functions and sources

Posted by GitBox <gi...@apache.org>.
sijie merged pull request #8523:
URL: https://github.com/apache/pulsar/pull/8523


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] wolfstudy commented on pull request #8523: Support key_based batch builder for functions and sources

Posted by GitBox <gi...@apache.org>.
wolfstudy commented on pull request #8523:
URL: https://github.com/apache/pulsar/pull/8523#issuecomment-727805738


   @sijie PTAL again thanks. I added the `--batch-builder` parameter to the existing integration test to confirm whether this parameter is correctly identified.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] wolfstudy commented on pull request #8523: Support key_based batch builder for functions and sources

Posted by GitBox <gi...@apache.org>.
wolfstudy commented on pull request #8523:
URL: https://github.com/apache/pulsar/pull/8523#issuecomment-725964147


   You mean, do we add another integration test on top of the unit test?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org