You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by "aloyszhang (via GitHub)" <gi...@apache.org> on 2024/07/05 03:19:08 UTC

[PR] [INLONG-10560][Sort] Support bounded pulsar source [inlong]

aloyszhang opened a new pull request, #10569:
URL: https://github.com/apache/inlong/pull/10569

   <!-- Prepare a Pull Request
   Change the title of pull request refer to the following example:
     [INLONG-XYZ][Component] Title of the pull request 
   -->
   
   <!-- Specify the issue this pull request going to fix.
   The following *XYZ* should be replaced by the actual [GitHub Issue](https://github.com/apache/inlong/issues) number)-->
   
   Fixes #10560 
   
   ### Motivation
   
   The pulsar source only supports un-bounded data sources, this pull request aims to add the support for bounded pulsar source.
   ### Modifications
   
   1. define the boundaries for the source
   2. make pulsar source support bounded data(this pull request only support time boundaries)
   
   This is worked for offline sync jobs with Flink batch runtime mode.
   Every time the Flink batch task begins, set the start and stop timestamp(which is the publish time of pulsar msg) of Pulsar source.
   
   Note: since the boundaries, especially the stop boundary, are defined by the pulsar message's publish time, the batch Flink job may not be finished if no messages are published into the Pulsar topic.
   
   ### Verifying this change
   
   *(Please pick either of the following options)*
   
   - [ ] This change is a trivial rework/code cleanup without any test coverage.
   
   
   ### Documentation
   
     - Does this pull request introduce a new feature? ( no)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [INLONG-10560][Sort] Support bounded pulsar source [inlong]

Posted by "aloyszhang (via GitHub)" <gi...@apache.org>.
aloyszhang merged PR #10569:
URL: https://github.com/apache/inlong/pull/10569


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@inlong.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org