You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/07/14 16:04:47 UTC

[GitHub] [beam] Lizzfox commented on pull request #17111: [BEAM-14101] [CdapIO] Add ReceiverBuilder for SparkReceiverIO

Lizzfox commented on PR #17111:
URL: https://github.com/apache/beam/pull/17111#issuecomment-1184624311

   @aromanenko-dev 
   > why do we need a new dedicated IO for that since it doesn't provide any IO user API? Could you give more details about the next steps on this IO?
   
   The `SparkReceiverIO` will provide a user interface for reading data from the [Spark Streaming Custom Receiver](https://spark.apache.org/docs/latest/streaming-custom-receivers.html) in Apache Beam pipelines. Users will be able to use this IO to read data from their own Spark Receivers.
   
   The next step is to implement the read interface. We have [SparkReceiverIO Read draft PR](https://github.com/apache/beam/pull/17828) that is in progress for now. I hope it would clarify the details. 
   
   We just decided to divide PRs to make them easier to review. So the current PR doesn't implement any user interface 🙂
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org