[jira] [Work logged] (BEAM-14101) [CdapIO] Design and implement Spark Receiver Builder

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-14101?focusedWorklogId=768370&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768370 ]

ASF GitHub Bot logged work on BEAM-14101:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 10/May/22 09:01
            Start Date: 10/May/22 09:01
    Worklog Time Spent: 10m 
      Work Description: Amar3tto commented on PR #17111:
URL: https://github.com/apache/beam/pull/17111#issuecomment-1122123477

   Run Java_PVR_Flink_Batch PreCommit




Issue Time Tracking
-------------------

    Worklog Id:     (was: 768370)
    Time Spent: 9h 40m  (was: 9.5h)

> [CdapIO] Design and implement Spark Receiver Builder
> ----------------------------------------------------
>
>                 Key: BEAM-14101
>                 URL: https://issues.apache.org/jira/browse/BEAM-14101
>             Project: Beam
>          Issue Type: Task
>          Components: io-java-cdap
>            Reporter: Elizaveta Lomteva
>            Assignee: Elizaveta Lomteva
>            Priority: P2
>              Labels: cdap-io-sprint-4
>          Time Spent: 9h 40m
>  Remaining Estimate: 0h
>
> h3. Context:
> CDAP plugins that support streaming sources include Receiver classes (ex. [HubSpotReceiver|https://github.com/data-integrations/hubspot/blob/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming/HubspotReceiver.java]) that extend {{org.apache.spark.streaming.receiver.Receiver}} abstract class. Receiver classes are used by plugin Streaming Utils classes (ex. [HubSpotStreamingUtils|https://github.com/data-integrations/hubspot/blob/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming/HubspotStreamingUtil.java]) to provide {{getStream()}} method to Streaming Source classes (ex. [HubSpotStreamingSource|https://github.com/data-integrations/hubspot/blob/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming/HubspotStreamingSource.java]) and usually placed in {{cdap/plugin/\{<plugin name>}/plugin/source/streaming}} folder (ex.[ HubSpot plugin repo|https://github.com/data-integrations/hubspot/tree/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming]).
> Reference information:
>  * [Streaming plug-ins integration|https://docs.google.com/document/d/1T-bhd0Qk7DBePIfgHEPagYiA1oLP4z5kYEd0S1SOGxQ/edit#heading=h.o88i6p9b13o9] (Apache CDAP Connection Design Doc)
>  * [Plugin integration process description|https://docs.google.com/document/d/1T-bhd0Qk7DBePIfgHEPagYiA1oLP4z5kYEd0S1SOGxQ/edit#heading=h.1h6udb1b52xc] (Apache CDAP Connection Design Doc)
>  * [Streaming wrapper design|https://docs.google.com/document/d/1T-bhd0Qk7DBePIfgHEPagYiA1oLP4z5kYEd0S1SOGxQ/edit#heading=h.fcafz0ydsso1] (Apache CDAP Connection Design Doc)
> h3. Task Description:
> Required to design custom Spark receivers builder class so that the custom receivers can be used in Apache Beam connector via SparkReceiverIO interface (used in CDAP IO as a dependency).
> h3. Acceptance criteria:
> Design of builder class(es) that will create custom Spark receivers in Apache Beam connectors ({{{}SparkReceiverIO{}}}).
> h4. Note:
> It is necessary that this builder class be independent of CDAP receivers and can be used by any other custom Spark receiver like a part of SparkReceiverIO.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)