[jira] [Assigned] (BEAM-14101) [CdapIO] Design and implement Proxy for Spark Receivers

["Vitaly Terentyev (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-14101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vitaly Terentyev reassigned BEAM-14101:
---------------------------------------

    Assignee: Vitaly Terentyev

> [CdapIO] Design and implement Proxy for Spark Receivers
> -------------------------------------------------------
>
>                 Key: BEAM-14101
>                 URL: https://issues.apache.org/jira/browse/BEAM-14101
>             Project: Beam
>          Issue Type: Task
>          Components: io-java-cdap
>            Reporter: Elizaveta Lomteva
>            Assignee: Vitaly Terentyev
>            Priority: P2
>              Labels: cdap-io-sprint-2
>
> h3. Context:
> CDAP plugins that support streaming sources include Receiver classes (ex. [HubSpotReceiver|https://github.com/data-integrations/hubspot/blob/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming/HubspotReceiver.java]) that extend {{org.apache.spark.streaming.receiver.Receiver}} abstract class. Receiver classes are used by plugin Streaming Utils classes (ex. [HubSpotStreamingUtils|https://github.com/data-integrations/hubspot/blob/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming/HubspotStreamingUtil.java]) to provide {{getStream()}} method to Streaming Source classes (ex. [HubSpotStreamingSource|https://github.com/data-integrations/hubspot/blob/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming/HubspotStreamingSource.java]) and usually placed in {{cdap/plugin/\{<plugin name>}/plugin/source/streaming}} folder (ex.[ HubSpot plugin repo|https://github.com/data-integrations/hubspot/tree/develop/src/main/java/io/cdap/plugin/hubspot/source/streaming]).
> Reference information:
>  * [Streaming plug-ins integration|https://docs.google.com/document/d/1T-bhd0Qk7DBePIfgHEPagYiA1oLP4z5kYEd0S1SOGxQ/edit#heading=h.o88i6p9b13o9] (Apache CDAP Connection Design Doc)
>  * [Plugin integration process description|https://docs.google.com/document/d/1T-bhd0Qk7DBePIfgHEPagYiA1oLP4z5kYEd0S1SOGxQ/edit#heading=h.1h6udb1b52xc] (Apache CDAP Connection Design Doc)
>  * [Streaming wrapper design|https://docs.google.com/document/d/1T-bhd0Qk7DBePIfgHEPagYiA1oLP4z5kYEd0S1SOGxQ/edit#heading=h.fcafz0ydsso1] (Apache CDAP Connection Design Doc)
> h3. Task Description:
> Required to design custom Spark receivers proxy class so that the custom receivers can be used in Apache Beam connector via SparkReceiverIO interface (used in CDAP IO as a dependency).
> h3. Acceptance criteria:
> Design of proxy class(es) that will provide custom Spark receivers functionality to Apache Beam connectors ({{{}SparkReceiverIO{}}}).
> h4. Note:
> It is necessary that this proxy class be independent of CDAP receivers and can be used by any other custom Spark receiver.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)