You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Chamikara Madhusanka Jayalath (Jira)" <ji...@apache.org> on 2021/03/22 21:31:00 UTC

[jira] [Created] (BEAM-12025) Environment specific dependency provisioning for Dataflow

Chamikara Madhusanka Jayalath created BEAM-12025:
----------------------------------------------------

             Summary: Environment specific dependency provisioning for Dataflow
                 Key: BEAM-12025
                 URL: https://issues.apache.org/jira/browse/BEAM-12025
             Project: Beam
          Issue Type: New Feature
          Components: cross-language, runner-dataflow, sdk-java-harness, sdk-py-harness
            Reporter: Chamikara Madhusanka Jayalath
            Assignee: Heejong Lee


Currently Dataflow (Runner v2) artifacts are staged to a single GCS location and all artifacts are downloaded to all SDK Harnesses using a bare bones artifact retrieval service.

We need to implement proper artifact staging/retrieval services for Dataflow and separate out dependencies by environment.

Doc: [https://docs.google.com/document/d/1jq8TgiUxvQlJCpY7Q38MtZahX_XbVAdcUcQ_qXLKfgE/edit]

 

Assigning to Heejong who hopes to look into this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)