[GitHub] [beam] kennknowles opened a new issue, #18650: JobService#prepareJob should be one-to-many with JobService#run

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #18650:
URL: https://github.com/apache/beam/issues/18650

   More specifically, prepare should not require a job; instead, it should return endpoints for any pre-job work that the client must perform (e.g. staging artifacts). Once the client has completed any pre-job work, those endpoints should return tokens by which the results of the pre-job work can be referenced. #run should then consume those tokens and use them to ensure that those results are available during execution.
   
    
   
   The simple example is that #prepare should return an artifact staging endpoint, and #run should take tokens returned by that endpoint that it can understand when executing the job. This permits multiple pipelines to be run with the same artifacts, among other benefits, and reduces the statefulness of the prepare-run call sequence.
   
   Imported from Jira [BEAM-3536](https://issues.apache.org/jira/browse/BEAM-3536). Original Jira may contain additional context.
   Reported by: tgroh.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org