[jira] [Comment Edited] (BEAM-4636) Make beam.Run() (and/or friends) thread-safe.

["Henning Rohde (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-4636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524344#comment-16524344 ] 

Henning Rohde edited comment on BEAM-4636 at 6/26/18 11:43 PM:
---------------------------------------------------------------

We do launch Go pipelines in parallel in the test driver: 

  https://github.com/apache/beam/blob/5c4c81d82d044b25a29ba7c3743cad36662bb37c/sdks/go/test/integration/driver.go#L83

[~exm] Are you seeing errors or is it because you need different flags for each pipeline?


was (Author: herohde):
We do launch Go pipelines in parallel in the test driver: 

  https://github.com/apache/beam/blob/5c4c81d82d044b25a29ba7c3743cad36662bb37c/sdks/go/test/integration/driver.go#L83

Are you seeing errors or is it because you need different flags for each pipeline?

> Make beam.Run() (and/or friends) thread-safe.
> ---------------------------------------------
>
>                 Key: BEAM-4636
>                 URL: https://issues.apache.org/jira/browse/BEAM-4636
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-go
>            Reporter: Eduardo Morales
>            Assignee: Henning Rohde
>            Priority: Minor
>
> It would be nice to be able to launch dataflow pipelines in parallel. 
> For example, here is my use case:
>  * I consume data produced by my clients/customers.
>  * I need to launch a pipeline on an event dispatch.
>  * I may receive multiple events, from multiple customers at the same time.
>  * Go code could be simpler if synchronization/cooperation wouldn't be needed from goroutines handling each customer. In particular, setting options through flags is cumbersome.
>  * Launching dataflow pipelines serially may not scale if I am able to sign up many customers. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)