Re: python Multiprocessing started with in a do function

[Robert Bradshaw <ro...@google.com>]

I suspect this may be due to long-standing bugs regarding forking a
process that has grpc channels. See, e.g.
https://github.com/grpc/grpc/issues/18321

On Wed, Feb 26, 2020 at 9:02 AM laxman reddy <dv...@gmail.com> wrote:
>
> Hello Team,
>                 i am using beam for experimenting for my project usecase
> i have specific steps for a pipeline. for each step to process i am using do function in pardo invocation in pipeline
> in one of the steps i had implemented multiprocessing implemented
>
> with out multiprocessing the step is working fine
> when i introduce multiprocessing in pardo transform step is blocking execution
>
>
> if use beam supported multiprocessing option vis direct runner number of workers it is not recognizing my module function
> if use multi threading it is again blocking infinitely
>
>
> can you suggest me how to resolve issue with in the pardo do fn
> or any other suggestion when multiprocessing
>
>
>
>
>