You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Stefan Wachter <st...@gmx.de> on 2021/10/12 10:23:59 UTC

side input terribly slow when using Flink runner

Hi,

I have a pipeline where are PCollection is fed as View.asMap into a
later stage. The performance is terribly bad. The attached flame graph
of the stage that uses that side input reveals that nearly all the time
is spent by the Flink runner to serialize pipeline options. Is there
something going terribly wrong?

--Stefan

Re: side input terribly slow when using Flink runner

Posted by Jan Lukavský <je...@seznam.cz>.

Hi Stefan,

could you verify what is the coder you use for the PCollection, which 
you materialize as side-input? I'm not sure from the flame-graph itself, 
but could it be SerializableCoder?

  Jan

On 10/12/21 12:23 PM, Stefan Wachter wrote:
> Hi,
>
> I have a pipeline where are PCollection is fed as View.asMap into a
> later stage. The performance is terribly bad. The attached flame graph
> of the stage that uses that side input reveals that nearly all the time
> is spent by the Flink runner to serialize pipeline options. Is there
> something going terribly wrong?
>
> --Stefan
>