You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Ifat Afek (Jira)" <ji...@apache.org> on 2022/05/09 08:29:00 UTC

[jira] [Created] (BEAM-14438) Beam slowness compared to flink-native

Ifat Afek created BEAM-14438:
--------------------------------

             Summary: Beam slowness compared to flink-native
                 Key: BEAM-14438
                 URL: https://issues.apache.org/jira/browse/BEAM-14438
             Project: Beam
          Issue Type: Bug
          Components: runner-flink
    Affects Versions: 2.38.0
            Reporter: Ifat Afek


I tried to compare a very simple beam pipeline with an equivalent flink-native pipeline. Both pipelines read strings from one kafka topic and write them to another topic. I ran each pipeline separately on a single task manager with a single slot and parallelism 1.

Flink native runs 5 times faster than beam: 150,000 strings per second in flink comparing to 30,000 in beam.

When using Avro and schema registry the difference is even more significant - flink native runs 30-80 times faster than beam.

Attached is the java code of both string-to-string pipelines.

 

 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)