[jira] [Updated] (BEAM-14438) Beam slowness compared to flink-native

["Ifat Afek (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ifat Afek updated BEAM-14438:
-----------------------------
    Attachment: BeamConvertToStringRecordExample.java
                BeamPipelineStringToStringFullExample.java

> Beam slowness compared to flink-native
> --------------------------------------
>
>                 Key: BEAM-14438
>                 URL: https://issues.apache.org/jira/browse/BEAM-14438
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>    Affects Versions: 2.38.0
>            Reporter: Ifat Afek
>            Priority: P2
>         Attachments: BeamConvertToStringRecordExample.java, BeamPipelineStringToStringFullExample.java, FlinkStringProcessingServiceExample.java
>
>
> I tried to compare a very simple beam pipeline with an equivalent flink-native pipeline. Both pipelines read strings from one kafka topic and write them to another topic. I ran each pipeline separately on a single task manager with a single slot and parallelism 1.
> Flink native runs 5 times faster than beam: 150,000 strings per second in flink comparing to 30,000 in beam.
> When using Avro and schema registry the difference is even more significant - flink native runs 30-80 times faster than beam.
> Attached is the java code of both string-to-string pipelines.
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.7#820007)