You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Soumabrata Chakraborty (Jira)" <ji...@apache.org> on 2020/03/04 19:35:00 UTC

[jira] [Created] (BEAM-9440) Performance Issue with Beam

Soumabrata Chakraborty created BEAM-9440:
--------------------------------------------

             Summary: Performance Issue with Beam
                 Key: BEAM-9440
                 URL: https://issues.apache.org/jira/browse/BEAM-9440
             Project: Beam
          Issue Type: Bug
          Components: runner-spark
            Reporter: Soumabrata Chakraborty


While doing a performance evaluation of Apache Beam with Spark Runner - I found that even for a simple word count problem on a text file – Beam with Spark runner was slower by a factor of 5 times as compared to Spark for a dataset as small as 14 GB.

You will find more details on this evaluation here - [https://github.com/soumabrata-chakraborty/spark-vs-beam/blob/master/README.md]

I also came across this analysis called _**Quantitative Impact Evaluation of an Abstraction Layer for Data Stream Processing Systems_ ([https://arxiv.org/pdf/1907.08302.pdf])

According to it, the observation was that for most scenarios the slowdown was at least a factor of 3 with the worse case being a factor of 58!

While it is understood that an abstraction layer would come with some performance cost - the current performance cost seems to be very high.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)