You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Sri Ganesh Venkataraman <sr...@gmail.com> on 2024/03/22 08:40:36 UTC

[Question] Apache Beam Spark Runner Support - Spark 3.5, Scala 2.13 Environment

Hello,

Does the latest Apache Beam version support Spark 3.5, Scala 2.13
environment for spark runner ?
Based on initial analysis -
Spark runner supports Spark 3.5.0 version
*Reference*:
https://github.com/apache/beam/blob/f660f49b6cf5d241285b156ccc4a5f52d82cfa63/runners/spark/3/build.gradle#L36

Would like to know compatibility with Scala version 2.13

Based on the below references, support is extended only to Scala version
2.12 (Libraries are compiled with Scala version 2.12)
https://github.com/apache/beam/blob/0dc330e6d53cce51b13bcb652e80859c1b3a5975/runners/spark/3/build.gradle#L24
https://mvnrepository.com/artifact/org.apache.beam/beam-runners-spark-3/2.54.0

Any insight/future roadmap plans on Scala 2.13 support will be very helpful

Thanks,
Sri Ganesh V