[jira] [Commented] (BEAM-7420) Including the Flink runner causes exceptions unless running in a flink environment

["Mike Kaplinskiy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-7420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16859306#comment-16859306 ] 

Mike Kaplinskiy commented on BEAM-7420:
---------------------------------------

Sorry for the late response. I manually excluded the flink-runner bits (I'm on a somewhat unorthodox build system (Buck) so I simply removed the dependencies). When the job runs in a flink cluster, the cluster includes all the necessary flink jars (as well as some others - e.g. to write checkpoints/savepoints to S3).

I'm curious - would it make sense to mark the flink dependencies in beam as {{provided}}? I'm not 100% sure it's possible to run a flink cluster from just the java class files - you probably need the rest of the flink distribution anyway, which definitely comes with the flink jars?

> Including the Flink runner causes exceptions unless running in a flink environment
> ----------------------------------------------------------------------------------
>
>                 Key: BEAM-7420
>                 URL: https://issues.apache.org/jira/browse/BEAM-7420
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>            Reporter: Mike Kaplinskiy
>            Assignee: Maximilian Michels
>            Priority: Major
>
> The {{FlinkPipelineOptions}} imports various Flink enums which make it impossible to e.g. run the direct runner with the same classpath but without the flink runtime. The fix is potentially easy - make the arguments strings and convert them to enums at the callsites.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)