[GitHub] [beam] kennknowles opened a new issue, #18192: Flink runner: submit job to a Flink-on-YARN cluster

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #18192:
URL: https://github.com/apache/beam/issues/18192

   As far as I understand, running Beam pipelines on a Flink cluster can be done in two ways:
   * Run directly with a Flink runner, and specifying `--flinkMaster` pipeline option via, say, `mvn exec`.
   * Produce a bundled JAR, and use `bin/flink` to submit the same pipeline.
   
   These two ways are equivalent, and work well on a standalone Flink cluster.
   
   Submitting to a Flink-on-YARN is more complicated. You can still produce a bundled JAR, and use `bin/flink -yid <applicationid>` to submit such a job. However, that seems impossible with a Flink runner directly.
   
   If so, we should add the ability to the Flink runner to submit a job to a Flink-on-YARN cluster directly.
   
   Imported from Jira [BEAM-1631](https://issues.apache.org/jira/browse/BEAM-1631). Original Jira may contain additional context.
   Reported by: davor.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org