Ship files to flink on yarn-cluster mode

[Guy Harmach <Gu...@Amdocs.com>]

Hi,

Is there a way to ship files to a flink job running on YARN (similar to -files flag in Spark)?
According the flink cli usage there is a  -yarnship flag, but no documentation how to use it and retrieve the files on the flink job.
If it is possible, I'd appreciate a reference to an example of sending a file and reading it.

Thanks, Guy
This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer <https://www.amdocs.com/about/email-disclaimer>

Re: Ship files to flink on yarn-cluster mode

[Till Rohrmann <tr...@apache.org>]

Hi Guy,

you can use the -yarnship command line option to specify a directory whose
content will be shipped to the launched yarn application. Thus, all files
contained in this directory will be copied to HDFS from where they are
accessible by Flink.

Cheers,
Till

On Thu, Jul 13, 2017 at 4:40 PM, Guy Harmach <Gu...@amdocs.com> wrote:

> Hi,
>
> Is there a way to ship files to a flink job running on YARN (similar to
> -files flag in Spark)?
> According the flink cli usage there is a  -yarnship flag, but no
> documentation how to use it and retrieve the files on the flink job.
> If it is possible, I'd appreciate a reference to an example of sending a
> file and reading it.
>
> Thanks, Guy
> This message and the information contained herein is proprietary and
> confidential and subject to the Amdocs policy statement,
>
> you may review at https://www.amdocs.com/about/email-disclaimer <
> https://www.amdocs.com/about/email-disclaimer>
>