[jira] [Commented] (BEAM-6058) Support flink config directory for flink runner.

["Thomas Weise (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-6058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688253#comment-16688253 ] 

Thomas Weise commented on BEAM-6058:
------------------------------------

I don't see why old Flink runner and job server server should behave differently. Both see the same environment and can load any configuration present.

Flink has an environment variable to identify the config dir: [https://github.com/apache/flink/blob/1a9675d54fda7c6d7c519935dde05f47eb449401/flink-core/src/main/java/org/apache/flink/configuration/GlobalConfiguration.java#L66]

That could be set in either case and should cause the desired configuration to be loaded.

Passing a local file reference as pipeline option seems odd, since it implies the user having knowledge about the job server environment.

 

 

> Support flink config directory for flink runner.
> ------------------------------------------------
>
>                 Key: BEAM-6058
>                 URL: https://issues.apache.org/jira/browse/BEAM-6058
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>            Reporter: Ankur Goenka
>            Assignee: Ankur Goenka
>            Priority: Major
>          Time Spent: 4h 50m
>  Remaining Estimate: 0h
>
> Give user option to use flink configuration from a config directory.
> Earlier we only used jars to submit flink jobs which by default picks the cluster flink properties.
> Portability uses  remote environment to submit flink job. In Remote execution, flink uses the client config instead of the cluster config. This require us to submit the custom flink properties via flink client.
> Suggestion is to read the flink config from a flink configuration folder and set them to the environment.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)