[jira] [Commented] (FLINK-21024) Dynamic properties get exposed to job's main method if user parameters are passed

["Xintong Song (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-21024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17267780#comment-17267780 ] 

Xintong Song commented on FLINK-21024:
--------------------------------------

I think the problem is that, `standalone-job.sh` failed to make sure all flink options comes before the job arguments.

{{ClusterEntrypoint}} reads flink options from the beginning of the JVM process argument list. It stops at the first unrecognized option, and treat the rest as the job arguments.

This is not reproduced on a standalone session cluster because job arguments are provided at submitting the job rather than at starting the cluster.

I can try to provide a fix.

> Dynamic properties get exposed to job's main method if user parameters are passed
> ---------------------------------------------------------------------------------
>
>                 Key: FLINK-21024
>                 URL: https://issues.apache.org/jira/browse/FLINK-21024
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Configuration
>    Affects Versions: 1.12.1
>            Reporter: Matthias
>            Priority: Major
>              Labels: starter
>         Attachments: WordCount.jar
>
>
> A bug was identified in the [user ML|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Application-cluster-standalone-job-some-JVM-Options-added-to-Program-Arguments-td40719.html] by Alexey exposing dynamic properties into the job user code.
> I was able to reproduce this issue by slightly adapting the WordCount example ({{org.apache.flink.streaming.examples.wordcount.WordCount2}} in attached  [^WordCount.jar] ).
> Initiating a standalone job without using the {{--input}} parameter will result in printing an empty array:
> {code}
> ./bin/standalone-job.sh start --job-classname org.apache.flink.streaming.examples.wordcount.WordCount2
> {code}
> The corresponding {{*.out}} file looks like this:
> {code}
> []
> Executing WordCount2 example with default input data set.
> Use --input to specify file input.
> Printing result to stdout. Use --output to specify output path.
> {code}
> In contrast, initiating the standalone job using the {{--input}} parameter will expose the dynamic properties:
> {code}
> ./bin/standalone-job.sh start --job-classname org.apache.flink.streaming.examples.wordcount.WordCount2 --input /opt/flink/config/flink-conf.yaml
> {code}
> Resulting in the following output:
> {code}
> [--input, /opt/flink/config/flink-conf.yaml, -D, jobmanager.memory.off-heap.size=134217728b, -D, jobmanager.memory.jvm-overhead.min=201326592b, -D, jobmanager.memory.jvm-metaspace.size=268435456b, -D, jobmanager.memory.heap.size=1073741824b, -D, jobmanager.memory.jvm-overhead.max=201326592b]
> Printing result to stdout. Use --output to specify output path.
> {code}
> Interestingly, this cannot be reproduced on a local standalone session cluster.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)