You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "V N, Suchithra (Nokia - IN/Bangalore)" <su...@nokia.com> on 2021/05/10 12:15:53 UTC
Query on passing memory parameters
Hello,
We are using Apache flink 1.12.1 version and using standalone-job.sh to start the per-job cluster mode flink. In the logs we could see some memory configurations parameters are being added as dynamic parameters.
When checked the Java process system properties we can see the below output:
sun.java.command=org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint --configDir /opt/flink/conf --job-classname com.nokia.csf.ccom.flink.poc.FlinkPoCMain --output-topic singleStream1 --profile_result /tmp/profileResult1.csv --kafka.bootstrap.servers kf-kafka-headless.flkist.svc.cluster.local\:9092 --proto udp --port 2099 --kafka.batch.size 3000 -D jobmanager.memory.off-heap.size\=134217728b -D jobmanager.memory.jvm-overhead.min\=201326592b -D jobmanager.memory.jvm-metaspace.size\=268435456b -D jobmanager.memory.heap.size\=1073741824b -D jobmanager.memory.jvm-overhead.max\=201326592b
As highlighted in the above captured output, we can see the extra space between -D and following word. Is this expected ? Whereas the log configuration settings are passed in below format by flink without space.
-Dlog.file=/opt/flink/log/flink--standalonejob-0-f1-flink-jobmanager-6886f8745b-mzm69.log
Jcmd output:
log.file=/opt/flink/log/flink--standalonejob-0-f1-flink-jobmanager-bd46c76d9-9hdgx.log
Please share your inputs.
Regards,
Suchithra
Re: Query on passing memory parameters
Posted by Chesnay Schepler <ch...@apache.org>.
yes, the space is expected. The logging settings do not have a space
because they are set as environment variables, whereas the memory
options are essentially just set as program arguments.
On 5/10/2021 2:15 PM, V N, Suchithra (Nokia - IN/Bangalore) wrote:
>
> Hello,
>
> We are using Apache flink 1.12.1 version and using standalone-job.sh
> to start the per-job cluster mode flink. In the logs we could see some
> memory configurations parameters are being added as dynamic parameters.
>
> When checked the Java process system properties we can see the below
> output:
>
> sun.java.command=org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint
> --configDir /opt/flink/conf --job-classname
> com.nokia.csf.ccom.flink.poc.FlinkPoCMain --output-topic singleStream1
> --profile_result /tmp/profileResult1.csv --kafka.bootstrap.servers
> kf-kafka-headless.flkist.svc.cluster.local\:9092 --proto udp --port
> 2099 --kafka.batch.size 3000 -D
> jobmanager.memory.off-heap.size\=134217728b -D
> jobmanager.memory.jvm-overhead.min\=201326592b -D
> jobmanager.memory.jvm-metaspace.size\=268435456b -D
> jobmanager.memory.heap.size\=1073741824b -D
> jobmanager.memory.jvm-overhead.max\=201326592b
>
> As highlighted in the above captured output, we can see the extra
> space between -D and following word. Is this expected ? Whereas the
> log configuration settings are passed in below format by flink without
> space.
>
> -Dlog.file=/opt/flink/log/flink--standalonejob-0-f1-flink-jobmanager-6886f8745b-mzm69.log
>
> Jcmd output:
>
> log.file=/opt/flink/log/flink--standalonejob-0-f1-flink-jobmanager-bd46c76d9-9hdgx.log
>
>
> Please share your inputs.
>
> Regards,
>
> Suchithra
>