How to give lesser memory(-xmx) for logwriter process and higher for worker process?

[saurabh mimani <mi...@gmail.com>]

Version Info:
   "org.apache.storm" % "storm-core" % "1.2.1"
   "org.apache.storm" % "storm-kafka-client" % "1.2.1"

With this version of storm, when running topology, I see two processes for
each worker: one for logwriter and other for worker, as mentioned here
<https://stackoverflow.com/a/42517250/1610034> as well.

The problem is I want to give less memory to logwriter process(say
-Xmx512mb) and higher to worker process(say -Xmx12gb). But I am not able to
achieve this, I have tried playing with different config.

There is config topology.worker.logwriter.childopts, which is taken by
logwriter process, but than I have given some config to worker.childopts ,
but that also gets applied to both logwriter and worker process. Similary I
tried using topology.worker.childopts and that also gets applied to both
logwriter and worker process.

How to solve this to give lesser memory for logwriter process and higher
for worker process?

Best Regards

Saurabh Kumar Mimani

Re: How to give lesser memory(-xmx) for logwriter process and higher for worker process?

[Stig Rohde Døssing <st...@gmail.com>]

Are you sure topology.worker.childopts are actually being applied to the
logwriter JVM? Keep in mind that the entire worker launch command is being
passed through as command line parameters to the logwriter main method, so
topology.worker.childopts will appear in the command line for the
logwriter, but the parameters should appear after the
"org.apache.storm.LogWriter"
part, so shouldn't be applied to the logwriter JVM.

See
https://github.com/apache/storm/blob/eaed3cbf3610bbefb9466f2c40db1e0e2a20ca9c/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L643
(this is for 2.0.0, but I'd imagine the same thing happens in 1.x)

Den lør. 10. nov. 2018 kl. 04.48 skrev saurabh mimani <
mimani.saurabh@gmail.com>:

>
>
>
>
> Version Info:
>    "org.apache.storm" % "storm-core" % "1.2.1"
>    "org.apache.storm" % "storm-kafka-client" % "1.2.1"
>
> With this version of storm, when running topology, I see two processes for
> each worker: one for logwriter and other for worker, as mentioned here
> <https://stackoverflow.com/a/42517250/1610034> as well.
>
> The problem is I want to give less memory to logwriter process(say
> -Xmx512mb) and higher to worker process(say -Xmx12gb). But I am not able to
> achieve this, I have tried playing with different config.
>
> There is config topology.worker.logwriter.childopts, which is taken by
> logwriter process, but than I have given some config to worker.childopts ,
> but that also gets applied to both logwriter and worker process. Similary I
> tried using topology.worker.childopts and that also gets applied to both
> logwriter and worker process.
>
> How to solve this to give lesser memory for logwriter process and higher
> for worker process?
>
> Best Regards
>
> Saurabh Kumar Mimani
>
>
>