You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Joe Percivall <jp...@apache.org> on 2018/08/30 17:54:20 UTC
Best practices for running Apache NiFi in production in a Docker container
Hey everyone,
I was recently searching for a best practice guide for running a production
instance of Apache NiFi within a Docker container and couldn't find
anything specific other than the normal guidance for best practices of a
high-performance instance[1]. I did expand my search for best practices on
running the JVM within a container and found a couple good articles[2][3].
The first of which explains why the JVM will take up more than is set via
"Xmx" and the second is about 2 JVM options which were backported from Java
9 to JDK 8u131 specifically for configuring the JVM heap for running in a
"VM".
So with that, a couple questions:
1: Does anyone have any best practices or lessons learned specifically for
running NiFi in a container?
2: "UseCGroupMemoryLimitForHeap" and "MaxRAMFraction" are technically
"Experimental VM Options", has anyone used them in practice?
[1]
https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.html
[2]
https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/#more-433899
[3]
https://blog.csanchez.org/2017/05/31/running-a-jvm-in-a-container-without-getting-killed/
Thanks,
Joe
--
*Joe Percivall*
linkedin.com/in/Percivall
e: jpercivall@apache.com
Re: Best practices for running Apache NiFi in production in a Docker container
Posted by Joe Percivall <jp...@apache.org>.
Hi Peter,
Thanks for the follow-up. Yup, I agree with the relationship between
Xmx/Xms and UseCGroupMemoryLimitForHeap/MaxRAMFraction.
For the MaxRAMFraction=1 though, it seems that it does leave at least some
room for off-heap memory that said, that may not be enough for NiFi and
it's normal use-cases. Has anyone run the Native Memory Tracker[1] on a
"real-world" system? It gives a nice dump of where/how the JVM is using
memory[2].
[1]
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html
[2] http://trustmeiamadeveloper.com/2016/03/18/where-is-my-memory-java/
Joe
On Fri, Aug 31, 2018 at 5:10 AM Peter Wilcsinszky <
peterwilcsinszky@gmail.com> wrote:
> Hi,
>
> I haven't done extensive research in this area but ran through the
> articles and also found another one [1]. From what I understand
> UseCGroupMemoryLimitForHeap is just the dynamic version of setting memory
> limits manually using Xmx and Xms which is currently done by the NiFi start
> script explicitly. In an environment where it should be done in a more
> dynamic fashion the UseCGroupMemoryLimitForHeap with proper MaxRAMFraction
> should be used but for caveats check the comments here: [1] and here: [2]
> (My understanding: MaxRAMFraction=1 considered to be unsafe,
> MaxRAMFraction=2 leaves half the memory unused)
>
> [1] https://banzaicloud.com/blog/java-resource-limits/
> [2]
> https://stackoverflow.com/questions/49854237/is-xxmaxramfraction-1-safe-for-production-in-a-containered-environment
>
>
> On Thu, Aug 30, 2018 at 7:54 PM Joe Percivall <jp...@apache.org>
> wrote:
>
>> Hey everyone,
>>
>> I was recently searching for a best practice guide for running a
>> production instance of Apache NiFi within a Docker container and couldn't
>> find anything specific other than the normal guidance for best practices of
>> a high-performance instance[1]. I did expand my search for best practices
>> on running the JVM within a container and found a couple good
>> articles[2][3]. The first of which explains why the JVM will take up more
>> than is set via "Xmx" and the second is about 2 JVM options which were
>> backported from Java 9 to JDK 8u131 specifically for configuring the JVM
>> heap for running in a "VM".
>>
>> So with that, a couple questions:
>> 1: Does anyone have any best practices or lessons learned specifically
>> for running NiFi in a container?
>> 2: "UseCGroupMemoryLimitForHeap" and "MaxRAMFraction" are technically
>> "Experimental VM Options", has anyone used them in practice?
>>
>> [1]
>> https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.html
>>
>> [2]
>> https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/#more-433899
>> [3]
>> https://blog.csanchez.org/2017/05/31/running-a-jvm-in-a-container-without-getting-killed/
>>
>> Thanks,
>> Joe
>> --
>> *Joe Percivall*
>> linkedin.com/in/Percivall
>> e: jpercivall@apache.com
>>
>
--
*Joe Percivall*
linkedin.com/in/Percivall
e: jpercivall@apache.com
Re: Best practices for running Apache NiFi in production in a Docker container
Posted by Peter Wilcsinszky <pe...@gmail.com>.
Hi,
I haven't done extensive research in this area but ran through the articles
and also found another one [1]. From what I understand
UseCGroupMemoryLimitForHeap is just the dynamic version of setting memory
limits manually using Xmx and Xms which is currently done by the NiFi start
script explicitly. In an environment where it should be done in a more
dynamic fashion the UseCGroupMemoryLimitForHeap with proper MaxRAMFraction
should be used but for caveats check the comments here: [1] and here: [2]
(My understanding: MaxRAMFraction=1 considered to be unsafe,
MaxRAMFraction=2 leaves half the memory unused)
[1] https://banzaicloud.com/blog/java-resource-limits/
[2]
https://stackoverflow.com/questions/49854237/is-xxmaxramfraction-1-safe-for-production-in-a-containered-environment
On Thu, Aug 30, 2018 at 7:54 PM Joe Percivall <jp...@apache.org> wrote:
> Hey everyone,
>
> I was recently searching for a best practice guide for running a
> production instance of Apache NiFi within a Docker container and couldn't
> find anything specific other than the normal guidance for best practices of
> a high-performance instance[1]. I did expand my search for best practices
> on running the JVM within a container and found a couple good
> articles[2][3]. The first of which explains why the JVM will take up more
> than is set via "Xmx" and the second is about 2 JVM options which were
> backported from Java 9 to JDK 8u131 specifically for configuring the JVM
> heap for running in a "VM".
>
> So with that, a couple questions:
> 1: Does anyone have any best practices or lessons learned specifically for
> running NiFi in a container?
> 2: "UseCGroupMemoryLimitForHeap" and "MaxRAMFraction" are technically
> "Experimental VM Options", has anyone used them in practice?
>
> [1]
> https://community.hortonworks.com/articles/7882/hdfnifi-best-practices-for-setting-up-a-high-perfo.html
>
> [2]
> https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/#more-433899
> [3]
> https://blog.csanchez.org/2017/05/31/running-a-jvm-in-a-container-without-getting-killed/
>
> Thanks,
> Joe
> --
> *Joe Percivall*
> linkedin.com/in/Percivall
> e: jpercivall@apache.com
>