[jira] [Updated] (NIFI-6908) PutKudu 1.10.0 Memory Leak

["Joe Witt (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joe Witt updated NIFI-6908:
---------------------------
    Fix Version/s: 1.11.0

> PutKudu 1.10.0 Memory Leak
> --------------------------
>
>                 Key: NIFI-6908
>                 URL: https://issues.apache.org/jira/browse/NIFI-6908
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.10.0
>         Environment: NiFi 1.10.0 8-Node Cluster; Kudu 1.10.0
>            Reporter: Josef Zahner
>            Assignee: Grant Henke
>            Priority: Blocker
>              Labels: heap, kudu, oom
>             Fix For: 1.11.0
>
>         Attachments: PutKudu_Properties.png, PutKudu_Scheduling.png, PutKudu_Settings.png, memory_leak.png
>
>
> PutKudu 1.10.0 eats up all the heap memory and garbage collection can't anymore free up memory after a few hours.
> We have an NiFi 8-Node cluster (31GB java max memory configured) with a streaming source which generates constantly about 2'500 flowfiles/2.5GB data in 5 minutes. In our example the streaming source was running on "nifi-05" (green line). As you can see between 00:00 and 04:00 the memory grows and grows and at the end the node became instable and the dreaded "java.lang.OutOfMemoryError: Java heap space" message appeared. We tried to do a manual garbage collection with visualvm profiler, but it didn't helped.  
> !memory_leak.png!
> We are sure that the PutKudu is the culprit, as we have now taken the codebase from PutKudu 1.9.2 and use it now in NiFi 1.10.0 without any leaks at all.
> With the official PutKudu 1.10.0 processor our cluster crashed within 5-6 hours with our current load as the memory was completely full.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)