You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Grant Henke (Jira)" <ji...@apache.org> on 2019/11/28 00:03:00 UTC
[jira] [Assigned] (NIFI-6908) PutKudu 1.10.0 Memory Leak
[ https://issues.apache.org/jira/browse/NIFI-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Henke reassigned NIFI-6908:
---------------------------------
Assignee: Grant Henke
> PutKudu 1.10.0 Memory Leak
> --------------------------
>
> Key: NIFI-6908
> URL: https://issues.apache.org/jira/browse/NIFI-6908
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.10.0
> Environment: NiFi 1.10.0 8-Node Cluster; Kudu 1.10.0
> Reporter: Josef Zahner
> Assignee: Grant Henke
> Priority: Blocker
> Labels: heap, kudu, oom
> Attachments: memory_leak.png
>
>
> PutKudu 1.10.0 eats up all the heap memory and garbage collection can't anymore free up memory after a few hours.
> We have an NiFi 8-Node cluster (31GB java max memory configured) with a streaming source which generates constantly about 2'500 flowfiles/2.5GB data in 5 minutes. In our example the streaming source was running on "nifi-05" (green line). As you can see between 00:00 and 04:00 the memory grows and grows and at the end the node became instable and the dreaded "java.lang.OutOfMemoryError: Java heap space" message appeared. We tried to do a manual garbage collection with visualvm profiler, but it didn't helped.
> !memory_leak.png!
> We are sure that the PutKudu is the culprit, as we have now taken the codebase from PutKudu 1.9.2 and use it now in NiFi 1.10.0 without any leaks at all.
> With the official PutKudu 1.10.0 processor our cluster crashed within 5-6 hours with our current load as the memory was completely full.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)