[jira] [Commented] (NIFI-9767) PutKudu OutOfMemory Exception (GC overhead limit exceeded)

["David Handermann (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-9767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502275#comment-17502275 ] 

David Handermann commented on NIFI-9767:
----------------------------------------

Thanks for reporting this issue [~jian659].  Can you provide some additional details about your system configuration?  In particular, providing the amount of physical memory, along with the maximum heap size configured in bootstrap.conf would be helpful.

The {{PutKudu}} processor includes the following system resource consideration note regarding memory:

??An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.??

The Kudu client leverages the Netty library, which can result in using notable amounts of memory due to thread pool configuration.  Each PutKudu processor instances creates a new thread pool, which directly impacts memory consumption.  For this reason, the number of PutKudu processors in a given NiFi flow should be as limited as possible.  The {{Kudu Client Worker Count}} property controls the number of client threads for the processor instance.  The default value is two times the number of reported CPU cores.  When configuring multiple instances of PutKudu, the worker count property should be reduced to lessen the impact on memory consumption.  The worker count property should generally align with the concurrent task configuration for the processor.

With that background, limiting the number of PutKudu processor instances, and reducing the worker count for each processor, should help address memory consumption.


> PutKudu OutOfMemory Exception (GC overhead limit exceeded)
> ----------------------------------------------------------
>
>                 Key: NIFI-9767
>                 URL: https://issues.apache.org/jira/browse/NIFI-9767
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.15.0
>         Environment: NiFi 1.15.0
>            Reporter: jason
>            Priority: Blocker
>         Attachments: free_heap.png, pipeline.png, total_threads.png
>
>
> PutKudu 1.15.0 eats up all the heap memory. Must restart NiFi to free heap.
> We create twenty the same of pipelines for test, will out of memory on few hours.Pipeline as shown in pipeline.png
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)