[jira] [Commented] (NIFI-4837) Thread leak on HandleHTTPRequest processor

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355894#comment-16355894 ] 

ASF GitHub Bot commented on NIFI-4837:
--------------------------------------

GitHub user mcgilman opened a pull request:

    https://github.com/apache/nifi/pull/2455

    NIFI-4837: Addressing thread leak in HandleHTTPRequest

    NIFI-4837:
    - When Jetty initializes fails, performing a shutdown sequence to ensure all allocated resources are released.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mcgilman/nifi NIFI-4837

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/nifi/pull/2455.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2455
    
----
commit 535fadab4830de0634617eb66698fc9204987b69
Author: Matt Gilman <ma...@...>
Date:   2018-02-07T18:52:02Z

    NIFI-4837:
    - When Jetty initializes fails, performing a shutdown sequence to ensure all allocated resources are released.

----


> Thread leak on HandleHTTPRequest processor
> ------------------------------------------
>
>                 Key: NIFI-4837
>                 URL: https://issues.apache.org/jira/browse/NIFI-4837
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0
>         Environment: CENTOS 7
>            Reporter: Matthew Clarke
>            Assignee: Matt Gilman
>            Priority: Blocker
>         Attachments: image-2018-02-02-11-14-51-964.png, image-2018-02-02-11-16-52-389.png
>
>
> When you have multiple HandleHTTPRequest processors trying to listen on the same port, for every Listen attempt NiFi builds a new thread and never recycles the old thread which eventually leads to NiFi shutting down when reaching the OS limit of the number of threads (default is 10.000). 
> The following error can be seen in nifi-app.log: 
> Caused by: java.lang.OutOfMemoryError: unable to create new native thread 
> at java.lang.Thread.start0(Native Method) 
> at java.lang.Thread.start(Thread.java:714) 
> This has happened before with version 1.2 and probably even with older versions. but I could also replicate the issue with the latest 1.5 version. 
> Steps to replicate the issue: 
> 1) build a simple flow with 2 HandleHTTPRequest processors listening on the same port. 
> !image-2018-02-02-11-14-51-964.png!
> 2) Start the processors. 
>   —  The second HandleHTTPRequest processor starts logging following as expected:
> 2018-02-02 16:18:29,518 ERROR [Timer-Driven Process Thread-3] o.a.n.p.standard.HandleHttpRequest HandleHttpRequest[id=af013c62-b26f-1eeb-ae81-8423c70bdc7f] Failed to process session due to org.apache.nifi.processor.exception.ProcessException: Failed to initialize the server: {}
> org.apache.nifi.processor.exception.ProcessException: Failed to initialize the server
> ....
> Caused by: java.net.BindException: Address already in use
> ...
> ... 12 common frames omitted
>  
> 3) Go to the Summary section in NiFi and watch the number of threads going up to 9959.
> !image-2018-02-02-11-16-52-389.png!
>  
> With above, I had processors scheduled on primary node only as to not affect every node.
> If you stop the second HandleHTTPRequest processor the threads stop climbing, but are not released.
>  
> After this, NiFi will soon stop.
>  
> A restart of NIFi is required to release these threads.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)