[jira] [Resolved] (NIFI-3978) Site-to-Site HTTP client tests fails on machines with too many cores

["Koji Kawamura (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Koji Kawamura resolved NIFI-3978.
---------------------------------
       Resolution: Fixed
    Fix Version/s: 1.3.0

> Site-to-Site HTTP client tests fails on machines with too many cores
> --------------------------------------------------------------------
>
>                 Key: NIFI-3978
>                 URL: https://issues.apache.org/jira/browse/NIFI-3978
>             Project: Apache NiFi
>          Issue Type: Bug
>            Reporter: Joey Frazee
>            Assignee: Joey Frazee
>            Priority: Minor
>             Fix For: 1.3.0
>
>
> The S2S http client tests will fail on machines with many cores because the [thread pool limits|https://github.com/apache/nifi/blob/master/nifi-commons/nifi-site-to-site-client/src/test/java/org/apache/nifi/remote/client/http/TestHttpClient.java#L435] are too low (see error below).
> Looks like the reason for this is that Jetty will use acceptors + selectors + request=1 threads, but calculates both the number of acceptors and selectors based on the number of hyper threads available:
> * The default number of acceptor tasks is the minimum of 1 and the number of available CPUs divided by 8
> * The default number of selectors is equal to half of the number of processors available to the JVM
> So suppose you have a 12 core 2 HT machine. Then you'll use 12 * 2 / 8 + 12 * 2 / 2 + 1 = 16 threads and all is well with the current limit of 20. But if you have a 16 core machine, you'll use 16 * 2 / 8 + 16 * 2 / 2 + 1 = 21 and go over the limit.
> We should bump this up to 50 which should accommodate builds on 2 x 16 core.
> {code}
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient
> org.apache.nifi.remote.client.http.TestHttpClient  Time elapsed: 0 sec  <<< ERROR!
> java.lang.IllegalStateException: Insufficient threads: max=20 < needed(acceptors=4 + selectors=16 + request=1)
> 	at org.eclipse.jetty.server.Server.doStart(Server.java:414)
> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> 	at org.apache.nifi.remote.client.http.TestHttpClient.setup(TestHttpClient.java:501)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)