You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Dima Kovalyov (JIRA)" <ji...@apache.org> on 2017/03/06 23:40:32 UTC
[jira] [Comment Edited] (NIFI-2699) Improve handling of response
timeouts in cluster
[ https://issues.apache.org/jira/browse/NIFI-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898390#comment-15898390 ]
Dima Kovalyov edited comment on NIFI-2699 at 3/6/17 11:40 PM:
--------------------------------------------------------------
I hit this problem all the time when I work with large NiFi flows, 6 groups, up-to 8 processors in each with 30000+ flow files in queue.
was (Author: dima_k):
I hit this problem all the time when i work with large NiFi flows, 6 groups, up-to 8 processors in each with 30000+ flow files in queue.
> Improve handling of response timeouts in cluster
> ------------------------------------------------
>
> Key: NIFI-2699
> URL: https://issues.apache.org/jira/browse/NIFI-2699
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Core UI
> Reporter: Jeff Storck
> Priority: Minor
>
> When running as a cluster, if a node is unable to respond within the socket timeout (eg, hitting a breakpoint while debugging), an IllegalClusterStateException will be thrown that causes the UI to show the "check config and fix errors" page. Once the node is communicating with the cluster again (i.e., breakpoint in the code is passed), the UI can be reloaded and the cluster recovers from the timeout without any user intervention at the service level. However, user experience could be improved. If a user initiates a replicated request to a node that is unable to respond within the socket timeout duration, the user might think NiFi crashed, when it in fact didn't.
> Here is the stack trace that was encountered during testing:
> {code}
> 2016-08-29 11:36:59,041 DEBUG [NiFi Web Server-22] o.a.n.w.a.c.IllegalClusterStateExceptionMapper
> org.apache.nifi.cluster.manager.exception.IllegalClusterStateException: Node localhost:8443 is unable to fulfill this request due to: Unexpected Response Code 500
> at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$2.onCompletion(ThreadPoolRequestReplicator.java:471) ~[nifi-framework-cluster-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:729) ~[nifi-framework-cluster-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_92]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_92]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_92]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_92]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_92]
> Caused by: com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out
> at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:560) ~[jersey-client-1.19.jar:1.19]
> at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:537) ~[nifi-framework-cluster-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:720) ~[nifi-framework-cluster-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> ... 5 common frames omitted
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.8.0_92]
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[na:1.8.0_92]
> at java.net.SocketInputStream.read(SocketInputStream.java:170) ~[na:1.8.0_92]
> at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[na:1.8.0_92]
> at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) ~[na:1.8.0_92]
> at sun.security.ssl.InputRecord.read(InputRecord.java:503) ~[na:1.8.0_92]
> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) ~[na:1.8.0_92]
> at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930) ~[na:1.8.0_92]
> at sun.security.ssl.AppInputStream.read(AppInputStream.java:105) ~[na:1.8.0_92]
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[na:1.8.0_92]
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[na:1.8.0_92]
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[na:1.8.0_92]
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704) ~[na:1.8.0_92]
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) ~[na:1.8.0_92]
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536) ~[na:1.8.0_92]
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) ~[na:1.8.0_92]
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[na:1.8.0_92]
> at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338) ~[na:1.8.0_92]
> at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19]
> ... 11 common frames omitted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)