You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Koji Kawamura (JIRA)" <ji...@apache.org> on 2017/09/11 09:06:00 UTC
[jira] [Updated] (NIFI-4376) Expired requests stay in
StansdardRootGroupPort request queue and block new request to be offered
[ https://issues.apache.org/jira/browse/NIFI-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Kawamura updated NIFI-4376:
--------------------------------
Description:
If the remote NiFi instance slows down for some reason, and S2S requests have been timed out continuously, those timed out requests will remain in the internal request queue at StandardRootGroupPort.
Once number of queued requests becomes 1,000, newly received request can not be offered to the queue, and RequestExpiredException is thrown. A S2S client receives following error in this case:
{code}
2017-09-11 16:32:38,137 ERROR [Timer-Driven Process Thread-4] o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=output,target=http://localhost:8080/nifi] failed to communicate with http://localhost:8080/nifi due to java.io.IOException: Unexpected response code: 500 errCode:Abort errMessage:<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /nifi-api/data-transfer/output-ports/6fc7fa6d-015e-1000-27cb-d118b1b0fe13/transactions/73da9bcf-87ec-4594-8e45-a41965c1ed82/flow-files. Reason:
<pre> Server Error</pre></p><h3>Caused by:</h3><pre>java.io.IOException: Failed to process the request.
at org.apache.nifi.web.api.DataTransferResource$1.write(DataTransferResource.java:676)
at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:71)
...
<h3>Caused by:</h3><pre>org.apache.nifi.remote.exception.RequestExpiredException
at org.apache.nifi.remote.StandardRootGroupPort.transferFlowFiles(StandardRootGroupPort.java:562)
at org.apache.nifi.web.api.DataTransferResource$1.write(DataTransferResource.java:668)
{code}
Once a NiFi instance owning the root group port enters this state, it will not be able to recover automatically as its client keeps retrying too often. And the returned error message is not clear to understand what is happening.
The remote NiFi instance should remove already expired requests from its internal queue, so that newly received requests are processed correctly, and return 'read timeout' error response if it can not be processed and expired.
was:
If the remote NiFi instance slows down for some reason, and S2S requests have been timed out continuously, those timed out requests will remain in the internal request queue at StandardRootGroupPort.
Once number of queued requests becomes 1,000, newly received request can not be offered to the queue, and RequestExpiredException is thrown. A S2S client receives following error in this case:
{code}
2017-09-11 16:32:38,137 ERROR [Timer-Driven Process Thread-4] o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=output,target=http://localhost:8080/nifi] failed to communicate with http://localhost:8080/nifi due to java.io.IOException: Unexpected response code: 500 errCode:Abort errMessage:<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 500 Server Error</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /nifi-api/data-transfer/output-ports/6fc7fa6d-015e-1000-27cb-d118b1b0fe13/transactions/73da9bcf-87ec-4594-8e45-a41965c1ed82/flow-files. Reason:
<pre> Server Error</pre></p><h3>Caused by:</h3><pre>java.io.IOException: Failed to process the request.
at org.apache.nifi.web.api.DataTransferResource$1.write(DataTransferResource.java:676)
at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:71)
...
<h3>Caused by:</h3><pre>org.apache.nifi.remote.exception.RequestExpiredException
at org.apache.nifi.remote.StandardRootGroupPort.transferFlowFiles(StandardRootGroupPort.java:562)
at org.apache.nifi.web.api.DataTransferResource$1.write(DataTransferResource.java:668)
{code}
Once a NiFi instance owning the root group port enters this state, it will not be able to recover automatically as its client keeps retrying too often. And the returned error message is not clear to understand what is happening.
The remote NiFi instance should remove already expired requests from its internal queue, so that newly received requests are processed correctly, and return 'read timeout' error response if it can not be processed and expired.
Clients can penalize the peer when it receives 'read timeout' error.
> Expired requests stay in StansdardRootGroupPort request queue and block new request to be offered
> -------------------------------------------------------------------------------------------------
>
> Key: NIFI-4376
> URL: https://issues.apache.org/jira/browse/NIFI-4376
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.0.1
> Reporter: Koji Kawamura
> Assignee: Koji Kawamura
>
> If the remote NiFi instance slows down for some reason, and S2S requests have been timed out continuously, those timed out requests will remain in the internal request queue at StandardRootGroupPort.
> Once number of queued requests becomes 1,000, newly received request can not be offered to the queue, and RequestExpiredException is thrown. A S2S client receives following error in this case:
> {code}
> 2017-09-11 16:32:38,137 ERROR [Timer-Driven Process Thread-4] o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=output,target=http://localhost:8080/nifi] failed to communicate with http://localhost:8080/nifi due to java.io.IOException: Unexpected response code: 500 errCode:Abort errMessage:<html>
> <head>
> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
> <title>Error 500 Server Error</title>
> </head>
> <body><h2>HTTP ERROR 500</h2>
> <p>Problem accessing /nifi-api/data-transfer/output-ports/6fc7fa6d-015e-1000-27cb-d118b1b0fe13/transactions/73da9bcf-87ec-4594-8e45-a41965c1ed82/flow-files. Reason:
> <pre> Server Error</pre></p><h3>Caused by:</h3><pre>java.io.IOException: Failed to process the request.
> at org.apache.nifi.web.api.DataTransferResource$1.write(DataTransferResource.java:676)
> at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:71)
> ...
> <h3>Caused by:</h3><pre>org.apache.nifi.remote.exception.RequestExpiredException
> at org.apache.nifi.remote.StandardRootGroupPort.transferFlowFiles(StandardRootGroupPort.java:562)
> at org.apache.nifi.web.api.DataTransferResource$1.write(DataTransferResource.java:668)
> {code}
> Once a NiFi instance owning the root group port enters this state, it will not be able to recover automatically as its client keeps retrying too often. And the returned error message is not clear to understand what is happening.
> The remote NiFi instance should remove already expired requests from its internal queue, so that newly received requests are processed correctly, and return 'read timeout' error response if it can not be processed and expired.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)