You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@nifi.apache.org by Jeremy Pemberton-Pigott <fu...@gmail.com> on 2020/06/30 04:04:16 UTC

Session state in cluster HandleHttpRequest and HandleHttpReponse

Hi,

I have a cluster of 3 nodes and the incoming request on one node's
HandleHttpRequest may be replied to by a different node's
HandleHttpResponse, in between there is a Spark streaming job process.  Is
there any example how to do that, maybe with DistributeMapCacheService?  So
that I can still reply back to the client with the necessary data in the
response.

Jeremy

Re: Session state in cluster HandleHttpRequest and HandleHttpReponse

Posted by Jeremy Pemberton-Pigott <fu...@gmail.com>.

Thanks Peter that makes sense. I'll try a wait/notify using an identifier for the node in the Spark messages being monitored so that the same node will receive the reply from Spark and respond to the client that initiated the connection. 

Regards,

Jeremy

On 30 Jun 2020, at 22:41, Peter Turcsanyi <tu...@apache.org> wrote:

Hi Jeremy,

I don't think you can accept the request in one node and send back the response from another node. 
There is an open HTTP connection between the client and the NiFi node while the HandleHttpRequest -> ... -> HandleHttpResponse flow is running.
Even if we passed the request/response context object between the NiFi nodes via a distributed cache, it would not be possible to send back the HTTP response to a client that originally connected and sent the request to another node. At least I don't know any solution to it in/outside NiFi.

Best,
Peter

> On Tue, Jun 30, 2020 at 6:04 AM Jeremy Pemberton-Pigott <fu...@gmail.com> wrote:
> Hi,
> 
> I have a cluster of 3 nodes and the incoming request on one node's HandleHttpRequest may be replied to by a different node's HandleHttpResponse, in between there is a Spark streaming job process.  Is there any example how to do that, maybe with DistributeMapCacheService?  So that I can still reply back to the client with the necessary data in the response.
> 
> Jeremy

Re: Session state in cluster HandleHttpRequest and HandleHttpReponse

Posted by Peter Turcsanyi <tu...@apache.org>.

Hi Jeremy,

I don't think you can accept the request in one node and send back the
response from another node.
There is an open HTTP connection between the client and the NiFi node while
the HandleHttpRequest -> ... -> HandleHttpResponse flow is running.
Even if we passed the request/response context object between the NiFi
nodes via a distributed cache, it would not be possible to send back the
HTTP response to a client that originally connected and sent the request to
another node. At least I don't know any solution to it in/outside NiFi.

Best,
Peter

On Tue, Jun 30, 2020 at 6:04 AM Jeremy Pemberton-Pigott <
fuzzychaos@gmail.com> wrote:

> Hi,
>
> I have a cluster of 3 nodes and the incoming request on one node's
> HandleHttpRequest may be replied to by a different node's
> HandleHttpResponse, in between there is a Spark streaming job process.  Is
> there any example how to do that, maybe with DistributeMapCacheService?  So
> that I can still reply back to the client with the necessary data in the
> response.
>
> Jeremy
>