You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Jean-Sebastien Vachon <js...@brizodata.com> on 2019/03/22 15:41:45 UTC

No load on second node

Hi all,

What's the best strategy to get load balancing working properly? I've configure one of the very first connection of my flow to use one of the load balancing option so that flows are processed on both machines.

However, one of my two nodes is not doing anything. The load one the first server is way higher than the load on the second (60 vs 1).

I tailed the logs on both servers and there is not much information except for the following:

2019-03-22 15:40:17,579 ERROR [Load-Balanced Client Thread-7] o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Unable to connect to 10.0.2.132:8088 for load balancing
java.net.ConnectException: Connection timed out

I used telnet to connect and everything seems fine. I recently upgraded from Nifi 1.7.. could it be that I'm missing some configuration ?

Thanks

Re: No load on second node

Posted by Jean-Sebastien Vachon <js...@brizodata.com>.
Ah that's what is missing from my configuration. Like I said, I've upgraded from 1.7 and didn't check for new configuration options.

Thanks, that will make it a lot easier to fix
________________________________
From: Bryan Bende <bb...@gmail.com>
Sent: Friday, March 22, 2019 1:38 PM
To: users@nifi.apache.org
Subject: Re: No load on second node

Hello,

The host and port used for load balancing are defined in
nifi.properties of each node with the following properties:

# cluster load balancing properties #
nifi.cluster.load.balance.host=
nifi.cluster.load.balance.port=6342
nifi.cluster.load.balance.connections.per.node=4
nifi.cluster.load.balance.max.thread.count=8
nifi.cluster.load.balance.comms.timeout=30 sec

Thanks,

Bryan

On Fri, Mar 22, 2019 at 1:37 PM Jean-Sebastien Vachon
<js...@brizodata.com> wrote:
>
> I am living in a very strict environment...  is a new port being used for load balancing?
>
> The logs show 8088 (which is my Jetty port) which is open and available for the node not receiving anything.
>
> I've read these two articles:
> https://blogs.apache.org/nifi/entry/load-balancing-across-the-cluster
>
> and
>
> https://pierrevillard.com/2018/10/29/nifi-1-8-revolutionizing-the-list-fetch-pattern-and-more/
>
> and I don't see any mention in this regard...
>
> My last hope will be to upgrade to the latest version (1.9.1)
>
> Thanks
>
>
> ________________________________
> From: Jean-Sebastien Vachon <js...@brizodata.com>
> Sent: Friday, March 22, 2019 11:41 AM
> To: users@nifi.apache.org
> Subject: No load on second node
>
> Hi all,
>
> What's the best strategy to get load balancing working properly? I've configure one of the very first connection of my flow to use one of the load balancing option so that flows are processed on both machines.
>
> However, one of my two nodes is not doing anything. The load one the first server is way higher than the load on the second (60 vs 1).
>
> I tailed the logs on both servers and there is not much information except for the following:
>
> 2019-03-22 15:40:17,579 ERROR [Load-Balanced Client Thread-7] o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Unable to connect to 10.0.2.132:8088 for load balancing
> java.net.ConnectException: Connection timed out
>
> I used telnet to connect and everything seems fine. I recently upgraded from Nifi 1.7.. could it be that I'm missing some configuration ?
>
> Thanks

Re: No load on second node

Posted by Bryan Bende <bb...@gmail.com>.
Hello,

The host and port used for load balancing are defined in
nifi.properties of each node with the following properties:

# cluster load balancing properties #
nifi.cluster.load.balance.host=
nifi.cluster.load.balance.port=6342
nifi.cluster.load.balance.connections.per.node=4
nifi.cluster.load.balance.max.thread.count=8
nifi.cluster.load.balance.comms.timeout=30 sec

Thanks,

Bryan

On Fri, Mar 22, 2019 at 1:37 PM Jean-Sebastien Vachon
<js...@brizodata.com> wrote:
>
> I am living in a very strict environment...  is a new port being used for load balancing?
>
> The logs show 8088 (which is my Jetty port) which is open and available for the node not receiving anything.
>
> I've read these two articles:
> https://blogs.apache.org/nifi/entry/load-balancing-across-the-cluster
>
> and
>
> https://pierrevillard.com/2018/10/29/nifi-1-8-revolutionizing-the-list-fetch-pattern-and-more/
>
> and I don't see any mention in this regard...
>
> My last hope will be to upgrade to the latest version (1.9.1)
>
> Thanks
>
>
> ________________________________
> From: Jean-Sebastien Vachon <js...@brizodata.com>
> Sent: Friday, March 22, 2019 11:41 AM
> To: users@nifi.apache.org
> Subject: No load on second node
>
> Hi all,
>
> What's the best strategy to get load balancing working properly? I've configure one of the very first connection of my flow to use one of the load balancing option so that flows are processed on both machines.
>
> However, one of my two nodes is not doing anything. The load one the first server is way higher than the load on the second (60 vs 1).
>
> I tailed the logs on both servers and there is not much information except for the following:
>
> 2019-03-22 15:40:17,579 ERROR [Load-Balanced Client Thread-7] o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Unable to connect to 10.0.2.132:8088 for load balancing
> java.net.ConnectException: Connection timed out
>
> I used telnet to connect and everything seems fine. I recently upgraded from Nifi 1.7.. could it be that I'm missing some configuration ?
>
> Thanks

Re: No load on second node

Posted by Jean-Sebastien Vachon <js...@brizodata.com>.
I am living in a very strict environment...  is a new port being used for load balancing?

The logs show 8088 (which is my Jetty port) which is open and available for the node not receiving anything.

I've read these two articles:
https://blogs.apache.org/nifi/entry/load-balancing-across-the-cluster

and

https://pierrevillard.com/2018/10/29/nifi-1-8-revolutionizing-the-list-fetch-pattern-and-more/

and I don't see any mention in this regard...

My last hope will be to upgrade to the latest version (1.9.1)

Thanks


________________________________
From: Jean-Sebastien Vachon <js...@brizodata.com>
Sent: Friday, March 22, 2019 11:41 AM
To: users@nifi.apache.org
Subject: No load on second node

Hi all,

What's the best strategy to get load balancing working properly? I've configure one of the very first connection of my flow to use one of the load balancing option so that flows are processed on both machines.

However, one of my two nodes is not doing anything. The load one the first server is way higher than the load on the second (60 vs 1).

I tailed the logs on both servers and there is not much information except for the following:

2019-03-22 15:40:17,579 ERROR [Load-Balanced Client Thread-7] o.a.n.c.q.c.c.a.n.NioAsyncLoadBalanceClient Unable to connect to 10.0.2.132:8088 for load balancing
java.net.ConnectException: Connection timed out

I used telnet to connect and everything seems fine. I recently upgraded from Nifi 1.7.. could it be that I'm missing some configuration ?

Thanks