You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Thane Durey <td...@harding.edu> on 2020/11/19 17:14:19 UTC
Unable to Receive TCP Traffic in NiFi
All,
I'm attempting to setup a Load Balancer (LB) in front of my NiFi cluster using another NiFi server. I've seen a successful example used online and it was very straight forward.
1. Receive data on LB
2. And then send it to the cluster using RoundRobin
Pretty straight forward.
FYI, here is my setup:
All machines are CentOS7
I'm using NiFi version 1.12.1 on LB and Cluster
LB's PutTCP:
- Max Size of Socket Send Buffer: 10MB
Cluster ListenTCP and ListenSyslog:
- Receive Buffer Size: 128 MB
- Max Size of Message Queue: 1,000,000
- Max Size of Socket Buffer: 100 MB
On my Load Balancer, I'm using Round Robin and sending my TCP data to my NiFi Cluster. I understand this isn't the best solution for the long term, and it's not easily scalable. Regardless, I'm sending the TCP data to my NiFi Cluster using PutTCP.
On my NiFi Cluster, I'm trying to receive this TCP traffic (sent on a specific port), but nothing is coming through on my ListenTCP (same port). This confused me, and my first assumption was I'm not receiving TCP data at all. After running a tcpdump, I can confirm I'm receiving TCP data over the specified port from my Load Balancer. I even ran tcpdump with the -A flag so I can see the actual data. So it's not empty TCP traffic (even though I can see the length of the data is greater than 0).
So, because I can see the TCP traffic on my NiFi nodes, why wouldn't I be able to receive the data using ListenTCP? I'm even using ListenSyslog to see if that would work. Nothing.
Thoughts? I really appreciate any time you put into this. Thank you in advance.
Re: Unable to Receive TCP Traffic in NiFi
Posted by Thane Durey <td...@harding.edu>.
I figured out the issue.
I needed to set my PutTCP's "Outgoing Message Delimiter" to the ListenTCP or ListenSyslog's default "Message Delimiter", which is the new line character "\n".
On 2020/11/19 17:14:19, Thane Durey <td...@harding.edu> wrote:
> All,
>
> I'm attempting to setup a Load Balancer (LB) in front of my NiFi cluster using another NiFi server. I've seen a successful example used online and it was very straight forward.
> 1. Receive data on LB
> 2. And then send it to the cluster using RoundRobin
>
> Pretty straight forward.
>
> FYI, here is my setup:
> All machines are CentOS7
> I'm using NiFi version 1.12.1 on LB and Cluster
>
> LB's PutTCP:
> - Max Size of Socket Send Buffer: 10MB
>
> Cluster ListenTCP and ListenSyslog:
> - Receive Buffer Size: 128 MB
> - Max Size of Message Queue: 1,000,000
> - Max Size of Socket Buffer: 100 MB
>
> On my Load Balancer, I'm using Round Robin and sending my TCP data to my NiFi Cluster. I understand this isn't the best solution for the long term, and it's not easily scalable. Regardless, I'm sending the TCP data to my NiFi Cluster using PutTCP.
>
> On my NiFi Cluster, I'm trying to receive this TCP traffic (sent on a specific port), but nothing is coming through on my ListenTCP (same port). This confused me, and my first assumption was I'm not receiving TCP data at all. After running a tcpdump, I can confirm I'm receiving TCP data over the specified port from my Load Balancer. I even ran tcpdump with the -A flag so I can see the actual data. So it's not empty TCP traffic (even though I can see the length of the data is greater than 0).
>
> So, because I can see the TCP traffic on my NiFi nodes, why wouldn't I be able to receive the data using ListenTCP? I'm even using ListenSyslog to see if that would work. Nothing.
>
>
> Thoughts? I really appreciate any time you put into this. Thank you in advance.
>