NIFI ListenTCP Processor

[Venkatesh Nandigam <ve...@bridgera.com>]

Hi Team,

We started using NIFI data flow in our current project by replacing node js
tcp listeners. we are using nifi listenTCP processor.

our use case: we have some devices they will send message to tcp port, from
nifi we have to get message and then place data into kafka.

Processors we are using:

ListenTCP processor: to receive data from port and send to kafka topic

PutKafka: place data into kafka topic.

Problem we are facing:

from device side we have message delimiter as "$" but nifi listenTCP
processor is accepting only "/n". we changed the delimeter in nifi admin
its not reflecting.. by looking at nifi code we saw TCP_DELIMETER field as
final. then we changed that class with our delimiter its working fine..

Path:
https://github.com/apache/nifi/blob/master/nifi-commons/nifi-processor-utilities/src/main/java/org/apache/nifi/processor/util/listen/handler/socket/SocketChannelHandler.java

Is this expected behavior? or we missing something?.. if it is excepted
behavior is they any chance to change that modifiable field?.

Thanks,
Venkat

Re: NIFI ListenTCP Processor

[venkat <ve...@bridgera.com>]

Thanks Brayn for creating JIRA task. yes its very useful for us if we have
option to change delimiter and processor can pick that delimiter. currently
exact match scenario works well for my scenario.

Thanks,
Venkat



--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/NIFI-ListenTCP-Processor-tp11212p11277.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: NIFI ListenTCP Processor

[Bryan Bende <bb...@gmail.com>]

Venkat,

You are correct that right now the incoming message delimiter is hard-coded
as "/n".

The property you see in the UI for message delimiter is actually for the
output of the processor. It is used when you increase the batch size > 1
and it writes multiple messages to a single flow file, it uses the value of
that property to separate them.

We definitely want to expose the incoming delimiter as something that can
be set through a property on the processor. I think one reason it hasn't
been done yet is because there are several strategies we'd like to support:

- exact match - this would be your case where you specify "$"
- pattern match - this would be reading until a pattern is seen to help
capture multi-line log messages that start with date patterns
- size match - this would be reading until a specified number of bytes have
been read

That being said, we shouldn't need to do all of these at once so I created
this JIRA for your scenario:

https://issues.apache.org/jira/browse/NIFI-1985

-Bryan



On Wed, Jun 8, 2016 at 2:25 AM, Venkatesh Nandigam <
venkat.nandigam@bridgera.com> wrote:

> Hi Team,
>
> We started using NIFI data flow in our current project by replacing node js
> tcp listeners. we are using nifi listenTCP processor.
>
> our use case: we have some devices they will send message to tcp port, from
> nifi we have to get message and then place data into kafka.
>
> Processors we are using:
>
> ListenTCP processor: to receive data from port and send to kafka topic
>
> PutKafka: place data into kafka topic.
>
> Problem we are facing:
>
> from device side we have message delimiter as "$" but nifi listenTCP
> processor is accepting only "/n". we changed the delimeter in nifi admin
> its not reflecting.. by looking at nifi code we saw TCP_DELIMETER field as
> final. then we changed that class with our delimiter its working fine..
>
> Path:
>
> https://github.com/apache/nifi/blob/master/nifi-commons/nifi-processor-utilities/src/main/java/org/apache/nifi/processor/util/listen/handler/socket/SocketChannelHandler.java
>
> Is this expected behavior? or we missing something?.. if it is excepted
> behavior is they any chance to change that modifiable field?.
>
> Thanks,
> Venkat
>