NiFi as a Web Services server

[Boris Tyukin <bo...@boristyukin.com>]

I wonder how practical it is to use NiFi as a server to serve web service
calls and return data over REST to consumers. It seems like a natural fit,
given NiFi's scalability and redundancy and a host of data integration and
transformation processors.

I am thinking to create a quick pilot using Kafka as a message broker (and
to decouple NiFi flows) and NiFi as transformation/integration engine and
also as a web server to expose flows via NiFi Httphandler/listener
processors.

Is anyone using NiFi for that?

Thanks,
Boris

Re: NiFi as a Web Services server

[Boris Tyukin <bo...@boristyukin.com>]

thanks Mike, you've reasoned my concerns as well.

On Mon, Mar 26, 2018 at 5:57 PM, Mike Thomsen <mi...@gmail.com>
wrote:

> I think you would hit two big barriers in design:
>
> 1. NiFi just isn't designed to be an app server for additional service
> layer components a la Tomcat.
> 2. Synchronizing between the REST services and NiFi's highly asynchronous
> processing would be a logistical nightmare if your goal is to confine NiFi
> processing to the request/response cycle of HTTP.
>
> If you want to really integrate NiFi into other apps, what you should do
> is focus on the inputs and outputs and use something very real-time like
> Kafka + WebSockets at the end so that you can stream output to consumers.
>
> Thanks,
>
> Mike
>
> On Mon, Mar 26, 2018 at 3:07 PM, Boris Tyukin <bo...@boristyukin.com>
> wrote:
>
>> I wonder how practical it is to use NiFi as a server to serve web service
>> calls and return data over REST to consumers. It seems like a natural fit,
>> given NiFi's scalability and redundancy and a host of data integration and
>> transformation processors.
>>
>> I am thinking to create a quick pilot using Kafka as a message broker
>> (and to decouple NiFi flows) and NiFi as transformation/integration engine
>> and also as a web server to expose flows via NiFi Httphandler/listener
>> processors.
>>
>> Is anyone using NiFi for that?
>>
>> Thanks,
>> Boris
>>
>>
>>
>>
>

Re: NiFi as a Web Services server

[Mike Thomsen <mi...@gmail.com>]

I think you would hit two big barriers in design:

1. NiFi just isn't designed to be an app server for additional service
layer components a la Tomcat.
2. Synchronizing between the REST services and NiFi's highly asynchronous
processing would be a logistical nightmare if your goal is to confine NiFi
processing to the request/response cycle of HTTP.

If you want to really integrate NiFi into other apps, what you should do is
focus on the inputs and outputs and use something very real-time like Kafka
+ WebSockets at the end so that you can stream output to consumers.

Thanks,

Mike

On Mon, Mar 26, 2018 at 3:07 PM, Boris Tyukin <bo...@boristyukin.com> wrote:

> I wonder how practical it is to use NiFi as a server to serve web service
> calls and return data over REST to consumers. It seems like a natural fit,
> given NiFi's scalability and redundancy and a host of data integration and
> transformation processors.
>
> I am thinking to create a quick pilot using Kafka as a message broker (and
> to decouple NiFi flows) and NiFi as transformation/integration engine and
> also as a web server to expose flows via NiFi Httphandler/listener
> processors.
>
> Is anyone using NiFi for that?
>
> Thanks,
> Boris
>
>
>
>