You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Kelsey RIDER <ke...@ineat-conseil.fr> on 2018/06/27 12:38:42 UTC
NiFi as RESTful server?
Hello,
I have a project where I'd like NiFi to expose about 4 different RESTful endpoints.
The flow behind each of these involves a half-dozen steps and is different in each case.
I wanted to set it up so that each flow started with a HandleHTTPRequest processor bound to the appropriate URL path. However, it turns out that two such processors cannot cohabitate because each one tries to start its own server bound to the given port.
I'm wondering:
* Is there a better way of doing this, without having to have a single HandleHTTPRequest and making my flows more complicated with all the required connections to link it to my current entry points?
* Would it be technically possible to have some sort of "HTTPService" controller that just starts the server and binds to the port, and then multiple "HTTPServiceListener" processors that use it? Or is it not possible for a Controller Service to "wake up" a Processor/initiate a flow?
Thanks for the help,
Kelsey
Suite ? l'?volution des dispositifs de r?glementation du travail, si vous recevez ce mail avant 7h00, en soir?e, durant le week-end ou vos cong?s merci, sauf cas d'urgence exceptionnelle, de ne pas le traiter ni d'y r?pondre imm?diatement.
RE: NiFi as RESTful server?
Posted by Kelsey RIDER <ke...@ineat-conseil.fr>.
Ah, that’s interesting. I had a look at the API but I didn’t find any easy way to send a flowFile and kick off processing…
I did find this link: https://stackoverflow.com/questions/43228475/how-to-transfer-flowfiles-in-nifi-using-nifi-api
Is that what you ended up implementing?
Basically, I have two use cases: in one case, I’m uploading a file that NiFi will transform and store in a DB; in the other case, I’m requesting a file (whose data mostly comes from the DB). Do you know if this can be done with the API?
From: Martijn Dekkers <ma...@dekkers.org.uk>
Sent: mercredi 27 juin 2018 17:23
To: users@nifi.apache.org
Subject: Re: NiFi as RESTful server?
Hey Kelsey,
we were looking at doing something similar, and like Shawn mentioned, we also implemented RouteonAttribute. We quickly hit some limitations specific to our needs, and decided to just manipulate the flow via the API.
On 27 June 2018 at 14:38, Kelsey RIDER <ke...@ineat-conseil.fr>> wrote:
Hello,
I have a project where I’d like NiFi to expose about 4 different RESTful endpoints.
The flow behind each of these involves a half-dozen steps and is different in each case.
I wanted to set it up so that each flow started with a HandleHTTPRequest processor bound to the appropriate URL path. However, it turns out that two such processors cannot cohabitate because each one tries to start its own server bound to the given port.
I’m wondering:
* Is there a better way of doing this, without having to have a single HandleHTTPRequest and making my flows more complicated with all the required connections to link it to my current entry points?
* Would it be technically possible to have some sort of “HTTPService” controller that just starts the server and binds to the port, and then multiple “HTTPServiceListener” processors that use it? Or is it not possible for a Controller Service to “wake up” a Processor/initiate a flow?
Thanks for the help,
Kelsey
Suite à l’évolution des dispositifs de réglementation du travail, si vous recevez ce mail avant 7h00, en soirée, durant le week-end ou vos congés merci, sauf cas d’urgence exceptionnelle, de ne pas le traiter ni d’y répondre immédiatement.
Re: NiFi as RESTful server?
Posted by Martijn Dekkers <ma...@dekkers.org.uk>.
Hey Kelsey,
we were looking at doing something similar, and like Shawn mentioned, we
also implemented RouteonAttribute. We quickly hit some limitations specific
to our needs, and decided to just manipulate the flow via the API.
On 27 June 2018 at 14:38, Kelsey RIDER <ke...@ineat-conseil.fr>
wrote:
> Hello,
>
>
>
> I have a project where I’d like NiFi to expose about 4 different RESTful
> endpoints.
>
> The flow behind each of these involves a half-dozen steps and is different
> in each case.
>
>
>
> I wanted to set it up so that each flow started with a HandleHTTPRequest
> processor bound to the appropriate URL path. However, it turns out that two
> such processors cannot cohabitate because each one tries to start its own
> server bound to the given port.
>
> I’m wondering:
>
> - Is there a better way of doing this, without having to have a single
> HandleHTTPRequest and making my flows more complicated with all the
> required connections to link it to my current entry points?
> - Would it be technically possible to have some sort of “HTTPService”
> controller that just starts the server and binds to the port, and then
> multiple “HTTPServiceListener” processors that use it? Or is it not
> possible for a Controller Service to “wake up” a Processor/initiate a flow?
>
>
>
> Thanks for the help,
>
>
>
> Kelsey
> Suite à l’évolution des dispositifs de réglementation du travail, si vous
> recevez ce mail avant 7h00, en soirée, durant le week-end ou vos congés
> merci, sauf cas d’urgence exceptionnelle, de ne pas le traiter ni d’y
> répondre immédiatement.
>
Re: NiFi as RESTful server?
Posted by Boris Tyukin <bo...@boristyukin.com>.
Kelsey,
Take it as an opinion from someone new to NiFi (using it for 1 year or so).
I've looked into doing the same thing. The best examples I found were
https://pierrevillard.com/2016/04/10/url-shortener-service-with-apache-nifi/comment-page-1/
https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates
but in the end we decided not to go this route. First, I did not feel good
about using embedded Jetty server which I am guessing leaves in the same
JVM as NiFi. Second, if you run NiFi cluster, things will be more tricky to
manage - how do you load balance requests? how do you provide redundancy if
one NiFi node goes down? Probably there are good ways to address these
concerns but we decided to use Kafka instead - luckily in our case, the
consumer application is flexible to either use web services over rest or
can listen to Kafka topics.
My conclusion was that it was doable but probably it felt it would be a
pain to support production-grade use cases.
On Wed, Jun 27, 2018 at 8:38 AM Kelsey RIDER <ke...@ineat-conseil.fr>
wrote:
> Hello,
>
>
>
> I have a project where I’d like NiFi to expose about 4 different RESTful
> endpoints.
>
> The flow behind each of these involves a half-dozen steps and is different
> in each case.
>
>
>
> I wanted to set it up so that each flow started with a HandleHTTPRequest
> processor bound to the appropriate URL path. However, it turns out that two
> such processors cannot cohabitate because each one tries to start its own
> server bound to the given port.
>
> I’m wondering:
>
> - Is there a better way of doing this, without having to have a single
> HandleHTTPRequest and making my flows more complicated with all the
> required connections to link it to my current entry points?
> - Would it be technically possible to have some sort of “HTTPService”
> controller that just starts the server and binds to the port, and then
> multiple “HTTPServiceListener” processors that use it? Or is it not
> possible for a Controller Service to “wake up” a Processor/initiate a flow?
>
>
>
> Thanks for the help,
>
>
>
> Kelsey
> Suite à l’évolution des dispositifs de réglementation du travail, si vous
> recevez ce mail avant 7h00, en soirée, durant le week-end ou vos congés
> merci, sauf cas d’urgence exceptionnelle, de ne pas le traiter ni d’y
> répondre immédiatement.
>
Re: NiFi as RESTful server?
Posted by Shawn Weeks <sw...@weeksconsulting.us>.
I'm already doing something like that with a single HandleHTTPRequest and a RouteOnAttribute just after to send it to the appropriate location. You can have multiple HandleHTTPResponses so it's really not all that complicated.
Thanks
Shawn
________________________________
From: Kelsey RIDER <ke...@ineat-conseil.fr>
Sent: Wednesday, June 27, 2018 7:38:42 AM
To: users@nifi.apache.org
Subject: NiFi as RESTful server?
Hello,
I have a project where I’d like NiFi to expose about 4 different RESTful endpoints.
The flow behind each of these involves a half-dozen steps and is different in each case.
I wanted to set it up so that each flow started with a HandleHTTPRequest processor bound to the appropriate URL path. However, it turns out that two such processors cannot cohabitate because each one tries to start its own server bound to the given port.
I’m wondering:
* Is there a better way of doing this, without having to have a single HandleHTTPRequest and making my flows more complicated with all the required connections to link it to my current entry points?
* Would it be technically possible to have some sort of “HTTPService” controller that just starts the server and binds to the port, and then multiple “HTTPServiceListener” processors that use it? Or is it not possible for a Controller Service to “wake up” a Processor/initiate a flow?
Thanks for the help,
Kelsey
Suite à l’évolution des dispositifs de réglementation du travail, si vous recevez ce mail avant 7h00, en soirée, durant le week-end ou vos congés merci, sauf cas d’urgence exceptionnelle, de ne pas le traiter ni d’y répondre immédiatement.