You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@nifi.apache.org by David Hoffer <dh...@gmail.com> on 2020/05/11 21:46:53 UTC

NiFi Swagger/OpenAPI Processor

I'm looking for a Swagger/OpenAPI v3 processor where I can use NiFi as an
embedded REST server per the Swagger/OpenAPI schema/spec.

I realize that NiFi has the HandleHttpRequest and HandleHttpResponse but
that appears to be either a global (accept everything) or specific for one
endpoint/action.  I want a dynamic REST service that exposes all the
endpoints/actions that are defined in the schema/spec and nothing else.

If this is not something that someone has done with NiFi already is there a
way I could embed my own REST service inside of a generic NiFi
process/etc?  E.g. can NiFi host some arbitrary code?

The reason I'd like to do this is because we already have NiFi flows for
this same 'business functionality' but now I also need a new REST service
for it that might not be connected to the existing flow but I'd like all
the logic in one place...so NiFi to host everything.

Can someone point me in the right direction?

-Dave

Re: NiFi Swagger/OpenAPI Processor

Posted by Mark Payne <ma...@hotmail.com>.

David,

Fair enough.

The attributes that it adds are documented. If you add the processor to your NiFi canvas and right-click on it and View Usage, you will get the processor’s documentation that indicates each of the attributes that are added along with an explanation of it. Or you could go to [1] for the docs on the latest version of that processor and see them there, under the “Writes Attributes” section.

I would caution you, though, that in general processors are not meant to be used as base classes that can be extended. You may have trouble trying to extend from that processor. It may make more sense to create your own or else create a processor that uses the attributes provided.

Hope this helps!
-Mark

[1] http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.HandleHttpRequest/index.html

On May 12, 2020, at 9:29 AM, David Hoffer <dh...@gmail.com>> wrote:

Thanks for the suggestions. I'm trying to find a way to handle this in a dynamic way based on the OpenAPI spec. Obviously I need to have a connected processor that can then generate the correct response for the request. You mentioned that that the HandleHttpRequest processor adds attributes for just about everything in the request. Is this documented somewhere? Perhaps I can use that info in my custom processor to handle correctly. Ideally I want the HandleHttpRequest processor to add the OpenAPI metadata as attributes as well. Perhaps I could extend the HandleHttpRequest to do that?

My OpenAPI spec might grow to dozens of operations/actions so I don't want to have to manually keep the NiFi flow up-to-date for each change. I understand I will have to code the behavior for each operation/action.

So basically what I need is an OpenApiHandleHttpRequest processor that is fully OpenAPI aware where I can provide the behavior in one location.

Also I need NiFi to host the static Swagger UI as well. This is just a folder of HTML/JS. I'd want OpenApiHandleHttpRequest to handle that as well.

Thanks,
-Dave

On Mon, May 11, 2020 at 9:37 PM Mark Payne <ma...@hotmail.com>> wrote:
Dave,

You may also want to consider using RouteOnAttribute. The HandleHttpRequest processor adds attributes for just about everything in the request. You can easily use those to route the requests not matching the path/method you want to a HandleHttpResponse that responds with a 404 or 405. Now that said, you’d have to configure those methods/paths that are allowed in the RouteOnAttribute rather than providing a swagger spec, but it may be worth considering.

Thanks
-Mark

Sent from my iPhone

On May 11, 2020, at 11:02 PM, Ami Goldenberg <am...@gmail.com>> wrote:

Maybe put an nginx in front that only passes through the requests you need? Or some other reverse proxy.

I'm not affiliated with AWS but know their API gateway has this functionality

https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-import-api.html

On Tue, May 12, 2020, 04:20 Otto Fowler <ot...@gmail.com>> wrote:
If nobody has done this you can look at:

https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html
https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-providingCodeOrDocumentationContributionProvidingcodeordocumentationcontributions

On May 11, 2020 at 17:47:10, David Hoffer (dhoffer6@gmail.com<ma...@gmail.com>) wrote:

I'm looking for a Swagger/OpenAPI v3 processor where I can use NiFi as an embedded REST server per the Swagger/OpenAPI schema/spec.

I realize that NiFi has the HandleHttpRequest and HandleHttpResponse but that appears to be either a global (accept everything) or specific for one endpoint/action. I want a dynamic REST service that exposes all the endpoints/actions that are defined in the schema/spec and nothing else.

If this is not something that someone has done with NiFi already is there a way I could embed my own REST service inside of a generic NiFi process/etc? E.g. can NiFi host some arbitrary code?

The reason I'd like to do this is because we already have NiFi flows for this same 'business functionality' but now I also need a new REST service for it that might not be connected to the existing flow but I'd like all the logic in one place...so NiFi to host everything.

Can someone point me in the right direction?

-Dave

Re: NiFi Swagger/OpenAPI Processor

Posted by David Hoffer <dh...@gmail.com>.

Thanks for the suggestions.  I'm trying to find a way to handle this in a
dynamic way based on the OpenAPI spec.  Obviously I need to have a
connected processor that can then generate the correct response for the
request.  You mentioned that that the HandleHttpRequest  processor adds
attributes for just about everything in the request.  Is this documented
somewhere?  Perhaps I can use that info in my custom processor to handle
correctly.  Ideally I want the HandleHttpRequest processor to add the
OpenAPI metadata as attributes as well.  Perhaps I could extend the
HandleHttpRequest to do that?

My OpenAPI spec might grow to dozens of operations/actions so I don't want
to have to manually keep the NiFi flow up-to-date for each change.  I
understand I will have to code the behavior for each operation/action.

So basically what I need is an OpenApiHandleHttpRequest processor that is
fully OpenAPI aware where I can provide the behavior in one location.

Also I need NiFi to host the static Swagger UI as well.  This is just a
folder of HTML/JS.  I'd want OpenApiHandleHttpRequest to handle that as
well.

Thanks,
-Dave

On Mon, May 11, 2020 at 9:37 PM Mark Payne <ma...@hotmail.com> wrote:

> Dave,
>
> You may also want to consider using RouteOnAttribute. The
> HandleHttpRequest processor adds attributes for just about everything in
> the request. You can easily use those to route the requests not matching
> the path/method you want to a HandleHttpResponse that responds with a 404
> or 405. Now that said, you’d have to configure those methods/paths that are
> allowed in the RouteOnAttribute rather than providing a swagger spec, but
> it may be worth considering.
>
> Thanks
> -Mark
>
> Sent from my iPhone
>
> On May 11, 2020, at 11:02 PM, Ami Goldenberg <am...@gmail.com> wrote:
>
> 
> Maybe put an nginx in front that only passes through the requests you
> need? Or some other reverse proxy.
>
> I'm not affiliated with AWS but know their API gateway has this
> functionality
>
>
> https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-import-api.html
>
> On Tue, May 12, 2020, 04:20 Otto Fowler <ot...@gmail.com> wrote:
>
>> If nobody has done this you can look at:
>>
>> https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html
>>
>> https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-providingCodeOrDocumentationContributionProvidingcodeordocumentationcontributions
>>
>> On May 11, 2020 at 17:47:10, David Hoffer (dhoffer6@gmail.com) wrote:
>>
>> I'm looking for a Swagger/OpenAPI v3 processor where I can use NiFi as an
>> embedded REST server per the Swagger/OpenAPI schema/spec.
>>
>> I realize that NiFi has the HandleHttpRequest and HandleHttpResponse but
>> that appears to be either a global (accept everything) or specific for one
>> endpoint/action.  I want a dynamic REST service that exposes all the
>> endpoints/actions that are defined in the schema/spec and nothing else.
>>
>> If this is not something that someone has done with NiFi already is there
>> a way I could embed my own REST service inside of a generic NiFi
>> process/etc?  E.g. can NiFi host some arbitrary code?
>>
>> The reason I'd like to do this is because we already have NiFi flows for
>> this same 'business functionality' but now I also need a new REST service
>> for it that might not be connected to the existing flow but I'd like all
>> the logic in one place...so NiFi to host everything.
>>
>> Can someone point me in the right direction?
>>
>> -Dave
>>
>>

Re: NiFi Swagger/OpenAPI Processor

Posted by Mark Payne <ma...@hotmail.com>.

Dave,

Thanks
-Mark

Sent from my iPhone

On May 11, 2020, at 11:02 PM, Ami Goldenberg <am...@gmail.com> wrote:

Maybe put an nginx in front that only passes through the requests you need? Or some other reverse proxy.

I'm not affiliated with AWS but know their API gateway has this functionality

https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-import-api.html

On Tue, May 12, 2020, 04:20 Otto Fowler <ot...@gmail.com>> wrote:
If nobody has done this you can look at:

On May 11, 2020 at 17:47:10, David Hoffer (dhoffer6@gmail.com<ma...@gmail.com>) wrote:

I'm looking for a Swagger/OpenAPI v3 processor where I can use NiFi as an embedded REST server per the Swagger/OpenAPI schema/spec.

If this is not something that someone has done with NiFi already is there a way I could embed my own REST service inside of a generic NiFi process/etc? E.g. can NiFi host some arbitrary code?

Can someone point me in the right direction?

-Dave

Re: NiFi Swagger/OpenAPI Processor

Posted by Ami Goldenberg <am...@gmail.com>.

Maybe put an nginx in front that only passes through the requests you need?
Or some other reverse proxy.

I'm not affiliated with AWS but know their API gateway has this
functionality

https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-import-api.html

On Tue, May 12, 2020, 04:20 Otto Fowler <ot...@gmail.com> wrote:

> If nobody has done this you can look at:
>
> https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html
>
> https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-providingCodeOrDocumentationContributionProvidingcodeordocumentationcontributions
>
> On May 11, 2020 at 17:47:10, David Hoffer (dhoffer6@gmail.com) wrote:
>
> I'm looking for a Swagger/OpenAPI v3 processor where I can use NiFi as an
> embedded REST server per the Swagger/OpenAPI schema/spec.
>
> I realize that NiFi has the HandleHttpRequest and HandleHttpResponse but
> that appears to be either a global (accept everything) or specific for one
> endpoint/action.  I want a dynamic REST service that exposes all the
> endpoints/actions that are defined in the schema/spec and nothing else.
>
> If this is not something that someone has done with NiFi already is there
> a way I could embed my own REST service inside of a generic NiFi
> process/etc?  E.g. can NiFi host some arbitrary code?
>
> The reason I'd like to do this is because we already have NiFi flows for
> this same 'business functionality' but now I also need a new REST service
> for it that might not be connected to the existing flow but I'd like all
> the logic in one place...so NiFi to host everything.
>
> Can someone point me in the right direction?
>
> -Dave
>
>

Re: NiFi Swagger/OpenAPI Processor

Posted by Otto Fowler <ot...@gmail.com>.

 If nobody has done this you can look at:

https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html
https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-providingCodeOrDocumentationContributionProvidingcodeordocumentationcontributions

On May 11, 2020 at 17:47:10, David Hoffer (dhoffer6@gmail.com) wrote:

I'm looking for a Swagger/OpenAPI v3 processor where I can use NiFi as an
embedded REST server per the Swagger/OpenAPI schema/spec.

I realize that NiFi has the HandleHttpRequest and HandleHttpResponse but
that appears to be either a global (accept everything) or specific for one
endpoint/action.  I want a dynamic REST service that exposes all the
endpoints/actions that are defined in the schema/spec and nothing else.

If this is not something that someone has done with NiFi already is there a
way I could embed my own REST service inside of a generic NiFi
process/etc?  E.g. can NiFi host some arbitrary code?

The reason I'd like to do this is because we already have NiFi flows for
this same 'business functionality' but now I also need a new REST service
for it that might not be connected to the existing flow but I'd like all
the logic in one place...so NiFi to host everything.

Can someone point me in the right direction?

-Dave