You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Ralph Vercauteren <rr...@qad.com> on 2021/02/04 09:31:54 UTC
To create a REST server
Hi all,
Not sure if I need to ask this in this mailing list or the list of the
developers.
I want to create a REST server with NIFI.
I have a Kafka queue storing messages. When something is connected to the
REST server, the Kafka queue has to be read and all waiting messages should
be collected and send back as a response to the REST client,
I tried to google for a hint or solution, but I couldn't find it.
With regards,
Mit freundlichem Gruß,
Met vriendelijke groet,
*Ralph Vercauteren*
Technical Architect QAD Automation Solutions
Mobile NL: +31 6 5397 7230
rrv@qad.com
This e-mail may contain QAD proprietary information and should be treated
as confidential.
Re: To create a REST server
Posted by Bryan Bende <bb...@gmail.com>.
I'd also keep in mind how partitioning of kafka topics works with
consumers, and whether or not you have a clustered nifi (most people would
for production).
If you have a topic with 1 partition and 3 nifi nodes, only one of the nifi
nodes has a consumer assigned to this partition (assuming 1 processor, 1
concurrent task), and you wouldn't know which node to call
ListenHttp/HandleHttpRequest on.
If you have a topic with 3 partitions and 3 nifi nodes, most likely 1
partition is assigned to the consumer on each node, so then you could call
the end-point on any node, but you'd only be consuming from 1 partition,
not all the messages available in the topic.
On Thu, Feb 4, 2021 at 10:05 AM David Handermann <ex...@gmail.com>
wrote:
> Thanks Chris, after looking more closely at the ConsumeKafka processors, I
> realized that they don't accept incoming flow files.
>
> It makes the flow more complex, and subject to potential queuing volume
> concerns, but another option might be to use the Wait and Notify
> processors. At a basic level, ConsumeKafka would retrieve files and queue
> them up with the Wait processor. The HandleHttpRequest processor would
> trigger the Notify processor, which would allow the Wait processor to
> release the files for HandleHttpResponse. Depending on the volume of
> messages from Kafka, and the interval at which the REST client connects,
> this could be problematic, but it is one potential option.
>
> Regards,
> David Handermann
>
> On Thu, Feb 4, 2021 at 8:50 AM Ralph Vercauteren <rr...@qad.com> wrote:
>
>> Thanks, Chris,
>>
>> This is the same problem I figured,
>>
>> Is it possible to create an executeScript reading Kafka?
>>
>> With regards,
>> Mit freundlichem Gruß,
>> Met vriendelijke groet,
>>
>> *Ralph Vercauteren*
>> Technical Architect QAD Automation Solutions
>> Mobile NL: +31 6 5397 7230
>> rrv@qad.com
>>
>> This e-mail may contain QAD proprietary information and should be treated
>> as confidential.
>>
>>
>> On Thu, Feb 4, 2021 at 3:45 PM Chris Sampson <ch...@naimuri.com>
>> wrote:
>>
>>> Suspect the problem there would be that the ConsumeKafka processors
>>> (according to the documentation) don't accept incoming connections, so
>>> you've no way of sending the FlowFile from InvokeHttp/HandleHttpRequest to
>>> ConsumeKafka to fetch the messages (which you'd possibly then want to
>>> MergeContent/MergeRecord to send everything back as a single response).
>>>
>>> I'm not very familiar with Kafka processors though, so hopefully someone
>>> else can suggest an alternative (or correct me and the NiFi docs if
>>> ConsumeKafka isn't just a source processor).
>>>
>>>
>>>
>>> ---
>>> *Chris Sampson*
>>> IT Consultant
>>> chris.sampson@naimuri.com
>>> <https://www.naimuri.com/>
>>>
>>>
>>> On Thu, 4 Feb 2021 at 14:26, David Handermann <
>>> exceptionfactory@gmail.com> wrote:
>>>
>>>> Ralph,
>>>>
>>>> Thanks for the reply and additional details. I may be
>>>> misunderstanding, but just for clarification, in your diagram, are you
>>>> expecting NiFi to act as the REST server? If that is the case, then you
>>>> should be able to use one of the ConsumeKafka processors to retrieve
>>>> messages and return them to the REST client. At a very basic level, the
>>>> flow would go from HandleHttpRequest to ConsumeKafka to
>>>> HandleHttpResponse. Depending on the format of the messages retrieved from
>>>> Kafka, you may need to introduce additional processors prior to
>>>> HandleHttpResponse in order to format the output for delivery to the REST
>>>> client.
>>>>
>>>> Regards,
>>>> David Handermann
>>>>
>>>> On Thu, Feb 4, 2021 at 7:47 AM Ralph Vercauteren <rr...@qad.com> wrote:
>>>>
>>>>> Hi David,
>>>>>
>>>>> Thanks for responding.
>>>>>
>>>>> I understand that part, still have the issue to see how we get the
>>>>> kafka messages from a topic.
>>>>>
>>>>> [image: image.png]
>>>>>
>>>>> With regards,
>>>>> Mit freundlichem Gruß,
>>>>> Met vriendelijke groet,
>>>>>
>>>>> *Ralph Vercauteren*
>>>>> Technical Architect QAD Automation Solutions
>>>>> Mobile NL: +31 6 5397 7230
>>>>> rrv@qad.com
>>>>>
>>>>> This e-mail may contain QAD proprietary information and should be
>>>>> treated as confidential.
>>>>>
>>>>>
>>>>> On Thu, Feb 4, 2021 at 2:21 PM David Handermann <
>>>>> exceptionfactory@gmail.com> wrote:
>>>>>
>>>>>> Hi Ralph,
>>>>>>
>>>>>> If you are looking to create a REST service for receiving and
>>>>>> processing messages in NiFi, take a look at the ListenHTTP processor, as
>>>>>> well as the HandleHttpRequest and HandleHttpResponse processors. The
>>>>>> ListenHTTP processor supports receiving HTTP requests and sending the
>>>>>> contents to other processors. The ListenHTTP processor provides the
>>>>>> easiest approach for getting started. If you need to perform validation or
>>>>>> additional processing before sending back a response to REST clients, then
>>>>>> using HandleHttpRequest and HandleHttpResponse with other processors in
>>>>>> between can support that approach. When using HandleHttpRequest and
>>>>>> HandleHttpResponse, it is important to optimize intermediate as much as
>>>>>> possible to avoid HTTP clients timing out while waiting for a response.
>>>>>> Both processors support a number of configuration properties, so be sure to
>>>>>> read through the documentation for each one in order to determine the
>>>>>> optimal settings for your configuration.
>>>>>>
>>>>>> Regards,
>>>>>> David Handermann
>>>>>>
>>>>>> On Thu, Feb 4, 2021 at 3:32 AM Ralph Vercauteren <rr...@qad.com> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Not sure if I need to ask this in this mailing list or the list of
>>>>>>> the developers.
>>>>>>>
>>>>>>> I want to create a REST server with NIFI.
>>>>>>>
>>>>>>> I have a Kafka queue storing messages. When something is connected
>>>>>>> to the REST server, the Kafka queue has to be read and all waiting messages
>>>>>>> should be collected and send back as a response to the REST client,
>>>>>>>
>>>>>>> I tried to google for a hint or solution, but I couldn't find it.
>>>>>>>
>>>>>>> With regards,
>>>>>>> Mit freundlichem Gruß,
>>>>>>> Met vriendelijke groet,
>>>>>>>
>>>>>>> *Ralph Vercauteren*
>>>>>>> Technical Architect QAD Automation Solutions
>>>>>>> Mobile NL: +31 6 5397 7230
>>>>>>> rrv@qad.com
>>>>>>>
>>>>>>> This e-mail may contain QAD proprietary information and should be
>>>>>>> treated as confidential.
>>>>>>>
>>>>>>
Re: To create a REST server
Posted by David Handermann <ex...@gmail.com>.
Thanks Chris, after looking more closely at the ConsumeKafka processors, I
realized that they don't accept incoming flow files.
It makes the flow more complex, and subject to potential queuing volume
concerns, but another option might be to use the Wait and Notify
processors. At a basic level, ConsumeKafka would retrieve files and queue
them up with the Wait processor. The HandleHttpRequest processor would
trigger the Notify processor, which would allow the Wait processor to
release the files for HandleHttpResponse. Depending on the volume of
messages from Kafka, and the interval at which the REST client connects,
this could be problematic, but it is one potential option.
Regards,
David Handermann
On Thu, Feb 4, 2021 at 8:50 AM Ralph Vercauteren <rr...@qad.com> wrote:
> Thanks, Chris,
>
> This is the same problem I figured,
>
> Is it possible to create an executeScript reading Kafka?
>
> With regards,
> Mit freundlichem Gruß,
> Met vriendelijke groet,
>
> *Ralph Vercauteren*
> Technical Architect QAD Automation Solutions
> Mobile NL: +31 6 5397 7230
> rrv@qad.com
>
> This e-mail may contain QAD proprietary information and should be treated
> as confidential.
>
>
> On Thu, Feb 4, 2021 at 3:45 PM Chris Sampson <ch...@naimuri.com>
> wrote:
>
>> Suspect the problem there would be that the ConsumeKafka processors
>> (according to the documentation) don't accept incoming connections, so
>> you've no way of sending the FlowFile from InvokeHttp/HandleHttpRequest to
>> ConsumeKafka to fetch the messages (which you'd possibly then want to
>> MergeContent/MergeRecord to send everything back as a single response).
>>
>> I'm not very familiar with Kafka processors though, so hopefully someone
>> else can suggest an alternative (or correct me and the NiFi docs if
>> ConsumeKafka isn't just a source processor).
>>
>>
>>
>> ---
>> *Chris Sampson*
>> IT Consultant
>> chris.sampson@naimuri.com
>> <https://www.naimuri.com/>
>>
>>
>> On Thu, 4 Feb 2021 at 14:26, David Handermann <ex...@gmail.com>
>> wrote:
>>
>>> Ralph,
>>>
>>> Thanks for the reply and additional details. I may be misunderstanding,
>>> but just for clarification, in your diagram, are you expecting NiFi to act
>>> as the REST server? If that is the case, then you should be able to use
>>> one of the ConsumeKafka processors to retrieve messages and return them to
>>> the REST client. At a very basic level, the flow would go from
>>> HandleHttpRequest to ConsumeKafka to HandleHttpResponse. Depending on the
>>> format of the messages retrieved from Kafka, you may need to introduce
>>> additional processors prior to HandleHttpResponse in order to format the
>>> output for delivery to the REST client.
>>>
>>> Regards,
>>> David Handermann
>>>
>>> On Thu, Feb 4, 2021 at 7:47 AM Ralph Vercauteren <rr...@qad.com> wrote:
>>>
>>>> Hi David,
>>>>
>>>> Thanks for responding.
>>>>
>>>> I understand that part, still have the issue to see how we get the
>>>> kafka messages from a topic.
>>>>
>>>> [image: image.png]
>>>>
>>>> With regards,
>>>> Mit freundlichem Gruß,
>>>> Met vriendelijke groet,
>>>>
>>>> *Ralph Vercauteren*
>>>> Technical Architect QAD Automation Solutions
>>>> Mobile NL: +31 6 5397 7230
>>>> rrv@qad.com
>>>>
>>>> This e-mail may contain QAD proprietary information and should be
>>>> treated as confidential.
>>>>
>>>>
>>>> On Thu, Feb 4, 2021 at 2:21 PM David Handermann <
>>>> exceptionfactory@gmail.com> wrote:
>>>>
>>>>> Hi Ralph,
>>>>>
>>>>> If you are looking to create a REST service for receiving and
>>>>> processing messages in NiFi, take a look at the ListenHTTP processor, as
>>>>> well as the HandleHttpRequest and HandleHttpResponse processors. The
>>>>> ListenHTTP processor supports receiving HTTP requests and sending the
>>>>> contents to other processors. The ListenHTTP processor provides the
>>>>> easiest approach for getting started. If you need to perform validation or
>>>>> additional processing before sending back a response to REST clients, then
>>>>> using HandleHttpRequest and HandleHttpResponse with other processors in
>>>>> between can support that approach. When using HandleHttpRequest and
>>>>> HandleHttpResponse, it is important to optimize intermediate as much as
>>>>> possible to avoid HTTP clients timing out while waiting for a response.
>>>>> Both processors support a number of configuration properties, so be sure to
>>>>> read through the documentation for each one in order to determine the
>>>>> optimal settings for your configuration.
>>>>>
>>>>> Regards,
>>>>> David Handermann
>>>>>
>>>>> On Thu, Feb 4, 2021 at 3:32 AM Ralph Vercauteren <rr...@qad.com> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Not sure if I need to ask this in this mailing list or the list of
>>>>>> the developers.
>>>>>>
>>>>>> I want to create a REST server with NIFI.
>>>>>>
>>>>>> I have a Kafka queue storing messages. When something is connected to
>>>>>> the REST server, the Kafka queue has to be read and all waiting messages
>>>>>> should be collected and send back as a response to the REST client,
>>>>>>
>>>>>> I tried to google for a hint or solution, but I couldn't find it.
>>>>>>
>>>>>> With regards,
>>>>>> Mit freundlichem Gruß,
>>>>>> Met vriendelijke groet,
>>>>>>
>>>>>> *Ralph Vercauteren*
>>>>>> Technical Architect QAD Automation Solutions
>>>>>> Mobile NL: +31 6 5397 7230
>>>>>> rrv@qad.com
>>>>>>
>>>>>> This e-mail may contain QAD proprietary information and should be
>>>>>> treated as confidential.
>>>>>>
>>>>>
Re: To create a REST server
Posted by Ralph Vercauteren <rr...@qad.com>.
Thanks, Chris,
This is the same problem I figured,
Is it possible to create an executeScript reading Kafka?
With regards,
Mit freundlichem Gruß,
Met vriendelijke groet,
*Ralph Vercauteren*
Technical Architect QAD Automation Solutions
Mobile NL: +31 6 5397 7230
rrv@qad.com
This e-mail may contain QAD proprietary information and should be treated
as confidential.
On Thu, Feb 4, 2021 at 3:45 PM Chris Sampson <ch...@naimuri.com>
wrote:
> Suspect the problem there would be that the ConsumeKafka processors
> (according to the documentation) don't accept incoming connections, so
> you've no way of sending the FlowFile from InvokeHttp/HandleHttpRequest to
> ConsumeKafka to fetch the messages (which you'd possibly then want to
> MergeContent/MergeRecord to send everything back as a single response).
>
> I'm not very familiar with Kafka processors though, so hopefully someone
> else can suggest an alternative (or correct me and the NiFi docs if
> ConsumeKafka isn't just a source processor).
>
>
>
> ---
> *Chris Sampson*
> IT Consultant
> chris.sampson@naimuri.com
> <https://www.naimuri.com/>
>
>
> On Thu, 4 Feb 2021 at 14:26, David Handermann <ex...@gmail.com>
> wrote:
>
>> Ralph,
>>
>> Thanks for the reply and additional details. I may be misunderstanding,
>> but just for clarification, in your diagram, are you expecting NiFi to act
>> as the REST server? If that is the case, then you should be able to use
>> one of the ConsumeKafka processors to retrieve messages and return them to
>> the REST client. At a very basic level, the flow would go from
>> HandleHttpRequest to ConsumeKafka to HandleHttpResponse. Depending on the
>> format of the messages retrieved from Kafka, you may need to introduce
>> additional processors prior to HandleHttpResponse in order to format the
>> output for delivery to the REST client.
>>
>> Regards,
>> David Handermann
>>
>> On Thu, Feb 4, 2021 at 7:47 AM Ralph Vercauteren <rr...@qad.com> wrote:
>>
>>> Hi David,
>>>
>>> Thanks for responding.
>>>
>>> I understand that part, still have the issue to see how we get the kafka
>>> messages from a topic.
>>>
>>> [image: image.png]
>>>
>>> With regards,
>>> Mit freundlichem Gruß,
>>> Met vriendelijke groet,
>>>
>>> *Ralph Vercauteren*
>>> Technical Architect QAD Automation Solutions
>>> Mobile NL: +31 6 5397 7230
>>> rrv@qad.com
>>>
>>> This e-mail may contain QAD proprietary information and should be
>>> treated as confidential.
>>>
>>>
>>> On Thu, Feb 4, 2021 at 2:21 PM David Handermann <
>>> exceptionfactory@gmail.com> wrote:
>>>
>>>> Hi Ralph,
>>>>
>>>> If you are looking to create a REST service for receiving and
>>>> processing messages in NiFi, take a look at the ListenHTTP processor, as
>>>> well as the HandleHttpRequest and HandleHttpResponse processors. The
>>>> ListenHTTP processor supports receiving HTTP requests and sending the
>>>> contents to other processors. The ListenHTTP processor provides the
>>>> easiest approach for getting started. If you need to perform validation or
>>>> additional processing before sending back a response to REST clients, then
>>>> using HandleHttpRequest and HandleHttpResponse with other processors in
>>>> between can support that approach. When using HandleHttpRequest and
>>>> HandleHttpResponse, it is important to optimize intermediate as much as
>>>> possible to avoid HTTP clients timing out while waiting for a response.
>>>> Both processors support a number of configuration properties, so be sure to
>>>> read through the documentation for each one in order to determine the
>>>> optimal settings for your configuration.
>>>>
>>>> Regards,
>>>> David Handermann
>>>>
>>>> On Thu, Feb 4, 2021 at 3:32 AM Ralph Vercauteren <rr...@qad.com> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Not sure if I need to ask this in this mailing list or the list of the
>>>>> developers.
>>>>>
>>>>> I want to create a REST server with NIFI.
>>>>>
>>>>> I have a Kafka queue storing messages. When something is connected to
>>>>> the REST server, the Kafka queue has to be read and all waiting messages
>>>>> should be collected and send back as a response to the REST client,
>>>>>
>>>>> I tried to google for a hint or solution, but I couldn't find it.
>>>>>
>>>>> With regards,
>>>>> Mit freundlichem Gruß,
>>>>> Met vriendelijke groet,
>>>>>
>>>>> *Ralph Vercauteren*
>>>>> Technical Architect QAD Automation Solutions
>>>>> Mobile NL: +31 6 5397 7230
>>>>> rrv@qad.com
>>>>>
>>>>> This e-mail may contain QAD proprietary information and should be
>>>>> treated as confidential.
>>>>>
>>>>
Re: To create a REST server
Posted by Chris Sampson <ch...@naimuri.com>.
Suspect the problem there would be that the ConsumeKafka processors
(according to the documentation) don't accept incoming connections, so
you've no way of sending the FlowFile from InvokeHttp/HandleHttpRequest to
ConsumeKafka to fetch the messages (which you'd possibly then want to
MergeContent/MergeRecord to send everything back as a single response).
I'm not very familiar with Kafka processors though, so hopefully someone
else can suggest an alternative (or correct me and the NiFi docs if
ConsumeKafka isn't just a source processor).
---
*Chris Sampson*
IT Consultant
chris.sampson@naimuri.com
<https://www.naimuri.com/>
On Thu, 4 Feb 2021 at 14:26, David Handermann <ex...@gmail.com>
wrote:
> Ralph,
>
> Thanks for the reply and additional details. I may be misunderstanding,
> but just for clarification, in your diagram, are you expecting NiFi to act
> as the REST server? If that is the case, then you should be able to use
> one of the ConsumeKafka processors to retrieve messages and return them to
> the REST client. At a very basic level, the flow would go from
> HandleHttpRequest to ConsumeKafka to HandleHttpResponse. Depending on the
> format of the messages retrieved from Kafka, you may need to introduce
> additional processors prior to HandleHttpResponse in order to format the
> output for delivery to the REST client.
>
> Regards,
> David Handermann
>
> On Thu, Feb 4, 2021 at 7:47 AM Ralph Vercauteren <rr...@qad.com> wrote:
>
>> Hi David,
>>
>> Thanks for responding.
>>
>> I understand that part, still have the issue to see how we get the kafka
>> messages from a topic.
>>
>> [image: image.png]
>>
>> With regards,
>> Mit freundlichem Gruß,
>> Met vriendelijke groet,
>>
>> *Ralph Vercauteren*
>> Technical Architect QAD Automation Solutions
>> Mobile NL: +31 6 5397 7230
>> rrv@qad.com
>>
>> This e-mail may contain QAD proprietary information and should be treated
>> as confidential.
>>
>>
>> On Thu, Feb 4, 2021 at 2:21 PM David Handermann <
>> exceptionfactory@gmail.com> wrote:
>>
>>> Hi Ralph,
>>>
>>> If you are looking to create a REST service for receiving and processing
>>> messages in NiFi, take a look at the ListenHTTP processor, as well as the
>>> HandleHttpRequest and HandleHttpResponse processors. The ListenHTTP
>>> processor supports receiving HTTP requests and sending the contents to
>>> other processors. The ListenHTTP processor provides the easiest approach
>>> for getting started. If you need to perform validation or additional
>>> processing before sending back a response to REST clients, then using
>>> HandleHttpRequest and HandleHttpResponse with other processors in between
>>> can support that approach. When using HandleHttpRequest and
>>> HandleHttpResponse, it is important to optimize intermediate as much as
>>> possible to avoid HTTP clients timing out while waiting for a response.
>>> Both processors support a number of configuration properties, so be sure to
>>> read through the documentation for each one in order to determine the
>>> optimal settings for your configuration.
>>>
>>> Regards,
>>> David Handermann
>>>
>>> On Thu, Feb 4, 2021 at 3:32 AM Ralph Vercauteren <rr...@qad.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Not sure if I need to ask this in this mailing list or the list of the
>>>> developers.
>>>>
>>>> I want to create a REST server with NIFI.
>>>>
>>>> I have a Kafka queue storing messages. When something is connected to
>>>> the REST server, the Kafka queue has to be read and all waiting messages
>>>> should be collected and send back as a response to the REST client,
>>>>
>>>> I tried to google for a hint or solution, but I couldn't find it.
>>>>
>>>> With regards,
>>>> Mit freundlichem Gruß,
>>>> Met vriendelijke groet,
>>>>
>>>> *Ralph Vercauteren*
>>>> Technical Architect QAD Automation Solutions
>>>> Mobile NL: +31 6 5397 7230
>>>> rrv@qad.com
>>>>
>>>> This e-mail may contain QAD proprietary information and should be
>>>> treated as confidential.
>>>>
>>>
Re: To create a REST server
Posted by David Handermann <ex...@gmail.com>.
Ralph,
Thanks for the reply and additional details. I may be misunderstanding,
but just for clarification, in your diagram, are you expecting NiFi to act
as the REST server? If that is the case, then you should be able to use
one of the ConsumeKafka processors to retrieve messages and return them to
the REST client. At a very basic level, the flow would go from
HandleHttpRequest to ConsumeKafka to HandleHttpResponse. Depending on the
format of the messages retrieved from Kafka, you may need to introduce
additional processors prior to HandleHttpResponse in order to format the
output for delivery to the REST client.
Regards,
David Handermann
On Thu, Feb 4, 2021 at 7:47 AM Ralph Vercauteren <rr...@qad.com> wrote:
> Hi David,
>
> Thanks for responding.
>
> I understand that part, still have the issue to see how we get the kafka
> messages from a topic.
>
> [image: image.png]
>
> With regards,
> Mit freundlichem Gruß,
> Met vriendelijke groet,
>
> *Ralph Vercauteren*
> Technical Architect QAD Automation Solutions
> Mobile NL: +31 6 5397 7230
> rrv@qad.com
>
> This e-mail may contain QAD proprietary information and should be treated
> as confidential.
>
>
> On Thu, Feb 4, 2021 at 2:21 PM David Handermann <
> exceptionfactory@gmail.com> wrote:
>
>> Hi Ralph,
>>
>> If you are looking to create a REST service for receiving and processing
>> messages in NiFi, take a look at the ListenHTTP processor, as well as the
>> HandleHttpRequest and HandleHttpResponse processors. The ListenHTTP
>> processor supports receiving HTTP requests and sending the contents to
>> other processors. The ListenHTTP processor provides the easiest approach
>> for getting started. If you need to perform validation or additional
>> processing before sending back a response to REST clients, then using
>> HandleHttpRequest and HandleHttpResponse with other processors in between
>> can support that approach. When using HandleHttpRequest and
>> HandleHttpResponse, it is important to optimize intermediate as much as
>> possible to avoid HTTP clients timing out while waiting for a response.
>> Both processors support a number of configuration properties, so be sure to
>> read through the documentation for each one in order to determine the
>> optimal settings for your configuration.
>>
>> Regards,
>> David Handermann
>>
>> On Thu, Feb 4, 2021 at 3:32 AM Ralph Vercauteren <rr...@qad.com> wrote:
>>
>>> Hi all,
>>>
>>> Not sure if I need to ask this in this mailing list or the list of the
>>> developers.
>>>
>>> I want to create a REST server with NIFI.
>>>
>>> I have a Kafka queue storing messages. When something is connected to
>>> the REST server, the Kafka queue has to be read and all waiting messages
>>> should be collected and send back as a response to the REST client,
>>>
>>> I tried to google for a hint or solution, but I couldn't find it.
>>>
>>> With regards,
>>> Mit freundlichem Gruß,
>>> Met vriendelijke groet,
>>>
>>> *Ralph Vercauteren*
>>> Technical Architect QAD Automation Solutions
>>> Mobile NL: +31 6 5397 7230
>>> rrv@qad.com
>>>
>>> This e-mail may contain QAD proprietary information and should be
>>> treated as confidential.
>>>
>>
Re: To create a REST server
Posted by Ralph Vercauteren <rr...@qad.com>.
Hi David,
Thanks for responding.
I understand that part, still have the issue to see how we get the kafka
messages from a topic.
[image: image.png]
With regards,
Mit freundlichem Gruß,
Met vriendelijke groet,
*Ralph Vercauteren*
Technical Architect QAD Automation Solutions
Mobile NL: +31 6 5397 7230
rrv@qad.com
This e-mail may contain QAD proprietary information and should be treated
as confidential.
On Thu, Feb 4, 2021 at 2:21 PM David Handermann <ex...@gmail.com>
wrote:
> Hi Ralph,
>
> If you are looking to create a REST service for receiving and processing
> messages in NiFi, take a look at the ListenHTTP processor, as well as the
> HandleHttpRequest and HandleHttpResponse processors. The ListenHTTP
> processor supports receiving HTTP requests and sending the contents to
> other processors. The ListenHTTP processor provides the easiest approach
> for getting started. If you need to perform validation or additional
> processing before sending back a response to REST clients, then using
> HandleHttpRequest and HandleHttpResponse with other processors in between
> can support that approach. When using HandleHttpRequest and
> HandleHttpResponse, it is important to optimize intermediate as much as
> possible to avoid HTTP clients timing out while waiting for a response.
> Both processors support a number of configuration properties, so be sure to
> read through the documentation for each one in order to determine the
> optimal settings for your configuration.
>
> Regards,
> David Handermann
>
> On Thu, Feb 4, 2021 at 3:32 AM Ralph Vercauteren <rr...@qad.com> wrote:
>
>> Hi all,
>>
>> Not sure if I need to ask this in this mailing list or the list of the
>> developers.
>>
>> I want to create a REST server with NIFI.
>>
>> I have a Kafka queue storing messages. When something is connected to the
>> REST server, the Kafka queue has to be read and all waiting messages should
>> be collected and send back as a response to the REST client,
>>
>> I tried to google for a hint or solution, but I couldn't find it.
>>
>> With regards,
>> Mit freundlichem Gruß,
>> Met vriendelijke groet,
>>
>> *Ralph Vercauteren*
>> Technical Architect QAD Automation Solutions
>> Mobile NL: +31 6 5397 7230
>> rrv@qad.com
>>
>> This e-mail may contain QAD proprietary information and should be treated
>> as confidential.
>>
>
Re: To create a REST server
Posted by David Handermann <ex...@gmail.com>.
Hi Ralph,
If you are looking to create a REST service for receiving and processing
messages in NiFi, take a look at the ListenHTTP processor, as well as the
HandleHttpRequest and HandleHttpResponse processors. The ListenHTTP
processor supports receiving HTTP requests and sending the contents to
other processors. The ListenHTTP processor provides the easiest approach
for getting started. If you need to perform validation or additional
processing before sending back a response to REST clients, then using
HandleHttpRequest and HandleHttpResponse with other processors in between
can support that approach. When using HandleHttpRequest and
HandleHttpResponse, it is important to optimize intermediate as much as
possible to avoid HTTP clients timing out while waiting for a response.
Both processors support a number of configuration properties, so be sure to
read through the documentation for each one in order to determine the
optimal settings for your configuration.
Regards,
David Handermann
On Thu, Feb 4, 2021 at 3:32 AM Ralph Vercauteren <rr...@qad.com> wrote:
> Hi all,
>
> Not sure if I need to ask this in this mailing list or the list of the
> developers.
>
> I want to create a REST server with NIFI.
>
> I have a Kafka queue storing messages. When something is connected to the
> REST server, the Kafka queue has to be read and all waiting messages should
> be collected and send back as a response to the REST client,
>
> I tried to google for a hint or solution, but I couldn't find it.
>
> With regards,
> Mit freundlichem Gruß,
> Met vriendelijke groet,
>
> *Ralph Vercauteren*
> Technical Architect QAD Automation Solutions
> Mobile NL: +31 6 5397 7230
> rrv@qad.com
>
> This e-mail may contain QAD proprietary information and should be treated
> as confidential.
>