[jira] [Commented] (NIFI-7505) Add InvokeHTTPRecord processor

["Peter Spiller (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-7505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17748486#comment-17748486 ] 

Peter Spiller commented on NIFI-7505:
-------------------------------------

I've had a go at creating an InvokeHTTPRecord processor, as record-based HTTP requests are something I've been having issues with for a while. It's based on the InvokeHTTP processor, but with modifications for record-based operation (e.g. varying request parameters come from individual records instead of flowfile attributes):

https://github.com/apache/nifi/pull/7532

It's possible to achieve something like this by using a LookupRecord processor combined with a RestLookupService, but I've had problems with this approach. RestLookupService is designed around looking up data, not making requests that potentially have side-effects. If HTTP requests encounter transient errors, retrying them is difficult and requires splitting and merging record-based flowfiles. Semantically, using a LookupService to make requests with side-effects doesn't feel great.

The PR isn't complete yet (it still needs tests), but I've created it in case anybody has any feedback on the approach I'm taking.

> Add InvokeHTTPRecord processor
> ------------------------------
>
>                 Key: NIFI-7505
>                 URL: https://issues.apache.org/jira/browse/NIFI-7505
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>    Affects Versions: 1.11.4
>            Reporter: Andy LoPresto
>            Priority: Major
>              Labels: Record, http, processor, rest
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Some users have recently requested being able to invoke a specific URL via {{InvokeHTTP}} on every record in a flowfile. Currently, the {{InvokeHTTP}} processor can only handle one piece of data per flowfile. There are some workarounds available for specific patterns with {{LookupRecord}} + {{RestLookupService}}, but this is not a complete solution. I propose introducing an {{InvokeHTTPRecord}} processor, providing the {{InvokeHTTP}} functionality in conjunction with the record processing behavior. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)