[jira] [Commented] (NIFI-16399) Enable attribute updates from ExecuteStreamCommand

["David Handermann (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-16399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18121421#comment-18121421 ] 

David Handermann commented on NIFI-16399:
-----------------------------------------

Thanks for the additional consideration of options.

Introducing a separate temporary file with JSON content is a protocol contract, but it is out of band from the primary stream. This is where I think a new Processor is probably the better path forward.

Rather than attempting to force something into ExecuteStreamCommand, what do you think about a new Processor? The new Processor would have a similar shape, but expect a new type of protocol. Although I mentioned HTTP as an analogy, after some evaluation, the MIME format, used for email, seems like a good possibility. It has been around for a long time, it provides a clear header and body structure, and it is supported across many languages. It is also simple enough that it could be implemented without any library at all. Such a Processor could be named something like ExecuteFlowFileCommand or perhaps ExecuteMimeCommand. This provides the opportunity to make the script contract clear, and then scripts requiring the ability to write FlowFile attributes could use this new Processor, and others could continue using ExecuteStreamCommand.

I also considered protocols like STOMP, which have a similar header-then-body structure, but MIME seems better to avoid the network-based implications.

If this sounds like a reasonable approach, I would be glad to draft an implementation for further consideration.

> Enable attribute updates from ExecuteStreamCommand
> --------------------------------------------------
>
>                 Key: NIFI-16399
>                 URL: https://issues.apache.org/jira/browse/NIFI-16399
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Mike Thomsen
>            Assignee: Mike Thomsen
>            Priority: Major
>          Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> This change will create an optional mode where ExecuteStreamCommand will write a temp file to the file system that can be used by the executed command to communicate updates to the flowfile attributes. It will be a flat json document.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)