You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Lehel Boér (Jira)" <ji...@apache.org> on 2024/02/12 20:27:00 UTC

[jira] [Commented] (NIFI-12781) Remove file location from UPLOAD provenance event

    [ https://issues.apache.org/jira/browse/NIFI-12781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816785#comment-17816785 ] 

Lehel Boér commented on NIFI-12781:
-----------------------------------

Thanks, there was a typo in the title, I fixed it. The first usage of UPLOAD event will be in PutAzureBlobStorage_v12 when used with FileResourceService. When the FileResourceService is used, the content of the FlowFile is replaced by external data (FETCH) and we upload external data to an external system (UPLOAD), so these provenance events should be emitted in this case.

 

So far the UPLOAD event got a ProvenanceFileResource object as an argument which contained the size of the resource in bytes and the location. The initial and most important idea was to include the size of the uploaded file too in the provenance event and use that size in Status History instead of the FlowFile size. The location was additional. 

 

Adding the location would introduce additional changes to FileResourceService which was intentionally designed without storing the file location. This would further complicate and delay the completion of this task, that's why the easiest way is to remove the ProvenanceFileResource(size, location) class and instead simply store the size of the resource. This would simply the matter and preserve the solution for the initial cause.

> Remove file location from UPLOAD provenance event
> -------------------------------------------------
>
>                 Key: NIFI-12781
>                 URL: https://issues.apache.org/jira/browse/NIFI-12781
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Lehel Boér
>            Assignee: Lehel Boér
>            Priority: Major
>
> Introducing a file location in the UPLOAD provenance event adds complexity, particularly in regard to modifications in the FileResourceService. Given that the primary concern is displaying file size in the Status History, for now it's advisable to remove the ProvenanceFileResource class. Since the UPLOAD event hasn't been utilized, there are no backward compatibility issues.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)