[jira] [Commented] (NIFI-6851) Improve Provenance RECEIVE and FETCH events to allow for specifics on how flowfile was RECEIVE'ed/FETCH'ed

["Nissim Shiman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17030129#comment-17030129 ] 

Nissim Shiman commented on NIFI-6851:
-------------------------------------

Regarding [~joewitt] 's question:
"would provenance events generated by an older nifi version be read by this one? And would provenance events written by this one be readable by an older nifi? [in case someone rolls back]"

The answer is yes to both.  Both have scenarios have been tested with both java 8 and java 11.

Test 1 was a follows:

Create simple graph on apache nifi 1.10.0
GetFile -> UpdateAttribute
run a few times.
Check Data Provenance

then using nifi compiled from the  new/NIFI-6851 version of apache nifi 
move over flow.xml.gz as well as the content_repository, database_repository, flowfile_repository, provenance_repository from 1.10.0 version to this  new/NIFI-6851 nifi version

start up nifi
the Data Provenance looks exactly as it did before (and is searchable, actions can be replayed etc)




 

Test 2 was as follows:

Create simple graph with this code (i.e. NIFI-6851) included in the build.
GetFile -> UpdateAttribute
run a few times.
Check Data Provenance

then using an older version of nifi (in my case, I did 1.10.0)
move over flow.xml.gz as well as the content_repository, database_repository, flowfile_repository, provenance_repository from the new/NIFI-6851 version to this older version
start up nifi
the Data Provenance looks exactly as it did before (and is searchable, actions can be replayed etc)

> Improve Provenance RECEIVE and FETCH events to allow for specifics on how flowfile was RECEIVE'ed/FETCH'ed
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-6851
>                 URL: https://issues.apache.org/jira/browse/NIFI-6851
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>    Affects Versions: 1.10.0
>            Reporter: Nissim Shiman
>            Assignee: Michael Hogue
>            Priority: Major
>          Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> Currently there is no way to know if a Provenance RECEIVE or FETCH event is the result of an active (i.e. GetFTP, FetchFTP, GetHTTP), passive (i.e. ListenHTTP)  or query (i.e. GetMongo) event.
> There was a discussion on the dev mailing lists regarding the most graceful way to do this [1] [2], 
>  
> At this time the smoothest path forward is to keep the current method signatures of receive() and fetch() as is in ProvenanceReporter.java [3]
> and to add new fetch() and receive()'s with a new parameter:
> receive(...,ACTIVE|PASSIVE|QUERY)
> If not specified it would be UNSPECIFIED. 
>  (Thank You [~joewitt] for this solution)
>  
>  
> [1] [https://mail-archives.apache.org/mod_mbox/nifi-dev/201910.mbox/browser] , topic of PULL ProvenanceEvent
> [2] [https://mail-archives.apache.org/mod_mbox/nifi-dev/201911.mbox/browser] , topic of Re: PULL ProvenanceEvent
> [3] [https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/provenance/ProvenanceReporter.java]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)