RE: [EXT] Re: After 1.7.1 upgrade, no Provenance data is visible

["Peter Wicks (pwicks)" <pw...@micron.com>]

Thanks Mike, that fixed it.

--Peter

From: Michael Moser [mailto:moser.mw@gmail.com]
Sent: Friday, August 10, 2018 3:49 PM
To: users@nifi.apache.org
Subject: [EXT] Re: After 1.7.1 upgrade, no Provenance data is visible

Hi Peter,

There was a change to provenance related access policies in 1.7.0.  Check out the Migration Guide [1] for 1.7.0.  It talks about what you'll need to do.

[1] - https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance

-- Mike


On Fri, Aug 10, 2018 at 5:39 PM Peter Wicks (pwicks) <pw...@micron.com>> wrote:
After upgrading our NiFi instances to 1.7.1 we are not able to see Provenance data anymore in the UI. We see this across about a dozen instances.
In the UI it tells me provenance is available for about the last 24 hours, and I can see that files have moved in and out of the processor in the last 5 min. In the logs, I can see it query provenance, and that the query returns 0 results.

Thoughts? I saw a few tickets related to Provenance in 1.7, but not sure if they have an impact.

Here are our properties:

# Provenance Repository Properties
nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
nifi.provenance.repository.debug.frequency=1_000_000
nifi.provenance.repository.encryption.key.provider.implementation=
nifi.provenance.repository.encryption.key.provider.location=
nifi.provenance.repository.encryption.key.id<http://nifi.provenance.repository.encryption.key.id>=
nifi.provenance.repository.encryption.key=

# Persistent Provenance Repository Properties
nifi.provenance.repository.directory.default=/data/nifi/repositories/provenance_repository
nifi.provenance.repository.max.storage.time=24 hours
nifi.provenance.repository.max.storage.size=1 GB
nifi.provenance.repository.rollover.time=30 secs
nifi.provenance.repository.rollover.size=100 MB
nifi.provenance.repository.query.threads=2
nifi.provenance.repository.index.threads=2
nifi.provenance.repository.compress.on.rollover=true
nifi.provenance.repository.always.sync=false
nifi.provenance.repository.journal.count=16
# Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are:
# EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, Relationship, Details
nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship
# FlowFile Attributes that should be indexed and made searchable.  Some examples to consider are filename, uuid, mime.type
nifi.provenance.repository.indexed.attributes=
# Large values for the shard size will result in more Java heap usage when searching the Provenance Repository
# but should provide better performance
nifi.provenance.repository.index.shard.size=500 MB
# Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from
# the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved.
nifi.provenance.repository.max.attribute.length=65536
nifi.provenance.repository.concurrent.merge.threads=2
nifi.provenance.repository.warm.cache.frequency=1 hour

Thanks,
  Peter