[jira] [Comment Edited] (NIFI-10235) Provenance replay fails when repository encryption is enabled

["Peter Kimberley (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-10235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17567177#comment-17567177 ] 

Peter Kimberley edited comment on NIFI-10235 at 7/15/22 10:16 AM:
------------------------------------------------------------------

As an update, I've done a completely fresh, traditional install on a pristine Rocky Linux 8.5 VM. No containers / Kubernetes.

Java version: OpenJDK 1.8.0.332

Same issue - once repository encryption is enabled, attempting to replay fails, with a similar log message recorded. See attached: [^error-base-install.log]

NiFi flow is attached: [^NiFi_Flow.json]


was (Author: JIRAUSER292873):
As an update, I've done a completely fresh, traditional install on a pristine Rocky Linux 8.5 VM. No containers / Kubernetes.

Java version: OpenJDK 1.8.0.332

Same issue - once repository encryption is enabled, attempting to replay fails, with a similar log message recorded. See attached {*}error-base-install.log{*}.

NiFi flow is attached[^NiFi_Flow.json]

> Provenance replay fails when repository encryption is enabled
> -------------------------------------------------------------
>
>                 Key: NIFI-10235
>                 URL: https://issues.apache.org/jira/browse/NIFI-10235
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework, Security
>    Affects Versions: 1.16.3
>         Environment: RHEL 8.5
>            Reporter: Peter Kimberley
>            Priority: Major
>              Labels: encryption, provenance, replay
>         Attachments: NiFi_Flow.json, error-base-install.log, error.log
>
>
> h3. Problem summary
> When repository encryption is enabled, replaying a DROP provenance record fails, with the following error appearing in the logs:
> {quote}org.apache.nifi.processor.exception.FlowFileAccessException: Failed to export StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default, section=23], offset=379, length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576] to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650 due to java.io.EOFException: *Attempted to copy {color:#ff8b00}1048576{color} bytes but only {color:#ff8b00}1048197{color} bytes were available*
> {quote}
>  
> I've observed that the difference between the sizes mentioned in the log is {+}*always 379 bytes*{+}, regardless of the length of the input file.
>  
> With repository encryption disabled, provenance replay works as expected.
> h3. Configuration
>  # NiFi v1.16.3 running as a three-node cluster in Kubernetes.
>  # Each node has up to 8GB memory and 4 CPUs available to it.
>  # Testing has included both NFS and ephemeral (emptyDir) storage.
>  # The encryption key was generated by the following command, using the same JDK version:
>  ## keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore repository.p12 -storetype PKCS12
> h4. nifi.properties
> {quote}nifi.repository.encryption.protocol.version=1
> nifi.repository.encryption.key.id=key-1
> nifi.repository.encryption.key.provider=KEYSTORE
> nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
> nifi.repository.encryption.key.provider.keystore.password=<password>
> {quote}
> h3. Processor group
> GenerateFlowFile processor generating 1MB random files every second to a PutFile processor. Have also tested with InvokeHTTP.
> h3. Other comments
> With repository encryption enabled, I am able to download files via the provenance UI (suggesting that encryption/decryption works). The processor group also performs all other actions as expected.
> Not having the ability to replay provenance records is a blocker for our deployment, which requires data to be encrypted at rest and in transit.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)