[jira] [Commented] (NIFI-11670) Encrypted Content Repository is very slow when FlowFiles have a non-zero Content Claim Offset

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-11670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731658#comment-17731658 ] 

ASF subversion and git services commented on NIFI-11670:
--------------------------------------------------------

Commit 702c6350344e4ae4cd31349747e96c1384017ed4 in nifi's branch refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=702c635034 ]

NIFI-11670 Refactored Content Repo OutputStream to create new Claim per FlowFile

Rather than creating many FlowFiles with the same Content Claim, refactored content repos' OutputStreams and ClaimWriteCache so that a new ContentClaim is created for each FlowFile. This ensures that we have a content claim offset of 0. The poor performance was due to having to use StreamUtils.skip() in conjunction with the CipherInputStream, which would only skip a max of 511 bytes at a time. By using a separate Content Claim per FlowFile, we no longer need to seek after creating the CipherInputStream

This closes #7363

Signed-off-by: David Handermann <ex...@apache.org>


> Encrypted Content Repository is very slow when FlowFiles have a non-zero Content Claim Offset
> ---------------------------------------------------------------------------------------------
>
>                 Key: NIFI-11670
>                 URL: https://issues.apache.org/jira/browse/NIFI-11670
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Major
>             Fix For: 1.latest, 2.latest
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> To replicate, create a flow with GenerateFlowFile -> MergeContent
> Configure GenerateFlowFile to generate 25 KB FlowFiles with a batch size of 1,000. It's important in order to replicate, that batch size be used.
> Configure MergeContent to merge bins of 1,000 FlowFiles.
> Merging the files when using unencrypted/default content repository takes milliseconds. Using Encrypted Repo it takes nearly a minute.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)