You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2022/07/20 22:17:09 UTC
[GitHub] [nifi] exceptionfactory opened a new pull request, #6228: NIFI-10235 Set Replay ContentClaim Length from Content Repository
exceptionfactory opened a new pull request, #6228:
URL: https://github.com/apache/nifi/pull/6228
# Summary
[NIFI-10235](https://issues.apache.org/jira/browse/NIFI-10235) Changes the behavior of Provenance Replay handling to resolve issues when running with encrypted repositories enabled.
When running with encrypted repositories, the content stored in the repository is larger than the file size itself, since it contains an encryption metadata header. This difference in size causes Provenance Replays to fail due to setting the replay ContentClaim length based on the previous file size. Changing the approach to set the ContentClaim length based on the ResourceClaim size that the Content Repository reports allows replays to function as expected.
# Tracking
Please complete the following tracking steps prior to pull request creation.
### Issue Tracking
- [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
### Pull Request Tracking
- [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
- [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
### Pull Request Formatting
- [X] Pull Request based on current revision of the `main` branch
- [X] Pull Request refers to a feature branch with one commit containing changes
# Verification
Please indicate the verification steps performed prior to pull request creation.
### Build
- [X] Build completed using `mvn clean install -P contrib-check`
- [X] JDK 8
- [ ] JDK 11
- [ ] JDK 17
### Licensing
- [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
- [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
### Documentation
- [ ] Documentation formatting appears as expected in rendered files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] NissimShiman commented on pull request #6228: NIFI-10235 Set Replay ContentClaim Length from Content Repository
Posted by GitBox <gi...@apache.org>.
NissimShiman commented on PR #6228:
URL: https://github.com/apache/nifi/pull/6228#issuecomment-1204307327
@exceptionfactory The initial fix works as I was able to verify (using a pkcs12 keystore) that trying to replay a SEND for InvokeHTTP (to ListenHTTP) did not work before the fix, but does work after this fix.
I see where the code is now looking at ContentClaim size to get a more accurate size. Very nice diagnosis.
LGTM +1,
but I'll plan to circle back if/when additional work is done.
The BCFKS issue is even when using a real password (and doesn't occur when following the instructions for PKCS12 keys - found in the first link of the SECRET KEY GENERATION section), so I am not sure what it happening with that.
Error is:
keytool error: java.io.IOException: BCKFS KeyStore corrupted: MAC calculation failed
using java:
openjdk version 1.8.0_332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #6228: NIFI-10235 Set Replay ContentClaim Length from Content Repository
Posted by GitBox <gi...@apache.org>.
exceptionfactory closed pull request #6228: NIFI-10235 Set Replay ContentClaim Length from Content Repository
URL: https://github.com/apache/nifi/pull/6228
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] NissimShiman commented on pull request #6228: NIFI-10235 Set Replay ContentClaim Length from Content Repository
Posted by GitBox <gi...@apache.org>.
NissimShiman commented on PR #6228:
URL: https://github.com/apache/nifi/pull/6228#issuecomment-1204089507
@exceptionfactory Working to recreate setup before and after fix...
When trying instructions for creating storetype of type BCFKS https://exceptionfactory.com/posts/2021/11/10/configuring-apache-nifi-repository-encryption (under SECRET KEY GENERATION) the
keytool -genseckey
followed by the
keytool -list
command
has output saying that there is an invalid MAC
This is not a blocker on this (as ticket was done with PKCS12), but I noticed you were the author of that article so maybe that could be looked into at some later time (and by the way, thank you for your very informative articles in general :) )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6228: NIFI-10235 Set Replay ContentClaim Length from Content Repository
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on PR #6228:
URL: https://github.com/apache/nifi/pull/6228#issuecomment-1241160368
Thanks again for the feedback on this @NissimShiman, closing for now and will reopen pending a more comprehensive solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6228: NIFI-10235 Set Replay ContentClaim Length from Content Repository
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on PR #6228:
URL: https://github.com/apache/nifi/pull/6228#issuecomment-1204100417
> When trying instructions for creating storetype of type BCFKS https://exceptionfactory.com/posts/2021/11/10/configuring-apache-nifi-repository-encryption (under SECRET KEY GENERATION) the keytool -genseckey followed by the keytool -list command has output saying that there is an invalid MAC
Thanks for the feedback @NissimShiman!
When verifying the the BCFKS keystore, it is necessary to provide the store password in the command. The example has the word `PLACEHOLDER`, which must be changed to the actual password.
I am still evaluating the issues associated with encrypted repositories. The current version of this pull request resolves the problem for simple use scenarios, but there is an additional issue related to content claim length determination when the framework writes multiple FlowFiles to a single resource. I am planning to update the pull request as soon as I have a working solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org