You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Brian Ghigiarelli (Jira)" <ji...@apache.org> on 2024/01/25 01:46:00 UTC
[jira] [Updated] (NIFI-12668) Test for Git-backed Registry fails with global Git config value
[ https://issues.apache.org/jira/browse/NIFI-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Ghigiarelli updated NIFI-12668:
-------------------------------------
Description:
(!) This may be OBE per NIFI-12667
----
The Git-backed Registry provider, GitFlowPersistenceProvider, does not digitally sign commits to Git, nor does it specify the GPG configuration that would be used if it were to sign commits. See [GitFlowMetaData.java:478|https://github.com/apache/nifi/blob/89836f32d017d77972a4de09c4e864b0e11899a8/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowMetaData.java#L478].
When the GPG configuration is not specified, the underlying jgit library uses the default value of openpgp for the GPG format when it performs the commit. This works fine except when the system's global Git configuration has `gpg.format` set to `ssh`. Available options are:
* openpgp (default)
* x509
* ssh
The underlying jgit library does not recognize `ssh` as a valid GPG format. jgit validates the GPG configuration when committing to git, whether it signs the commit or not.
This issue manifests when a user attempts to run NiFi's tests with their global Git configuration has `gpg.format` set to `ssh`. It would also occur if the NiFi Registry ran on a server with Git installed and configured in the same manner.
h3. Steps to reproduce
From the root NiFi source code directory, run:
{code:java}
git config --global gpg.format ssh
cd ./nifi-registry/nifi-registry-core/nifi-registry-framework
mvn clean test{code}
h3. Observed Failure
{code:java}
[ERROR] Tests run: 6, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 3.387 s <<< FAILURE! -- in org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider
[ERROR] org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadLargeFlow -- Time elapsed: 3.273 s <<< ERROR!
java.lang.IllegalArgumentException: Invalid value: gpg.format=ssh
at org.eclipse.jgit.lib.DefaultTypedConfigGetter.getEnum(DefaultTypedConfigGetter.java:103)
at org.eclipse.jgit.lib.Config.getEnum(Config.java:453)
at org.eclipse.jgit.lib.GpgConfig.<init>(GpgConfig.java:86)
at org.eclipse.jgit.api.CommitCommand.processOptions(CommitCommand.java:662)
at org.eclipse.jgit.api.CommitCommand.call(CommitCommand.java:189)
at org.apache.nifi.registry.provider.flow.git.GitFlowMetaData.commit(GitFlowMetaData.java:481)
at org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider.saveFlowContent(GitFlowPersistenceProvider.java:187)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.lambda$testLoadLargeFlow$14(TestGitFlowPersistenceProvider.java:311)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.assertProvider(TestGitFlowPersistenceProvider.java:112)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadLargeFlow(TestGitFlowPersistenceProvider.java:300)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
[ERROR] org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadCommitHistories -- Time elapsed: 0.008 s <<< ERROR!
java.lang.IllegalArgumentException: Invalid value: gpg.format=ssh
at org.eclipse.jgit.lib.DefaultTypedConfigGetter.getEnum(DefaultTypedConfigGetter.java:103)
at org.eclipse.jgit.lib.Config.getEnum(Config.java:453)
at org.eclipse.jgit.lib.GpgConfig.<init>(GpgConfig.java:86)
at org.eclipse.jgit.api.CommitCommand.processOptions(CommitCommand.java:662)
at org.eclipse.jgit.api.CommitCommand.call(CommitCommand.java:189)
at org.apache.nifi.registry.provider.flow.git.GitFlowMetaData.commit(GitFlowMetaData.java:481)
at org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider.saveFlowContent(GitFlowPersistenceProvider.java:187)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.lambda$testLoadCommitHistories$6(TestGitFlowPersistenceProvider.java:153)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.assertProvider(TestGitFlowPersistenceProvider.java:112)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadCommitHistories(TestGitFlowPersistenceProvider.java:140)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) {code}
...
{code:java}
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestGitFlowPersistenceProvider.testLoadCommitHistories:140->assertProvider:112->lambda$testLoadCommitHistories$6:153 » IllegalArgument Invalid value: gpg.format=ssh
[ERROR] TestGitFlowPersistenceProvider.testLoadLargeFlow:300->assertProvider:112->lambda$testLoadLargeFlow$14:311 » IllegalArgument Invalid value: gpg.format=ssh
[INFO]
[ERROR] Tests run: 300, Failures: 0, Errors: 2, Skipped: 0 {code}
h3. Resolution
There are at least a few options here:
# add a no-op GPG configuration to the existing commit call so that it can pass validation
# configure jgit to not read the system .gitconfig ([may not be viable|https://www.eclipse.org/forums/index.php/m/1855720/#msg_1855720])
# update the jgit library (out of scope for NiFi, and has been an issue with jgit since Nov 2021 when [Git 2.34|https://github.blog/2021-11-15-highlights-from-git-2-34/] introduced SSH signing support)
# replace jgit with an updated alternative that supports the full range of signing options
was:
The Git-backed Registry provider, GitFlowPersistenceProvider, does not digitally sign commits to Git, nor does it specify the GPG configuration that would be used if it were to sign commits. See [GitFlowMetaData.java:478|https://github.com/apache/nifi/blob/89836f32d017d77972a4de09c4e864b0e11899a8/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowMetaData.java#L478].
When the GPG configuration is not specified, the underlying jgit library uses the default value of openpgp for the GPG format when it performs the commit. This works fine except when the system's global Git configuration has `gpg.format` set to `ssh`. Available options are:
* openpgp (default)
* x509
* ssh
The underlying jgit library does not recognize `ssh` as a valid GPG format. jgit validates the GPG configuration when committing to git, whether it signs the commit or not.
This issue manifests when a user attempts to run NiFi's tests with their global Git configuration has `gpg.format` set to `ssh`. It would also occur if the NiFi Registry ran on a server with Git installed and configured in the same manner.
h3. Steps to reproduce
From the root NiFi source code directory, run:
{code:java}
git config --global gpg.format ssh
cd ./nifi-registry/nifi-registry-core/nifi-registry-framework
mvn clean test{code}
h3. Observed Failure
{code:java}
[ERROR] Tests run: 6, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 3.387 s <<< FAILURE! -- in org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider
[ERROR] org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadLargeFlow -- Time elapsed: 3.273 s <<< ERROR!
java.lang.IllegalArgumentException: Invalid value: gpg.format=ssh
at org.eclipse.jgit.lib.DefaultTypedConfigGetter.getEnum(DefaultTypedConfigGetter.java:103)
at org.eclipse.jgit.lib.Config.getEnum(Config.java:453)
at org.eclipse.jgit.lib.GpgConfig.<init>(GpgConfig.java:86)
at org.eclipse.jgit.api.CommitCommand.processOptions(CommitCommand.java:662)
at org.eclipse.jgit.api.CommitCommand.call(CommitCommand.java:189)
at org.apache.nifi.registry.provider.flow.git.GitFlowMetaData.commit(GitFlowMetaData.java:481)
at org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider.saveFlowContent(GitFlowPersistenceProvider.java:187)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.lambda$testLoadLargeFlow$14(TestGitFlowPersistenceProvider.java:311)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.assertProvider(TestGitFlowPersistenceProvider.java:112)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadLargeFlow(TestGitFlowPersistenceProvider.java:300)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
[ERROR] org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadCommitHistories -- Time elapsed: 0.008 s <<< ERROR!
java.lang.IllegalArgumentException: Invalid value: gpg.format=ssh
at org.eclipse.jgit.lib.DefaultTypedConfigGetter.getEnum(DefaultTypedConfigGetter.java:103)
at org.eclipse.jgit.lib.Config.getEnum(Config.java:453)
at org.eclipse.jgit.lib.GpgConfig.<init>(GpgConfig.java:86)
at org.eclipse.jgit.api.CommitCommand.processOptions(CommitCommand.java:662)
at org.eclipse.jgit.api.CommitCommand.call(CommitCommand.java:189)
at org.apache.nifi.registry.provider.flow.git.GitFlowMetaData.commit(GitFlowMetaData.java:481)
at org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider.saveFlowContent(GitFlowPersistenceProvider.java:187)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.lambda$testLoadCommitHistories$6(TestGitFlowPersistenceProvider.java:153)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.assertProvider(TestGitFlowPersistenceProvider.java:112)
at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadCommitHistories(TestGitFlowPersistenceProvider.java:140)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) {code}
...
{code:java}
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] TestGitFlowPersistenceProvider.testLoadCommitHistories:140->assertProvider:112->lambda$testLoadCommitHistories$6:153 » IllegalArgument Invalid value: gpg.format=ssh
[ERROR] TestGitFlowPersistenceProvider.testLoadLargeFlow:300->assertProvider:112->lambda$testLoadLargeFlow$14:311 » IllegalArgument Invalid value: gpg.format=ssh
[INFO]
[ERROR] Tests run: 300, Failures: 0, Errors: 2, Skipped: 0 {code}
h3. Resolution
There are at least a few options here:
# add a no-op GPG configuration to the existing commit call so that it can pass validation
# configure jgit to not read the system .gitconfig ([may not be viable|https://www.eclipse.org/forums/index.php/m/1855720/#msg_1855720])
# update the jgit library (out of scope for NiFi, and has been an issue with jgit since Nov 2021 when [Git 2.34|https://github.blog/2021-11-15-highlights-from-git-2-34/] introduced SSH signing support)
# replace jgit with an updated alternative that supports the full range of signing options
> Test for Git-backed Registry fails with global Git config value
> ---------------------------------------------------------------
>
> Key: NIFI-12668
> URL: https://issues.apache.org/jira/browse/NIFI-12668
> Project: Apache NiFi
> Issue Type: Improvement
> Components: NiFi Registry
> Affects Versions: 2.0.0-M1
> Reporter: Brian Ghigiarelli
> Priority: Minor
>
> (!) This may be OBE per NIFI-12667
> ----
> The Git-backed Registry provider, GitFlowPersistenceProvider, does not digitally sign commits to Git, nor does it specify the GPG configuration that would be used if it were to sign commits. See [GitFlowMetaData.java:478|https://github.com/apache/nifi/blob/89836f32d017d77972a4de09c4e864b0e11899a8/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/provider/flow/git/GitFlowMetaData.java#L478].
> When the GPG configuration is not specified, the underlying jgit library uses the default value of openpgp for the GPG format when it performs the commit. This works fine except when the system's global Git configuration has `gpg.format` set to `ssh`. Available options are:
> * openpgp (default)
> * x509
> * ssh
> The underlying jgit library does not recognize `ssh` as a valid GPG format. jgit validates the GPG configuration when committing to git, whether it signs the commit or not.
> This issue manifests when a user attempts to run NiFi's tests with their global Git configuration has `gpg.format` set to `ssh`. It would also occur if the NiFi Registry ran on a server with Git installed and configured in the same manner.
> h3. Steps to reproduce
> From the root NiFi source code directory, run:
> {code:java}
> git config --global gpg.format ssh
> cd ./nifi-registry/nifi-registry-core/nifi-registry-framework
> mvn clean test{code}
> h3. Observed Failure
> {code:java}
> [ERROR] Tests run: 6, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 3.387 s <<< FAILURE! -- in org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider
> [ERROR] org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadLargeFlow -- Time elapsed: 3.273 s <<< ERROR!
> java.lang.IllegalArgumentException: Invalid value: gpg.format=ssh
> at org.eclipse.jgit.lib.DefaultTypedConfigGetter.getEnum(DefaultTypedConfigGetter.java:103)
> at org.eclipse.jgit.lib.Config.getEnum(Config.java:453)
> at org.eclipse.jgit.lib.GpgConfig.<init>(GpgConfig.java:86)
> at org.eclipse.jgit.api.CommitCommand.processOptions(CommitCommand.java:662)
> at org.eclipse.jgit.api.CommitCommand.call(CommitCommand.java:189)
> at org.apache.nifi.registry.provider.flow.git.GitFlowMetaData.commit(GitFlowMetaData.java:481)
> at org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider.saveFlowContent(GitFlowPersistenceProvider.java:187)
> at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.lambda$testLoadLargeFlow$14(TestGitFlowPersistenceProvider.java:311)
> at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.assertProvider(TestGitFlowPersistenceProvider.java:112)
> at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadLargeFlow(TestGitFlowPersistenceProvider.java:300)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> [ERROR] org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadCommitHistories -- Time elapsed: 0.008 s <<< ERROR!
> java.lang.IllegalArgumentException: Invalid value: gpg.format=ssh
> at org.eclipse.jgit.lib.DefaultTypedConfigGetter.getEnum(DefaultTypedConfigGetter.java:103)
> at org.eclipse.jgit.lib.Config.getEnum(Config.java:453)
> at org.eclipse.jgit.lib.GpgConfig.<init>(GpgConfig.java:86)
> at org.eclipse.jgit.api.CommitCommand.processOptions(CommitCommand.java:662)
> at org.eclipse.jgit.api.CommitCommand.call(CommitCommand.java:189)
> at org.apache.nifi.registry.provider.flow.git.GitFlowMetaData.commit(GitFlowMetaData.java:481)
> at org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider.saveFlowContent(GitFlowPersistenceProvider.java:187)
> at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.lambda$testLoadCommitHistories$6(TestGitFlowPersistenceProvider.java:153)
> at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.assertProvider(TestGitFlowPersistenceProvider.java:112)
> at org.apache.nifi.registry.provider.flow.git.TestGitFlowPersistenceProvider.testLoadCommitHistories(TestGitFlowPersistenceProvider.java:140)
> at java.base/java.lang.reflect.Method.invoke(Method.java:580)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1596) {code}
> ...
> {code:java}
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR] TestGitFlowPersistenceProvider.testLoadCommitHistories:140->assertProvider:112->lambda$testLoadCommitHistories$6:153 » IllegalArgument Invalid value: gpg.format=ssh
> [ERROR] TestGitFlowPersistenceProvider.testLoadLargeFlow:300->assertProvider:112->lambda$testLoadLargeFlow$14:311 » IllegalArgument Invalid value: gpg.format=ssh
> [INFO]
> [ERROR] Tests run: 300, Failures: 0, Errors: 2, Skipped: 0 {code}
> h3. Resolution
> There are at least a few options here:
> # add a no-op GPG configuration to the existing commit call so that it can pass validation
> # configure jgit to not read the system .gitconfig ([may not be viable|https://www.eclipse.org/forums/index.php/m/1855720/#msg_1855720])
> # update the jgit library (out of scope for NiFi, and has been an issue with jgit since Nov 2021 when [Git 2.34|https://github.blog/2021-11-15-highlights-from-git-2-34/] introduced SSH signing support)
> # replace jgit with an updated alternative that supports the full range of signing options
--
This message was sent by Atlassian Jira
(v8.20.10#820010)