You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by th...@apache.org on 2022/11/04 16:48:41 UTC
[nifi] branch main updated: NIFI-10506 Moved Repository Encryption to Admin Guide
This is an automated email from the ASF dual-hosted git repository.
thenatog pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new f3cd416816 NIFI-10506 Moved Repository Encryption to Admin Guide
f3cd416816 is described below
commit f3cd416816ae0d0dc86d7bb7fc97601e4c315619
Author: exceptionfactory <ex...@apache.org>
AuthorDate: Wed Nov 2 12:30:02 2022 -0500
NIFI-10506 Moved Repository Encryption to Admin Guide
- Removed Repository Encryption section from User Guide
- Added Configuration Change Considerations section
Signed-off-by: Nathan Gough <th...@gmail.com>
This closes #6615.
---
.../src/main/asciidoc/administration-guide.adoc | 149 ++++++++++++++++++++-
nifi-docs/src/main/asciidoc/user-guide.adoc | 144 --------------------
2 files changed, 147 insertions(+), 146 deletions(-)
diff --git a/nifi-docs/src/main/asciidoc/administration-guide.adoc b/nifi-docs/src/main/asciidoc/administration-guide.adoc
index b4fd84dce1..5829b4dbf7 100644
--- a/nifi-docs/src/main/asciidoc/administration-guide.adoc
+++ b/nifi-docs/src/main/asciidoc/administration-guide.adoc
@@ -1599,11 +1599,156 @@ Being added to both the view and modify policies for the process group, User2 ca
image:user2-edit-connection.png["User2 Edit Connection"]
+[[repository-encryption]]
+== Repository Encryption
+
+NiFi supports encryption of local repositories using a configurable Key Provider to enable protection of information
+on the filesystem. Repository encryption configuration uses a version number to indicate the cipher algorithms, metadata
+format, and repository implementation classes. This approach provides a generalized method for configuration without the
+need to customize each repository implementation class.
+
+Repository encryption incurs a performance cost due to the overhead of cipher operations. Filesystem encryption at the
+operating system level provides an alternative solution, with different performance characteristics. For deployments
+where filesystem encryption is not configured, repository encryption provides an enhanced level of data protection.
+Due to increased performance requirements, more computing resources may be necessary to achieve sufficient throughput
+when enabling repository encryption.
+
+The security of repository encryption depends on a combination of the cipher algorithms and the protection of encryption
+keys. Key protection and key rotation are important parts of securing an encrypted repository configuration.
+Key protection involves limiting access to the Key Provider and key rotation requires manual updates to generate and
+specify a new encryption key.
+
+=== Configuration Change Considerations
+
+Disabling repository encryption on existing installations requires removing existing repository contents, and
+restarting the system after making configuration changes. For this reason, flow administrators should confirm that the
+system has processed all available FlowFiles to avoid losing information when disabling repository encryption.
+
+=== Repository Encryption Protocol Version 1
+
+The first version of support for repository encryption includes the following cipher algorithms:
+
+[options="header"]
+|=======================
+|Repository Type |Cipher Algorithm
+|Content Repository |AES/CTR/NoPadding
+|FlowFile Repository |AES/GCM/NoPadding
+|FlowFile Swap Manager |AES/GCM/NoPadding
+|Provenance Repository |AES/GCM/NoPadding
+|=======================
+
+The following classes provide the direct repository encryption implementation, extending standard classes:
+
+[options="header"]
+|=======================
+|Repository Type |Class
+|Content Repository |org.apache.nifi.content.EncryptedFileSystemRepository
+|FlowFile Repository |org.apache.nifi.wali.EncryptedSequentialAccessWriteAheadLog
+|FlowFile Swap Manager |org.apache.nifi.controller.EncryptedFileSystemSwapManager
+|Provenance Repository |org.apache.nifi.provenance.EncryptedWriteAheadProvenanceRepository
+|=======================
+
+==== Encryption Metadata Serialization
+
+Each repository implementation class leverages standard cipher operations to perform encryption and decryption. In order
+to support AES, the encryption process writes metadata associated with each encryption operation. Encryption protocol
+version 1 uses Java Object serialization to write objects containing the encryption Key Identifier, the cipher
+Initialization Vector, and other required properties. Serialized objects include the following required properties:
+
+[options="header"]
+|=====================
+|Property Name |Property Type |Description
+|keyId |String |Encryption key identifier
+|ivBytes |byte[] |Cipher initialization vector
+|algorithm |String |Cipher algorithm
+|version |String |Encryption protocol version
+|cipherByteLength |int |Length of enciphered record
+|=====================
+
+Metadata serialization uses the standard `java.io.ObjectOutputStream.writeObject()` method to write objects to a stream
+that can be converted to a byte array. The deserialization process uses a custom extension of the
+`java.io.ObjectInputStream` to read objects regardless of the original class name associated with the record. This
+approach requires the presence of the standard metadata properties, but provides a compatibility layer that avoids
+linking the implementation to a specific Java class.
+
+The initial implementation of encrypted repositories used different byte array markers when writing metadata. Each
+repository implementation uses the following byte array markers before writing a serialized metadata record:
+
+[options="header"]
+|=======================
+|Repository Type |Byte Array
+|Content Repository |byte[]{0x00, 0x00}
+|FlowFile Repository |byte[]{}
+|Provenance Repository |byte[]{0x01}
+|=======================
+
+=== Repository Encryption Configuration
+
+Configuring repository encryption requires specifying the encryption protocol version and the associated Key Provider
+properties. Repository encryption can be configured on new or existing installations using standard properties. Records
+in existing repositories should be readable using standard capabilities, and the encrypted repository will write new
+records using the specified configuration.
+
+==== Protocol Version Configuration
+
+Setting the following protocol version property enables encryption for all repositories:
+
+....
+nifi.repository.encryption.protocol.version=1
+....
+
+==== Key Provider Configuration
+
+All encrypted repositories require a Key Provider to perform encryption and decryption operations. NiFi supports
+configuring the Key Provider implementation as well as the Key Identifier that will be used for new encryption
+operations. Key Provider implementations can hold multiple keys to support using a new key while maintaining access to
+information encrypted using the previous key.
+
+Repository encryption supports access to secret keys using standard `java.security.KeyStore` files.
+See <<secret-key-generation-and-storage-using-keytool>> for details on supported KeyStore types, as well as examples of
+generating secret keys.
+
+The following configuration properties provide an example using a PKCS12 KeyStore file named `repository.p12` containing
+a secret key labeled with an alias of `primary-key`:
+
+....
+nifi.repository.encryption.key.id=primary-key
+nifi.repository.encryption.key.provider=KEYSTORE
+nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
+nifi.repository.encryption.key.provider.keystore.password=2fRKmwDyMYmT7P5L
+....
+
+[[secret-key-generation-and-storage-using-keytool]]
+==== Secret Key Generation and Storage using Keytool
+
+The `KeyStoreKeyProvider` supports reading from a `java.security.KeyStore` using a configured password to load AES Secret Key entries.
+The `KeyStoreKeyProvider` can be configured with any of the encrypted repository implementations.
+
+The provider supports the following KeyStore Types:
+
+* BCFKS
+* PKCS12
+
+The keystore filename extension must be either `.p12` indicating PKCS12 or `.bcfks` indicating BCFKS.
+
+The `keytool` command can be used to generate an AES-256 Secret Key stored in a PKCS12 file for repository encryption:
+
+ keytool -genseckey -alias primary-key -keyalg AES -keysize 256 -keystore repository.p12 -storetype PKCS12
+
+The `keytool` command requires additional arguments specifying the BouncyCastle Security Provider to store
+Secret Keys using BCFKS. The arguments must include a reference to the BouncyCastle Security Provider library, which
+is available in the `lib/bootstrap` directory under the NiFi installation.
+
+The following command can be used to generate an AES-256 Secret Key stored using BCFKS:
+
+ keytool -genseckey -alias primary-key -keyalg AES -keysize 256 -keystore repository.bcfks -storetype BCFKS -providerclass org.bouncycastle.jce.provider.BouncyCastleProvider -providerpath lib/bootstrap/bcprov-jdk15on-*.jar
+
+Enter a keystore password when prompted. The same value must be used for both the keystore password and key password.
+The keystore password will be used in the provider configuration properties.
+
[[encryption]]
== Encryption Configuration
-This section provides an overview of the capabilities of NiFi to encrypt and decrypt data.
-
The EncryptContent processor allows for the encryption and decryption of data, both internal to NiFi and integrated with external systems, such as `openssl` and other data sources and consumers.
[[key-derivation-functions]]
diff --git a/nifi-docs/src/main/asciidoc/user-guide.adoc b/nifi-docs/src/main/asciidoc/user-guide.adoc
index 959522ed8b..f62e86c53a 100644
--- a/nifi-docs/src/main/asciidoc/user-guide.adoc
+++ b/nifi-docs/src/main/asciidoc/user-guide.adoc
@@ -2970,150 +2970,6 @@ Once these property changes have been made, restart NiFi.
**Note:** Detailed descriptions for each of these properties can be found in <<administration-guide.adoc#system_properties,System Properties>>.
-== Repository Encryption
-
-NiFi supports encryption of local repositories using a configurable Key Provider to enable protection of information
-on the filesystem. Repository encryption configuration uses a version number to indicate the cipher algorithms, metadata
-format, and repository implementation classes. This approach provides a generalized method for configuration without the
-need to customize each repository implementation class.
-
-Repository encryption incurs a performance cost due to the overhead of cipher operations. Filesystem encryption at the
-operating system level provides an alternative solution, with different performance characteristics. For deployments
-where filesystem encryption is not configured, repository encryption provides an enhanced level of data protection.
-Due to increased performance requirements, more computing resources may be necessary to achieve sufficient throughput
-when enabling repository encryption.
-
-The security of repository encryption depends on a combination of the cipher algorithms and the protection of encryption
-keys. Key protection and key rotation are important parts of securing an encrypted repository configuration.
-Key protection involves limiting access to the Key Provider and key rotation requires manual updates to generate and
-specify a new encryption key.
-
-=== Repository Encryption Protocol Version 1
-
-The first version of support for repository encryption includes the following cipher algorithms:
-
-[options="header"]
-|=======================
-|Repository Type |Cipher Algorithm
-|Content Repository |AES/CTR/NoPadding
-|FlowFile Repository |AES/GCM/NoPadding
-|FlowFile Swap Manager |AES/GCM/NoPadding
-|Provenance Repository |AES/GCM/NoPadding
-|=======================
-
-The following classes provide the direct repository encryption implementation, extending standard classes:
-
-[options="header"]
-|=======================
-|Repository Type |Class
-|Content Repository |org.apache.nifi.content.EncryptedFileSystemRepository
-|FlowFile Repository |org.apache.nifi.wali.EncryptedSequentialAccessWriteAheadLog
-|FlowFile Swap Manager |org.apache.nifi.controller.EncryptedFileSystemSwapManager
-|Provenance Repository |org.apache.nifi.provenance.EncryptedWriteAheadProvenanceRepository
-|=======================
-
-==== Encryption Metadata Serialization
-
-Each repository implementation class leverages standard cipher operations to perform encryption and decryption. In order
-to support AES, the encryption process writes metadata associated with each encryption operation. Encryption protocol
-version 1 uses Java Object serialization to write objects containing the encryption Key Identifier, the cipher
-Initialization Vector, and other required properties. Serialized objects include the following required properties:
-
-[options="header"]
-|=====================
-|Property Name |Property Type |Description
-|keyId |String |Encryption key identifier
-|ivBytes |byte[] |Cipher initialization vector
-|algorithm |String |Cipher algorithm
-|version |String |Encryption protocol version
-|cipherByteLength |int |Length of enciphered record
-|=====================
-
-Metadata serialization uses the standard `java.io.ObjectOutputStream.writeObject()` method to write objects to a stream
-that can be converted to a byte array. The deserialization process uses a custom extension of the
-`java.io.ObjectInputStream` to read objects regardless of the original class name associated with the record. This
-approach requires the presence of the standard metadata properties, but provides a compatibility layer that avoids
-linking the implementation to a specific Java class.
-
-The initial implementation of encrypted repositories used different byte array markers when writing metadata. Each
-repository implementation uses the following byte array markers before writing a serialized metadata record:
-
-[options="header"]
-|=======================
-|Repository Type |Byte Array
-|Content Repository |byte[]{0x00, 0x00}
-|FlowFile Repository |byte[]{}
-|Provenance Repository |byte[]{0x01}
-|=======================
-
-=== Repository Encryption Configuration
-
-Configuring repository encryption requires specifying the encryption protocol version and the associated Key Provider
-properties. Repository encryption can be configured on new or existing installations using standard properties. Records
-in existing repositories should be readable using standard capabilities, and the encrypted repository will write new
-records using the specified configuration.
-
-Disabling repository encryption on existing installations requires removing existing repository contents, and
-restarting the system after making configuration changes. For this reason, flow administrators should confirm that the
-system has processed all available FlowFiles to avoid losing information when disabling repository encryption.
-
-==== Protocol Version Configuration
-
-Setting the following protocol version property enables encryption for all repositories:
-
-....
-nifi.repository.encryption.protocol.version=1
-....
-
-==== Key Provider Configuration
-
-All encrypted repositories require a Key Provider to perform encryption and decryption operations. NiFi supports
-configuring the Key Provider implementation as well as the Key Identifier that will be used for new encryption
-operations. Key Provider implementations can hold multiple keys to support using a new key while maintaining access to
-information encrypted using the previous key.
-
-Repository encryption supports access to secret keys using standard `java.security.KeyStore` files.
-See <<secret-key-generation-and-storage-using-keytool>> for details on supported KeyStore types, as well as examples of
-generating secret keys.
-
-The following configuration properties provide an example using a PKCS12 KeyStore file named `repository.p12` containing
-a secret key labeled with an alias of `primary-key`:
-
-....
-nifi.repository.encryption.key.id=primary-key
-nifi.repository.encryption.key.provider=KEYSTORE
-nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
-nifi.repository.encryption.key.provider.keystore.password=2fRKmwDyMYmT7P5L
-....
-
-[[secret-key-generation-and-storage-using-keytool]]
-==== Secret Key Generation and Storage using Keytool
-
-The `KeyStoreKeyProvider` supports reading from a `java.security.KeyStore` using a configured password to load AES Secret Key entries.
-The `KeyStoreKeyProvider` can be configured with any of the encrypted repository implementations.
-
-The provider supports the following KeyStore Types:
-
-* BCFKS
-* PKCS12
-
-The keystore filename extension must be either `.p12` indicating PKCS12 or `.bcfks` indicating BCFKS.
-
-The `keytool` command can be used to generate an AES-256 Secret Key stored in a PKCS12 file for repository encryption:
-
- keytool -genseckey -alias primary-key -keyalg AES -keysize 256 -keystore repository.p12 -storetype PKCS12
-
-The `keytool` command requires additional arguments specifying the BouncyCastle Security Provider to store
-Secret Keys using BCFKS. The arguments must include a reference to the BouncyCastle Security Provider library, which
-is available in the `lib/bootstrap` directory under the NiFi installation.
-
-The following command can be used to generate an AES-256 Secret Key stored using BCFKS:
-
- keytool -genseckey -alias primary-key -keyalg AES -keysize 256 -keystore repository.bcfks -storetype BCFKS -providerclass org.bouncycastle.jce.provider.BouncyCastleProvider -providerpath lib/bootstrap/bcprov-jdk15on-*.jar
-
-Enter a keystore password when prompted. The same value must be used for both the keystore password and key password.
-The keystore password will be used in the provider configuration properties.
-
[[experimental_warning]]
== Experimental Warning