You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by kd...@apache.org on 2019/04/22 21:13:21 UTC
[nifi-registry] branch master updated: NIFIREG-244 Updated Admin
Guide documentation
This is an automated email from the ASF dual-hosted git repository.
kdoran pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/nifi-registry.git
The following commit(s) were added to refs/heads/master by this push:
new ceca1e9 NIFIREG-244 Updated Admin Guide documentation
ceca1e9 is described below
commit ceca1e92323a2d60403a0871ba39360efe7e086d
Author: Bryan Bende <bb...@apache.org>
AuthorDate: Fri Apr 12 10:40:31 2019 -0400
NIFIREG-244 Updated Admin Guide documentation
- Added section describing Access Policies to Admin Guide
- Added sections to Admin Guide on extension bundle persistence providers
- Added Identity Mapping Properties section to Admin Guide
- Added Admin Guide section on URL aliasing
This closes #171.
Signed-off-by: Kevin Doran <kd...@apache.org>
---
.../src/main/asciidoc/administration-guide.adoc | 310 +++++++++++++++++++--
1 file changed, 286 insertions(+), 24 deletions(-)
diff --git a/nifi-registry-core/nifi-registry-docs/src/main/asciidoc/administration-guide.adoc b/nifi-registry-core/nifi-registry-docs/src/main/asciidoc/administration-guide.adoc
index f56da94..3c2cde3 100644
--- a/nifi-registry-core/nifi-registry-docs/src/main/asciidoc/administration-guide.adoc
+++ b/nifi-registry-core/nifi-registry-docs/src/main/asciidoc/administration-guide.adoc
@@ -616,6 +616,90 @@ In this example, the users and groups are loaded from LDAP but the servers are m
from an attribute in a LDAP entry based on the `User Identity Attribute`. The `NiFi Identity` values are established in the local file using the
`Initial User Identity` properties.
+=== Access Policies
+
+You can manage the ability for users and groups to view or modify NiFi Registry resources using 'access policies'. Access policies can be created to control access to buckets, as well as to grant special privileges to users for managing a NiFi Registry instance.
+
+==== Bucket Policies
+
+Bucket policies govern the following bucket level authorizations:
+
+|===
+|Policy |Privilege |Resource Descriptor
+
+| Read Bucket
+| Allows users to read items in the bucket
+| `resource="/buckets/<bucket-UUID>" action="R"`
+
+| Write Bucket
+| Allows users to write items to the bucket
+| `resource="/buckets/<bucket-UUID>" action="W"`
+
+| Delete Bucket
+| Allows users to delete the bucket
+| `resource="/buckets/<bucket-UUID>" action="D"`
+
+
+|===
+
+
+==== Special Privilege Policies
+
+Special privilege policies govern the following system level authorizations:
+
+|===
+|Policy |Privilege |Resource Descriptor
+
+| Can Manage Buckets (Read)
+| Allows users to read from all buckets
+| `resource="/buckets" action="R"`
+
+| Can Manage Buckets (Write)
+| Allows users to write to all buckets
+| `resource="/buckets" action="W"`
+
+| Can Manage Buckets (Delete)
+| Allows users to delete all buckets
+| `resource="/buckets" action="D"`
+
+| Can Manage Users (Read)
+| Allows users to view users
+| `resource="/tenants" action="R"`
+
+| Can Manage Users (Write)
+| Allows users to create and modify users
+| `resource="/tenants" action="W"`
+
+| Can Manage Users (Delete)
+| Allows users to delete users
+| `resource="/tenants" action="D"`
+
+| Can Manage Policies (Read)
+| Allows users to view policies
+| `resource="/policies" action="R"`
+
+| Can Manage Policies (Write)
+| Allows users to create and modify policies
+| `resource="/policies" action="W"`
+
+| Can Manage Policies (Delete)
+| Allows users to delete policies
+| `resource="/policies" action="D"`
+
+| Can Proxy Requests
+| Allows users to proxy requests
+| `resource="/proxy" action="W"`
+
+| View Swagger
+| Allows users to access the self-hosted Swagger UI
+| `resource="/swagger" action="R"`
+
+| View Actuator
+| Allows users to access the Spring Boot Actuator end-points
+| `resource="/actuator" action="R"`
+
+|===
+
== Encrypted Passwords in Configuration Files
@@ -906,6 +990,38 @@ These properties pertain to various security features in NiFi Registry. Many of
|`nifi.registry.security.identity.provider`|This indicates what type of identity provider to use. The default value is blank, can be set to the identifier from a provider in the file specified in `nifi.registry.security.identity.providers.configuration.file`. Setting this property will trigger NiFi Registry to support username/password authentication.
|====
+=== Identity Mapping Properties
+
+These properties can be utilized to normalize user identities. When implemented, identities authenticated by different identity providers (certificates, LDAP, Kerberos) are treated the same internally in NiFi Registry. As a result, duplicate users are avoided and user-specific configurations such as authorizations only need to be setup once per user.
+
+The following examples demonstrate normalizing DNs from certificates and principals from Kerberos:
+
+----
+nifi.registry.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$
+nifi.registry.security.identity.mapping.value.dn=$1@$2
+nifi.registry.security.identity.mapping.transform.dn=NONE
+nifi.registry.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$
+nifi.registry.security.identity.mapping.value.kerb=$1@$2
+nifi.registry.security.identity.mapping.transform.kerb=NONE
+----
+
+The last segment of each property is an identifier used to associate the pattern with the replacement value. When a user makes a request to NiFi Registry, their identity is checked to see if it matches each of those patterns in lexicographical order. For the first one that matches, the replacement specified in the `nifi.registry.security.identity.mapping.value.xxxx` property is used. So a login with `CN=localhost, OU=Apache NiFi, O=Apache, L=Santa Monica, ST=CA, C=US` matches the DN ma [...]
+
+In addition to mapping, a transform may be applied. The supported versions are `NONE` (no transform applied), `LOWER` (identity lowercased), and `UPPER` (identity uppercased). If not specified, the default value is `NONE`.
+
+NOTE: These mappings are also applied to the "Initial Admin Identity" in the _authorizers.xml_ file, as well as users imported from LDAP (See <<authorizers-setup>>).
+
+Group names can also be mapped. The following example will accept the existing group name but will lowercase it. This may be helpful when used in conjunction with an external authorizer.
+
+----
+nifi.registry.security.group.mapping.pattern.anygroup=^(.*)$
+nifi.registry.security.group.mapping.value.anygroup=$1
+nifi.registry.security.group.mapping.transform.anygroup=LOWER
+----
+
+NOTE: These mappings are applied to groups imported from LDAP.
+
+
=== Providers Properties
These properties pertain to flow persistence providers. NiFi Registry uses a pluggable flow persistence provider to store the
@@ -916,6 +1032,16 @@ content of the flows saved to the registry. For further details on persistence p
|`nifi.registry.providers.configuration.file`|This is the location of the file where flow persistence providers are configured. The default value is `./conf/providers.xml`.
|====
+=== Alias Properties
+
+These properties pertain to the support for URL aliasing. For further details, refer to <<URL Aliasing>>.
+
+|====
+|*Property*|*Description*
+|`nifi.registry.registry.alias.configuration.file`|This is the location of the file where URL aliases are configured. The default value is `./conf/registry-aliases.xml`.
+|====
+
+
=== Database Properties
These properties define the settings for the Registry database, which keeps track of metadata about buckets and all items stored in buckets.
@@ -978,30 +1104,28 @@ providing 2 total locations, including `nifi.registry.extension.dir.1`.
== Persistence Providers
-NiFi Registry uses a pluggable flow persistence provider to store the content of the flows saved to the registry. NiFi Registry provides `<<FileSystemFlowPersistenceProvider>>` and `<<GitFlowPersistenceProvider>>`.
+NiFi Registry uses a pluggable persistence provider to store the content of each versioned item. Each type of versioned item, such as a versioned flow or extension bundle, has its own persistence provider.
+
+Each persistence provider has its own configuration parameters, which can be configured in an XML file specified in _<<Providers Properties,nifi-registry.properties>>_.
+
+=== Flow Persistence Providers
-Each persistence provider has its own configuration parameters, those can be configured in a XML file specified in _<<Providers Properties,nifi-registry.properties>>_.
+The flow persistence provider stores the content of the flows saved to the registry. NiFi Registry provides `<<FileSystemFlowPersistenceProvider>>` and `<<GitFlowPersistenceProvider>>`.
The XML configuration file looks like below. It has a `flowPersistenceProvider` element in which qualified class name of a persistence provider implementation and its configuration properties are defined. See following sections for available configurations for each provider.
-.Example providers.xml
+.Example flow persistence provider in providers.xml
[source,xml]
....
-<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
-<providers>
-
- <flowPersistenceProvider>
- <class>persistence-provider-qualified-class-name</class>
- <property name="property-1">property-value-1</property>
- <property name="property-2">property-value-2</property>
- <property name="property-n">property-value-n</property>
- </flowPersistenceProvider>
-
-</providers>
+<flowPersistenceProvider>
+ <class>persistence-provider-qualified-class-name</class>
+ <property name="property-1">property-value-1</property>
+ <property name="property-2">property-value-2</property>
+ <property name="property-n">property-value-n</property>
+</flowPersistenceProvider>
....
-
-=== FileSystemFlowPersistenceProvider
+==== FileSystemFlowPersistenceProvider
FileSystemFlowPersistenceProvider simply stores serialized Flow contents into `{bucket-id}/{flow-id}/{version}` directories.
@@ -1025,7 +1149,7 @@ Qualified class name: `org.apache.nifi.registry.provider.flow.FileSystemFlowPers
|====
-=== GitFlowPersistenceProvider
+==== GitFlowPersistenceProvider
`GitFlowPersistenceProvider` stores flow contents under a Git directory.
@@ -1069,7 +1193,7 @@ link:https://git-scm.com/book/en/v2/Git-Internals-The-Refspec[https://git-scm.co
|`Remote Access Password`|Used with `Remote Access User`.
|====
-==== Initialize Git directory
+===== Initialize Git directory
In order to use `GitFlowPersistenceRepository`, you need to prepare a Git directory on the local file system. You can do so by initializing a directory with `git init` command, or clone an existing Git project from a remote Git repository by `git clone` command.
@@ -1078,7 +1202,7 @@ link:https://git-scm.com/docs/git-init[https://git-scm.com/docs/git-init^]
- `git clone` command
link:https://git-scm.com/docs/git-clone[https://git-scm.com/docs/git-clone^]
-==== Git user configuration
+===== Git user configuration
This persistence provider uses preconfigured Git user name and user email address when it creates Git commits. NiFi Registry user name is added to commit messages.
@@ -1100,7 +1224,7 @@ You can configure Git user name and email address by `git config` command.
link:https://git-scm.com/docs/git-config[https://git-scm.com/docs/git-config^]
-==== Git user authentication
+===== Git user authentication
By default, this persistence repository only create commits to local repository. No user authentication is needed to do so. However, if 'Commit To Push' is enabled, user authentication to the remote Git repository is required.
@@ -1123,9 +1247,9 @@ Host bitbucket.org
IdentityFile ~/.ssh/key-for-bitbucket
....
-=== Switching from other Persistence Provider
+==== Switching from other Flow Persistence Provider
-In order to switch the Persistence Provider to use, it is necessary to reset NiFi Registry.
+In order to switch the Flow Persistence Provider, it is necessary to reset NiFi Registry.
For example, to switch from `FileSystemFlowPersistenceProvider` to `GitFlowPersistenceProvider`, follow these steps:
. Stop version control on all ProcessGroups in NiFi
@@ -1136,7 +1260,7 @@ For example, to switch from `FileSystemFlowPersistenceProvider` to `GitFlowPersi
. Recreate any buckets
. Start version control on all ProcessGroups again
-=== Data model version of serialized Flow snapshots
+==== Data model version of serialized Flow snapshots
Serialized Flow snapshots saved by these persistence providers have versions, so that the data format and schema can evolve over time. Data model version update is done automatically by NiFi Registry when it reads and stores each Flow content.
@@ -1148,6 +1272,78 @@ Here is the data model version histories:
|1|0.1|Binary format having header bytes at the beginning followed by Flow content represented as XML.
|====
+=== Bundle Persistence Providers
+
+The bundle persistence provider stores the content of extension bundles saved to the registry. NiFi Registry provides `<<FileSystemBundlePersistenceProvider>>` and `<<S3BundlePersistenceProvider>>`.
+
+The XML configuration file looks like below. It has a `extensionBundlePersistenceProvider` element in which the qualified class name of a persistence provider implementation and its configuration properties are defined. See following sections for available configurations for each provider.
+
+.Example extension bundle persistence provider in providers.xml
+[source,xml]
+....
+<extensionBundlePersistenceProvider>
+ <class>persistence-provider-qualified-class-name</class>
+ <property name="property-1">property-value-1</property>
+ <property name="property-2">property-value-2</property>
+ <property name="property-n">property-value-n</property>
+</extensionBundlePersistenceProvider>
+....
+
+==== FileSystemBundlePersistenceProvider
+
+The FileSystemBundlePersistenceProvider stores the content of extension bundles on the local file-system. The bundles are organized in directories according to bucket id, group, artifact, and version.
+
+Example of persisted extension bundles:
+....
+Extension Bundle Storage Directory/
+├── {bucket-id}/
+ └── {group-id}/
+ └── {artifact-id}
+ └── {version}/{artifact-id}-{version}.{extension}
+├── d1beba88-32e9-45d1-bfe9-057cc41f7ce8/
+ └── org.apache.nifi
+ └── nifi-example-nar
+ └── 1.0.0/nifi-example-nar-1.0.0.nar
+ └── 2.0.0/nifi-example-nar-2.0.0.nar
+....
+
+===== Configuration
+
+Qualified class name: `org.apache.nifi.registry.provider.extension.FileSystemBundlePersistenceProvider`
+
+|====
+|*Property*|*Description*
+|`Extension Bundle Storage Directory`|REQUIRED: File system path for a directory where extension bundle contents files are persisted to. If the directory does not exist when NiFi Registry starts, it will be created. If the directory exists, it must be readable and writable from NiFi Registry.
+|====
+
+==== S3BundlePersistenceProvider
+
+The S3BundlePersistenceProvider stores the content of extension bundles in AWS S3 bucket. The bucket is expected to already exist and be accessible to the credentials provided to the persistence providcer.
+
+NOTE: This provider must be added to the classpath by specifying a custom extension directory in nifi-registry.properties, such as nifi.registry.extension.dir.aws=./ext/aws/lib, where ./ext/aws/ contains the contents of the extracted nifi-registry-aws-assembly-<version>-bin.zip.
+
+The key of an extension bundle in the S3 bucket will be the following:
+....
+/{registry-bucket-id}/{group-id}/{artifact-id}/{version}/{artifact-id}-{version}.{extension}
+....
+
+If an optional Key Prefix is specified, then that prefix will be applied to the beginning of the above key.
+
+===== Configuration
+
+Qualified class name: `org.apache.nifi.registry.aws.S3BundlePersistenceProvider`
+
+|====
+|*Property*|*Description*
+|`Region`|REQUIRED: The name of the S3 region where the bucket exists.
+|`Bucket Name`|REQUIRED: The name of an existing bucket to store extension bundles.
+|`Key Prefix`|An optional prefix that if specified will be added to the beginning of all S3 keys.
+|`Credentials Provider`|REQUIRED: Indicates how credentials will be provided, must be a value of DEFAULT_CHAIN or STATIC. DEFAULT_CHAIN will consider in order: Java system properties, environment variables, credential profiles (~/.aws/credentials). STATIC requires that "Access Key" and "Secret Access Key" be specified directly in this file.
+|`Access Key`| The access key to use when using STATIC credentials provider.
+|`Secret Access Key`| The secret access key to use when using STATIC credentials provider.
+|`Endpoint URL`| An optional URL that overrides the default AWS S3 endpoint URL. Set this when using an AWS S3 API compatible service hosted at a different URL.
+|====
+
== Event Hooks
Event hooks are an integration point that allows for custom code to to be triggered when NiFi Registry application events occur.
@@ -1209,4 +1405,70 @@ via NiFi Registry’s logback.xml, which by default contains an appender that wr
org.apache.nifi.registry.provider.hook.LoggingEventHookProvider
</class>
</eventHookProvider>
-....
\ No newline at end of file
+....
+
+== URL Aliasing
+A versioned item may contain the URL of a registry instance embedded in the content of the item. For example, flows with nested versioning contain the URL of the registry where the nested versioned flow is located. If the location of the registry instances changes, then the content is no longer accurate.
+
+URL aliasing can be used to dynamically handle this situation so that URLs are never written to the stored content, and can be re-written with the correct value when being retrieved by a client.
+
+The aliases are configured in an XML file which can be specified in _<<Alias Properties,nifi-registry.properties>>_.
+
+.Example aliases in registry-aliases.xml
+[source,xml]
+....
+<aliases>
+ <alias>
+ <internal>NIFI_REGISTRY_1</internal>
+ <external>http://registry1.nifi.apache.org:18080</external>
+ </alias>
+ <alias>
+ <internal>NIFI_REGISTRY_2</internal>
+ <external>http://registry2.nifi.apache.org:18080</external>
+ </alias>
+</aliases>
+....
+
+If a flow is saved to registry with two child process groups, each under version control, the incoming flow would contain something like the following:
+....
+"processGroups" : [ {
+ ...
+ "versionedFlowCoordinates" : {
+ "bucketId" : "ca20e058-f6e7-404c-aee0-e30833e792c7",
+ "flowId" : "178a6657-e1a7-4cce-8f83-4e615e38f57a",
+ "registryUrl" : "http://registry1.nifi.apache.org:18080",
+ "version" : 1
+ },
+ {
+ ...
+ "versionedFlowCoordinates" : {
+ "bucketId" : "ca20e058-f6e7-404c-aee0-e30833e792c7",
+ "flowId" : "985cb44b-3aec-32be-860f-d2a0f2c72aac",
+ "registryUrl" : "http://registry2.nifi.apache.org:18080",
+ "version" : 1
+ }
+]
+....
+
+With the example aliases configuration above, the URLs would be written to the FlowPersistenceProvider as the following:
+....
+"processGroups" : [ {
+ ...
+ "versionedFlowCoordinates" : {
+ "bucketId" : "ca20e058-f6e7-404c-aee0-e30833e792c7",
+ "flowId" : "178a6657-e1a7-4cce-8f83-4e615e38f57a",
+ "registryUrl" : "NIFI_REGISTRY_1",
+ "version" : 1
+ },
+ {
+ ...
+ "versionedFlowCoordinates" : {
+ "bucketId" : "ca20e058-f6e7-404c-aee0-e30833e792c7",
+ "flowId" : "985cb44b-3aec-32be-860f-d2a0f2c72aac",
+ "registryUrl" : "NIFI_REGISTRY_2",
+ "version" : 1
+ }
+]
+....
+
+When this flow is retrieved from any API call, the internal values would be rewritten to the external values.