You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by ex...@apache.org on 2024/07/05 16:24:53 UTC
(nifi) 04/04: NIFI-13511 Removed ShellUserGroupProvider from NiFi and Registry
This is an automated email from the ASF dual-hosted git repository.
exceptionfactory pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
commit 1d2b755edf428d465d50c4025824f636bc9c7c50
Author: Joseph Witt <jo...@apache.org>
AuthorDate: Thu Jul 4 19:37:16 2024 -0700
NIFI-13511 Removed ShellUserGroupProvider from NiFi and Registry
This closes #9046
Signed-off-by: David Handermann <ex...@apache.org>
---
nifi-code-coverage/pom.xml | 5 -
.../src/main/asciidoc/administration-guide.adoc | 24 +-
.../nifi-framework-nar-bom/pom.xml | 10 -
nifi-framework-bundle/nifi-framework-nar/pom.xml | 5 -
.../src/main/resources/conf/authorizers.xml | 25 -
.../nifi-framework/nifi-shell-authorizer/pom.xml | 38 --
.../nifi/authorization/NssShellCommands.java | 56 --
.../nifi/authorization/OsxShellCommands.java | 56 --
.../nifi/authorization/RemoteShellCommands.java | 62 --
.../nifi/authorization/ShellCommandsProvider.java | 67 --
.../nifi/authorization/ShellUserGroupProvider.java | 672 ---------------------
.../nifi/authorization/util/ShellRunner.java | 123 ----
...org.apache.nifi.authorization.UserGroupProvider | 15 -
nifi-framework-bundle/nifi-framework/pom.xml | 1 -
.../authorization/shell/NssShellCommands.java | 55 --
.../authorization/shell/OsxShellCommands.java | 55 --
.../authorization/shell/ShellCommandsProvider.java | 67 --
.../security/authorization/shell/ShellRunner.java | 123 ----
.../shell/ShellUserGroupProvider.java | 645 --------------------
...gistry.security.authorization.UserGroupProvider | 3 +-
.../src/main/resources/conf/authorizers.xml | 21 -
21 files changed, 2 insertions(+), 2126 deletions(-)
diff --git a/nifi-code-coverage/pom.xml b/nifi-code-coverage/pom.xml
index ceaf8df8a3..00bda5522f 100644
--- a/nifi-code-coverage/pom.xml
+++ b/nifi-code-coverage/pom.xml
@@ -596,11 +596,6 @@
<artifactId>nifi-runtime</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
- <dependency>
- <groupId>org.apache.nifi</groupId>
- <artifactId>nifi-shell-authorizer</artifactId>
- <version>2.0.0-SNAPSHOT</version>
- </dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-site-to-site</artifactId>
diff --git a/nifi-docs/src/main/asciidoc/administration-guide.adoc b/nifi-docs/src/main/asciidoc/administration-guide.adoc
index f8d1a8a469..0b2a0e5ebf 100644
--- a/nifi-docs/src/main/asciidoc/administration-guide.adoc
+++ b/nifi-docs/src/main/asciidoc/administration-guide.adoc
@@ -775,28 +775,6 @@ The LdapUserGroupProvider has the following properties:
NOTE: Any identity mapping rules specified in _nifi.properties_ will also be applied to the user identities. Group names are not mapped.
-==== ShellUserGroupProvider
-
-The ShellUserGroupProvider fetches user and group details from Unix-like systems using shell commands.
-
-This provider executes various shell pipelines with commands such as `getent` on Linux and `dscl` on macOS.
-
-Supported systems may be configured to retrieve users and groups from an external source, such as LDAP or NIS. In these cases the shell commands
-will return those external users and groups. This provides administrators another mechanism to integrate user and group directory services.
-
-The ShellUserGroupProvider has the following properties:
-
-[options="header,footer"]
-|==================================================================================================================================================
-| Property Name | Description
-|`Initial Refresh Delay` | Duration of initial delay before first user and group refresh. (i.e. `10 secs`). Default is `5 mins`.
-|`Refresh Delay` | Duration of delay between each user and group refresh. (i.e. `10 secs`). Default is `5 mins`.
-|`Exclude Groups` | Regular expression used to exclude groups. Default is '', which means no groups are excluded.
-|`Exclude Users` | Regular expression used to exclude users. Default is '', which means no users are excluded.
-|==================================================================================================================================================
-
-Like LdapUserGroupProvider, the ShellUserGroupProvider is commented out in the _authorizers.xml_ file. Refer to that comment for usage examples.
-
==== AzureGraphUserGroupProvider
The AzureGraphUserGroupProvider fetches users and groups from Azure Active Directory (AAD) using the Microsoft Graph API.
@@ -831,7 +809,7 @@ The AzureGraphUserGroupProvider has the following properties:
|`Claim for Username` | The property of the user directory object mapped to the NiFi user name field. Default is 'upn'. 'email' is another option when `nifi.security.user.oidc.fallback.claims.identifying.user` is set to 'upn'.
|==================================================================================================================================================
-Like LdapUserGroupProvider and ShellUserGroupProvider, the AzureGraphUserGroupProvider configuration is commented out in the _authorizers.xml_ file. Refer to the comment for a starter configuration.
+Like LdapUserGroupProvider, the AzureGraphUserGroupProvider configuration is commented out in the _authorizers.xml_ file. Refer to the comment for a starter configuration.
==== Composite Implementations
diff --git a/nifi-framework-bundle/nifi-framework-nar-bom/pom.xml b/nifi-framework-bundle/nifi-framework-nar-bom/pom.xml
index 65e3202b6a..3aa2756dd6 100644
--- a/nifi-framework-bundle/nifi-framework-nar-bom/pom.xml
+++ b/nifi-framework-bundle/nifi-framework-nar-bom/pom.xml
@@ -63,12 +63,6 @@
<version>2.0.0-SNAPSHOT</version>
<scope>provided</scope>
</dependency>
- <dependency>
- <groupId>org.apache.nifi</groupId>
- <artifactId>nifi-shell-authorizer</artifactId>
- <version>2.0.0-SNAPSHOT</version>
- <scope>provided</scope>
- </dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-authorizer</artifactId>
@@ -248,10 +242,6 @@
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-file-authorizer</artifactId>
</dependency>
- <dependency>
- <groupId>org.apache.nifi</groupId>
- <artifactId>nifi-shell-authorizer</artifactId>
- </dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-authorizer</artifactId>
diff --git a/nifi-framework-bundle/nifi-framework-nar/pom.xml b/nifi-framework-bundle/nifi-framework-nar/pom.xml
index a9ac2a114e..19a6f2ba8e 100644
--- a/nifi-framework-bundle/nifi-framework-nar/pom.xml
+++ b/nifi-framework-bundle/nifi-framework-nar/pom.xml
@@ -55,11 +55,6 @@
<artifactId>nifi-file-authorizer</artifactId>
<scope>compile</scope>
</dependency>
- <dependency>
- <groupId>org.apache.nifi</groupId>
- <artifactId>nifi-shell-authorizer</artifactId>
- <scope>compile</scope>
- </dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-authorizer</artifactId>
diff --git a/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/authorizers.xml b/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/authorizers.xml
index fdaf70dc9e..190a79f589 100644
--- a/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/authorizers.xml
+++ b/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/authorizers.xml
@@ -165,31 +165,6 @@
</userGroupProvider>
To enable the ldap-user-group-provider remove 2 lines. This is 2 of 2. -->
- <!--
- The ShellUserGroupProvider provides support for retrieving users and groups by way of shell commands
- on systems that support `sh`. Implementations available for Linux and Mac OS, and are selected by the
- provider based on the system property `os.name`.
-
- 'Refresh Delay' - duration to wait between subsequent refreshes. Default is '5 mins'.
- 'Exclude Groups' - regular expression used to exclude groups. Default is '', which means no groups are excluded.
- 'Exclude Users' - regular expression used to exclude users. Default is '', which means no users are excluded.
- 'Legacy Identifier Mode' - preserves the legacy behavior for id generation. Disabling this will ensure that
- user and group ids are differentiated to handle the case where a user and group have
- the same identity. Default is 'true', which means users and groups are not differentiated.
- 'Command Timeout' - amount of time to wait while executing a command before timing out
- -->
- <!-- To enable the shell-user-group-provider remove 2 lines. This is 1 of 2.
- <userGroupProvider>
- <identifier>shell-user-group-provider</identifier>
- <class>org.apache.nifi.authorization.ShellUserGroupProvider</class>
- <property name="Refresh Delay">5 mins</property>
- <property name="Exclude Groups"></property>
- <property name="Exclude Users"></property>
- <property name="Legacy Identifier Mode">true</property>
- <property name="Command Timeout">60 seconds</property>
- </userGroupProvider>
- To enable the shell-user-group-provider remove 2 lines. This is 2 of 2. -->
-
<!--
The AzureGraphUserGroupProvider fetches users and groups from Azure Active Directory (AAD) using the Microsoft Graph API.
diff --git a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/pom.xml b/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/pom.xml
deleted file mode 100644
index 5e56747194..0000000000
--- a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/pom.xml
+++ /dev/null
@@ -1,38 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
- <parent>
- <artifactId>nifi-framework</artifactId>
- <groupId>org.apache.nifi</groupId>
- <version>2.0.0-SNAPSHOT</version>
- </parent>
- <modelVersion>4.0.0</modelVersion>
- <artifactId>nifi-shell-authorizer</artifactId>
- <dependencies>
-
-
- <dependency>
- <groupId>org.apache.nifi</groupId>
- <artifactId>nifi-framework-api</artifactId>
- </dependency>
- <dependency>
- <groupId>org.apache.nifi</groupId>
- <artifactId>nifi-utils</artifactId>
- <version>2.0.0-SNAPSHOT</version>
- </dependency>
-
- </dependencies>
-</project>
diff --git a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/NssShellCommands.java b/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/NssShellCommands.java
deleted file mode 100644
index c5c3f61ef9..0000000000
--- a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/NssShellCommands.java
+++ /dev/null
@@ -1,56 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.authorization;
-
-
-/**
- * Provides shell commands to read users and groups on NSS-enabled systems.
- *
- * See `man 5 nsswitch.conf` for more info.
- */
-class NssShellCommands implements ShellCommandsProvider {
- /**
- * @return Shell command string that will return a list of users.
- */
- public String getUsersList() {
- return "getent passwd | cut -f 1,3,4 -d ':'";
- }
-
- /**
- * @return Shell command string that will return a list of groups.
- */
- public String getGroupsList() {
- return "getent group | cut -f 1,3 -d ':'";
- }
-
- /**
- * @param groupName name of group.
- * @return Shell command string that will return a list of users for a group.
- */
- public String getGroupMembers(String groupName) {
- return String.format("getent group %s | cut -f 4 -d ':'", groupName);
- }
-
- /**
- * This gives exit code 0 on all tested distributions.
- *
- * @return Shell command string that will exit normally (0) on a suitable system.
- */
- public String getSystemCheck() {
- return "getent --version";
- }
-}
diff --git a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/OsxShellCommands.java b/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/OsxShellCommands.java
deleted file mode 100644
index 90f405f428..0000000000
--- a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/OsxShellCommands.java
+++ /dev/null
@@ -1,56 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.authorization;
-
-
-/**
- * Provides shell commands to read users and groups on Mac OSX systems.
- *
- * See `man dscl` for more info.
- */
-class OsxShellCommands implements ShellCommandsProvider {
- /**
- * @return Shell command string that will return a list of users.
- */
- public String getUsersList() {
- return "dscl . -readall /Users UniqueID PrimaryGroupID | awk 'BEGIN { OFS = \":\"; ORS=\"\\n\"; i=0;} /RecordName: / {name = $2;i = 0;}" +
- "/PrimaryGroupID: / {gid = $2;} /^ / {if (i == 0) { i++; name = $1;}} /UniqueID: / {uid = $2;print name, uid, gid;}' | grep -v ^_";
- }
-
- /**
- * @return Shell command string that will return a list of groups.
- */
- public String getGroupsList() {
- return "dscl . -list /Groups PrimaryGroupID | grep -v '^_' | sed 's/ \\{1,\\}/:/g'";
- }
-
- /**
- *
- * @param groupName name of group.
- * @return Shell command string that will return a list of users for a group.
- */
- public String getGroupMembers(String groupName) {
- return String.format("dscl . -read /Groups/%s GroupMembership | cut -f 2- -d ' ' | sed 's/\\ /,/g'", groupName);
- }
-
- /**
- * @return Shell command string that will exit normally (0) on a suitable system.
- */
- public String getSystemCheck() {
- return "which dscl";
- }
-}
diff --git a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/RemoteShellCommands.java b/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/RemoteShellCommands.java
deleted file mode 100644
index 9a6d7fd11a..0000000000
--- a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/RemoteShellCommands.java
+++ /dev/null
@@ -1,62 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.authorization;
-
-
-class RemoteShellCommands implements ShellCommandsProvider {
- // Carefully crafted command replacement string:
- private final static String remoteCommand = "ssh " +
- "-o 'StrictHostKeyChecking no' " +
- "-o 'PasswordAuthentication no' " +
- "-o \"RemoteCommand %s\" " +
- "-i %s -p %s -l root %s";
-
- private ShellCommandsProvider innerProvider;
- private String privateKeyPath;
- private String remoteHost;
- private Integer remotePort;
-
- private RemoteShellCommands() {
- }
-
- public static ShellCommandsProvider wrapOtherProvider(ShellCommandsProvider otherProvider, String keyPath, String host, Integer port) {
- RemoteShellCommands remote = new RemoteShellCommands();
-
- remote.innerProvider = otherProvider;
- remote.privateKeyPath = keyPath;
- remote.remoteHost = host;
- remote.remotePort = port;
-
- return remote;
- }
-
- public String getUsersList() {
- return String.format(remoteCommand, innerProvider.getUsersList(), privateKeyPath, remotePort, remoteHost);
- }
-
- public String getGroupsList() {
- return String.format(remoteCommand, innerProvider.getGroupsList(), privateKeyPath, remotePort, remoteHost);
- }
-
- public String getGroupMembers(String groupName) {
- return String.format(remoteCommand, innerProvider.getGroupMembers(groupName), privateKeyPath, remotePort, remoteHost);
- }
-
- public String getSystemCheck() {
- return String.format(remoteCommand, innerProvider.getSystemCheck(), privateKeyPath, remotePort, remoteHost);
- }
-}
diff --git a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/ShellCommandsProvider.java b/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/ShellCommandsProvider.java
deleted file mode 100644
index 2879057e2c..0000000000
--- a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/ShellCommandsProvider.java
+++ /dev/null
@@ -1,67 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.authorization;
-
-/**
- * Common interface for shell command strings to read users and groups.
- *
- */
-interface ShellCommandsProvider {
- /**
- * Gets the command for listing users.
- *
- * When executed, this command should output one record per line in this format:
- *
- * `username:user-id:primary-group-id`
- *
- * @return Shell command string that will return a list of users.
- */
- String getUsersList();
-
- /**
- * Gets the command for listing groups.
- *
- * When executed, this command should output one record per line in this format:
- *
- * `group-name:group-id`
- *
- * @return Shell command string that will return a list of groups.
- */
- String getGroupsList();
-
- /**
- * Gets the command for listing the members of a group.
- *
- * When executed, this command should output one line in this format:
- *
- * `user-name-1,user-name-2,user-name-n`
- *
- * @param groupName name of group.
- * @return Shell command string that will return a list of users for a group.
- */
- String getGroupMembers(String groupName);
-
- /**
- * Gets the command for checking the suitability of the host system.
- *
- * The command is expected to exit with status 0 (zero) to indicate success, and any other status
- * to indicate failure.
- *
- * @return Shell command string that will exit normally (0) on a suitable system.
- */
- String getSystemCheck();
-}
diff --git a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/ShellUserGroupProvider.java b/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/ShellUserGroupProvider.java
deleted file mode 100644
index bd51f6676e..0000000000
--- a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/ShellUserGroupProvider.java
+++ /dev/null
@@ -1,672 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.authorization;
-
-import org.apache.nifi.authorization.exception.AuthorizationAccessException;
-import org.apache.nifi.authorization.exception.AuthorizerCreationException;
-import org.apache.nifi.authorization.exception.AuthorizerDestructionException;
-import org.apache.nifi.authorization.util.ShellRunner;
-import org.apache.nifi.components.PropertyValue;
-import org.apache.nifi.util.FormatUtils;
-import org.apache.nifi.util.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.IOException;
-import java.util.Comparator;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import java.util.TreeSet;
-import java.util.concurrent.Executors;
-import java.util.concurrent.ScheduledExecutorService;
-import java.util.concurrent.TimeUnit;
-import java.util.regex.Pattern;
-import java.util.regex.PatternSyntaxException;
-
-
-/*
- * ShellUserGroupProvider implements UserGroupProvider by way of shell commands.
- */
-public class ShellUserGroupProvider implements UserGroupProvider {
- private final static Logger logger = LoggerFactory.getLogger(ShellUserGroupProvider.class);
-
- private final static String OS_TYPE_ERROR = "Unsupported operating system.";
- private final static String SYS_CHECK_ERROR = "System check failed - cannot provide users and groups.";
- private final static Map<String, User> usersById = new HashMap<>(); // id == identifier
- private final static Map<String, User> usersByName = new HashMap<>(); // name == identity
- private final static Map<String, Group> groupsById = new HashMap<>();
- private final static Map<String, Group> groupsByName = new HashMap<>();
-
- public static final String REFRESH_DELAY_PROPERTY = "Refresh Delay";
- private static final long MINIMUM_SYNC_INTERVAL_MILLISECONDS = 10_000;
-
- public static final String EXCLUDE_USER_PROPERTY = "Exclude Users";
- public static final String EXCLUDE_GROUP_PROPERTY = "Exclude Groups";
- public static final String LEGACY_IDENTIFIER_MODE = "Legacy Identifier Mode";
- public static final String COMMAND_TIMEOUT_PROPERTY = "Command Timeout";
-
- private static final String DEFAULT_COMMAND_TIMEOUT = "60 seconds";
-
- private long fixedDelay;
- private Pattern excludeUsers;
- private Pattern excludeGroups;
- private boolean legacyIdentifierMode;
- private int timeoutSeconds;
-
- // Our scheduler has one thread for users, one for groups:
- private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(2);
-
- // Commands selected during initialization:
- private ShellCommandsProvider selectedShellCommands;
-
- private ShellRunner shellRunner;
-
- // Start of the UserGroupProvider implementation. Javadoc strings
- // copied from the interface definition for reference.
-
- /**
- * Retrieves all users. Must be non null
- *
- * @return a list of users
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public Set<User> getUsers() throws AuthorizationAccessException {
- synchronized (usersById) {
- logger.debug("getUsers has user set of size: {}", usersById.size());
- return new HashSet<>(usersById.values());
- }
- }
-
- /**
- * Retrieves the user with the given identifier.
- *
- * @param identifier the id of the user to retrieve
- * @return the user with the given id, or null if no matching user was found
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public User getUser(String identifier) throws AuthorizationAccessException {
- User user;
-
- synchronized (usersById) {
- user = usersById.get(identifier);
- }
-
- if (user == null) {
- logger.debug("getUser (by id) user not found: {}", identifier);
- } else {
- logger.debug("getUser (by id) found user: {} for id: {}", user, identifier);
- }
- return user;
- }
-
- /**
- * Retrieves the user with the given identity.
- *
- * @param identity the identity of the user to retrieve
- * @return the user with the given identity, or null if no matching user was found
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public User getUserByIdentity(String identity) throws AuthorizationAccessException {
- User user;
-
- synchronized (usersByName) {
- user = usersByName.get(identity);
- }
-
- if (user == null) {
- logger.debug("getUser (by name) user not found: {}", identity);
- } else {
- logger.debug("getUser (by name) found user: {} for name: {}", user.getIdentity(), identity);
- }
- return user;
- }
-
- /**
- * Retrieves all groups. Must be non null
- *
- * @return a list of groups
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public Set<Group> getGroups() throws AuthorizationAccessException {
- synchronized (groupsById) {
- logger.debug("getGroups has group set of size: {}", groupsById.size());
- return new HashSet<>(groupsById.values());
- }
- }
-
- /**
- * Retrieves a Group by Id.
- *
- * @param identifier the identifier of the Group to retrieve
- * @return the Group with the given identifier, or null if no matching group was found
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public Group getGroup(String identifier) throws AuthorizationAccessException {
- Group group;
-
- synchronized (groupsById) {
- group = groupsById.get(identifier);
- }
-
- if (group == null) {
- logger.debug("getGroup (by id) group not found: {}", identifier);
- } else {
- logger.debug("getGroup (by id) found group: {} for id: {}", group.getName(), identifier);
- }
- return group;
-
- }
-
- @Override
- public Group getGroupByName(String name) throws AuthorizationAccessException {
- Group group;
-
- synchronized (groupsByName) {
- group = groupsByName.get(name);
- }
-
- if (group == null) {
- logger.debug("getGroup (by name) group not found: {}", name);
- } else {
- logger.debug("getGroup (by name) found group: {} for name: {}", group.getName(), name);
- }
-
- return group;
- }
-
- /**
- * Gets a user and their groups.
- *
- * @return the UserAndGroups for the specified identity
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public UserAndGroups getUserAndGroups(String identity) throws AuthorizationAccessException {
- User user = getUserByIdentity(identity);
- logger.debug("Retrieved user {} for identity {}", user, identity);
-
- Set<Group> groups = new HashSet<>();
- if (user != null) {
- for (Group g : getGroups()) {
- if (g.getUsers().contains(user.getIdentifier())) {
- logger.debug("User {} belongs to group {}", user.getIdentity(), g.getName());
- groups.add(g);
- }
- }
- }
-
- if (groups.isEmpty()) {
- logger.debug("User {} belongs to no groups", user);
- }
-
- return new UserAndGroups() {
- @Override
- public User getUser() {
- return user;
- }
-
- @Override
- public Set<Group> getGroups() {
- return groups;
- }
- };
- }
-
- /**
- * Called immediately after instance creation for implementers to perform additional setup
- *
- * @param initializationContext in which to initialize
- */
- @Override
- public void initialize(UserGroupProviderInitializationContext initializationContext) throws AuthorizerCreationException {
- }
-
- /**
- * Called to configure the Authorizer.
- *
- * @param configurationContext at the time of configuration
- * @throws AuthorizerCreationException for any issues configuring the provider
- */
- @Override
- public void onConfigured(AuthorizerConfigurationContext configurationContext) throws AuthorizerCreationException {
- logger.info("Configuring ShellUserGroupProvider");
-
- fixedDelay = getDelayProperty(configurationContext, REFRESH_DELAY_PROPERTY, "5 mins");
- timeoutSeconds = getTimeoutProperty(configurationContext, COMMAND_TIMEOUT_PROPERTY, DEFAULT_COMMAND_TIMEOUT);
- shellRunner = new ShellRunner(timeoutSeconds);
- logger.debug("Configured ShellRunner with command timeout of '{}' seconds", timeoutSeconds);
-
- // Our next init step is to select the command set based on the operating system name:
- ShellCommandsProvider commands = getCommandsProvider();
-
- if (commands == null) {
- commands = getCommandsProviderFromName(null);
- setCommandsProvider(commands);
- }
-
- // Our next init step is to run the system check from that command set to determine if the other commands
- // will work on this host or not.
- try {
- shellRunner.runShell(commands.getSystemCheck(), "Supported System Check");
- } catch (final Exception e) {
- logger.error("initialize exception: system check command: {}", commands.getSystemCheck(), e);
- throw new AuthorizerCreationException(SYS_CHECK_ERROR, e);
- }
-
- // The next step is to add the user and group exclude regexes:
- try {
- excludeGroups = Pattern.compile(getProperty(configurationContext, EXCLUDE_GROUP_PROPERTY, ""));
- excludeUsers = Pattern.compile(getProperty(configurationContext, EXCLUDE_USER_PROPERTY, ""));
- } catch (final PatternSyntaxException e) {
- throw new AuthorizerCreationException(e);
- }
-
- // Get the value for Legacy Identifier Mo
- legacyIdentifierMode = Boolean.parseBoolean(getProperty(configurationContext, LEGACY_IDENTIFIER_MODE, "true"));
-
- // With our command set selected, and our system check passed, we can pull in the users and groups:
- refreshUsersAndGroups();
-
- // And finally, our last init step is to fire off the refresh thread:
- scheduler.scheduleWithFixedDelay(() -> {
- try {
- refreshUsersAndGroups();
- } catch (final Throwable t) {
- logger.error("", t);
- }
- }, fixedDelay, fixedDelay, TimeUnit.MILLISECONDS);
-
- logger.info("Completed configuration of ShellUserGroupProvider");
- }
-
- private static ShellCommandsProvider getCommandsProviderFromName(String osName) {
- if (osName == null) {
- osName = System.getProperty("os.name");
- }
-
- ShellCommandsProvider commands;
- if (osName.startsWith("Linux")) {
- logger.debug("Selected Linux command set.");
- commands = new NssShellCommands();
- } else if (osName.startsWith("Mac OS X")) {
- logger.debug("Selected OSX command set.");
- commands = new OsxShellCommands();
- } else {
- throw new AuthorizerCreationException(OS_TYPE_ERROR);
- }
- return commands;
- }
-
- private String getProperty(AuthorizerConfigurationContext authContext, String propertyName, String defaultValue) {
- final PropertyValue property = authContext.getProperty(propertyName);
- final String value;
-
- if (property != null && property.isSet()) {
- value = property.getValue();
- } else {
- value = defaultValue;
- }
- return value;
-
- }
-
- private long getDelayProperty(AuthorizerConfigurationContext authContext, String propertyName, String defaultValue) {
- final PropertyValue intervalProperty = authContext.getProperty(propertyName);
- final String propertyValue;
- final long syncInterval;
-
- if (intervalProperty.isSet()) {
- propertyValue = intervalProperty.getValue();
- } else {
- propertyValue = defaultValue;
- }
-
- try {
- syncInterval = Math.round(FormatUtils.getPreciseTimeDuration(propertyValue, TimeUnit.MILLISECONDS));
- } catch (final IllegalArgumentException ignored) {
- throw new AuthorizerCreationException(String.format("The %s '%s' is not a valid time interval.", propertyName, propertyValue));
- }
-
- if (syncInterval < MINIMUM_SYNC_INTERVAL_MILLISECONDS) {
- throw new AuthorizerCreationException(String.format("The %s '%s' is below the minimum value of '%d ms'", propertyName, propertyValue, MINIMUM_SYNC_INTERVAL_MILLISECONDS));
- }
- return syncInterval;
- }
-
- private int getTimeoutProperty(AuthorizerConfigurationContext authContext, String propertyName, String defaultValue) {
- final PropertyValue timeoutProperty = authContext.getProperty(propertyName);
-
- final String propertyValue;
- if (timeoutProperty.isSet()) {
- propertyValue = timeoutProperty.getValue();
- } else {
- propertyValue = defaultValue;
- }
-
- final long timeoutValue;
- try {
- timeoutValue = Math.round(FormatUtils.getPreciseTimeDuration(propertyValue, TimeUnit.SECONDS));
- } catch (final IllegalArgumentException ignored) {
- throw new AuthorizerCreationException(String.format("The %s '%s' is not a valid time interval.", propertyName, propertyValue));
- }
-
- return Math.toIntExact(timeoutValue);
- }
-
- /**
- * Called immediately before instance destruction for implementers to release resources.
- *
- * @throws AuthorizerDestructionException If pre-destruction fails.
- */
- @Override
- public void preDestruction() throws AuthorizerDestructionException {
- try {
- scheduler.shutdownNow();
- } catch (final Exception e) {
- logger.warn("Error shutting down refresh scheduler", e);
- }
- try {
- shellRunner.shutdown();
- } catch (final Exception e) {
- logger.warn("Error shutting down ShellRunner", e);
- }
- }
-
- public ShellCommandsProvider getCommandsProvider() {
- return selectedShellCommands;
- }
-
- public void setCommandsProvider(ShellCommandsProvider commandsProvider) {
- selectedShellCommands = commandsProvider;
- }
-
- /**
- * This is our entry point for user and group refresh. This method runs the top-level
- * `getUserList()` and `getGroupsList()` shell commands, then passes those results to the
- * other methods for record parse, extract, and object construction.
- */
- private void refreshUsersAndGroups() {
- final long startTime = System.currentTimeMillis();
-
- Map<String, User> uidToUser = new HashMap<>();
- Map<String, User> usernameToUser = new HashMap<>();
- Map<String, User> gidToUser = new HashMap<>();
- Map<String, Group> gidToGroup = new HashMap<>();
-
- List<String> userLines;
- List<String> groupLines;
-
- try {
- userLines = shellRunner.runShell(selectedShellCommands.getUsersList(), "Get Users List");
- groupLines = shellRunner.runShell(selectedShellCommands.getGroupsList(), "Get Groups List");
- } catch (final IOException ioexc) {
- logger.error("refreshUsersAndGroups shell exception", ioexc);
- return;
- }
-
- rebuildUsers(userLines, uidToUser, usernameToUser, gidToUser);
- rebuildGroups(groupLines, gidToGroup);
- reconcilePrimaryGroups(gidToUser, gidToGroup);
-
- synchronized (usersById) {
- usersById.clear();
- usersById.putAll(uidToUser);
-
- if (logger.isTraceEnabled()) {
- logger.trace("=== Users by id...");
- Set<User> sortedUsers = new TreeSet<>(Comparator.comparing(User::getIdentity));
- sortedUsers.addAll(usersById.values());
- sortedUsers.forEach(u -> logger.trace("=== {}", u));
- }
- }
-
- synchronized (usersByName) {
- usersByName.clear();
- usersByName.putAll(usernameToUser);
- logger.debug("users now size: {}", usersByName.size());
- }
-
- synchronized (groupsById) {
- groupsById.clear();
- groupsById.putAll(gidToGroup);
- logger.debug("groupsById now size: {}", groupsById.size());
-
- if (logger.isTraceEnabled()) {
- logger.trace("=== Groups by id...");
- Set<Group> sortedGroups = new TreeSet<>(Comparator.comparing(Group::getName));
- sortedGroups.addAll(groupsById.values());
- sortedGroups.forEach(g -> logger.trace("=== {}", g));
- }
- }
-
- synchronized (groupsByName) {
- groupsByName.clear();
- gidToGroup.values().forEach(g -> groupsByName.put(g.getName(), g));
- logger.debug("groupsByName now size: {}", groupsByName.size());
- }
-
- final long endTime = System.currentTimeMillis();
- logger.info("Refreshed users and groups, took {} seconds", (endTime - startTime) / 1000);
- }
-
- /**
- * This method parses the output of the `getUsersList()` shell command, where we expect the output
- * to look like `user-name:user-id:primary-group-id`.
- * <p>
- * This method splits each output line on the ":" and attempts to build a User object
- * from the resulting name, uid, and primary gid. Unusable records are logged.
- */
- private void rebuildUsers(List<String> userLines, Map<String, User> idToUser, Map<String, User> usernameToUser, Map<String, User> gidToUser) {
- userLines.forEach(line -> {
- logger.trace("Processing user: {}", line);
-
- String[] record = line.split(":");
- if (record.length > 2) {
- String userIdentity = record[0], userIdentifier = record[1], primaryGroupIdentifier = record[2];
-
- if (!StringUtils.isBlank(userIdentifier) && !StringUtils.isBlank(userIdentity) && !excludeUsers.matcher(userIdentity).matches()) {
- User user = new User.Builder()
- .identity(userIdentity)
- .identifierGenerateFromSeed(getUserIdentifierSeed(userIdentity))
- .build();
- idToUser.put(user.getIdentifier(), user);
- usernameToUser.put(userIdentity, user);
- logger.debug("Refreshed user {}", user);
-
- if (!StringUtils.isBlank(primaryGroupIdentifier)) {
- // create a temporary group to deterministically generate the group id and associate this user
- Group group = new Group.Builder()
- .name(primaryGroupIdentifier)
- .identifierGenerateFromSeed(getGroupIdentifierSeed(primaryGroupIdentifier))
- .build();
- gidToUser.put(group.getIdentifier(), user);
- logger.debug("Associated primary group {} with user {}", group.getIdentifier(), userIdentity);
- } else {
- logger.warn("Null or empty primary group id for: {}", userIdentity);
- }
-
- } else {
- logger.warn("Null, empty, or skipped user name: {} or id: {}", userIdentity, userIdentifier);
- }
- } else {
- logger.warn("Unexpected record format. Expected 3 or more colon separated values per line.");
- }
- });
- }
-
- /**
- * This method parses the output of the `getGroupsList()` shell command, where we expect the output
- * to look like `group-name:group-id`.
- * <p>
- * This method splits each output line on the ":" and attempts to build a Group object
- * from the resulting name and gid. Unusable records are logged.
- * <p>
- * This command also runs the `getGroupMembers(username)` command once per group. The expected output
- * of that command should look like `group-name-1,group-name-2`.
- */
- private void rebuildGroups(List<String> groupLines, Map<String, Group> groupsById) {
- groupLines.forEach(line -> {
- logger.trace("Processing group: {}", line);
-
- String[] record = line.split(":");
- if (record.length > 1) {
- Set<String> users = new HashSet<>();
- String groupName = record[0], groupIdentifier = record[1];
-
- try {
- String groupMembersCommand = selectedShellCommands.getGroupMembers(groupName);
- List<String> memberLines = shellRunner.runShell(groupMembersCommand, "Get Group Members");
- // Use the first line only, and log if the line count isn't exactly one:
- if (!memberLines.isEmpty()) {
- String memberLine = memberLines.get(0);
- if (!StringUtils.isBlank(memberLine)) {
- String[] members = memberLine.split(",");
- for (String userIdentity : members) {
- if (!StringUtils.isBlank(userIdentity)) {
- User tempUser = new User.Builder()
- .identity(userIdentity)
- .identifierGenerateFromSeed(getUserIdentifierSeed(userIdentity))
- .build();
- users.add(tempUser.getIdentifier());
- logger.debug("Added temp user {} for group {}", tempUser, groupName);
- }
- }
- } else {
- logger.debug("list membership returned no members");
- }
- } else {
- logger.debug("list membership returned zero lines.");
- }
- if (memberLines.size() > 1) {
- logger.error("list membership returned too many lines, only used the first.");
- }
-
- } catch (final IOException ioexc) {
- logger.error("list membership shell exception", ioexc);
- }
-
- if (!StringUtils.isBlank(groupIdentifier) && !StringUtils.isBlank(groupName) && !excludeGroups.matcher(groupName).matches()) {
- Group group = new Group.Builder()
- .name(groupName)
- .identifierGenerateFromSeed(getGroupIdentifierSeed(groupIdentifier))
- .addUsers(users)
- .build();
- groupsById.put(group.getIdentifier(), group);
- logger.debug("Refreshed group {}", group);
- } else {
- logger.warn("Null, empty, or skipped group name: {} or id: {}", groupName, groupIdentifier);
- }
- } else {
- logger.warn("Unexpected record format. Expected 1 or more comma separated values.");
- }
- });
- }
-
- /**
- * This method parses the output of the `getGroupsList()` shell command, where we expect the output
- * to look like `group-name:group-id`.
- * <p>
- * This method splits each output line on the ":" and attempts to build a Group object
- * from the resulting name and gid.
- */
- private void reconcilePrimaryGroups(Map<String, User> uidToUser, Map<String, Group> gidToGroup) {
- uidToUser.forEach((primaryGid, primaryUser) -> {
- Group primaryGroup = gidToGroup.get(primaryGid);
-
- if (primaryGroup == null) {
- logger.warn("Primary group {} not found for {}", primaryGid, primaryUser.getIdentity());
- } else if (!excludeGroups.matcher(primaryGroup.getName()).matches()) {
- Set<String> groupUsers = primaryGroup.getUsers();
- if (!groupUsers.contains(primaryUser.getIdentifier())) {
- Set<String> updatedUserIdentifiers = new HashSet<>(groupUsers);
- updatedUserIdentifiers.add(primaryUser.getIdentifier());
-
- Group updatedGroup = new Group.Builder()
- .identifier(primaryGroup.getIdentifier())
- .name(primaryGroup.getName())
- .addUsers(updatedUserIdentifiers)
- .build();
- gidToGroup.put(updatedGroup.getIdentifier(), updatedGroup);
- logger.debug("Added user {} to primary group {}", primaryUser, updatedGroup);
- } else {
- logger.debug("Primary group {} already contains user {}", primaryGroup, primaryUser);
- }
- } else {
- logger.debug("Primary group {} excluded from matcher for {}", primaryGroup.getName(), primaryUser.getIdentity());
- }
- });
- }
-
- private String getUserIdentifierSeed(final String userIdentifier) {
- return legacyIdentifierMode ? userIdentifier : userIdentifier + "-user";
- }
-
- private String getGroupIdentifierSeed(final String groupIdentifier) {
- return legacyIdentifierMode ? groupIdentifier : groupIdentifier + "-group";
- }
-
-
- /**
- * @return The fixed refresh delay.
- */
- public long getRefreshDelay() {
- return fixedDelay;
- }
-
- /**
- * Testing concession for clearing the internal caches.
- */
- void clearCaches() {
- synchronized (usersById) {
- usersById.clear();
- }
-
- synchronized (usersByName) {
- usersByName.clear();
- }
-
- synchronized (groupsById) {
- groupsById.clear();
- }
-
- synchronized (groupsByName) {
- groupsByName.clear();
- }
- }
-
- /**
- * @return The size of the internal user cache.
- */
- public int userCacheSize() {
- return usersById.size();
- }
-
- /**
- * @return The size of the internal group cache.
- */
- public int groupCacheSize() {
- return groupsById.size();
- }
-}
diff --git a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/util/ShellRunner.java b/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/util/ShellRunner.java
deleted file mode 100644
index fa1eceedf4..0000000000
--- a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/java/org/apache/nifi/authorization/util/ShellRunner.java
+++ /dev/null
@@ -1,123 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.authorization.util;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.BufferedReader;
-import java.io.IOException;
-import java.io.InputStreamReader;
-import java.io.Reader;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.concurrent.ExecutorService;
-import java.util.concurrent.Executors;
-import java.util.concurrent.ThreadFactory;
-import java.util.concurrent.TimeUnit;
-
-public class ShellRunner {
- private final static Logger logger = LoggerFactory.getLogger(ShellRunner.class);
-
- static String SHELL = "sh";
- static String OPTS = "-c";
-
- private final int timeoutSeconds;
- private final ExecutorService executor;
-
- public ShellRunner(final int timeoutSeconds) {
- this.timeoutSeconds = timeoutSeconds;
- this.executor = Executors.newFixedThreadPool(1, new ThreadFactory() {
- @Override
- public Thread newThread(final Runnable r) {
- final Thread t = Executors.defaultThreadFactory().newThread(r);
- t.setName("ShellRunner");
- t.setDaemon(true);
- return t;
- }
- });
- }
-
- public List<String> runShell(String command, String description) throws IOException {
- final ProcessBuilder builder = new ProcessBuilder(SHELL, OPTS, command);
- builder.redirectErrorStream(true);
-
- final List<String> builderCommand = builder.command();
- logger.debug("Run Command '{}': {}", description, builderCommand);
-
- final Process proc = builder.start();
-
- final List<String> lines = new ArrayList<>();
- executor.submit(() -> {
- try {
- try (final Reader stdin = new InputStreamReader(proc.getInputStream());
- final BufferedReader reader = new BufferedReader(stdin)) {
- logger.trace("Reading process input stream...");
-
- String line;
- int lineCount = 0;
- while ((line = reader.readLine()) != null) {
- if (logger.isTraceEnabled()) {
- logger.trace("{} - {}", (++lineCount), line);
- }
- lines.add(line.trim());
- }
-
- logger.trace("Finished reading process input stream");
- }
- } catch (IOException e) {
- logger.error(e.getMessage(), e);
- }
- });
-
- boolean completed;
- try {
- completed = proc.waitFor(timeoutSeconds, TimeUnit.SECONDS);
- } catch (InterruptedException irexc) {
- throw new IOException(irexc.getMessage(), irexc.getCause());
- }
-
- if (!completed) {
- logger.debug("Process did not complete in allotted time, attempting to forcibly destroy process...");
- try {
- proc.destroyForcibly();
- } catch (Exception e) {
- logger.debug("Process failed to destroy", e);
- }
- throw new IllegalStateException("Shell command '" + command + "' did not complete during the allotted time period");
- }
-
- if (proc.exitValue() != 0) {
- throw new IOException("Process exited with non-zero value: " + proc.exitValue());
- }
-
- return lines;
- }
-
- public void shutdown() {
- executor.shutdown();
- try {
- if (!executor.awaitTermination(5000L, TimeUnit.MILLISECONDS)) {
- logger.info("Failed to stop ShellRunner executor in 5 seconds. Terminating");
- executor.shutdownNow();
- }
- } catch (InterruptedException ie) {
- executor.shutdownNow();
- Thread.currentThread().interrupt();
- }
- }
-}
diff --git a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/resources/META-INF/services/org.apache.nifi.authorization.UserGroupProvider b/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/resources/META-INF/services/org.apache.nifi.authorization.UserGroupProvider
deleted file mode 100755
index fb3360ddb8..0000000000
--- a/nifi-framework-bundle/nifi-framework/nifi-shell-authorizer/src/main/resources/META-INF/services/org.apache.nifi.authorization.UserGroupProvider
+++ /dev/null
@@ -1,15 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-org.apache.nifi.authorization.ShellUserGroupProvider
diff --git a/nifi-framework-bundle/nifi-framework/pom.xml b/nifi-framework-bundle/nifi-framework/pom.xml
index 54b62d6cac..e51c11ead3 100644
--- a/nifi-framework-bundle/nifi-framework/pom.xml
+++ b/nifi-framework-bundle/nifi-framework/pom.xml
@@ -54,7 +54,6 @@
<module>nifi-properties-loader</module>
<module>nifi-standard-prioritizers</module>
<module>nifi-mock-authorizer</module>
- <module>nifi-shell-authorizer</module>
<module>nifi-headless-server</module>
</modules>
</project>
diff --git a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/NssShellCommands.java b/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/NssShellCommands.java
deleted file mode 100644
index 0d49a6e0a0..0000000000
--- a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/NssShellCommands.java
+++ /dev/null
@@ -1,55 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.registry.security.authorization.shell;
-
-/**
- * Provides shell commands to read users and groups on NSS-enabled systems.
- *
- * See `man 5 nsswitch.conf` for more info.
- */
-class NssShellCommands implements ShellCommandsProvider {
- /**
- * @return Shell command string that will return a list of users.
- */
- public String getUsersList() {
- return "getent passwd | cut -f 1,3,4 -d ':'";
- }
-
- /**
- * @return Shell command string that will return a list of groups.
- */
- public String getGroupsList() {
- return "getent group | cut -f 1,3 -d ':'";
- }
-
- /**
- * @param groupName name of group.
- * @return Shell command string that will return a list of users for a group.
- */
- public String getGroupMembers(String groupName) {
- return String.format("getent group %s | cut -f 4 -d ':'", groupName);
- }
-
- /**
- * This gives exit code 0 on all tested distributions.
- *
- * @return Shell command string that will exit normally (0) on a suitable system.
- */
- public String getSystemCheck() {
- return "getent --version";
- }
-}
\ No newline at end of file
diff --git a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/OsxShellCommands.java b/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/OsxShellCommands.java
deleted file mode 100644
index e4949ac2e5..0000000000
--- a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/OsxShellCommands.java
+++ /dev/null
@@ -1,55 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.registry.security.authorization.shell;
-
-/**
- * Provides shell commands to read users and groups on Mac OSX systems.
- *
- * See `man dscl` for more info.
- */
-class OsxShellCommands implements ShellCommandsProvider {
- /**
- * @return Shell command string that will return a list of users.
- */
- public String getUsersList() {
- return "dscl . -readall /Users UniqueID PrimaryGroupID | awk 'BEGIN { OFS = \":\"; ORS=\"\\n\"; i=0;} /RecordName: / {name = $2;i = 0;}" +
- "/PrimaryGroupID: / {gid = $2;} /^ / {if (i == 0) { i++; name = $1;}} /UniqueID: / {uid = $2;print name, uid, gid;}' | grep -v ^_";
- }
-
- /**
- * @return Shell command string that will return a list of groups.
- */
- public String getGroupsList() {
- return "dscl . -list /Groups PrimaryGroupID | grep -v '^_' | sed 's/ \\{1,\\}/:/g'";
- }
-
- /**
- *
- * @param groupName name of group.
- * @return Shell command string that will return a list of users for a group.
- */
- public String getGroupMembers(String groupName) {
- return String.format("dscl . -read /Groups/%s GroupMembership | cut -f 2- -d ' ' | sed 's/\\ /,/g'", groupName);
- }
-
- /**
- * @return Shell command string that will exit normally (0) on a suitable system.
- */
- public String getSystemCheck() {
- return "which dscl";
- }
-}
diff --git a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/ShellCommandsProvider.java b/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/ShellCommandsProvider.java
deleted file mode 100644
index 1038358637..0000000000
--- a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/ShellCommandsProvider.java
+++ /dev/null
@@ -1,67 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.registry.security.authorization.shell;
-
-/**
- * Common interface for shell command strings to read users and groups.
- *
- */
-interface ShellCommandsProvider {
- /**
- * Gets the command for listing users.
- *
- * When executed, this command should output one record per line in this format:
- *
- * `username:user-id:primary-group-id`
- *
- * @return Shell command string that will return a list of users.
- */
- String getUsersList();
-
- /**
- * Gets the command for listing groups.
- *
- * When executed, this command should output one record per line in this format:
- *
- * `group-name:group-id`
- *
- * @return Shell command string that will return a list of groups.
- */
- String getGroupsList();
-
- /**
- * Gets the command for listing the members of a group.
- *
- * When executed, this command should output one line in this format:
- *
- * `user-name-1,user-name-2,user-name-n`
- *
- * @param groupName name of group.
- * @return Shell command string that will return a list of users for a group.
- */
- String getGroupMembers(String groupName);
-
- /**
- * Gets the command for checking the suitability of the host system.
- *
- * The command is expected to exit with status 0 (zero) to indicate success, and any other status
- * to indicate failure.
- *
- * @return Shell command string that will exit normally (0) on a suitable system.
- */
- String getSystemCheck();
-}
diff --git a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/ShellRunner.java b/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/ShellRunner.java
deleted file mode 100644
index 8ad5b9ceb6..0000000000
--- a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/ShellRunner.java
+++ /dev/null
@@ -1,123 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.registry.security.authorization.shell;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.BufferedReader;
-import java.io.IOException;
-import java.io.InputStreamReader;
-import java.io.Reader;
-import java.util.ArrayList;
-import java.util.List;
-import java.util.concurrent.ExecutorService;
-import java.util.concurrent.Executors;
-import java.util.concurrent.ThreadFactory;
-import java.util.concurrent.TimeUnit;
-
-public class ShellRunner {
- private final static Logger logger = LoggerFactory.getLogger(ShellRunner.class);
-
- static String SHELL = "sh";
- static String OPTS = "-c";
-
- private final int timeoutSeconds;
- private final ExecutorService executor;
-
- public ShellRunner(final int timeoutSeconds) {
- this.timeoutSeconds = timeoutSeconds;
- this.executor = Executors.newFixedThreadPool(1, new ThreadFactory() {
- @Override
- public Thread newThread(final Runnable r) {
- final Thread t = Executors.defaultThreadFactory().newThread(r);
- t.setName("ShellRunner");
- t.setDaemon(true);
- return t;
- }
- });
- }
-
- public List<String> runShell(String command, String description) throws IOException {
- final ProcessBuilder builder = new ProcessBuilder(SHELL, OPTS, command);
- builder.redirectErrorStream(true);
-
- final List<String> builderCommand = builder.command();
- logger.debug("Run Command '{}': {}", description, builderCommand);
-
- final Process proc = builder.start();
-
- final List<String> lines = new ArrayList<>();
- executor.submit(() -> {
- try {
- try (final Reader stdin = new InputStreamReader(proc.getInputStream());
- final BufferedReader reader = new BufferedReader(stdin)) {
- logger.trace("Reading process input stream...");
-
- String line;
- int lineCount = 0;
- while ((line = reader.readLine()) != null) {
- if (logger.isTraceEnabled()) {
- logger.trace("{} - {}", (++lineCount), line);
- }
- lines.add(line.trim());
- }
-
- logger.trace("Finished reading process input stream");
- }
- } catch (IOException e) {
- logger.error(e.getMessage(), e);
- }
- });
-
- boolean completed;
- try {
- completed = proc.waitFor(timeoutSeconds, TimeUnit.SECONDS);
- } catch (InterruptedException irexc) {
- throw new IOException(irexc.getMessage(), irexc.getCause());
- }
-
- if (!completed) {
- logger.debug("Process did not complete in allotted time, attempting to forcibly destroy process...");
- try {
- proc.destroyForcibly();
- } catch (Exception e) {
- logger.debug("Process failed to destroy", e);
- }
- throw new IllegalStateException("Shell command '" + command + "' did not complete during the allotted time period");
- }
-
- if (proc.exitValue() != 0) {
- throw new IOException("Process exited with non-zero value: " + proc.exitValue());
- }
-
- return lines;
- }
-
- public void shutdown() {
- executor.shutdown();
- try {
- if (!executor.awaitTermination(5000L, TimeUnit.MILLISECONDS)) {
- logger.info("Failed to stop ShellRunner executor in 5 seconds. Terminating");
- executor.shutdownNow();
- }
- } catch (InterruptedException ie) {
- executor.shutdownNow();
- Thread.currentThread().interrupt();
- }
- }
-}
diff --git a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/ShellUserGroupProvider.java b/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/ShellUserGroupProvider.java
deleted file mode 100644
index 13f32f516c..0000000000
--- a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/shell/ShellUserGroupProvider.java
+++ /dev/null
@@ -1,645 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.registry.security.authorization.shell;
-
-import org.apache.commons.lang3.StringUtils;
-import org.apache.nifi.registry.security.authorization.AuthorizerConfigurationContext;
-import org.apache.nifi.registry.security.authorization.Group;
-import org.apache.nifi.registry.security.authorization.User;
-import org.apache.nifi.registry.security.authorization.UserAndGroups;
-import org.apache.nifi.registry.security.authorization.UserGroupProvider;
-import org.apache.nifi.registry.security.authorization.UserGroupProviderInitializationContext;
-import org.apache.nifi.registry.security.authorization.exception.AuthorizationAccessException;
-import org.apache.nifi.registry.security.exception.SecurityProviderCreationException;
-import org.apache.nifi.registry.security.exception.SecurityProviderDestructionException;
-import org.apache.nifi.registry.util.FormatUtils;
-import org.apache.nifi.registry.util.PropertyValue;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.io.IOException;
-import java.util.Comparator;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import java.util.TreeSet;
-import java.util.concurrent.Executors;
-import java.util.concurrent.ScheduledExecutorService;
-import java.util.concurrent.TimeUnit;
-import java.util.regex.Pattern;
-import java.util.regex.PatternSyntaxException;
-
-/*
- * ShellUserGroupProvider implements UserGroupProvider by way of shell commands.
- */
-public class ShellUserGroupProvider implements UserGroupProvider {
- private final static Logger logger = LoggerFactory.getLogger(ShellUserGroupProvider.class);
-
- private final static String OS_TYPE_ERROR = "Unsupported operating system.";
- private final static String SYS_CHECK_ERROR = "System check failed - cannot provide users and groups.";
- private final static Map<String, User> usersById = new HashMap<>(); // id == identifier
- private final static Map<String, User> usersByName = new HashMap<>(); // name == identity
- private final static Map<String, Group> groupsById = new HashMap<>();
-
- public static final String REFRESH_DELAY_PROPERTY = "Refresh Delay";
- private static final long MINIMUM_SYNC_INTERVAL_MILLISECONDS = 10_000;
-
- public static final String EXCLUDE_USER_PROPERTY = "Exclude Users";
- public static final String EXCLUDE_GROUP_PROPERTY = "Exclude Groups";
-
- public static final String COMMAND_TIMEOUT_PROPERTY = "Command Timeout";
-
- private static final String DEFAULT_COMMAND_TIMEOUT = "60 seconds";
-
- private long fixedDelay;
- private Pattern excludeUsers;
- private Pattern excludeGroups;
- private int timeoutSeconds;
-
- // Our scheduler has one thread for users, one for groups:
- private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(2);
-
- // Commands selected during initialization:
- private ShellCommandsProvider selectedShellCommands;
-
- private ShellRunner shellRunner;
-
- // Start of the UserGroupProvider implementation. Javadoc strings
- // copied from the interface definition for reference.
-
- /**
- * Retrieves all users. Must be non null
- *
- * @return a list of users
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public Set<User> getUsers() throws AuthorizationAccessException {
- synchronized (usersById) {
- logger.debug("getUsers has user set of size: {}", usersById.size());
- return new HashSet<>(usersById.values());
- }
- }
-
- /**
- * Retrieves the user with the given identifier.
- *
- * @param identifier the id of the user to retrieve
- * @return the user with the given id, or null if no matching user was found
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public User getUser(String identifier) throws AuthorizationAccessException {
- User user;
-
- synchronized (usersById) {
- user = usersById.get(identifier);
- }
-
- if (user == null) {
- logger.debug("getUser (by id) user not found: {}", identifier);
- } else {
- logger.debug("getUser (by id) found user: {} for id: {}", user, identifier);
- }
- return user;
- }
-
- /**
- * Retrieves the user with the given identity.
- *
- * @param identity the identity of the user to retrieve
- * @return the user with the given identity, or null if no matching user was found
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public User getUserByIdentity(String identity) throws AuthorizationAccessException {
- User user;
-
- synchronized (usersByName) {
- user = usersByName.get(identity);
- }
-
- if (user == null) {
- logger.debug("getUser (by name) user not found: {}", identity);
- } else {
- logger.debug("getUser (by name) found user: {} for name: {}", user.getIdentity(), identity);
- }
- return user;
- }
-
- /**
- * Retrieves all groups. Must be non null
- *
- * @return a list of groups
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public Set<Group> getGroups() throws AuthorizationAccessException {
- synchronized (groupsById) {
- logger.debug("getGroups has group set of size: {}", groupsById.size());
- return new HashSet<>(groupsById.values());
- }
- }
-
- /**
- * Retrieves a Group by Id.
- *
- * @param identifier the identifier of the Group to retrieve
- * @return the Group with the given identifier, or null if no matching group was found
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public Group getGroup(String identifier) throws AuthorizationAccessException {
- Group group;
-
- synchronized (groupsById) {
- group = groupsById.get(identifier);
- }
-
- if (group == null) {
- logger.debug("getGroup (by id) group not found: {}", identifier);
- } else {
- logger.debug("getGroup (by id) found group: {} for id: {}", group.getName(), identifier);
- }
- return group;
-
- }
-
- /**
- * Gets a user and their groups.
- *
- * @return the UserAndGroups for the specified identity
- * @throws AuthorizationAccessException if there was an unexpected error performing the operation
- */
- @Override
- public UserAndGroups getUserAndGroups(String identity) throws AuthorizationAccessException {
- User user = getUserByIdentity(identity);
- logger.debug("Retrieved user {} for identity {}", user, identity);
-
- Set<Group> groups = new HashSet<>();
- if (user != null) {
- for (Group g : getGroups()) {
- if (g.getUsers().contains(user.getIdentifier())) {
- logger.debug("User {} belongs to group {}", user.getIdentity(), g.getName());
- groups.add(g);
- }
- }
- }
-
- if (groups.isEmpty()) {
- logger.debug("User {} belongs to no groups", user);
- }
-
- return new UserAndGroups() {
- @Override
- public User getUser() {
- return user;
- }
-
- @Override
- public Set<Group> getGroups() {
- return groups;
- }
- };
- }
-
- /**
- * Called immediately after instance creation for implementers to perform additional setup
- *
- * @param initializationContext in which to initialize
- */
- @Override
- public void initialize(UserGroupProviderInitializationContext initializationContext) throws SecurityProviderCreationException {
- }
-
- /**
- * Called to configure the Authorizer.
- *
- * @param configurationContext at the time of configuration
- * @throws SecurityProviderCreationException for any issues configuring the provider
- */
- @Override
- public void onConfigured(AuthorizerConfigurationContext configurationContext) throws SecurityProviderCreationException {
- logger.info("Configuring ShellUserGroupProvider");
-
- fixedDelay = getDelayProperty(configurationContext, REFRESH_DELAY_PROPERTY, "5 mins");
- timeoutSeconds = getTimeoutProperty(configurationContext, COMMAND_TIMEOUT_PROPERTY, DEFAULT_COMMAND_TIMEOUT);
- shellRunner = new ShellRunner(timeoutSeconds);
- logger.debug("Configured ShellRunner with command timeout of '{}' seconds", timeoutSeconds);
-
-
- // Our next init step is to select the command set based on the operating system name:
- ShellCommandsProvider commands = getCommandsProvider();
-
- if (commands == null) {
- commands = getCommandsProviderFromName(null);
- setCommandsProvider(commands);
- }
-
- // Our next init step is to run the system check from that command set to determine if the other commands
- // will work on this host or not.
- try {
- shellRunner.runShell(commands.getSystemCheck(), "Supported System Check");
- } catch (final Exception e) {
- logger.error("initialize exception: system check command: {}", commands.getSystemCheck(), e);
- throw new SecurityProviderCreationException(SYS_CHECK_ERROR, e);
- }
-
- // The next step is to add the user and group exclude regexes:
- try {
- excludeGroups = Pattern.compile(getProperty(configurationContext, EXCLUDE_GROUP_PROPERTY, ""));
- excludeUsers = Pattern.compile(getProperty(configurationContext, EXCLUDE_USER_PROPERTY, ""));
- } catch (final PatternSyntaxException e) {
- throw new SecurityProviderCreationException(e);
- }
-
- // With our command set selected, and our system check passed, we can pull in the users and groups:
- refreshUsersAndGroups();
-
- // And finally, our last init step is to fire off the refresh thread:
- scheduler.scheduleWithFixedDelay(() -> {
- try {
- refreshUsersAndGroups();
- } catch (final Throwable t) {
- logger.error("", t);
- }
- }, fixedDelay, fixedDelay, TimeUnit.MILLISECONDS);
-
- logger.info("Completed configuration of ShellUserGroupProvider");
- }
-
- private static ShellCommandsProvider getCommandsProviderFromName(String osName) {
- if (osName == null) {
- osName = System.getProperty("os.name");
- }
-
- ShellCommandsProvider commands;
- if (osName.startsWith("Linux")) {
- logger.debug("Selected Linux command set.");
- commands = new NssShellCommands();
- } else if (osName.startsWith("Mac OS X")) {
- logger.debug("Selected OSX command set.");
- commands = new OsxShellCommands();
- } else {
- throw new SecurityProviderCreationException(OS_TYPE_ERROR);
- }
- return commands;
- }
-
- private String getProperty(AuthorizerConfigurationContext authContext, String propertyName, String defaultValue) {
- final PropertyValue property = authContext.getProperty(propertyName);
- final String value;
-
- if (property != null && property.isSet()) {
- value = property.getValue();
- } else {
- value = defaultValue;
- }
- return value;
-
- }
-
- private long getDelayProperty(AuthorizerConfigurationContext authContext, String propertyName, String defaultValue) {
- final PropertyValue intervalProperty = authContext.getProperty(propertyName);
- final String propertyValue;
- final long syncInterval;
-
- if (intervalProperty.isSet()) {
- propertyValue = intervalProperty.getValue();
- } else {
- propertyValue = defaultValue;
- }
-
- try {
- syncInterval = Math.round(FormatUtils.getPreciseTimeDuration(propertyValue, TimeUnit.MILLISECONDS));
- } catch (final IllegalArgumentException ignored) {
- throw new SecurityProviderCreationException(String.format("The %s '%s' is not a valid time interval.", propertyName, propertyValue));
- }
-
- if (syncInterval < MINIMUM_SYNC_INTERVAL_MILLISECONDS) {
- throw new SecurityProviderCreationException(String.format("The %s '%s' is below the minimum value of '%d ms'", propertyName, propertyValue, MINIMUM_SYNC_INTERVAL_MILLISECONDS));
- }
- return syncInterval;
- }
-
- private int getTimeoutProperty(AuthorizerConfigurationContext authContext, String propertyName, String defaultValue) {
- final PropertyValue timeoutProperty = authContext.getProperty(propertyName);
-
- final String propertyValue;
- if (timeoutProperty.isSet()) {
- propertyValue = timeoutProperty.getValue();
- } else {
- propertyValue = defaultValue;
- }
-
- final long timeoutValue;
- try {
- timeoutValue = Math.round(FormatUtils.getPreciseTimeDuration(propertyValue, TimeUnit.SECONDS));
- } catch (final IllegalArgumentException ignored) {
- throw new SecurityProviderCreationException(String.format("The %s '%s' is not a valid time interval.", propertyName, propertyValue));
- }
-
- return Math.toIntExact(timeoutValue);
- }
-
- /**
- * Called immediately before instance destruction for implementers to release resources.
- *
- * @throws SecurityProviderDestructionException If pre-destruction fails.
- */
- @Override
- public void preDestruction() throws SecurityProviderDestructionException {
- try {
- scheduler.shutdownNow();
- } catch (final Exception e) {
- logger.warn("Error shutting down refresh scheduler", e);
- }
- try {
- shellRunner.shutdown();
- } catch (final Exception e) {
- logger.warn("Error shutting down ShellRunner", e);
- }
- }
-
- public ShellCommandsProvider getCommandsProvider() {
- return selectedShellCommands;
- }
-
- public void setCommandsProvider(ShellCommandsProvider commandsProvider) {
- selectedShellCommands = commandsProvider;
- }
-
- /**
- * This is our entry point for user and group refresh. This method runs the top-level
- * `getUserList()` and `getGroupsList()` shell commands, then passes those results to the
- * other methods for record parse, extract, and object construction.
- */
- private void refreshUsersAndGroups() {
- final long startTime = System.currentTimeMillis();
-
- Map<String, User> uidToUser = new HashMap<>();
- Map<String, User> usernameToUser = new HashMap<>();
- Map<String, User> gidToUser = new HashMap<>();
- Map<String, Group> gidToGroup = new HashMap<>();
-
- List<String> userLines;
- List<String> groupLines;
-
- try {
- userLines = shellRunner.runShell(selectedShellCommands.getUsersList(), "Get Users List");
- groupLines = shellRunner.runShell(selectedShellCommands.getGroupsList(), "Get Groups List");
- } catch (final IOException ioexc) {
- logger.error("refreshUsersAndGroups shell exception", ioexc);
- return;
- }
-
- rebuildUsers(userLines, uidToUser, usernameToUser, gidToUser);
- rebuildGroups(groupLines, gidToGroup);
- reconcilePrimaryGroups(gidToUser, gidToGroup);
-
- synchronized (usersById) {
- usersById.clear();
- usersById.putAll(uidToUser);
-
- if (logger.isTraceEnabled()) {
- logger.trace("=== Users by id...");
- Set<User> sortedUsers = new TreeSet<>(Comparator.comparing(User::getIdentity));
- sortedUsers.addAll(usersById.values());
- sortedUsers.forEach(u -> logger.trace("=== {}", u));
- }
- }
-
- synchronized (usersByName) {
- usersByName.clear();
- usersByName.putAll(usernameToUser);
- logger.debug("users now size: {}", usersByName.size());
- }
-
- synchronized (groupsById) {
- groupsById.clear();
- groupsById.putAll(gidToGroup);
- logger.debug("groups now size: {}", groupsById.size());
-
- if (logger.isTraceEnabled()) {
- logger.trace("=== Groups by id...");
- Set<Group> sortedGroups = new TreeSet<>(Comparator.comparing(Group::getName));
- sortedGroups.addAll(groupsById.values());
- sortedGroups.forEach(g -> logger.trace("=== {}", g));
- }
- }
-
- final long endTime = System.currentTimeMillis();
- logger.info("Refreshed users and groups, took {} seconds", (endTime - startTime) / 1000);
- }
-
- /**
- * This method parses the output of the `getUsersList()` shell command, where we expect the output
- * to look like `user-name:user-id:primary-group-id`.
- * <p>
- * This method splits each output line on the ":" and attempts to build a User object
- * from the resulting name, uid, and primary gid. Unusable records are logged.
- */
- private void rebuildUsers(List<String> userLines, Map<String, User> idToUser, Map<String, User> usernameToUser, Map<String, User> gidToUser) {
- userLines.forEach(line -> {
- logger.trace("Processing user: {}", line);
-
- String[] record = line.split(":");
- if (record.length > 2) {
- String userIdentity = record[0], userIdentifier = record[1], primaryGroupIdentifier = record[2];
-
- if (!StringUtils.isBlank(userIdentifier) && !StringUtils.isBlank(userIdentity) && !excludeUsers.matcher(userIdentity).matches()) {
- User user = new User.Builder()
- .identity(userIdentity)
- .identifierGenerateFromSeed(getUserIdentifierSeed(userIdentity))
- .build();
- idToUser.put(user.getIdentifier(), user);
- usernameToUser.put(userIdentity, user);
- logger.debug("Refreshed user {}", user);
-
- if (!StringUtils.isBlank(primaryGroupIdentifier)) {
- // create a temporary group to deterministically generate the group id and associate this user
- Group group = new Group.Builder()
- .name(primaryGroupIdentifier)
- .identifierGenerateFromSeed(getGroupIdentifierSeed(primaryGroupIdentifier))
- .build();
- gidToUser.put(group.getIdentifier(), user);
- logger.debug("Associated primary group {} with user {}", group.getIdentifier(), userIdentity);
- } else {
- logger.warn("Null or empty primary group id for: {}", userIdentity);
- }
-
- } else {
- logger.warn("Null, empty, or skipped user name: {} or id: {}", userIdentity, userIdentifier);
- }
- } else {
- logger.warn("Unexpected record format. Expected 3 or more colon separated values per line.");
- }
- });
- }
-
- /**
- * This method parses the output of the `getGroupsList()` shell command, where we expect the output
- * to look like `group-name:group-id`.
- * <p>
- * This method splits each output line on the ":" and attempts to build a Group object
- * from the resulting name and gid. Unusable records are logged.
- * <p>
- * This command also runs the `getGroupMembers(username)` command once per group. The expected output
- * of that command should look like `group-name-1,group-name-2`.
- */
- private void rebuildGroups(List<String> groupLines, Map<String, Group> groupsById) {
- groupLines.forEach(line -> {
- logger.trace("Processing group: {}", line);
-
- String[] record = line.split(":");
- if (record.length > 1) {
- Set<String> users = new HashSet<>();
- String groupName = record[0], groupIdentifier = record[1];
-
- try {
- String groupMembersCommand = selectedShellCommands.getGroupMembers(groupName);
- List<String> memberLines = shellRunner.runShell(groupMembersCommand, "Get Group Members");
- // Use the first line only, and log if the line count isn't exactly one:
- if (!memberLines.isEmpty()) {
- String memberLine = memberLines.get(0);
- if (!StringUtils.isBlank(memberLine)) {
- String[] members = memberLine.split(",");
- for (String userIdentity : members) {
- if (!StringUtils.isBlank(userIdentity)) {
- User tempUser = new User.Builder()
- .identity(userIdentity)
- .identifierGenerateFromSeed(getUserIdentifierSeed(userIdentity))
- .build();
- users.add(tempUser.getIdentifier());
- logger.debug("Added temp user {} for group {}", tempUser, groupName);
- }
- }
- } else {
- logger.debug("list membership returned no members");
- }
- } else {
- logger.debug("list membership returned zero lines.");
- }
- if (memberLines.size() > 1) {
- logger.error("list membership returned too many lines, only used the first.");
- }
-
- } catch (final IOException ioexc) {
- logger.error("list membership shell exception", ioexc);
- }
-
- if (!StringUtils.isBlank(groupIdentifier) && !StringUtils.isBlank(groupName) && !excludeGroups.matcher(groupName).matches()) {
- Group group = new Group.Builder()
- .name(groupName)
- .identifierGenerateFromSeed(getGroupIdentifierSeed(groupIdentifier))
- .addUsers(users)
- .build();
- groupsById.put(group.getIdentifier(), group);
- logger.debug("Refreshed group {}", group);
- } else {
- logger.warn("Null, empty, or skipped group name: {} or id: {}", groupName, groupIdentifier);
- }
- } else {
- logger.warn("Unexpected record format. Expected 1 or more comma separated values.");
- }
- });
- }
-
- /**
- * This method parses the output of the `getGroupsList()` shell command, where we expect the output
- * to look like `group-name:group-id`.
- * <p>
- * This method splits each output line on the ":" and attempts to build a Group object
- * from the resulting name and gid.
- */
- private void reconcilePrimaryGroups(Map<String, User> uidToUser, Map<String, Group> gidToGroup) {
- uidToUser.forEach((primaryGid, primaryUser) -> {
- Group primaryGroup = gidToGroup.get(primaryGid);
-
- if (primaryGroup == null) {
- logger.warn("Primary group {} not found for {}", primaryGid, primaryUser.getIdentity());
- } else if (!excludeGroups.matcher(primaryGroup.getName()).matches()) {
- Set<String> groupUsers = primaryGroup.getUsers();
- if (!groupUsers.contains(primaryUser.getIdentifier())) {
- Set<String> updatedUserIdentifiers = new HashSet<>(groupUsers);
- updatedUserIdentifiers.add(primaryUser.getIdentifier());
-
- Group updatedGroup = new Group.Builder()
- .identifier(primaryGroup.getIdentifier())
- .name(primaryGroup.getName())
- .addUsers(updatedUserIdentifiers)
- .build();
- gidToGroup.put(updatedGroup.getIdentifier(), updatedGroup);
- logger.debug("Added user {} to primary group {}", primaryUser, updatedGroup);
- } else {
- logger.debug("Primary group {} already contains user {}", primaryGroup, primaryUser);
- }
- } else {
- logger.debug("Primary group {} excluded from matcher for {}", primaryGroup.getName(), primaryUser.getIdentity());
- }
- });
- }
-
- private String getUserIdentifierSeed(final String userIdentifier) {
- return userIdentifier + "-user";
- }
-
- private String getGroupIdentifierSeed(final String groupIdentifier) {
- return groupIdentifier + "-group";
- }
-
-
- /**
- * @return The fixed refresh delay.
- */
- public long getRefreshDelay() {
- return fixedDelay;
- }
-
- /**
- * Testing concession for clearing the internal caches.
- */
- void clearCaches() {
- synchronized (usersById) {
- usersById.clear();
- }
-
- synchronized (usersByName) {
- usersByName.clear();
- }
-
- synchronized (groupsById) {
- groupsById.clear();
- }
- }
-
- /**
- * @return The size of the internal user cache.
- */
- public int userCacheSize() {
- return usersById.size();
- }
-
- /**
- * @return The size of the internal group cache.
- */
- public int groupCacheSize() {
- return groupsById.size();
- }
-}
diff --git a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/resources/META-INF/services/org.apache.nifi.registry.security.authorization.UserGroupProvider b/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/resources/META-INF/services/org.apache.nifi.registry.security.authorization.UserGroupProvider
index 73ec0cf54c..6343af9460 100644
--- a/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/resources/META-INF/services/org.apache.nifi.registry.security.authorization.UserGroupProvider
+++ b/nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/resources/META-INF/services/org.apache.nifi.registry.security.authorization.UserGroupProvider
@@ -16,5 +16,4 @@ org.apache.nifi.registry.security.authorization.CompositeUserGroupProvider
org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider
org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider
org.apache.nifi.registry.security.ldap.tenants.LdapUserGroupProvider
-org.apache.nifi.registry.security.authorization.database.DatabaseUserGroupProvider
-org.apache.nifi.registry.security.authorization.shell.ShellUserGroupProvider
\ No newline at end of file
+org.apache.nifi.registry.security.authorization.database.DatabaseUserGroupProvider
\ No newline at end of file
diff --git a/nifi-registry/nifi-registry-core/nifi-registry-resources/src/main/resources/conf/authorizers.xml b/nifi-registry/nifi-registry-core/nifi-registry-resources/src/main/resources/conf/authorizers.xml
index 9db7acad82..ff01c8f073 100644
--- a/nifi-registry/nifi-registry-core/nifi-registry-resources/src/main/resources/conf/authorizers.xml
+++ b/nifi-registry/nifi-registry-core/nifi-registry-resources/src/main/resources/conf/authorizers.xml
@@ -179,27 +179,6 @@
</userGroupProvider>
To enable the ldap-user-group-provider remove 2 lines. This is 2 of 2. -->
- <!--
- The ShellUserGroupProvider provides support for retrieving users and groups by way of shell commands
- on systems that support `sh`. Implementations available for Linux and Mac OS, and are selected by the
- provider based on the system property `os.name`.
-
- 'Refresh Delay' - duration to wait between subsequent refreshes. Default is '5 mins'.
- 'Exclude Groups' - regular expression used to exclude groups. Default is '', which means no groups are excluded.
- 'Exclude Users' - regular expression used to exclude users. Default is '', which means no users are excluded.
- 'Command Timeout' - amount of time to wait while executing a command before timing out
- -->
- <!-- To enable the shell-user-group-provider remove 2 lines. This is 1 of 2.
- <userGroupProvider>
- <identifier>shell-user-group-provider</identifier>
- <class>org.apache.nifi.registry.security.authorization.shell.ShellUserGroupProvider</class>
- <property name="Refresh Delay">5 mins</property>
- <property name="Exclude Groups"></property>
- <property name="Exclude Users"></property>
- <property name="Command Timeout">60 seconds</property>
- </userGroupProvider>
- To enable the shell-user-group-provider remove 2 lines. This is 2 of 2. -->
-
<!--
The CompositeUserGroupProvider will provide support for retrieving users and groups from multiple sources.