You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2022/12/11 22:25:26 UTC
[GitHub] [nifi] turcsanyip opened a new pull request, #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
turcsanyip opened a new pull request, #6777:
URL: https://github.com/apache/nifi/pull/6777
…cessors
Also added Signer Override property in AWSCredentialsProviderControllerService with Custom Signer extension point. Made Assume Role related properties dependent on Assume Role ARN property in AWSCredentialsProviderControllerService. Fixed S3 IT tests.
# Summary
[NIFI-10969](https://issues.apache.org/jira/browse/NIFI-10969)
# Tracking
Please complete the following tracking steps prior to pull request creation.
### Issue Tracking
- [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
### Pull Request Tracking
- [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
- [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
### Pull Request Formatting
- [ ] Pull Request based on current revision of the `main` branch
- [ ] Pull Request refers to a feature branch with one commit containing changes
# Verification
Please indicate the verification steps performed prior to pull request creation.
### Build
- [ ] Build completed using `mvn clean install -P contrib-check`
- [ ] JDK 8
- [ ] JDK 11
- [ ] JDK 17
### Licensing
- [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
- [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
### Documentation
- [ ] Documentation formatting appears as expected in rendered files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] tpalfy commented on a diff in pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
Posted by GitBox <gi...@apache.org>.
tpalfy commented on code in PR #6777:
URL: https://github.com/apache/nifi/pull/6777#discussion_r1049665128
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/test/java/org/apache/nifi/processors/aws/credentials/provider/factory/TestCredentialsProviderFactory.java:
##########
@@ -296,8 +285,42 @@ public void testAssumeRoleMissingProxyPort() throws Throwable {
public void testAssumeRoleInvalidProxyPort() throws Throwable {
final TestRunner runner = TestRunners.newTestRunner(MockAWSProcessor.class);
runner.setProperty(CredentialPropertyDescriptors.CREDENTIALS_FILE, "src/test/resources/mock-aws-credentials.properties");
+ runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_ARN, "BogusArn");
+ runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_NAME, "BogusSession");
runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_PROXY_HOST, "proxy.company.com");
runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_PROXY_PORT, "notIntPort");
runner.assertNotValid();
}
+
+ @Test
+ public void testAssumeRoleCredentialsWithCustomSigner() throws Exception {
+ final TestRunner runner = TestRunners.newTestRunner(MockAWSProcessor.class);
+ runner.setProperty(CredentialPropertyDescriptors.CREDENTIALS_FILE, "src/test/resources/mock-aws-credentials.properties");
+ runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_ARN, "BogusArn");
+ runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_NAME, "BogusSession");
+ runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_STS_SIGNER_OVERRIDE, AwsSignerType.CUSTOM_SIGNER.getValue());
+ runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_STS_CUSTOM_SIGNER_CLASS_NAME, CustomSTSSigner.class.getName());
+ runner.assertValid();
+
+ final Map<PropertyDescriptor, String> properties = runner.getProcessContext().getProperties();
+ final CredentialsProviderFactory factory = new CredentialsProviderFactory();
+
+ final AWSCredentialsProvider credentialsProvider = factory.getCredentialsProvider(properties);
+
+ final Field stsClientField = credentialsProvider.getClass().getDeclaredField("securityTokenService");
+ stsClientField.setAccessible(true);
+ AWSSecurityTokenServiceClient stsClient = (AWSSecurityTokenServiceClient) stsClientField.get(credentialsProvider);
+
+ ClientConfiguration stsClientConfig = stsClient.getClientConfiguration();
+
+ final String signerName = stsClientConfig.getSignerOverride();
+ assertNotNull(signerName);
+ final Signer signer = SignerFactory.createSigner(signerName, new SignerParams("sts", "us-west-2"));
+ assertNotNull(signer);
+ assertSame(CustomSTSSigner.class, signer.getClass());
+ }
+
+ public static class CustomSTSSigner extends AWS4Signer {
+
+ }
Review Comment:
The `.setAccessible(true);` call should be avoided as it may even break in newer Java versions (16 and 17 onwards).
We can use a more traditional approach:
```suggestion
@Test
public void testAssumeRoleCredentialsWithCustomSigner() throws Exception {
final TestRunner runner = TestRunners.newTestRunner(MockAWSProcessor.class);
runner.setProperty(CredentialPropertyDescriptors.CREDENTIALS_FILE, "src/test/resources/mock-aws-credentials.properties");
runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_ARN, "BogusArn");
runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_NAME, "BogusSession");
runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_STS_SIGNER_OVERRIDE, AwsSignerType.CUSTOM_SIGNER.getValue());
runner.setProperty(CredentialPropertyDescriptors.ASSUME_ROLE_STS_CUSTOM_SIGNER_CLASS_NAME, CustomSTSSigner.class.getName());
runner.assertValid();
final Map<PropertyDescriptor, String> properties = runner.getProcessContext().getProperties();
final CredentialsProviderFactory factory = new CredentialsProviderFactory();
Signer signerChecker = Mockito.mock(Signer.class);
CustomSTSSigner.setSignerChecker(signerChecker);
final AWSCredentialsProvider credentialsProvider = factory.getCredentialsProvider(properties);
try {
credentialsProvider.getCredentials();
} catch (Exception e) {
// Expected to fail, we are only intersted in the Signer
}
verify(signerChecker).sign(any(), any());
}
public static class CustomSTSSigner extends AWS4Signer {
private static final ThreadLocal<Signer> SIGNER_CHECKER = new ThreadLocal<>();
public static void setSignerChecker(Signer signerChecker) {
SIGNER_CHECKER.set(signerChecker);
}
@Override
public void sign(SignableRequest<?> request, AWSCredentials credentials) {
SIGNER_CHECKER.get().sign(request, credentials);
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on code in PR #6777:
URL: https://github.com/apache/nifi/pull/6777#discussion_r1046061504
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/signer/AwsCustomSignerFactory.java:
##########
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.signer;
+
+import com.amazonaws.auth.Signer;
+
+public interface AwsCustomSignerFactory {
Review Comment:
This interface effectively introduces a new internal extension point, which seems like it could become difficult to maintain.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
URL: https://github.com/apache/nifi/pull/6777
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
Posted by GitBox <gi...@apache.org>.
turcsanyip commented on code in PR #6777:
URL: https://github.com/apache/nifi/pull/6777#discussion_r1046417139
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/signer/AwsCustomSignerFactory.java:
##########
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.signer;
+
+import com.amazonaws.auth.Signer;
+
+public interface AwsCustomSignerFactory {
Review Comment:
@exceptionfactory Thanks for your hint about the `sign()` method parameters!
Request URL is available on `SignableRequest` as you mentioned.
For region and service type, we need to type cast `SignableRequest` to `DefaultRequest`:
```
((DefaultRequest) request).getHandlerContext(HandlerContextKey.SIGNING_REGION)
((DefaultRequest) request).getHandlerContext(HandlerContextKey.SERVICE_ID)
```
As `DefaultRequest` is the only implementation of the interface (available in the dependencies we use), the type cast seems to be safe to use in custom signer implementations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
Posted by GitBox <gi...@apache.org>.
turcsanyip commented on code in PR #6777:
URL: https://github.com/apache/nifi/pull/6777#discussion_r1049004637
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/signer/AwsSignerType.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.signer;
+
+import com.amazonaws.auth.SignerFactory;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.DescribedValue;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public enum AwsSignerType implements DescribedValue {
+
+ // AWS_***_SIGNERs must follow the names in com.amazonaws.auth.SignerFactory and com.amazonaws.services.s3.AmazonS3Client
+ DEFAULT_SIGNER("Default Signature", "Default Signature"),
+ AWS_V4_SIGNER(SignerFactory.VERSION_FOUR_SIGNER, "Signature Version 4"),
+ AWS_S3_V4_SIGNER("AWSS3V4SignerType", "Signature Version 4"), // AmazonS3Client.S3_V4_SIGNER
+ AWS_S3_V2_SIGNER("S3SignerType", "Signature Version 2"), // AmazonS3Client.S3_SIGNER
+ CUSTOM_SIGNER("CustomSignerType", "Custom Signature");
+
+ private static final Map<String, AwsSignerType> LOOKUP_MAP = new HashMap<>();
+
+ static {
+ for (AwsSignerType signerType : AwsSignerType.values()) {
+ LOOKUP_MAP.put(signerType.getValue(), signerType);
+ }
+ }
+
+ private final String value;
+ private final String displayName;
+ private final String description;
+
+ AwsSignerType(String value, String displayName) {
+ this(value, displayName, null);
+ }
+
+ AwsSignerType(String value, String displayName, String description) {
+ this.value = value;
+ this.displayName = displayName;
+ this.description = description;
+ }
+
+ @Override
+ public String getValue() {
+ return value;
+ }
+
+ @Override
+ public String getDisplayName() {
+ return displayName;
+ }
+
+ @Override
+ public String getDescription() {
+ return description;
+ }
+
+ public AllowableValue getAllowableValue() {
+ return new AllowableValue(value, displayName, description);
+ }
Review Comment:
Nope, it is not anymore. Thanks for catching it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
Posted by GitBox <gi...@apache.org>.
turcsanyip commented on code in PR #6777:
URL: https://github.com/apache/nifi/pull/6777#discussion_r1046417139
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/signer/AwsCustomSignerFactory.java:
##########
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.signer;
+
+import com.amazonaws.auth.Signer;
+
+public interface AwsCustomSignerFactory {
Review Comment:
@exceptionfactory Thanks for your hint about the `sign()` method parameters!
Request URL is available on `SignableRequest` as you mentioned.
For region and service type we need to type cast `SignableRequest` to `DefaultRequest`:
```
((DefaultRequest) request).getHandlerContext(HandlerContextKey.SIGNING_REGION)
((DefaultRequest) request).getHandlerContext(HandlerContextKey.SERVICE_ID)
```
As `DefaultRequest` is the only implementation of the interface (available in the dependencies we use), the type cast seems to be safe to use in custom signer implementations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
Posted by GitBox <gi...@apache.org>.
turcsanyip commented on code in PR #6777:
URL: https://github.com/apache/nifi/pull/6777#discussion_r1049009041
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java:
##########
@@ -95,50 +100,29 @@ public boolean proxyVariablesValidForAssumeRole(final Map<PropertyDescriptor, St
@Override
public Collection<ValidationResult> validate(final ValidationContext validationContext,
final CredentialsStrategy primaryStrategy) {
- final boolean assumeRoleArnIsSet = validationContext.getProperty(ASSUME_ROLE_ARN).isSet();
- final boolean assumeRoleNameIsSet = validationContext.getProperty(ASSUME_ROLE_NAME).isSet();
- final Integer maxSessionTime = validationContext.getProperty(MAX_SESSION_TIME).asInteger();
- final boolean assumeRoleExternalIdIsSet = validationContext.getProperty(ASSUME_ROLE_EXTERNAL_ID).isSet();
- final boolean assumeRoleProxyHostIsSet = validationContext.getProperty(ASSUME_ROLE_PROXY_HOST).isSet();
- final boolean assumeRoleProxyPortIsSet = validationContext.getProperty(ASSUME_ROLE_PROXY_PORT).isSet();
- final boolean assumeRoleSTSEndpointIsSet = validationContext.getProperty(ASSUME_ROLE_STS_ENDPOINT).isSet();
-
- final Collection<ValidationResult> validationFailureResults = new ArrayList<ValidationResult>();
-
- // Both role and arn name are req if present
- if (assumeRoleArnIsSet ^ assumeRoleNameIsSet ) {
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role Arn and Name")
- .valid(false).explanation("Assume role requires both arn and name to be set").build());
- }
+ final Collection<ValidationResult> validationFailureResults = new ArrayList<>();
- // Session time only b/w 900 to 3600 sec (see sts session class)
- if ( maxSessionTime < 900 || maxSessionTime > 3600 )
- validationFailureResults.add(new ValidationResult.Builder().valid(false).input(maxSessionTime + "")
- .explanation(MAX_SESSION_TIME.getDisplayName() +
- " must be between 900 and 3600 seconds").build());
-
- // External ID should only be provided with viable Assume Role ARN and Name
- if (assumeRoleExternalIdIsSet && (!assumeRoleArnIsSet || !assumeRoleNameIsSet)) {
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role External ID")
- .valid(false)
- .explanation("Assume role requires both arn and name to be set with External ID")
- .build());
- }
-
- // STS Endpoint should only be provided with viable Assume Role ARN and Name
- if (assumeRoleSTSEndpointIsSet && (!assumeRoleArnIsSet || !assumeRoleNameIsSet)) {
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role STS Endpoint")
- .valid(false)
- .explanation("Assume role requires both arn and name to be set with STS Endpoint")
- .build());
- }
+ final boolean assumeRoleArnIsSet = validationContext.getProperty(ASSUME_ROLE_ARN).isSet();
- // Both proxy host and proxy port are required if present
- if (assumeRoleProxyHostIsSet ^ assumeRoleProxyPortIsSet){
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role Proxy Host and Port")
- .valid(false)
- .explanation("Assume role with proxy requires both host and port for the proxy to be set")
- .build());
+ if (assumeRoleArnIsSet) {
+ final Integer maxSessionTime = validationContext.getProperty(MAX_SESSION_TIME).asInteger();
+
+ // Session time only b/w 900 to 3600 sec (see sts session class)
+ if (maxSessionTime < 900 || maxSessionTime > 3600)
Review Comment:
I think the intent was here to check it at validation time instead of failing at runtime.
The 1 hour max value seems to be a safety approach because the session max time can be configured at role level on AWS side, and it can be 1 to 12 hours (1 is default). So 1 hour works for all settings.
As it is legacy code part and does not affect the current change, I would leave the logic as it is.
I only changed the syntax and the comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
Posted by GitBox <gi...@apache.org>.
turcsanyip commented on code in PR #6777:
URL: https://github.com/apache/nifi/pull/6777#discussion_r1046970850
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/signer/AwsCustomSignerFactory.java:
##########
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.signer;
+
+import com.amazonaws.auth.Signer;
+
+public interface AwsCustomSignerFactory {
Review Comment:
@exceptionfactory Updated the PR.
Found a more straightforward way to access region and service type: `AWS4Signer` implementation of the `Signer` interface has `serviceName` and `regionName` fields and this class may be used as a base class for custom signers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] tpalfy commented on pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
Posted by GitBox <gi...@apache.org>.
tpalfy commented on PR #6777:
URL: https://github.com/apache/nifi/pull/6777#issuecomment-1353476394
LGTM
Thank you for your work @turcsanyip and @exceptionfactory for the review.
Merged to main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6777: NIFI-10969: Created extension point for Signer Override in AWS S3 pro…
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on code in PR #6777:
URL: https://github.com/apache/nifi/pull/6777#discussion_r1047648938
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java:
##########
@@ -95,50 +100,29 @@ public boolean proxyVariablesValidForAssumeRole(final Map<PropertyDescriptor, St
@Override
public Collection<ValidationResult> validate(final ValidationContext validationContext,
final CredentialsStrategy primaryStrategy) {
- final boolean assumeRoleArnIsSet = validationContext.getProperty(ASSUME_ROLE_ARN).isSet();
- final boolean assumeRoleNameIsSet = validationContext.getProperty(ASSUME_ROLE_NAME).isSet();
- final Integer maxSessionTime = validationContext.getProperty(MAX_SESSION_TIME).asInteger();
- final boolean assumeRoleExternalIdIsSet = validationContext.getProperty(ASSUME_ROLE_EXTERNAL_ID).isSet();
- final boolean assumeRoleProxyHostIsSet = validationContext.getProperty(ASSUME_ROLE_PROXY_HOST).isSet();
- final boolean assumeRoleProxyPortIsSet = validationContext.getProperty(ASSUME_ROLE_PROXY_PORT).isSet();
- final boolean assumeRoleSTSEndpointIsSet = validationContext.getProperty(ASSUME_ROLE_STS_ENDPOINT).isSet();
-
- final Collection<ValidationResult> validationFailureResults = new ArrayList<ValidationResult>();
-
- // Both role and arn name are req if present
- if (assumeRoleArnIsSet ^ assumeRoleNameIsSet ) {
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role Arn and Name")
- .valid(false).explanation("Assume role requires both arn and name to be set").build());
- }
+ final Collection<ValidationResult> validationFailureResults = new ArrayList<>();
- // Session time only b/w 900 to 3600 sec (see sts session class)
- if ( maxSessionTime < 900 || maxSessionTime > 3600 )
- validationFailureResults.add(new ValidationResult.Builder().valid(false).input(maxSessionTime + "")
- .explanation(MAX_SESSION_TIME.getDisplayName() +
- " must be between 900 and 3600 seconds").build());
-
- // External ID should only be provided with viable Assume Role ARN and Name
- if (assumeRoleExternalIdIsSet && (!assumeRoleArnIsSet || !assumeRoleNameIsSet)) {
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role External ID")
- .valid(false)
- .explanation("Assume role requires both arn and name to be set with External ID")
- .build());
- }
-
- // STS Endpoint should only be provided with viable Assume Role ARN and Name
- if (assumeRoleSTSEndpointIsSet && (!assumeRoleArnIsSet || !assumeRoleNameIsSet)) {
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role STS Endpoint")
- .valid(false)
- .explanation("Assume role requires both arn and name to be set with STS Endpoint")
- .build());
- }
+ final boolean assumeRoleArnIsSet = validationContext.getProperty(ASSUME_ROLE_ARN).isSet();
- // Both proxy host and proxy port are required if present
- if (assumeRoleProxyHostIsSet ^ assumeRoleProxyPortIsSet){
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role Proxy Host and Port")
- .valid(false)
- .explanation("Assume role with proxy requires both host and port for the proxy to be set")
- .build());
+ if (assumeRoleArnIsSet) {
+ final Integer maxSessionTime = validationContext.getProperty(MAX_SESSION_TIME).asInteger();
+
+ // Session time only b/w 900 to 3600 sec (see sts session class)
+ if (maxSessionTime < 900 || maxSessionTime > 3600)
Review Comment:
Opening and closing braces should be used:
```suggestion
if (maxSessionTime < 900 || maxSessionTime > 3600) {
```
Is it necessary to implement this check if the logic is already implemented in AWS code?
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/strategies/AssumeRoleCredentialsStrategy.java:
##########
@@ -95,50 +100,29 @@ public boolean proxyVariablesValidForAssumeRole(final Map<PropertyDescriptor, St
@Override
public Collection<ValidationResult> validate(final ValidationContext validationContext,
final CredentialsStrategy primaryStrategy) {
- final boolean assumeRoleArnIsSet = validationContext.getProperty(ASSUME_ROLE_ARN).isSet();
- final boolean assumeRoleNameIsSet = validationContext.getProperty(ASSUME_ROLE_NAME).isSet();
- final Integer maxSessionTime = validationContext.getProperty(MAX_SESSION_TIME).asInteger();
- final boolean assumeRoleExternalIdIsSet = validationContext.getProperty(ASSUME_ROLE_EXTERNAL_ID).isSet();
- final boolean assumeRoleProxyHostIsSet = validationContext.getProperty(ASSUME_ROLE_PROXY_HOST).isSet();
- final boolean assumeRoleProxyPortIsSet = validationContext.getProperty(ASSUME_ROLE_PROXY_PORT).isSet();
- final boolean assumeRoleSTSEndpointIsSet = validationContext.getProperty(ASSUME_ROLE_STS_ENDPOINT).isSet();
-
- final Collection<ValidationResult> validationFailureResults = new ArrayList<ValidationResult>();
-
- // Both role and arn name are req if present
- if (assumeRoleArnIsSet ^ assumeRoleNameIsSet ) {
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role Arn and Name")
- .valid(false).explanation("Assume role requires both arn and name to be set").build());
- }
+ final Collection<ValidationResult> validationFailureResults = new ArrayList<>();
- // Session time only b/w 900 to 3600 sec (see sts session class)
- if ( maxSessionTime < 900 || maxSessionTime > 3600 )
- validationFailureResults.add(new ValidationResult.Builder().valid(false).input(maxSessionTime + "")
- .explanation(MAX_SESSION_TIME.getDisplayName() +
- " must be between 900 and 3600 seconds").build());
-
- // External ID should only be provided with viable Assume Role ARN and Name
- if (assumeRoleExternalIdIsSet && (!assumeRoleArnIsSet || !assumeRoleNameIsSet)) {
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role External ID")
- .valid(false)
- .explanation("Assume role requires both arn and name to be set with External ID")
- .build());
- }
-
- // STS Endpoint should only be provided with viable Assume Role ARN and Name
- if (assumeRoleSTSEndpointIsSet && (!assumeRoleArnIsSet || !assumeRoleNameIsSet)) {
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role STS Endpoint")
- .valid(false)
- .explanation("Assume role requires both arn and name to be set with STS Endpoint")
- .build());
- }
+ final boolean assumeRoleArnIsSet = validationContext.getProperty(ASSUME_ROLE_ARN).isSet();
- // Both proxy host and proxy port are required if present
- if (assumeRoleProxyHostIsSet ^ assumeRoleProxyPortIsSet){
- validationFailureResults.add(new ValidationResult.Builder().input("Assume Role Proxy Host and Port")
- .valid(false)
- .explanation("Assume role with proxy requires both host and port for the proxy to be set")
- .build());
+ if (assumeRoleArnIsSet) {
+ final Integer maxSessionTime = validationContext.getProperty(MAX_SESSION_TIME).asInteger();
+
+ // Session time only b/w 900 to 3600 sec (see sts session class)
Review Comment:
Recommend providing the STS Session class name instead of just `sts session class` for clearer reference point.
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/s3/PutS3Object.java:
##########
@@ -279,9 +279,9 @@ public class PutS3Object extends AbstractS3Processor {
public static final List<PropertyDescriptor> properties = Collections.unmodifiableList(
Arrays.asList(KEY, BUCKET, CONTENT_TYPE, CONTENT_DISPOSITION, CACHE_CONTROL, ACCESS_KEY, SECRET_KEY, CREDENTIALS_FILE, AWS_CREDENTIALS_PROVIDER_SERVICE,
OBJECT_TAGS_PREFIX, REMOVE_TAG_PREFIX, STORAGE_CLASS, REGION, TIMEOUT, EXPIRATION_RULE_ID, FULL_CONTROL_USER_LIST, READ_USER_LIST, WRITE_USER_LIST,
- READ_ACL_LIST, WRITE_ACL_LIST, OWNER, CANNED_ACL, SSL_CONTEXT_SERVICE, ENDPOINT_OVERRIDE, SIGNER_OVERRIDE, MULTIPART_THRESHOLD, MULTIPART_PART_SIZE,
- MULTIPART_S3_AGEOFF_INTERVAL, MULTIPART_S3_MAX_AGE, MULTIPART_TEMP_DIR, SERVER_SIDE_ENCRYPTION, ENCRYPTION_SERVICE, USE_CHUNKED_ENCODING,
- USE_PATH_STYLE_ACCESS, PROXY_CONFIGURATION_SERVICE, PROXY_HOST, PROXY_HOST_PORT, PROXY_USERNAME, PROXY_PASSWORD));
+ READ_ACL_LIST, WRITE_ACL_LIST, OWNER, CANNED_ACL, SSL_CONTEXT_SERVICE, ENDPOINT_OVERRIDE, SIGNER_OVERRIDE, S3_CUSTOM_SIGNER_CLASS_NAME, S3_CUSTOM_SIGNER_MODULE_LOCATION,
+ MULTIPART_THRESHOLD, MULTIPART_PART_SIZE, MULTIPART_S3_AGEOFF_INTERVAL, MULTIPART_S3_MAX_AGE, MULTIPART_TEMP_DIR, SERVER_SIDE_ENCRYPTION, ENCRYPTION_SERVICE,
+ USE_CHUNKED_ENCODING, USE_PATH_STYLE_ACCESS, PROXY_CONFIGURATION_SERVICE, PROXY_HOST, PROXY_HOST_PORT, PROXY_USERNAME, PROXY_PASSWORD));
Review Comment:
This would be a good opportunity to restructure the code to have one item per line.
##########
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/signer/AwsSignerType.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.signer;
+
+import com.amazonaws.auth.SignerFactory;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.DescribedValue;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public enum AwsSignerType implements DescribedValue {
+
+ // AWS_***_SIGNERs must follow the names in com.amazonaws.auth.SignerFactory and com.amazonaws.services.s3.AmazonS3Client
+ DEFAULT_SIGNER("Default Signature", "Default Signature"),
+ AWS_V4_SIGNER(SignerFactory.VERSION_FOUR_SIGNER, "Signature Version 4"),
+ AWS_S3_V4_SIGNER("AWSS3V4SignerType", "Signature Version 4"), // AmazonS3Client.S3_V4_SIGNER
+ AWS_S3_V2_SIGNER("S3SignerType", "Signature Version 2"), // AmazonS3Client.S3_SIGNER
+ CUSTOM_SIGNER("CustomSignerType", "Custom Signature");
+
+ private static final Map<String, AwsSignerType> LOOKUP_MAP = new HashMap<>();
+
+ static {
+ for (AwsSignerType signerType : AwsSignerType.values()) {
+ LOOKUP_MAP.put(signerType.getValue(), signerType);
+ }
+ }
+
+ private final String value;
+ private final String displayName;
+ private final String description;
+
+ AwsSignerType(String value, String displayName) {
+ this(value, displayName, null);
+ }
+
+ AwsSignerType(String value, String displayName, String description) {
+ this.value = value;
+ this.displayName = displayName;
+ this.description = description;
+ }
+
+ @Override
+ public String getValue() {
+ return value;
+ }
+
+ @Override
+ public String getDisplayName() {
+ return displayName;
+ }
+
+ @Override
+ public String getDescription() {
+ return description;
+ }
+
+ public AllowableValue getAllowableValue() {
+ return new AllowableValue(value, displayName, description);
+ }
Review Comment:
Is this method necessary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org