[GitHub] [beam] mosche commented on issue #23206: [Bug]: KinesisIO incorrectly selects Aws Credentials provider and crashes with NullPointerException on resource not found errors

[GitBox <gi...@apache.org>]

mosche commented on issue #23206:
URL: https://github.com/apache/beam/issues/23206#issuecomment-1248296235

   @oakad I acknowledge this is confusing and by far not optimal... `ProfileCredentialsProvider` allows for additional configuration options. However, these are currently not supported as documented in the Javadocs of `AwsOptions`.
   
   The issue is that credential providers have to be serialized, and there's no decent generic way to do that :/
   Unfortunately same applies to verifying the configuration of credentials providers if provided programmatically.
   But the latter could certainly be improved a lot and catch more such issues upfront!
   
   Would you be willing to submit a PR that adds support for a custom profile name?
   
   From the [Javadocs]:(https://beam.apache.org/releases/javadoc/2.41.0/org/apache/beam/sdk/io/aws2/options/AwsOptions.html#setAwsCredentialsProvider-software.amazon.awssdk.auth.credentials.AwsCredentialsProvider-):
   
   The class name of the provider must be set in the @type field. 
   
   Note: Not all available providers are supported and some configuration options might be ignored.
   
   Most providers rely on the system's environment to follow AWS conventions, there's no further configuration supported:
   
   - [DefaultCredentialsProvider](https://static.javadoc.io/software.amazon.awssdk/auth/2.17.127/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html?is-external=true)
   - [EnvironmentVariableCredentialsProvider](https://static.javadoc.io/software.amazon.awssdk/auth/2.17.127/software/amazon/awssdk/auth/credentials/EnvironmentVariableCredentialsProvider.html?is-external=true)
   - [SystemPropertyCredentialsProvider](https://static.javadoc.io/software.amazon.awssdk/auth/2.17.127/software/amazon/awssdk/auth/credentials/SystemPropertyCredentialsProvider.html?is-external=true)
   - [ProfileCredentialsProvider](https://static.javadoc.io/software.amazon.awssdk/auth/2.17.127/software/amazon/awssdk/auth/credentials/ProfileCredentialsProvider.html?is-external=true)
   - [ContainerCredentialsProvider](https://static.javadoc.io/software.amazon.awssdk/auth/2.17.127/software/amazon/awssdk/auth/credentials/ContainerCredentialsProvider.html?is-external=true)
     Example:
      ```
       --awsCredentialsProvider={"@type": "ProfileCredentialsProvider"}
      ```
   
   Some other providers require additional configuration:
   
   - [StaticCredentialsProvider](https://static.javadoc.io/software.amazon.awssdk/auth/2.17.127/software/amazon/awssdk/auth/credentials/StaticCredentialsProvider.html?is-external=true)
   - [StsAssumeRoleCredentialsProvider](https://static.javadoc.io/software.amazon.awssdk/sts/2.17.127/software/amazon/awssdk/services/sts/auth/StsAssumeRoleCredentialsProvider.html?is-external=true)
      Examples:
      ```
       --awsCredentialsProvider={
         "@type": "StaticCredentialsProvider",
         "awsAccessKeyId": "key_id_value",
         "awsSecretKey": "secret_value"
       }
      ```
      ```
       --awsCredentialsProvider={
         "@type": "StaticCredentialsProvider",
         "awsAccessKeyId": "key_id_value",
         "awsSecretKey": "secret_value",
         "sessionToken": "token_value"
       }
      ```
      ```
       --awsCredentialsProvider={
         "@type": "StsAssumeRoleCredentialsProvider",
         "roleArn": "role_arn_Value",
         "roleSessionName": "session_name_value",
         "policy": "policy_value",
         "durationSeconds": 3600
       }
      ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org