[jira] [Assigned] (SPARK-57899) Implement SparkOidcAwsCredentialsProvider

["Kousuke Saruta (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-57899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kousuke Saruta reassigned SPARK-57899:
--------------------------------------

    Assignee: Shrirang Mhalgi

> Implement SparkOidcAwsCredentialsProvider
> -----------------------------------------
>
>                 Key: SPARK-57899
>                 URL: https://issues.apache.org/jira/browse/SPARK-57899
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Spark Core
>    Affects Versions: 4.3.0
>            Reporter: Kousuke Saruta
>            Assignee: Shrirang Mhalgi
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.4.0
>
>
> This sub-task is part of the umbrella [SPARK-57703|https://issues.apache.org/jira/browse/SPARK-57703] (OIDC Credential Propagation).
> h2. Problem
> Executor-side S3A FileSystem needs to pick up refreshed credentials from the credential store without requiring FileSystem cache invalidation.
> h2. Goal
> Implement a dynamic AWS credentials provider that reads from the executor credential store on every call.
> h2. Scope
> * Implement {{software.amazon.awssdk.auth.credentials.AwsCredentialsProvider}} in {{connector/credential-aws}}
> * On each {{resolveCredentials()}} call, read current {{ServiceCredential}} from executor credential store
> * Never cache credentials internally (always return latest from store)
> * Auto-enabled when {{spark.security.credentials.enabled=true}} and user has not explicitly configured {{fs.s3a.aws.credentials.provider}}
> h2. Acceptance criteria
> * After credential refresh (store update), the next {{resolveCredentials()}} call returns new credentials.
> * No stale credentials are returned after a refresh.
> * User-provided {{fs.s3a.aws.credentials.provider}} is not overwritten.
> * Unit tests verify dynamic behavior with simulated credential store updates.
> Reference: [SPIP design document Appendix B|https://docs.google.com/document/d/1usJKncCPMiyFUg7aIdpZ0HQsklXIHow_sU_6dfFMjN0/edit].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org