[jira] [Created] (SPARK-57891) Add CredentialProvider SPI and ServiceLoader discovery

["Kousuke Saruta (Jira)" <ji...@apache.org>]

Kousuke Saruta created SPARK-57891:
--------------------------------------

             Summary: Add CredentialProvider SPI and ServiceLoader discovery
                 Key: SPARK-57891
                 URL: https://issues.apache.org/jira/browse/SPARK-57891
             Project: Spark
          Issue Type: Sub-task
          Components: Spark Core
    Affects Versions: 4.3.0
            Reporter: Kousuke Saruta


This sub-task is part of the umbrella [SPARK-XXXXX|https://issues.apache.org/jira/browse/SPARK-XXXXX] (OIDC Credential Pro
pagation).

h2. Problem

There is no pluggable SPI for exchanging an identity token for short-lived service credentials.

h2. Goal

Introduce a {{CredentialProvider}} SPI that providers (AWS STS, Azure, GCP, etc.) can implement to resolve credentials from a user identity.

h2. Scope

* Define {{CredentialProvider}} interface in {{org.apache.spark.security}}:
{code:java}
@DeveloperApi
public interface CredentialProvider {
    void init(Map<String, String> conf);
    Set<String> supportedSchemes();
    ServiceCredential resolve(UserContext user, URI target)
        throws CredentialResolutionException;
    default Duration suggestedTtl() { return Duration.ofMinutes(15); }
}
{code}
* {{CredentialProviderLoader}}: discovers providers via {{ServiceLoader}}, resolves provider for a scheme based on {{spark.security.credentials.provider.<scheme>}}
* {{FakeCredentialProvider}} under {{core/src/test}} for testing

h2. Acceptance criteria

* {{CredentialProviderLoader}} discovers {{FakeCredentialProvider}} via ServiceLoader.
* Correct provider is selected based on scheme configuration.
* Multiple providers for the same scheme are handled (configuration-based priority).
* Unit tests for discovery, scheme resolution, init, and resolve.

Reference: [SPIP design document Appendix A|https://docs.google.com/document/d/1usJKncCPMiyFUg7aIdpZ0HQsklXIHow_sU_6dfFMjN0/edit].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org