You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "XJDKC (via GitHub)" <gi...@apache.org> on 2025/10/09 01:07:07 UTC
[PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
XJDKC opened a new pull request, #2782:
URL: https://github.com/apache/polaris/pull/2782
<!--
Possible security vulnerabilities: STOP here and contact security@apache.org instead!
Please update the title of the PR with a meaningful message - do not leave it "empty" or "generated"
Please update this summary field:
The summary should cover these topics, if applicable:
* the motivation for the change
* a description of the status quo, for example the current behavior
* the desired behavior
* etc
PR checklist:
- Do a self-review of your code before opening a pull request
- Make sure that there's good test coverage for the changes included in this PR
- Run tests locally before pushing a PR (./gradlew check)
- Code should have comments where applicable. Particularly hard-to-understand
areas deserve good in-line documentation.
- Include changes and enhancements to the documentation (in site/content/in-dev/unreleased)
- For Work In Progress Pull Requests, please use the Draft PR feature.
Make sure to add the information BELOW this comment.
Everything in this comment will NOT be added to the PR description.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422241467
##########
runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java:
##########
@@ -335,4 +337,64 @@ public ProductionReadinessCheck checkOverlappingSiblingCheckSettings(
? ProductionReadinessCheck.OK
: ProductionReadinessCheck.of(errors.toArray(new Error[0]));
}
+
+ @Produces
+ @SuppressWarnings("unchecked")
+ public ProductionReadinessCheck checkConnectionCredentialVendors(
+ Instance<ConnectionCredentialVendor> credentialVendors,
+ FeaturesConfiguration featureConfiguration) {
+ var mapper = new ObjectMapper();
+ var defaults = featureConfiguration.parseDefaults(mapper);
+ var realmOverrides = featureConfiguration.parseRealmOverrides(mapper);
+
+ var federationKey = FeatureConfiguration.ENABLE_CATALOG_FEDERATION.key();
+ var authTypesKey = FeatureConfiguration.SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES.key();
+ var federationEnabled =
+ Boolean.parseBoolean(defaults.getOrDefault(federationKey, false).toString());
+ var defaultAuthTypes = (List<String>) defaults.getOrDefault(authTypesKey, List.of());
+
+ var allAuthTypes = new java.util.HashSet<String>();
+ if (federationEnabled) allAuthTypes.addAll(defaultAuthTypes);
+
+ realmOverrides.forEach(
+ (id, overrides) -> {
+ if (Boolean.parseBoolean(
+ overrides.getOrDefault(federationKey, federationEnabled).toString())) {
+ allAuthTypes.addAll(
+ (List<String>)
+ (overrides.containsKey(authTypesKey)
+ ? overrides.get(authTypesKey)
+ : defaultAuthTypes));
+ }
+ });
+
+ var errors = new ArrayList<Error>();
+ for (var name : allAuthTypes) {
+ try {
+ var type = org.apache.polaris.core.connection.AuthenticationType.valueOf(name);
+ if (credentialVendors.select(AuthType.Literal.of(type)).isUnsatisfied()) {
+ errors.add(
+ Error.of(
+ format(
+ "Catalog federation is enabled but no ConnectionCredentialVendor found for "
+ + "authentication type '%s'. External catalog connections using this "
+ + "authentication type will fail.",
+ type),
Review Comment:
Yeah, naming needs improvement here. An `error` production readiness check will not block startup. Only `severe` case break startup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2427414222
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/OAuthConnectionCredentialVendor.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Joiner;
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import java.time.Instant;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.connection.OAuthClientCredentialsParametersDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for OAuth 2.0 Client Credentials authentication.
+ *
+ * <p>This vendor handles OAuth 2.0 client credentials flow by reading the client secret from the
+ * secrets manager and formatting it as an OAuth credential for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the OAuth credential (formatted as "clientId:clientSecret"). When
+ * connecting to the remote catalog, Iceberg SDK will use this credential to fetch OAuth tokens
+ * automatically.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.OAUTH)
+@Priority(100)
+public class OAuthConnectionCredentialVendor implements ConnectionCredentialVendor {
+
+ private static final Joiner COLON_JOINER = Joiner.on(":");
+
+ private final UserSecretsManager secretsManager;
+
+ @Inject
+ public OAuthConnectionCredentialVendor(UserSecretsManager secretsManager) {
+ this.secretsManager = secretsManager;
+ }
+
+ @Override
+ public @Nonnull ConnectionCredentials getConnectionCredentials(
+ @Nonnull ConnectionConfigInfoDpo connectionConfig) {
+
+ // Validate authentication parameters type
+ Preconditions.checkArgument(
+ connectionConfig.getAuthenticationParameters()
+ instanceof OAuthClientCredentialsParametersDpo,
+ "Expected OAuthClientCredentialsParametersDpo, got: %s",
+ connectionConfig.getAuthenticationParameters().getClass().getName());
+
+ OAuthClientCredentialsParametersDpo oauthParams =
+ (OAuthClientCredentialsParametersDpo) connectionConfig.getAuthenticationParameters();
+
+ // Read the client secret from secrets manager
+ String clientSecret = secretsManager.readSecret(oauthParams.getClientSecretReference());
+
+ // Format credential as "clientId:clientSecret"
+ String credential = COLON_JOINER.join(oauthParams.getClientId(), clientSecret);
+
+ // Return the OAuth credential
+ // OAuth credentials don't expire - set expiration to Instant.MAX to indicate infinite validity
+ // If the credential expires, users need to update the catalog entity to rotate the credential.
+ return ConnectionCredentials.builder()
+ .putCredential(CatalogAccessProperty.OAUTH2_CREDENTIAL.getPropertyName(), credential)
Review Comment:
Renamed `OAuthConnectionCredentialVendor` to `OAuthClientCredentialVendor`!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2426859509
##########
polaris-core/src/main/java/org/apache/polaris/core/connection/BearerAuthenticationParametersDpo.java:
##########
@@ -51,9 +49,10 @@ public BearerAuthenticationParametersDpo(
@Override
public @Nonnull Map<String, String> asIcebergCatalogProperties(
- UserSecretsManager secretsManager, PolarisCredentialManager credentialManager) {
- String bearerToken = secretsManager.readSecret(getBearerTokenReference());
- return Map.of(OAuth2Properties.TOKEN, bearerToken);
+ PolarisCredentialManager credentialManager) {
+ // Return only metadata properties - credentials are handled by ConnectionCredentialVendor
+ // Bearer auth has no metadata properties
+ return Map.of();
Review Comment:
Sure!
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/BearerConnectionCredentialVendor.java:
##########
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.BearerAuthenticationParametersDpo;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for Bearer token authentication.
+ *
+ * <p>This vendor handles Bearer token authentication by reading the bearer token from the secrets
+ * manager and providing it for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the bearer token. When connecting to the remote catalog, Iceberg SDK
+ * will use this token as-is in HTTP Authorization headers. Bearer tokens typically have a limited
+ * lifetime and should be refreshed by the user when they expire.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.BEARER)
+@Priority(100)
Review Comment:
Makes sense!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2421965864
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/OAuthConnectionCredentialVendor.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Joiner;
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import java.time.Instant;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.connection.OAuthClientCredentialsParametersDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for OAuth 2.0 Client Credentials authentication.
+ *
+ * <p>This vendor handles OAuth 2.0 client credentials flow by reading the client secret from the
+ * secrets manager and formatting it as an OAuth credential for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the OAuth credential (formatted as "clientId:clientSecret"). When
+ * connecting to the remote catalog, Iceberg SDK will use this credential to fetch OAuth tokens
+ * automatically.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.OAUTH)
+@Priority(100)
+public class OAuthConnectionCredentialVendor implements ConnectionCredentialVendor {
+
+ private static final Joiner COLON_JOINER = Joiner.on(":");
+
+ private final UserSecretsManager secretsManager;
+
+ @Inject
+ public OAuthConnectionCredentialVendor(UserSecretsManager secretsManager) {
+ this.secretsManager = secretsManager;
+ }
+
+ @Override
+ public @Nonnull ConnectionCredentials getConnectionCredentials(
+ @Nonnull ConnectionConfigInfoDpo connectionConfig) {
+
+ // Validate authentication parameters type
+ Preconditions.checkArgument(
+ connectionConfig.getAuthenticationParameters()
+ instanceof OAuthClientCredentialsParametersDpo,
+ "Expected OAuthClientCredentialsParametersDpo, got: %s",
+ connectionConfig.getAuthenticationParameters().getClass().getName());
+
+ OAuthClientCredentialsParametersDpo oauthParams =
+ (OAuthClientCredentialsParametersDpo) connectionConfig.getAuthenticationParameters();
+
+ // Read the client secret from secrets manager
+ String clientSecret = secretsManager.readSecret(oauthParams.getClientSecretReference());
+
+ // Format credential as "clientId:clientSecret"
+ String credential = COLON_JOINER.join(oauthParams.getClientId(), clientSecret);
+
+ // Return the OAuth credential
+ // OAuth credentials don't expire - set expiration to Instant.MAX to indicate infinite validity
+ // If the credential expires, users need to update the catalog entity to rotate the credential.
+ return ConnectionCredentials.builder()
+ .putCredential(CatalogAccessProperty.OAUTH2_CREDENTIAL.getPropertyName(), credential)
Review Comment:
Yes, you're right, this class currently just retrieves the OAuth credential from `UserSecretsManager` and returns it. The `PolarisCredentialManager` will then cache it in `AccessCredsCache` (not yet implemented, I plan to add that in a follow-up PR) so that we don't need to fetch the credential from `UserSecretsManager` every time we make a call, especially if the secret manager doesn't have its own cache.
The main goal of this PR is to make `PolarisCredentialManager` the single entry point for credential retrieval so we can centralize the caching logic in one place.
Regarding your suggestion, I agree, it's still beneficial to include OAuth in the class name to make the purpose of this class clear at a glance.
Right now, the class only returns the stored OAuth credential, but in the future, we could extend it to **perform the client credentials flow to obtain a temporary token directly here**, rather than relying on the Iceberg SDK to fetch a new token for every request.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422241467
##########
runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java:
##########
@@ -335,4 +337,64 @@ public ProductionReadinessCheck checkOverlappingSiblingCheckSettings(
? ProductionReadinessCheck.OK
: ProductionReadinessCheck.of(errors.toArray(new Error[0]));
}
+
+ @Produces
+ @SuppressWarnings("unchecked")
+ public ProductionReadinessCheck checkConnectionCredentialVendors(
+ Instance<ConnectionCredentialVendor> credentialVendors,
+ FeaturesConfiguration featureConfiguration) {
+ var mapper = new ObjectMapper();
+ var defaults = featureConfiguration.parseDefaults(mapper);
+ var realmOverrides = featureConfiguration.parseRealmOverrides(mapper);
+
+ var federationKey = FeatureConfiguration.ENABLE_CATALOG_FEDERATION.key();
+ var authTypesKey = FeatureConfiguration.SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES.key();
+ var federationEnabled =
+ Boolean.parseBoolean(defaults.getOrDefault(federationKey, false).toString());
+ var defaultAuthTypes = (List<String>) defaults.getOrDefault(authTypesKey, List.of());
+
+ var allAuthTypes = new java.util.HashSet<String>();
+ if (federationEnabled) allAuthTypes.addAll(defaultAuthTypes);
+
+ realmOverrides.forEach(
+ (id, overrides) -> {
+ if (Boolean.parseBoolean(
+ overrides.getOrDefault(federationKey, federationEnabled).toString())) {
+ allAuthTypes.addAll(
+ (List<String>)
+ (overrides.containsKey(authTypesKey)
+ ? overrides.get(authTypesKey)
+ : defaultAuthTypes));
+ }
+ });
+
+ var errors = new ArrayList<Error>();
+ for (var name : allAuthTypes) {
+ try {
+ var type = org.apache.polaris.core.connection.AuthenticationType.valueOf(name);
+ if (credentialVendors.select(AuthType.Literal.of(type)).isUnsatisfied()) {
+ errors.add(
+ Error.of(
+ format(
+ "Catalog federation is enabled but no ConnectionCredentialVendor found for "
+ + "authentication type '%s'. External catalog connections using this "
+ + "authentication type will fail.",
+ type),
Review Comment:
Yeah, naming needs improvement here. An `error` production readiness check will not block startup. Only `severe` cases break startup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422234071
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/SigV4ConnectionCredentialVendor.java:
##########
@@ -58,7 +58,7 @@
* <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
* override this by providing a higher priority value.
*/
-@ApplicationScoped
+@RequestScoped
Review Comment:
It's `DefaultServiceIdentityProvider`, I think the only related class is this
Specifically, a `ApplicationScoped` `SigV4ConnectionCredentialVendor` refers to a `RequestScoped` `DefaultServiceIdentityProvider`.
https://github.com/apache/polaris/blob/main/runtime/service/src/main/java/org/apache/polaris/service/identity/provider/DefaultServiceIdentityProvider.java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2421965864
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/OAuthConnectionCredentialVendor.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Joiner;
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import java.time.Instant;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.connection.OAuthClientCredentialsParametersDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for OAuth 2.0 Client Credentials authentication.
+ *
+ * <p>This vendor handles OAuth 2.0 client credentials flow by reading the client secret from the
+ * secrets manager and formatting it as an OAuth credential for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the OAuth credential (formatted as "clientId:clientSecret"). When
+ * connecting to the remote catalog, Iceberg SDK will use this credential to fetch OAuth tokens
+ * automatically.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.OAUTH)
+@Priority(100)
+public class OAuthConnectionCredentialVendor implements ConnectionCredentialVendor {
+
+ private static final Joiner COLON_JOINER = Joiner.on(":");
+
+ private final UserSecretsManager secretsManager;
+
+ @Inject
+ public OAuthConnectionCredentialVendor(UserSecretsManager secretsManager) {
+ this.secretsManager = secretsManager;
+ }
+
+ @Override
+ public @Nonnull ConnectionCredentials getConnectionCredentials(
+ @Nonnull ConnectionConfigInfoDpo connectionConfig) {
+
+ // Validate authentication parameters type
+ Preconditions.checkArgument(
+ connectionConfig.getAuthenticationParameters()
+ instanceof OAuthClientCredentialsParametersDpo,
+ "Expected OAuthClientCredentialsParametersDpo, got: %s",
+ connectionConfig.getAuthenticationParameters().getClass().getName());
+
+ OAuthClientCredentialsParametersDpo oauthParams =
+ (OAuthClientCredentialsParametersDpo) connectionConfig.getAuthenticationParameters();
+
+ // Read the client secret from secrets manager
+ String clientSecret = secretsManager.readSecret(oauthParams.getClientSecretReference());
+
+ // Format credential as "clientId:clientSecret"
+ String credential = COLON_JOINER.join(oauthParams.getClientId(), clientSecret);
+
+ // Return the OAuth credential
+ // OAuth credentials don't expire - set expiration to Instant.MAX to indicate infinite validity
+ // If the credential expires, users need to update the catalog entity to rotate the credential.
+ return ConnectionCredentials.builder()
+ .putCredential(CatalogAccessProperty.OAUTH2_CREDENTIAL.getPropertyName(), credential)
Review Comment:
Yes, you're right, this class currently just retrieves the OAuth credential from `UserSecretsManager` and returns it. The `PolarisCredentialManager` will then cache it in `AccessCredsCache` (not yet implemented, I plan to add that in a follow-up PR) so that we don't need to fetch the credential from `UserSecretsManager` every time we make a call, especially if the secret manager doesn't have its own cache.
The main goal of this PR is to make `PolarisCredentialManager` the single entry point for credential retrieval so we can centralize the caching logic in one place.
Regarding your suggestion, I agree, it's still beneficial to include OAuth in the class name to make the purpose of this class clear at a glance.
Right now, the class only returns the stored OAuth credential, but in the future, we could extend it to perform the client credentials flow to obtain a temporary token directly here, rather than relying on the Iceberg SDK to fetch a new token for every request.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422182379
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/SigV4ConnectionCredentialVendor.java:
##########
@@ -58,7 +58,7 @@
* <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
* override this by providing a higher priority value.
*/
-@ApplicationScoped
+@RequestScoped
Review Comment:
`DefaultServiceIdentityRegistry` does not exist in code? :thinking: Did you mean `DefaultServiceIdentityProvider`?
Referencing an application-scoped object from a request scope is usually ok :thinking:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422244649
##########
runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java:
##########
@@ -335,4 +337,64 @@ public ProductionReadinessCheck checkOverlappingSiblingCheckSettings(
? ProductionReadinessCheck.OK
: ProductionReadinessCheck.of(errors.toArray(new Error[0]));
}
+
+ @Produces
+ @SuppressWarnings("unchecked")
+ public ProductionReadinessCheck checkConnectionCredentialVendors(
+ Instance<ConnectionCredentialVendor> credentialVendors,
+ FeaturesConfiguration featureConfiguration) {
+ var mapper = new ObjectMapper();
+ var defaults = featureConfiguration.parseDefaults(mapper);
+ var realmOverrides = featureConfiguration.parseRealmOverrides(mapper);
+
+ var federationKey = FeatureConfiguration.ENABLE_CATALOG_FEDERATION.key();
+ var authTypesKey = FeatureConfiguration.SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES.key();
+ var federationEnabled =
+ Boolean.parseBoolean(defaults.getOrDefault(federationKey, false).toString());
+ var defaultAuthTypes = (List<String>) defaults.getOrDefault(authTypesKey, List.of());
+
+ var allAuthTypes = new java.util.HashSet<String>();
+ if (federationEnabled) allAuthTypes.addAll(defaultAuthTypes);
+
+ realmOverrides.forEach(
+ (id, overrides) -> {
+ if (Boolean.parseBoolean(
+ overrides.getOrDefault(federationKey, federationEnabled).toString())) {
+ allAuthTypes.addAll(
+ (List<String>)
+ (overrides.containsKey(authTypesKey)
+ ? overrides.get(authTypesKey)
+ : defaultAuthTypes));
+ }
+ });
+
+ var errors = new ArrayList<Error>();
+ for (var name : allAuthTypes) {
+ try {
+ var type = org.apache.polaris.core.connection.AuthenticationType.valueOf(name);
+ if (credentialVendors.select(AuthType.Literal.of(type)).isUnsatisfied()) {
+ errors.add(
+ Error.of(
+ format(
+ "Catalog federation is enabled but no ConnectionCredentialVendor found for "
+ + "authentication type '%s'. External catalog connections using this "
+ + "authentication type will fail.",
+ type),
Review Comment:
I'm not sure the check is logically correct. A particular deployment only needs to provide `ConnectionCredentialVendor` implementations for auth types that are actually used by created catalogs.
I think the vendor availability is implicitly checked on catalog creation already.
On startup, I think we only need to check types for existing catalogs, not for all types listed in the enum. The latter would put an undue burden for custom deployments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2426858961
##########
runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java:
##########
@@ -335,4 +337,64 @@ public ProductionReadinessCheck checkOverlappingSiblingCheckSettings(
? ProductionReadinessCheck.OK
: ProductionReadinessCheck.of(errors.toArray(new Error[0]));
}
+
+ @Produces
+ @SuppressWarnings("unchecked")
+ public ProductionReadinessCheck checkConnectionCredentialVendors(
+ Instance<ConnectionCredentialVendor> credentialVendors,
+ FeaturesConfiguration featureConfiguration) {
+ var mapper = new ObjectMapper();
+ var defaults = featureConfiguration.parseDefaults(mapper);
+ var realmOverrides = featureConfiguration.parseRealmOverrides(mapper);
+
+ var federationKey = FeatureConfiguration.ENABLE_CATALOG_FEDERATION.key();
+ var authTypesKey = FeatureConfiguration.SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES.key();
+ var federationEnabled =
+ Boolean.parseBoolean(defaults.getOrDefault(federationKey, false).toString());
+ var defaultAuthTypes = (List<String>) defaults.getOrDefault(authTypesKey, List.of());
+
+ var allAuthTypes = new java.util.HashSet<String>();
+ if (federationEnabled) allAuthTypes.addAll(defaultAuthTypes);
+
+ realmOverrides.forEach(
+ (id, overrides) -> {
+ if (Boolean.parseBoolean(
+ overrides.getOrDefault(federationKey, federationEnabled).toString())) {
+ allAuthTypes.addAll(
+ (List<String>)
+ (overrides.containsKey(authTypesKey)
+ ? overrides.get(authTypesKey)
+ : defaultAuthTypes));
+ }
+ });
+
+ var errors = new ArrayList<Error>();
+ for (var name : allAuthTypes) {
+ try {
+ var type = org.apache.polaris.core.connection.AuthenticationType.valueOf(name);
+ if (credentialVendors.select(AuthType.Literal.of(type)).isUnsatisfied()) {
+ errors.add(
+ Error.of(
+ format(
+ "Catalog federation is enabled but no ConnectionCredentialVendor found for "
+ + "authentication type '%s'. External catalog connections using this "
+ + "authentication type will fail.",
+ type),
Review Comment:
I see, I will switch to `severe`.
> A particular deployment only needs to provide ConnectionCredentialVendor implementations for auth types that are actually used by created catalogs. I think we only need to check types for existing catalogs, not for all types listed in the enum
The logic is: if we enable the catalog federation then all the enabled auth types must have a corresponding `ConnectionCredentialVendor`. IIRC, we don't want to defer the check during the runtime based on the feedback in the previous PR.
If vendors only implement one / two auth types for catalog federation, they need to only enable them explicitly, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422195202
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/OAuthConnectionCredentialVendor.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Joiner;
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import java.time.Instant;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.connection.OAuthClientCredentialsParametersDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for OAuth 2.0 Client Credentials authentication.
+ *
+ * <p>This vendor handles OAuth 2.0 client credentials flow by reading the client secret from the
+ * secrets manager and formatting it as an OAuth credential for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the OAuth credential (formatted as "clientId:clientSecret"). When
+ * connecting to the remote catalog, Iceberg SDK will use this credential to fetch OAuth tokens
+ * automatically.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.OAUTH)
+@Priority(100)
+public class OAuthConnectionCredentialVendor implements ConnectionCredentialVendor {
+
+ private static final Joiner COLON_JOINER = Joiner.on(":");
+
+ private final UserSecretsManager secretsManager;
+
+ @Inject
+ public OAuthConnectionCredentialVendor(UserSecretsManager secretsManager) {
+ this.secretsManager = secretsManager;
+ }
+
+ @Override
+ public @Nonnull ConnectionCredentials getConnectionCredentials(
+ @Nonnull ConnectionConfigInfoDpo connectionConfig) {
+
+ // Validate authentication parameters type
+ Preconditions.checkArgument(
+ connectionConfig.getAuthenticationParameters()
+ instanceof OAuthClientCredentialsParametersDpo,
+ "Expected OAuthClientCredentialsParametersDpo, got: %s",
+ connectionConfig.getAuthenticationParameters().getClass().getName());
+
+ OAuthClientCredentialsParametersDpo oauthParams =
+ (OAuthClientCredentialsParametersDpo) connectionConfig.getAuthenticationParameters();
+
+ // Read the client secret from secrets manager
+ String clientSecret = secretsManager.readSecret(oauthParams.getClientSecretReference());
+
+ // Format credential as "clientId:clientSecret"
+ String credential = COLON_JOINER.join(oauthParams.getClientId(), clientSecret);
+
+ // Return the OAuth credential
+ // OAuth credentials don't expire - set expiration to Instant.MAX to indicate infinite validity
+ // If the credential expires, users need to update the catalog entity to rotate the credential.
+ return ConnectionCredentials.builder()
+ .putCredential(CatalogAccessProperty.OAUTH2_CREDENTIAL.getPropertyName(), credential)
Review Comment:
I fully support implementing OAuth2 flows in the future.
However, in its current form this credential vendor provides a simple pair of values, but claims the type of `AuthenticationType.OAUTH`. I personally find it misleading :thinking:
Even if Polaris supported OAuth2 flows to obtain a token for a catalog connection, this class populates Iceberg's `credential` property, which is not a token. This property cannot hold an OAuth2 token. Instead it instruct the client to perform some activities internally in order to obtain the access token. From the Polaris POV OAuth is not apparent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422195202
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/OAuthConnectionCredentialVendor.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Joiner;
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import java.time.Instant;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.connection.OAuthClientCredentialsParametersDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for OAuth 2.0 Client Credentials authentication.
+ *
+ * <p>This vendor handles OAuth 2.0 client credentials flow by reading the client secret from the
+ * secrets manager and formatting it as an OAuth credential for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the OAuth credential (formatted as "clientId:clientSecret"). When
+ * connecting to the remote catalog, Iceberg SDK will use this credential to fetch OAuth tokens
+ * automatically.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.OAUTH)
+@Priority(100)
+public class OAuthConnectionCredentialVendor implements ConnectionCredentialVendor {
+
+ private static final Joiner COLON_JOINER = Joiner.on(":");
+
+ private final UserSecretsManager secretsManager;
+
+ @Inject
+ public OAuthConnectionCredentialVendor(UserSecretsManager secretsManager) {
+ this.secretsManager = secretsManager;
+ }
+
+ @Override
+ public @Nonnull ConnectionCredentials getConnectionCredentials(
+ @Nonnull ConnectionConfigInfoDpo connectionConfig) {
+
+ // Validate authentication parameters type
+ Preconditions.checkArgument(
+ connectionConfig.getAuthenticationParameters()
+ instanceof OAuthClientCredentialsParametersDpo,
+ "Expected OAuthClientCredentialsParametersDpo, got: %s",
+ connectionConfig.getAuthenticationParameters().getClass().getName());
+
+ OAuthClientCredentialsParametersDpo oauthParams =
+ (OAuthClientCredentialsParametersDpo) connectionConfig.getAuthenticationParameters();
+
+ // Read the client secret from secrets manager
+ String clientSecret = secretsManager.readSecret(oauthParams.getClientSecretReference());
+
+ // Format credential as "clientId:clientSecret"
+ String credential = COLON_JOINER.join(oauthParams.getClientId(), clientSecret);
+
+ // Return the OAuth credential
+ // OAuth credentials don't expire - set expiration to Instant.MAX to indicate infinite validity
+ // If the credential expires, users need to update the catalog entity to rotate the credential.
+ return ConnectionCredentials.builder()
+ .putCredential(CatalogAccessProperty.OAUTH2_CREDENTIAL.getPropertyName(), credential)
Review Comment:
I fully support implementing OAuth2 flows in the future.
However, in its current form this credential vendor provides simple pair of values, but claims the type of `AuthenticationType.OAUTH`. I personally find it misleading :thinking:
Even if Polaris supported OAuth2 flows to obtain a credential for a catalog connection, this class populates Iceberg's `credential` property, which is not a token. This property cannot hold an OAuth2 token. Instead it instruct the client to perform some activities internally in order to obtain the access token. From the Polaris POV OAuth is not apparent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2421952242
##########
polaris-core/src/main/java/org/apache/polaris/core/credentials/connection/CatalogAccessProperty.java:
##########
@@ -28,15 +29,23 @@
* Catalog service.
*/
public enum CatalogAccessProperty {
+ // OAuth
+ OAUTH2_CREDENTIAL(String.class, OAuth2Properties.CREDENTIAL, "the OAuth2 credential", true),
+
+ // Bearer
+ BEARER_TOKEN(String.class, OAuth2Properties.TOKEN, "the bearer token", true),
+
+ // SigV4
AWS_ACCESS_KEY_ID(String.class, AwsProperties.REST_ACCESS_KEY_ID, "the aws access key id", true),
AWS_SECRET_ACCESS_KEY(
- String.class, AwsProperties.REST_SECRET_ACCESS_KEY, "the aws access key secret", true),
- AWS_SESSION_TOKEN(
- String.class, AwsProperties.REST_SESSION_TOKEN, "the aws scoped access token", true),
+ String.class, AwsProperties.REST_SECRET_ACCESS_KEY, "the aws secret access key", true),
+ AWS_SESSION_TOKEN(String.class, AwsProperties.REST_SESSION_TOKEN, "the aws session token", true),
+
+ // Metadata
EXPIRATION_TIME(
Review Comment:
Revised the `CatalogAccessProperty` a bit, follow the pattern in `StorageAccessCredential`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422244649
##########
runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java:
##########
@@ -335,4 +337,64 @@ public ProductionReadinessCheck checkOverlappingSiblingCheckSettings(
? ProductionReadinessCheck.OK
: ProductionReadinessCheck.of(errors.toArray(new Error[0]));
}
+
+ @Produces
+ @SuppressWarnings("unchecked")
+ public ProductionReadinessCheck checkConnectionCredentialVendors(
+ Instance<ConnectionCredentialVendor> credentialVendors,
+ FeaturesConfiguration featureConfiguration) {
+ var mapper = new ObjectMapper();
+ var defaults = featureConfiguration.parseDefaults(mapper);
+ var realmOverrides = featureConfiguration.parseRealmOverrides(mapper);
+
+ var federationKey = FeatureConfiguration.ENABLE_CATALOG_FEDERATION.key();
+ var authTypesKey = FeatureConfiguration.SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES.key();
+ var federationEnabled =
+ Boolean.parseBoolean(defaults.getOrDefault(federationKey, false).toString());
+ var defaultAuthTypes = (List<String>) defaults.getOrDefault(authTypesKey, List.of());
+
+ var allAuthTypes = new java.util.HashSet<String>();
+ if (federationEnabled) allAuthTypes.addAll(defaultAuthTypes);
+
+ realmOverrides.forEach(
+ (id, overrides) -> {
+ if (Boolean.parseBoolean(
+ overrides.getOrDefault(federationKey, federationEnabled).toString())) {
+ allAuthTypes.addAll(
+ (List<String>)
+ (overrides.containsKey(authTypesKey)
+ ? overrides.get(authTypesKey)
+ : defaultAuthTypes));
+ }
+ });
+
+ var errors = new ArrayList<Error>();
+ for (var name : allAuthTypes) {
+ try {
+ var type = org.apache.polaris.core.connection.AuthenticationType.valueOf(name);
+ if (credentialVendors.select(AuthType.Literal.of(type)).isUnsatisfied()) {
+ errors.add(
+ Error.of(
+ format(
+ "Catalog federation is enabled but no ConnectionCredentialVendor found for "
+ + "authentication type '%s'. External catalog connections using this "
+ + "authentication type will fail.",
+ type),
Review Comment:
I'm not sure the check is logically correct. A particular deployment only needs to provide `ConnectionCredentialVendor` implementations for auth types that are actually used by created catalogs.
I think the vendor availability is implicitly checked on catalog creation already.
On startup, I think we only need to check types for existing catalogs, not for all types listed in the enum. The latter would put an undue burden on custom deployments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422238423
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/OAuthConnectionCredentialVendor.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Joiner;
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import java.time.Instant;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.connection.OAuthClientCredentialsParametersDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for OAuth 2.0 Client Credentials authentication.
+ *
+ * <p>This vendor handles OAuth 2.0 client credentials flow by reading the client secret from the
+ * secrets manager and formatting it as an OAuth credential for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the OAuth credential (formatted as "clientId:clientSecret"). When
+ * connecting to the remote catalog, Iceberg SDK will use this credential to fetch OAuth tokens
+ * automatically.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.OAUTH)
+@Priority(100)
+public class OAuthConnectionCredentialVendor implements ConnectionCredentialVendor {
+
+ private static final Joiner COLON_JOINER = Joiner.on(":");
+
+ private final UserSecretsManager secretsManager;
+
+ @Inject
+ public OAuthConnectionCredentialVendor(UserSecretsManager secretsManager) {
+ this.secretsManager = secretsManager;
+ }
+
+ @Override
+ public @Nonnull ConnectionCredentials getConnectionCredentials(
+ @Nonnull ConnectionConfigInfoDpo connectionConfig) {
+
+ // Validate authentication parameters type
+ Preconditions.checkArgument(
+ connectionConfig.getAuthenticationParameters()
+ instanceof OAuthClientCredentialsParametersDpo,
+ "Expected OAuthClientCredentialsParametersDpo, got: %s",
+ connectionConfig.getAuthenticationParameters().getClass().getName());
+
+ OAuthClientCredentialsParametersDpo oauthParams =
+ (OAuthClientCredentialsParametersDpo) connectionConfig.getAuthenticationParameters();
+
+ // Read the client secret from secrets manager
+ String clientSecret = secretsManager.readSecret(oauthParams.getClientSecretReference());
+
+ // Format credential as "clientId:clientSecret"
+ String credential = COLON_JOINER.join(oauthParams.getClientId(), clientSecret);
+
+ // Return the OAuth credential
+ // OAuth credentials don't expire - set expiration to Instant.MAX to indicate infinite validity
+ // If the credential expires, users need to update the catalog entity to rotate the credential.
+ return ConnectionCredentials.builder()
+ .putCredential(CatalogAccessProperty.OAUTH2_CREDENTIAL.getPropertyName(), credential)
Review Comment:
Correct, how about renaming it to `OAuthClientCredentialVendor` then add a real `OAuthConnectionCredentialVendor` to perform the oauth2 flow and get a token?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2427413410
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/BearerConnectionCredentialVendor.java:
##########
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.BearerAuthenticationParametersDpo;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for Bearer token authentication.
+ *
+ * <p>This vendor handles Bearer token authentication by reading the bearer token from the secrets
+ * manager and providing it for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the bearer token. When connecting to the remote catalog, Iceberg SDK
+ * will use this token as-is in HTTP Authorization headers. Bearer tokens typically have a limited
+ * lifetime and should be refreshed by the user when they expire.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.BEARER)
+@Priority(100)
Review Comment:
Done!
##########
polaris-core/src/main/java/org/apache/polaris/core/connection/BearerAuthenticationParametersDpo.java:
##########
@@ -51,9 +49,10 @@ public BearerAuthenticationParametersDpo(
@Override
public @Nonnull Map<String, String> asIcebergCatalogProperties(
- UserSecretsManager secretsManager, PolarisCredentialManager credentialManager) {
- String bearerToken = secretsManager.readSecret(getBearerTokenReference());
- return Map.of(OAuth2Properties.TOKEN, bearerToken);
+ PolarisCredentialManager credentialManager) {
+ // Return only metadata properties - credentials are handled by ConnectionCredentialVendor
+ // Bearer auth has no metadata properties
+ return Map.of();
Review Comment:
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2429957970
##########
runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java:
##########
@@ -335,4 +337,64 @@ public ProductionReadinessCheck checkOverlappingSiblingCheckSettings(
? ProductionReadinessCheck.OK
: ProductionReadinessCheck.of(errors.toArray(new Error[0]));
}
+
+ @Produces
+ @SuppressWarnings("unchecked")
+ public ProductionReadinessCheck checkConnectionCredentialVendors(
+ Instance<ConnectionCredentialVendor> credentialVendors,
+ FeaturesConfiguration featureConfiguration) {
+ var mapper = new ObjectMapper();
+ var defaults = featureConfiguration.parseDefaults(mapper);
+ var realmOverrides = featureConfiguration.parseRealmOverrides(mapper);
+
+ var federationKey = FeatureConfiguration.ENABLE_CATALOG_FEDERATION.key();
+ var authTypesKey = FeatureConfiguration.SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES.key();
+ var federationEnabled =
+ Boolean.parseBoolean(defaults.getOrDefault(federationKey, false).toString());
+ var defaultAuthTypes = (List<String>) defaults.getOrDefault(authTypesKey, List.of());
+
+ var allAuthTypes = new java.util.HashSet<String>();
+ if (federationEnabled) allAuthTypes.addAll(defaultAuthTypes);
+
+ realmOverrides.forEach(
+ (id, overrides) -> {
+ if (Boolean.parseBoolean(
+ overrides.getOrDefault(federationKey, federationEnabled).toString())) {
+ allAuthTypes.addAll(
+ (List<String>)
+ (overrides.containsKey(authTypesKey)
+ ? overrides.get(authTypesKey)
+ : defaultAuthTypes));
+ }
+ });
+
+ var errors = new ArrayList<Error>();
+ for (var name : allAuthTypes) {
+ try {
+ var type = org.apache.polaris.core.connection.AuthenticationType.valueOf(name);
+ if (credentialVendors.select(AuthType.Literal.of(type)).isUnsatisfied()) {
+ errors.add(
+ Error.of(
+ format(
+ "Catalog federation is enabled but no ConnectionCredentialVendor found for "
+ + "authentication type '%s'. External catalog connections using this "
+ + "authentication type will fail.",
+ type),
Review Comment:
SGTM ๐
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422230219
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/SigV4ConnectionCredentialVendor.java:
##########
@@ -58,7 +58,7 @@
* <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
* override this by providing a higher priority value.
*/
-@ApplicationScoped
+@RequestScoped
Review Comment:
> [...] an application-scoped object referencing a request-scoped one.
What exactly is the application-scoped object in this case? I could not figure it out from previous comments :sweat_smile:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2421327498
##########
polaris-core/src/main/java/org/apache/polaris/core/connection/BearerAuthenticationParametersDpo.java:
##########
@@ -51,9 +49,10 @@ public BearerAuthenticationParametersDpo(
@Override
public @Nonnull Map<String, String> asIcebergCatalogProperties(
- UserSecretsManager secretsManager, PolarisCredentialManager credentialManager) {
- String bearerToken = secretsManager.readSecret(getBearerTokenReference());
- return Map.of(OAuth2Properties.TOKEN, bearerToken);
+ PolarisCredentialManager credentialManager) {
+ // Return only metadata properties - credentials are handled by ConnectionCredentialVendor
+ // Bearer auth has no metadata properties
+ return Map.of();
Review Comment:
Could we make this a default implementation in abstract class `AuthenticationParametersDpo` or the interface?
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/BearerConnectionCredentialVendor.java:
##########
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.BearerAuthenticationParametersDpo;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for Bearer token authentication.
+ *
+ * <p>This vendor handles Bearer token authentication by reading the bearer token from the secrets
+ * manager and providing it for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the bearer token. When connecting to the remote catalog, Iceberg SDK
+ * will use this token as-is in HTTP Authorization headers. Bearer tokens typically have a limited
+ * lifetime and should be refreshed by the user when they expire.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.BEARER)
+@Priority(100)
Review Comment:
Could we extract the priority to some constants class, like `FilterPriorities`. It will be easier for custom implementation to use and managed the prioty.
##########
runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java:
##########
@@ -335,4 +337,64 @@ public ProductionReadinessCheck checkOverlappingSiblingCheckSettings(
? ProductionReadinessCheck.OK
: ProductionReadinessCheck.of(errors.toArray(new Error[0]));
}
+
+ @Produces
+ @SuppressWarnings("unchecked")
+ public ProductionReadinessCheck checkConnectionCredentialVendors(
+ Instance<ConnectionCredentialVendor> credentialVendors,
+ FeaturesConfiguration featureConfiguration) {
+ var mapper = new ObjectMapper();
+ var defaults = featureConfiguration.parseDefaults(mapper);
+ var realmOverrides = featureConfiguration.parseRealmOverrides(mapper);
+
+ var federationKey = FeatureConfiguration.ENABLE_CATALOG_FEDERATION.key();
+ var authTypesKey = FeatureConfiguration.SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES.key();
+ var federationEnabled =
+ Boolean.parseBoolean(defaults.getOrDefault(federationKey, false).toString());
+ var defaultAuthTypes = (List<String>) defaults.getOrDefault(authTypesKey, List.of());
+
+ var allAuthTypes = new java.util.HashSet<String>();
+ if (federationEnabled) allAuthTypes.addAll(defaultAuthTypes);
+
+ realmOverrides.forEach(
+ (id, overrides) -> {
+ if (Boolean.parseBoolean(
+ overrides.getOrDefault(federationKey, federationEnabled).toString())) {
+ allAuthTypes.addAll(
+ (List<String>)
+ (overrides.containsKey(authTypesKey)
+ ? overrides.get(authTypesKey)
+ : defaultAuthTypes));
+ }
+ });
+
+ var errors = new ArrayList<Error>();
+ for (var name : allAuthTypes) {
+ try {
+ var type = org.apache.polaris.core.connection.AuthenticationType.valueOf(name);
+ if (credentialVendors.select(AuthType.Literal.of(type)).isUnsatisfied()) {
+ errors.add(
+ Error.of(
+ format(
+ "Catalog federation is enabled but no ConnectionCredentialVendor found for "
+ + "authentication type '%s'. External catalog connections using this "
+ + "authentication type will fail.",
+ type),
Review Comment:
Just curious, would this check be too strong that we only consider it to be production ready if all auth types need the vendor, or this is more like a warning despite it is called "Error"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2418218843
##########
runtime/service/src/test/java/org/apache/polaris/service/credentials/connection/BearerConnectionCredentialVendorTest.java:
##########
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.time.Instant;
+import org.apache.polaris.core.connection.BearerAuthenticationParametersDpo;
+import org.apache.polaris.core.connection.iceberg.IcebergRestConnectionConfigInfoDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.SecretReference;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** Tests for {@link BearerConnectionCredentialVendor}. */
+public class BearerConnectionCredentialVendorTest {
+
+ private BearerConnectionCredentialVendor bearerVendor;
+ private UserSecretsManager mockSecretsManager;
+
+ @BeforeEach
+ void setup() {
+ mockSecretsManager = mock(UserSecretsManager.class);
+ bearerVendor = new BearerConnectionCredentialVendor(mockSecretsManager);
+ }
+
+ @Test
+ public void testGetConnectionCredentials() {
+ // Setup
+ SecretReference bearerTokenRef =
+ new SecretReference("urn:polaris-secret:test:bearer-token", java.util.Map.of());
+ when(mockSecretsManager.readSecret(bearerTokenRef)).thenReturn("my-bearer-token-value");
+
+ BearerAuthenticationParametersDpo authParams =
+ new BearerAuthenticationParametersDpo(bearerTokenRef);
+
+ IcebergRestConnectionConfigInfoDpo connectionConfig =
+ new IcebergRestConnectionConfigInfoDpo(
+ "https://catalog.example.com", authParams, null, "test-catalog");
+
+ // Execute
+ ConnectionCredentials credentials = bearerVendor.getConnectionCredentials(connectionConfig);
+
+ // Verify - only bearer token is provided
+ Assertions.assertThat(credentials.credentials())
+ .hasSize(1)
+ .containsEntry(
+ CatalogAccessProperty.BEARER_TOKEN.getPropertyName(), "my-bearer-token-value");
+ Assertions.assertThat(credentials.expiresAt()).contains(Instant.MAX);
+ }
+
+ @Test
+ public void testGetConnectionCredentialsWithWrongAuthType() {
+ // Setup - use a mock with wrong authentication type
+ org.apache.polaris.core.connection.ConnectionConfigInfoDpo mockConfig =
Review Comment:
import?
##########
polaris-core/src/main/java/org/apache/polaris/core/credentials/connection/CatalogAccessProperty.java:
##########
@@ -28,15 +29,23 @@
* Catalog service.
*/
public enum CatalogAccessProperty {
+ // OAuth
+ OAUTH2_CREDENTIAL(String.class, OAuth2Properties.CREDENTIAL, "the OAuth2 credential", true),
+
+ // Bearer
+ BEARER_TOKEN(String.class, OAuth2Properties.TOKEN, "the bearer token", true),
+
+ // SigV4
AWS_ACCESS_KEY_ID(String.class, AwsProperties.REST_ACCESS_KEY_ID, "the aws access key id", true),
AWS_SECRET_ACCESS_KEY(
- String.class, AwsProperties.REST_SECRET_ACCESS_KEY, "the aws access key secret", true),
- AWS_SESSION_TOKEN(
- String.class, AwsProperties.REST_SESSION_TOKEN, "the aws scoped access token", true),
+ String.class, AwsProperties.REST_SECRET_ACCESS_KEY, "the aws secret access key", true),
+ AWS_SESSION_TOKEN(String.class, AwsProperties.REST_SESSION_TOKEN, "the aws session token", true),
+
+ // Metadata
EXPIRATION_TIME(
Review Comment:
This enum value does not appear to be used anywhere.
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/SigV4ConnectionCredentialVendor.java:
##########
@@ -58,7 +58,7 @@
* <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
* override this by providing a higher priority value.
*/
-@ApplicationScoped
+@RequestScoped
Review Comment:
Why do we need this change?
##########
runtime/service/src/test/java/org/apache/polaris/service/credentials/connection/OAuthConnectionCredentialVendorTest.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.time.Instant;
+import java.util.List;
+import org.apache.polaris.core.connection.OAuthClientCredentialsParametersDpo;
+import org.apache.polaris.core.connection.iceberg.IcebergRestConnectionConfigInfoDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.SecretReference;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** Tests for {@link OAuthConnectionCredentialVendor}. */
+public class OAuthConnectionCredentialVendorTest {
+
+ private OAuthConnectionCredentialVendor oauthVendor;
+ private UserSecretsManager mockSecretsManager;
+
+ @BeforeEach
+ void setup() {
+ mockSecretsManager = mock(UserSecretsManager.class);
+ oauthVendor = new OAuthConnectionCredentialVendor(mockSecretsManager);
+ }
+
+ @Test
+ public void testGetConnectionCredentials() {
+ // Setup
+ SecretReference clientSecretRef =
+ new SecretReference("urn:polaris-secret:test:oauth-client-secret", java.util.Map.of());
+ when(mockSecretsManager.readSecret(clientSecretRef)).thenReturn("my-client-secret");
+
+ OAuthClientCredentialsParametersDpo authParams =
+ new OAuthClientCredentialsParametersDpo(
+ "https://auth.example.com/token",
+ "my-client-id",
+ clientSecretRef,
+ List.of("catalog", "read:data"));
+
+ IcebergRestConnectionConfigInfoDpo connectionConfig =
+ new IcebergRestConnectionConfigInfoDpo(
+ "https://catalog.example.com", authParams, null, "test-catalog");
+
+ // Execute
+ ConnectionCredentials credentials = oauthVendor.getConnectionCredentials(connectionConfig);
+
+ // Verify - only OAuth credential is provided
+ Assertions.assertThat(credentials.credentials())
+ .hasSize(1)
+ .containsEntry(
+ CatalogAccessProperty.OAUTH2_CREDENTIAL.getPropertyName(),
+ "my-client-id:my-client-secret");
+ Assertions.assertThat(credentials.expiresAt()).contains(Instant.MAX);
+ }
+
+ @Test
+ public void testGetConnectionCredentialsWithWrongAuthType() {
+ // Setup - use a mock with wrong authentication type
+ org.apache.polaris.core.connection.ConnectionConfigInfoDpo mockConfig =
Review Comment:
import?
##########
runtime/service/src/test/java/org/apache/polaris/service/credentials/connection/ImplicitConnectionCredentialVendorTest.java:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.time.Instant;
+import org.apache.polaris.core.connection.ImplicitAuthenticationParametersDpo;
+import org.apache.polaris.core.connection.iceberg.IcebergRestConnectionConfigInfoDpo;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** Tests for {@link ImplicitConnectionCredentialVendor}. */
+public class ImplicitConnectionCredentialVendorTest {
+
+ private ImplicitConnectionCredentialVendor implicitVendor;
+
+ @BeforeEach
+ void setup() {
+ implicitVendor = new ImplicitConnectionCredentialVendor();
+ }
+
+ @Test
+ public void testGetConnectionCredentials() {
+ // Setup
+ ImplicitAuthenticationParametersDpo authParams = new ImplicitAuthenticationParametersDpo();
+
+ IcebergRestConnectionConfigInfoDpo connectionConfig =
+ new IcebergRestConnectionConfigInfoDpo(
+ "https://catalog.example.com", authParams, null, "test-catalog");
+
+ // Execute
+ ConnectionCredentials credentials = implicitVendor.getConnectionCredentials(connectionConfig);
+
+ // Verify - no credentials are provided, but expiration is set to infinite
+ Assertions.assertThat(credentials.credentials()).isEmpty();
+ Assertions.assertThat(credentials.expiresAt()).contains(Instant.MAX);
+ }
+
+ @Test
+ public void testGetConnectionCredentialsWithWrongAuthType() {
+ // Setup - use a mock with wrong authentication type
+ org.apache.polaris.core.connection.ConnectionConfigInfoDpo mockConfig =
Review Comment:
import?
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/OAuthConnectionCredentialVendor.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Joiner;
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import java.time.Instant;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.connection.OAuthClientCredentialsParametersDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for OAuth 2.0 Client Credentials authentication.
+ *
+ * <p>This vendor handles OAuth 2.0 client credentials flow by reading the client secret from the
+ * secrets manager and formatting it as an OAuth credential for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the OAuth credential (formatted as "clientId:clientSecret"). When
+ * connecting to the remote catalog, Iceberg SDK will use this credential to fetch OAuth tokens
+ * automatically.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.OAUTH)
+@Priority(100)
+public class OAuthConnectionCredentialVendor implements ConnectionCredentialVendor {
+
+ private static final Joiner COLON_JOINER = Joiner.on(":");
+
+ private final UserSecretsManager secretsManager;
+
+ @Inject
+ public OAuthConnectionCredentialVendor(UserSecretsManager secretsManager) {
+ this.secretsManager = secretsManager;
+ }
+
+ @Override
+ public @Nonnull ConnectionCredentials getConnectionCredentials(
+ @Nonnull ConnectionConfigInfoDpo connectionConfig) {
+
+ // Validate authentication parameters type
+ Preconditions.checkArgument(
+ connectionConfig.getAuthenticationParameters()
+ instanceof OAuthClientCredentialsParametersDpo,
+ "Expected OAuthClientCredentialsParametersDpo, got: %s",
+ connectionConfig.getAuthenticationParameters().getClass().getName());
+
+ OAuthClientCredentialsParametersDpo oauthParams =
+ (OAuthClientCredentialsParametersDpo) connectionConfig.getAuthenticationParameters();
+
+ // Read the client secret from secrets manager
+ String clientSecret = secretsManager.readSecret(oauthParams.getClientSecretReference());
+
+ // Format credential as "clientId:clientSecret"
+ String credential = COLON_JOINER.join(oauthParams.getClientId(), clientSecret);
+
+ // Return the OAuth credential
+ // OAuth credentials don't expire - set expiration to Instant.MAX to indicate infinite validity
+ // If the credential expires, users need to update the catalog entity to rotate the credential.
+ return ConnectionCredentials.builder()
+ .putCredential(CatalogAccessProperty.OAUTH2_CREDENTIAL.getPropertyName(), credential)
Review Comment:
There is nothing in this class that actually relates to the OAuth2 spec as far as I can tell. It only relates to the Iceberg `OAuth2Properties.CREDENTIAL` constant, which again has very little to do with OAuth2 RFCs as far as connection configuration is concerned... It's just a property name plus format spec ๐คท
What the client does with this property (whether it's OAuth2 or something else) is beyond the scope of this credential vendor. At this level we do not know how the credential is used.
I'd propose to rename this to "Iceberg Credential Vendor".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2426858961
##########
runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java:
##########
@@ -335,4 +337,64 @@ public ProductionReadinessCheck checkOverlappingSiblingCheckSettings(
? ProductionReadinessCheck.OK
: ProductionReadinessCheck.of(errors.toArray(new Error[0]));
}
+
+ @Produces
+ @SuppressWarnings("unchecked")
+ public ProductionReadinessCheck checkConnectionCredentialVendors(
+ Instance<ConnectionCredentialVendor> credentialVendors,
+ FeaturesConfiguration featureConfiguration) {
+ var mapper = new ObjectMapper();
+ var defaults = featureConfiguration.parseDefaults(mapper);
+ var realmOverrides = featureConfiguration.parseRealmOverrides(mapper);
+
+ var federationKey = FeatureConfiguration.ENABLE_CATALOG_FEDERATION.key();
+ var authTypesKey = FeatureConfiguration.SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES.key();
+ var federationEnabled =
+ Boolean.parseBoolean(defaults.getOrDefault(federationKey, false).toString());
+ var defaultAuthTypes = (List<String>) defaults.getOrDefault(authTypesKey, List.of());
+
+ var allAuthTypes = new java.util.HashSet<String>();
+ if (federationEnabled) allAuthTypes.addAll(defaultAuthTypes);
+
+ realmOverrides.forEach(
+ (id, overrides) -> {
+ if (Boolean.parseBoolean(
+ overrides.getOrDefault(federationKey, federationEnabled).toString())) {
+ allAuthTypes.addAll(
+ (List<String>)
+ (overrides.containsKey(authTypesKey)
+ ? overrides.get(authTypesKey)
+ : defaultAuthTypes));
+ }
+ });
+
+ var errors = new ArrayList<Error>();
+ for (var name : allAuthTypes) {
+ try {
+ var type = org.apache.polaris.core.connection.AuthenticationType.valueOf(name);
+ if (credentialVendors.select(AuthType.Literal.of(type)).isUnsatisfied()) {
+ errors.add(
+ Error.of(
+ format(
+ "Catalog federation is enabled but no ConnectionCredentialVendor found for "
+ + "authentication type '%s'. External catalog connections using this "
+ + "authentication type will fail.",
+ type),
Review Comment:
I see, I will switch to `severe`.
> A particular deployment only needs to provide ConnectionCredentialVendor implementations for auth types that are actually used by created catalogs. I think we only need to check types for existing catalogs, not for all types listed in the enum
The logic is: if we enable the catalog federation and all the enabled auth types must have a corresponding `ConnectionCredentialVendor`. IIRC, we don't want to defer the check during the runtime based on the feedback in the previous PR.
If vendors only implement one / two auth types for catalog federation, they need to only enable them, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on PR #2782:
URL: https://github.com/apache/polaris/pull/2782#issuecomment-3403411752
Seems we have all the comments resolved. Thanks @dimas-b for reviewing this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422195202
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/OAuthConnectionCredentialVendor.java:
##########
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import com.google.common.base.Joiner;
+import com.google.common.base.Preconditions;
+import jakarta.annotation.Nonnull;
+import jakarta.annotation.Priority;
+import jakarta.enterprise.context.RequestScoped;
+import jakarta.inject.Inject;
+import java.time.Instant;
+import org.apache.polaris.core.connection.AuthenticationType;
+import org.apache.polaris.core.connection.ConnectionConfigInfoDpo;
+import org.apache.polaris.core.connection.OAuthClientCredentialsParametersDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentialVendor;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+
+/**
+ * Connection credential vendor for OAuth 2.0 Client Credentials authentication.
+ *
+ * <p>This vendor handles OAuth 2.0 client credentials flow by reading the client secret from the
+ * secrets manager and formatting it as an OAuth credential for connecting to external catalogs.
+ *
+ * <p>The vendor provides only the OAuth credential (formatted as "clientId:clientSecret"). When
+ * connecting to the remote catalog, Iceberg SDK will use this credential to fetch OAuth tokens
+ * automatically.
+ *
+ * <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
+ * override this by providing a higher priority value.
+ */
+@RequestScoped
+@AuthType(AuthenticationType.OAUTH)
+@Priority(100)
+public class OAuthConnectionCredentialVendor implements ConnectionCredentialVendor {
+
+ private static final Joiner COLON_JOINER = Joiner.on(":");
+
+ private final UserSecretsManager secretsManager;
+
+ @Inject
+ public OAuthConnectionCredentialVendor(UserSecretsManager secretsManager) {
+ this.secretsManager = secretsManager;
+ }
+
+ @Override
+ public @Nonnull ConnectionCredentials getConnectionCredentials(
+ @Nonnull ConnectionConfigInfoDpo connectionConfig) {
+
+ // Validate authentication parameters type
+ Preconditions.checkArgument(
+ connectionConfig.getAuthenticationParameters()
+ instanceof OAuthClientCredentialsParametersDpo,
+ "Expected OAuthClientCredentialsParametersDpo, got: %s",
+ connectionConfig.getAuthenticationParameters().getClass().getName());
+
+ OAuthClientCredentialsParametersDpo oauthParams =
+ (OAuthClientCredentialsParametersDpo) connectionConfig.getAuthenticationParameters();
+
+ // Read the client secret from secrets manager
+ String clientSecret = secretsManager.readSecret(oauthParams.getClientSecretReference());
+
+ // Format credential as "clientId:clientSecret"
+ String credential = COLON_JOINER.join(oauthParams.getClientId(), clientSecret);
+
+ // Return the OAuth credential
+ // OAuth credentials don't expire - set expiration to Instant.MAX to indicate infinite validity
+ // If the credential expires, users need to update the catalog entity to rotate the credential.
+ return ConnectionCredentials.builder()
+ .putCredential(CatalogAccessProperty.OAUTH2_CREDENTIAL.getPropertyName(), credential)
Review Comment:
I fully support implementing OAuth2 flows in the future.
However, in its current form this credential vendor provides a simple pair of values, but claims the type of `AuthenticationType.OAUTH`. I personally find it misleading :thinking:
Even if Polaris supported OAuth2 flows to obtain a token for a catalog connection, this class populates Iceberg's `credential` property, which is not a token. This property cannot hold an OAuth2 token. Instead it instruct the (Iceberg) client to perform some activities internally in order to obtain the access token. From the Polaris POV OAuth is not apparent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422239947
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/SigV4ConnectionCredentialVendor.java:
##########
@@ -58,7 +58,7 @@
* <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
* override this by providing a higher priority value.
*/
-@ApplicationScoped
+@RequestScoped
Review Comment:
`DefaultServiceIdentityRegistry` is the previous name, I mistyped it in my comment๐
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on PR #2782:
URL: https://github.com/apache/polaris/pull/2782#issuecomment-3403458039
@HonahX @dimas-b Thanks for reviewing the PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2426858961
##########
runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java:
##########
@@ -335,4 +337,64 @@ public ProductionReadinessCheck checkOverlappingSiblingCheckSettings(
? ProductionReadinessCheck.OK
: ProductionReadinessCheck.of(errors.toArray(new Error[0]));
}
+
+ @Produces
+ @SuppressWarnings("unchecked")
+ public ProductionReadinessCheck checkConnectionCredentialVendors(
+ Instance<ConnectionCredentialVendor> credentialVendors,
+ FeaturesConfiguration featureConfiguration) {
+ var mapper = new ObjectMapper();
+ var defaults = featureConfiguration.parseDefaults(mapper);
+ var realmOverrides = featureConfiguration.parseRealmOverrides(mapper);
+
+ var federationKey = FeatureConfiguration.ENABLE_CATALOG_FEDERATION.key();
+ var authTypesKey = FeatureConfiguration.SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES.key();
+ var federationEnabled =
+ Boolean.parseBoolean(defaults.getOrDefault(federationKey, false).toString());
+ var defaultAuthTypes = (List<String>) defaults.getOrDefault(authTypesKey, List.of());
+
+ var allAuthTypes = new java.util.HashSet<String>();
+ if (federationEnabled) allAuthTypes.addAll(defaultAuthTypes);
+
+ realmOverrides.forEach(
+ (id, overrides) -> {
+ if (Boolean.parseBoolean(
+ overrides.getOrDefault(federationKey, federationEnabled).toString())) {
+ allAuthTypes.addAll(
+ (List<String>)
+ (overrides.containsKey(authTypesKey)
+ ? overrides.get(authTypesKey)
+ : defaultAuthTypes));
+ }
+ });
+
+ var errors = new ArrayList<Error>();
+ for (var name : allAuthTypes) {
+ try {
+ var type = org.apache.polaris.core.connection.AuthenticationType.valueOf(name);
+ if (credentialVendors.select(AuthType.Literal.of(type)).isUnsatisfied()) {
+ errors.add(
+ Error.of(
+ format(
+ "Catalog federation is enabled but no ConnectionCredentialVendor found for "
+ + "authentication type '%s'. External catalog connections using this "
+ + "authentication type will fail.",
+ type),
Review Comment:
I see, I will switch to `severe`.
> A particular deployment only needs to provide ConnectionCredentialVendor implementations for auth types that are actually used by created catalogs. I think we only need to check types for existing catalogs, not for all types listed in the enum
The logic is: if we enable the catalog federation then all the enabled auth types must have a corresponding `ConnectionCredentialVendor`. IIRC, we don't want to defer the check during the runtime based on the feedback in the previous PR.
If vendors only implement one / two auth types for catalog federation, they need to only enable them, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2421689703
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/SigV4ConnectionCredentialVendor.java:
##########
@@ -58,7 +58,7 @@
* <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
* override this by providing a higher priority value.
*/
-@ApplicationScoped
+@RequestScoped
Review Comment:
Because I just realized that the `DefaultServiceIdentityRegistry` is a `RequestScoped` bean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2421949641
##########
runtime/service/src/test/java/org/apache/polaris/service/credentials/connection/OAuthConnectionCredentialVendorTest.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.polaris.service.credentials.connection;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.time.Instant;
+import java.util.List;
+import org.apache.polaris.core.connection.OAuthClientCredentialsParametersDpo;
+import org.apache.polaris.core.connection.iceberg.IcebergRestConnectionConfigInfoDpo;
+import org.apache.polaris.core.credentials.connection.CatalogAccessProperty;
+import org.apache.polaris.core.credentials.connection.ConnectionCredentials;
+import org.apache.polaris.core.secrets.SecretReference;
+import org.apache.polaris.core.secrets.UserSecretsManager;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** Tests for {@link OAuthConnectionCredentialVendor}. */
+public class OAuthConnectionCredentialVendorTest {
+
+ private OAuthConnectionCredentialVendor oauthVendor;
+ private UserSecretsManager mockSecretsManager;
+
+ @BeforeEach
+ void setup() {
+ mockSecretsManager = mock(UserSecretsManager.class);
+ oauthVendor = new OAuthConnectionCredentialVendor(mockSecretsManager);
+ }
+
+ @Test
+ public void testGetConnectionCredentials() {
+ // Setup
+ SecretReference clientSecretRef =
+ new SecretReference("urn:polaris-secret:test:oauth-client-secret", java.util.Map.of());
+ when(mockSecretsManager.readSecret(clientSecretRef)).thenReturn("my-client-secret");
+
+ OAuthClientCredentialsParametersDpo authParams =
+ new OAuthClientCredentialsParametersDpo(
+ "https://auth.example.com/token",
+ "my-client-id",
+ clientSecretRef,
+ List.of("catalog", "read:data"));
+
+ IcebergRestConnectionConfigInfoDpo connectionConfig =
+ new IcebergRestConnectionConfigInfoDpo(
+ "https://catalog.example.com", authParams, null, "test-catalog");
+
+ // Execute
+ ConnectionCredentials credentials = oauthVendor.getConnectionCredentials(connectionConfig);
+
+ // Verify - only OAuth credential is provided
+ Assertions.assertThat(credentials.credentials())
+ .hasSize(1)
+ .containsEntry(
+ CatalogAccessProperty.OAUTH2_CREDENTIAL.getPropertyName(),
+ "my-client-id:my-client-secret");
+ Assertions.assertThat(credentials.expiresAt()).contains(Instant.MAX);
+ }
+
+ @Test
+ public void testGetConnectionCredentialsWithWrongAuthType() {
+ // Setup - use a mock with wrong authentication type
+ org.apache.polaris.core.connection.ConnectionConfigInfoDpo mockConfig =
Review Comment:
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422233188
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/SigV4ConnectionCredentialVendor.java:
##########
@@ -58,7 +58,7 @@
* <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
* override this by providing a higher priority value.
*/
-@ApplicationScoped
+@RequestScoped
Review Comment:
So, this is about `DefaultServiceIdentityProvider` - all clear now :sweat_smile:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422230219
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/SigV4ConnectionCredentialVendor.java:
##########
@@ -58,7 +58,7 @@
* <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
* override this by providing a higher priority value.
*/
-@ApplicationScoped
+@RequestScoped
Review Comment:
> [...] an application-scoped object referencing a request-scoped one.
What exactly is the other object in this case? I could not figure it out from previous comments :sweat_smile:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422237996
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/SigV4ConnectionCredentialVendor.java:
##########
@@ -58,7 +58,7 @@
* <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
* override this by providing a higher priority value.
*/
-@ApplicationScoped
+@RequestScoped
Review Comment:
Ah yeah, sorry I did not notice the previous confusion there. IDE just popped this class when I searched first few characters ๐
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2422215071
##########
runtime/service/src/main/java/org/apache/polaris/service/credentials/connection/SigV4ConnectionCredentialVendor.java:
##########
@@ -58,7 +58,7 @@
* <p>This is the default implementation with {@code @Priority(100)}. Custom implementations can
* override this by providing a higher priority value.
*/
-@ApplicationScoped
+@RequestScoped
Review Comment:
> Referencing an application-scoped object from a request scope is usually ok
I think in case of the SigV4ConenctionCredentialVendor is the reverse? That an application-scoped object referencing a request-scoped one. The request-scoped one will always be empty or contain wrong context I suppose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "XJDKC (via GitHub)" <gi...@apache.org>.
XJDKC commented on code in PR #2782:
URL: https://github.com/apache/polaris/pull/2782#discussion_r2426858961
##########
runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java:
##########
@@ -335,4 +337,64 @@ public ProductionReadinessCheck checkOverlappingSiblingCheckSettings(
? ProductionReadinessCheck.OK
: ProductionReadinessCheck.of(errors.toArray(new Error[0]));
}
+
+ @Produces
+ @SuppressWarnings("unchecked")
+ public ProductionReadinessCheck checkConnectionCredentialVendors(
+ Instance<ConnectionCredentialVendor> credentialVendors,
+ FeaturesConfiguration featureConfiguration) {
+ var mapper = new ObjectMapper();
+ var defaults = featureConfiguration.parseDefaults(mapper);
+ var realmOverrides = featureConfiguration.parseRealmOverrides(mapper);
+
+ var federationKey = FeatureConfiguration.ENABLE_CATALOG_FEDERATION.key();
+ var authTypesKey = FeatureConfiguration.SUPPORTED_EXTERNAL_CATALOG_AUTHENTICATION_TYPES.key();
+ var federationEnabled =
+ Boolean.parseBoolean(defaults.getOrDefault(federationKey, false).toString());
+ var defaultAuthTypes = (List<String>) defaults.getOrDefault(authTypesKey, List.of());
+
+ var allAuthTypes = new java.util.HashSet<String>();
+ if (federationEnabled) allAuthTypes.addAll(defaultAuthTypes);
+
+ realmOverrides.forEach(
+ (id, overrides) -> {
+ if (Boolean.parseBoolean(
+ overrides.getOrDefault(federationKey, federationEnabled).toString())) {
+ allAuthTypes.addAll(
+ (List<String>)
+ (overrides.containsKey(authTypesKey)
+ ? overrides.get(authTypesKey)
+ : defaultAuthTypes));
+ }
+ });
+
+ var errors = new ArrayList<Error>();
+ for (var name : allAuthTypes) {
+ try {
+ var type = org.apache.polaris.core.connection.AuthenticationType.valueOf(name);
+ if (credentialVendors.select(AuthType.Literal.of(type)).isUnsatisfied()) {
+ errors.add(
+ Error.of(
+ format(
+ "Catalog federation is enabled but no ConnectionCredentialVendor found for "
+ + "authentication type '%s'. External catalog connections using this "
+ + "authentication type will fail.",
+ type),
Review Comment:
I see, I will switch to `severe`.
> A particular deployment only needs to provide ConnectionCredentialVendor implementations for auth types that are actually used by created catalogs. I think we only need to check types for existing catalogs, not for all types listed in the enum
The logic is: if we enable the catalog federation and all the enabled auth types must have a corresponding `ConnectionCredentialVendor`. IIRC, we don't want to defer the check during the runtime based on the feedback in the previous PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Catalog Federation] Add Connection Credential Vendors for Other Auth Types [polaris]
Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX merged PR #2782:
URL: https://github.com/apache/polaris/pull/2782
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org