[PR] feat(csharp/test/Drivers/Databricks): Add mandatory token exchange [arrow-adbc]

["alexguo-db (via GitHub)" <gi...@apache.org>]

alexguo-db opened a new pull request, #3192:
URL: https://github.com/apache/arrow-adbc/pull/3192

   ## Motivation
   
   Databricks will eventually require that all non-inhouse OAuth tokens be exchanged for Databricks OAuth tokens before accessing resources. This change implements mandatory token exchange before sending Thrift requests. This check and exchange is performed in the background for now to reduce latency, but it will eventually need to be blocking if non-inhouse OAuth tokens will fail to access Databricks resources in the future. 
   
   ## Key Components
   
     1. JWT Token Decoder - Decodes JWT tokens to inspect the issuer claim and determine if token exchange is necessary
     2. MandatoryTokenExchangeDelegatingHandler - HTTP handler that intercepts requests and performs token exchange when required
     3. TokenExchangeClient - Handles the token exchange logic with the same /oidc/v1/token endpoint as token refresh, with slightly different parameters
   
   ## Changes
   
     - Added new connection string parameter: IdentityFederationClientId for service principal workload identity federation scenarios
     - Implemented token exchange logic that checks JWT issuer against workspace host
     - Introduced fallback behavior to maintain backward compatibility if token exchange fails
   
   ## Testing
   `dotnet test --filter "FullyQualifiedName~MandatoryTokenExchangeDelegatingHandlerTests"`
   
   `dotnet test --filter "FullyQualifiedName~TokenExchangeClientTests"`
   
   
   `dotnet test --filter "FullyQualifiedName~JwtTokenDecoderTests"`
   
   
   
   Also tested E2E manually with AAD tokens for Azure Databricks workspaces, AAD tokens for AWS Databricks workspaces, and service principal workload identity federation tokens


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(csharp/test/Drivers/Databricks): Add mandatory token exchange [arrow-adbc]

["CurtHagenlocher (via GitHub)" <gi...@apache.org>]

CurtHagenlocher commented on code in PR #3192:
URL: https://github.com/apache/arrow-adbc/pull/3192#discussion_r2240558548


##########
csharp/src/Drivers/Databricks/DatabricksParameters.cs:
##########
@@ -212,6 +212,12 @@ public class DatabricksParameters : SparkParameters
         /// Default value is 0 (disabled) if not specified.
         /// </summary>
         public const string TokenRenewLimit = "adbc.databricks.token_renew_limit";
+
+        /// <summary>
+        /// The client ID of the service principal when using workload identity federation.
+        /// Default value is empty if not specified.
+        /// </summary>
+        public const string IdentityFederationClientId = "adbc.databricks.identity_federation_client_id";

Review Comment:
   Can this also be added to the documentation at csharp\src\Drivers\Databricks\readme.md?
   
   I haven't always been paying attention to that, so there may be a bit of a gap now which could be addressed all at once with a separate PR.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(csharp/test/Drivers/Databricks): Add mandatory token exchange [arrow-adbc]

["CurtHagenlocher (via GitHub)" <gi...@apache.org>]

CurtHagenlocher merged PR #3192:
URL: https://github.com/apache/arrow-adbc/pull/3192


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org