You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@pinot.apache.org by "Akanksha-kedia (via GitHub)" <gi...@apache.org> on 2026/02/06 10:26:30 UTC

[PR] GENAI=NO TLNSDD-371 Complete Vault integration for Apache Pinot [pinot]

Akanksha-kedia opened a new pull request, #17651:
URL: https://github.com/apache/pinot/pull/17651

   This commit implements comprehensive Vault integration for secure token management across all Pinot components (Controller, Broker, Server, Minion).
   
   Key Features:
   - Vault-based authentication with automatic token refresh
   - AuthProviderFactory for centralized auth provider creation
   - Comprehensive error handling and logging
   - Thread-safe token caching with configurable TTL
   - Support for multiple Vault authentication methods (AppRole, Token)
   
   Components Added:
   - VaultAuth: Core Vault authentication logic
   - VaultConfig: Configuration management for Vault settings
   - VaultResponse: Response parsing and validation
   - VaultStartupManager: Singleton initialization manager
   - VaultTokenAuthProvider: Auth provider implementation
   - VaultTokenCache: Thread-safe token caching
   - VaultUtil: Utility methods for Vault operations
   - AuthProviderFactory: Factory pattern for auth providers
   
   Enhancements:
   - Updated AuthProviderUtils with Vault support
   - Enhanced StaticTokenAuthProvider with null checks
   - Integrated Vault initialization in all component starters
   - Added HttpSegmentFetcher auth support
   
   Testing:
   - AuthProviderFactoryTest: 390 lines of comprehensive tests
   - VaultTokenAuthProviderTest: 214 lines of provider tests
   - VaultUtilTest: 69 lines of utility tests
   
   Security:
   - Fixed security vulnerabilities
   - Removed debug/development tags
   - Proper secret handling and token lifecycle management
   
   PR Title
   Distribute Service Tokens Across Pinot Components with Static and Vault‑Based Auth Providers
   
   ✅ PR Description
   This change introduces a clear and explicit service‑token distribution model for Apache Pinot components (Controller, Broker, Server, Minion) and documents it visually and configurationally.
   
   🔐 Service Token Distribution
   All Pinot components now support service‑level authentication tokens that are independent of user authentication mechanisms (Basic, ZK, LDAP).
   
   For simplicity and clarity:
   
   Admin credentials are reused as service tokens
   In production deployments, service credentials must be separated from admin users
   ✅ Supported Auth Provider Modes (Final)
   The following parameter is introduced and treated as authoritative plain text:
   
   
   
   auth.provider.type = static | null | vault
   
   
   This parameter is not optional, not experimental, and not open for discussion.
   
   🧩 Behavior by Mode
   1️⃣ auth.provider.type=static
   Service tokens are configured manually in component config files
   No external dependency
   Example:
   
   
   # Enable the controller to fetch segments using a service token
   controller.segment.fetcher.auth.token=Basic YWRtaW46dmVyeXNlY3JldA
   
   Basic  + base64encode(admin:verysecret)
   ✅ Tokens must NOT be surrounded by quotes
   ✅ Restart affected components for changes to take effect
   
   2️⃣ auth.provider.type=vault
   Controller, Broker, Server, and Minion:
   Authenticate to Vault at startup
   Fetch credentials (username/password)
   Generate a Basic Auth service token
   Cache the token in memory
   No Vault calls at runtime
   Example configuration:
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org