You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "jackye1995 (via GitHub)" <gi...@apache.org> on 2025/08/23 04:08:58 UTC

[PR] feat(services/gcs): add configuration aliases for better Arrow object_store compatibility [opendal]

jackye1995 opened a new pull request, #6526:
URL: https://github.com/apache/opendal/pull/6526

   ## Summary
   
   This PR adds serde aliases to GcsConfig fields to support multiple configuration key naming conventions, enabling seamless migration from Arrow object_store and other file system libraries. Users can now use either OpenDAL's native field names or Google-prefixed alternatives.
   
   ## Changes
   
   Added `#[serde(alias = "...")]` attributes to 5 key GCS configuration fields, following the exact same patterns used by [Apache Arrow's object_store library](https://github.com/apache/arrow-rs/blob/main/object_store/src/gcp/builder.rs).
   
   ### Example Usage
   
   ```json
   // Original OpenDAL style (still works)
   {
     "bucket": "my-bucket",
     "service_account": "/path/to/sa.json",
     "credential": "service-account-key-json"
   }
   
   // Google SDK style (now supported!)
   {
     "google_bucket": "my-bucket", 
     "google_service_account": "/path/to/sa.json",
     "google_service_account_key": "service-account-key-json"
   }
   ```
   
   ## Added Aliases
   
   | OpenDAL Field | Added Aliases | Arrow Reference |
   |---------------|---------------|-----------------|
   | `bucket` | `google_bucket`, `google_bucket_name`, `bucket_name` | [L156-L160](https://github.com/apache/arrow-rs/blob/main/object_store/src/gcp/builder.rs#L156-L160) |
   | `service_account` | `google_service_account`, `google_service_account_path`, `service_account_path` | [L136-L142](https://github.com/apache/arrow-rs/blob/main/object_store/src/gcp/builder.rs#L136-L142) |
   | `credential` | `google_service_account_key`, `service_account_key` | [L146-L149](https://github.com/apache/arrow-rs/blob/main/object_store/src/gcp/builder.rs#L146-L149) |
   | `credential_path` | `google_application_credentials` | [L164-L165](https://github.com/apache/arrow-rs/blob/main/object_store/src/gcp/builder.rs#L164-L165) |
   | `allow_anonymous` | `google_skip_signature`, `skip_signature` | [L167-L168](https://github.com/apache/arrow-rs/blob/main/object_store/src/gcp/builder.rs#L167-L168) |
   
   ## Missing Configurations (Future Work)
   
   These Arrow configurations are not yet available in OpenDAL and could be added in future PRs:
   
   | Arrow Configuration | Missing in OpenDAL | Arrow Reference | Description |
   |-------------------|-------------------|-----------------|-------------|
   | Client configurations | Various client options | [L170-L172](https://github.com/apache/arrow-rs/blob/main/object_store/src/gcp/builder.rs#L170-L172) | HTTP client configuration options |
   
   ## Testing
   
   - ✅ Added comprehensive unit tests covering all aliases
   - ✅ All existing GCS tests pass
   - ✅ Code compiles without warnings  
   - ✅ Clippy passes without issues
   
   ## Backward Compatibility
   
   This change is fully backward compatible. All existing configurations continue to work exactly as before, with the new aliases providing additional flexibility.
   
   Similar to #6524


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(services/gcs): add configuration aliases for better Arrow object_store compatibility [opendal]

Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo merged PR #6526:
URL: https://github.com/apache/opendal/pull/6526


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org