You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "WangXiaoBao1222 (via GitHub)" <gi...@apache.org> on 2026/04/20 09:26:45 UTC
[PR] feat(services/cos): support cosn scheme and STS security token [opendal]
WangXiaoBao1222 opened a new pull request, #7418:
URL: https://github.com/apache/opendal/pull/7418
# Which issue does this PR close?
Closes #7417.
# Rationale for this change
This PR adds two related improvements to the Tencent Cloud COS (`services/cos`) backend:
1. **`cosn://` scheme alias** — In the Hadoop / Spark ecosystem, Tencent COS is
commonly accessed via the `cosn://` protocol (see the official
[hadoop-cos] document). Users migrating Lance / Spark workloads to
OpenDAL expect the same URI style to work out of the box, so we register
`cosn` as an alias of `cos` at the service layer.
2. **STS temporary-credential (`security_token`) support** — Production
deployments frequently use Tencent Cloud STS (`GetFederationToken` /
`AssumeRole`) to issue short-lived credentials consisting of a
`secret_id`, `secret_key`, **and a `security_token`**. Previously, only
the first two could be passed via config; the third one had to be
injected via environment variables, which is awkward in embedded or
multi-tenant scenarios.
[hadoop-cos]: https://cloud.tencent.com/document/product/436/6884
# What changes are included in this PR?
- Register `cosn` as an alias of `cos` in `services/cos/src/lib.rs`.
- Add `CosConfig::security_token` config field.
- Add `CosBuilder::security_token(...)` setter with doc comments describing
the precedence rules.
- In `CosBuilder::build`, when both `secret_id` and `secret_key` are
provided, resolve the session token with this precedence:
1. explicit `config.security_token` set by the user;
2. environment variables (`TENCENTCLOUD_TOKEN`,
`TENCENTCLOUD_SECURITY_TOKEN`, `QCLOUD_SECRET_TOKEN`), only when
`disable_config_load` is **not** set.
The resolved token is then forwarded to
`StaticCredentialProvider::with_security_token` so the
`x-cos-security-token` header is attached automatically when signing.
- Redact `security_token` in the `Debug` impl of `CosConfig` to avoid
leaking secrets into logs.
- Add unit tests covering:
- `CosConfig::from_uri` correctly parses both `cos://` and `cosn://`
URIs, including `bucket`, `root`, and `security_token`;
- `security_token` is redacted in `Debug` output.
Verified locally:
- `cargo fmt --all -- --check` ✅
- `cargo clippy -p opendal-service-cos --all-targets -- -D warnings` ✅
- `cargo test -p opendal-service-cos` ✅
- Manual end-to-end test against a real Tencent COS bucket using STS
temporary credentials from `sts.tencentcloudapi.com` — read / write /
list / delete all succeeded via both `cos://` and `cosn://` schemes.
# Are there any user-facing changes?
Yes, but **fully backward-compatible**:
- New optional builder method `CosBuilder::security_token(...)`.
- New optional config field `CosConfig::security_token`.
- `cosn://` is now accepted wherever `cos://` was previously accepted.
No existing behavior is removed or changed for users who do not set
`security_token` or use `cosn://`, so no `breaking-changes` label is
required.
# AI Usage Statement
Parts of this change (code scaffolding, unit tests, and this PR
description) were drafted with the assistance of an AI coding agent
(Anthropic Claude, `claude-4.7-opus`). All changes were reviewed, tested,
and verified by the author against a real Tencent Cloud COS bucket before
submission.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/cos): support cosn scheme and STS security token [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo merged PR #7418:
URL: https://github.com/apache/opendal/pull/7418
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/cos): support cosn scheme and STS security token [opendal]
Posted by "WangXiaoBao1222 (via GitHub)" <gi...@apache.org>.
WangXiaoBao1222 commented on code in PR #7418:
URL: https://github.com/apache/opendal/pull/7418#discussion_r3286506824
##########
core/services/cos/src/backend.rs:
##########
@@ -199,14 +220,21 @@ impl Builder for CosBuilder {
self.config.secret_id.as_deref(),
self.config.secret_key.as_deref(),
) {
- let security_token = envs
- .get("TENCENTCLOUD_TOKEN")
- .or_else(|| envs.get("TENCENTCLOUD_SECURITY_TOKEN"))
- .or_else(|| envs.get("QCLOUD_SECRET_TOKEN"));
+ // Precedence:
+ // 1. explicit `config.security_token` set by the user;
+ // 2. environment variables when config load is NOT disabled.
+ let env_token = if self.config.disable_config_load {
+ None
+ } else {
+ envs.get("TENCENTCLOUD_TOKEN")
Review Comment:
Done , removed the redundant env-reading logic from backend.rs; it's already handled by reqsign's DefaultCredentialProvider. Verified locally with STS temporary credentials. PTAL, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/cos): support cosn scheme and STS security token [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #7418:
URL: https://github.com/apache/opendal/pull/7418#discussion_r3235244417
##########
core/services/cos/src/backend.rs:
##########
@@ -199,14 +220,21 @@ impl Builder for CosBuilder {
self.config.secret_id.as_deref(),
self.config.secret_key.as_deref(),
) {
- let security_token = envs
- .get("TENCENTCLOUD_TOKEN")
- .or_else(|| envs.get("TENCENTCLOUD_SECURITY_TOKEN"))
- .or_else(|| envs.get("QCLOUD_SECRET_TOKEN"));
+ // Precedence:
+ // 1. explicit `config.security_token` set by the user;
+ // 2. environment variables when config load is NOT disabled.
+ let env_token = if self.config.disable_config_load {
+ None
+ } else {
+ envs.get("TENCENTCLOUD_TOKEN")
Review Comment:
Those logic should be in reqsign, can you double check there?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org