You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "howardlau1999 (via GitHub)" <gi...@apache.org> on 2025/08/12 09:39:15 UTC
[PR] WIP: feat(services/oss): allow users to specify endpoint addressing style [opendal]
howardlau1999 opened a new pull request, #6504:
URL: https://github.com/apache/opendal/pull/6504
# Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
Closes #.
# Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
Allow users to access OSS using custom CNAME endpoints or self-hosted proxies.
There are three styles for addressing a bucket:
1. Virtual-host: the bucket name will be prepended to the host. This is the common case when a user accesses OSS using official `oss-*.aliyuncs.com` endpoints.
2. Path: the bucket name will be the first segment of the path. This is the case when a user sets up a reverse proxy and a custom domain to access OSS but does not bind the domain to a specific bucket.
3. Cname: the domain name itself is bound to a specific bucket, both host and path should not be changed. This is the case when a user accesses OSS using a custom domain which is bound to a specific bucket. The path itself is the object name.
# What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
Change the endpoint parsing function accordingly.
# Are there any user-facing changes?
Two config options are added.
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!---
If there are any breaking changes to public APIs, please add the `breaking-changes` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2275727995
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
What's the difference of `CNAME` compared to `path-style`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "howardlau1999 (via GitHub)" <gi...@apache.org>.
howardlau1999 commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2269745924
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
> How about an enum?
I was using an enum at first but it seems that the config parser does not support parsing enums so I changed this to a string. Do you have better suggestions?
https://github.com/apache/opendal/commit/72bf31203575f475f2718fcc23f7404ccd980a3d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "howardlau1999 (via GitHub)" <gi...@apache.org>.
howardlau1999 commented on PR #6504:
URL: https://github.com/apache/opendal/pull/6504#issuecomment-3260314920
@Xuanwo I have addressed your comments. PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] WIP: feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "howardlau1999 (via GitHub)" <gi...@apache.org>.
howardlau1999 commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2269437134
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
If only two addressing style can be added in OpenDAL, I think the cname one will be more useful
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
If only two addressing styles can be added in OpenDAL, I think the cname one will be more useful
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "howardlau1999 (via GitHub)" <gi...@apache.org>.
howardlau1999 commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2269745924
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
> How about an enum?
I was using an enum at first but it seems that the config parser does not support parse enums so I changed this to a string. Do you have better suggestions?
https://github.com/apache/opendal/commit/72bf31203575f475f2718fcc23f7404ccd980a3d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #6504:
URL: https://github.com/apache/opendal/pull/6504#issuecomment-3245530282
> I am quite occupied these days. I will address your comments this weekend.
>
>
>
> 获取Outlook for Android<https://aka.ms/AAb9ysg>
>
> ________________________________
>
> From: Xuanwo ***@***.***>
>
> Sent: Tuesday, September 2, 2025 5:46:28 PM
>
> To: apache/opendal ***@***.***>
>
> Cc: Howard Lau ***@***.***>; Mention ***@***.***>
>
> Subject: Re: [apache/opendal] feat(services/oss): allow users to specify endpoint addressing style (PR #6504)
>
>
>
> [https://avatars.githubusercontent.com/u/5351546?s=20&v=4]Xuanwo left a comment (apache/opendal#6504)<https://github.com/apache/opendal/pull/6504#issuecomment-3244594169>
>
>
>
> Thank you for the explanation. The PR looks good to me. The only thing we need is to add a private enum, parse it, return meaningful errors to users, and then use it in the OSS.
>
>
>
> Hi, @howardlau1999<https://github.com/howardlau1999> the only thing left for this PR to get merged.
>
>
>
> ―
>
> Reply to this email directly, view it on GitHub<https://github.com/apache/opendal/pull/6504#issuecomment-3244594169>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABIB3ORV7X2BE4TC4QUPPDT3QVRPJAVCNFSM6AAAAACDV44TO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTENBUGU4TIMJWHE>.
>
> You are receiving this because you were mentioned.Message ID: ***@***.***>
>
>
Really appreciated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "howardlau1999 (via GitHub)" <gi...@apache.org>.
howardlau1999 commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2275785600
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
> What's the difference of `CNAME` compared to `path-style`?
For path style, the bucket name will appear in the URL path, i.e. https://oss-<region>.aliyuncs.com/mybucket/objectname. However, this is normally forbidden. But for some historical reasons, some buckets are allowed to be accessed using this method.
For cname, the user should bind a bucket to a custom domain and point the domain to official endpoints using CNAME records. For example, if the user binds a bucket named `mybucket` to a domain `storage.foo.com` and sets up a CNAME record to point `storage.foo.com` to an official endpoint (e.g. `oss-us-east-1.aliyuncs.com`), then no bucket name will be needed to appear when using this domain to access objects. The URL is just https://storage.foo.com/objectname.
Technically, this is all about the `Host` header in the request. If a custom domain is not bound to a bucket in OSS, the access will be denied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #6504:
URL: https://github.com/apache/opendal/pull/6504#issuecomment-3244594169
> Thank you for the explanation. The PR looks good to me. The only thing we need is to add a private enum, parse it, return meaningful errors to users, and then use it in the OSS.
Hi, @howardlau1999 the only thing left for this PR to get merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] WIP: feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2269398684
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
Thank for working on this but I think adding an `addressing_style` is too much for us. How about adding an option called `enable_path_style`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] WIP: feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "howardlau1999 (via GitHub)" <gi...@apache.org>.
howardlau1999 commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2269434510
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
As documented in the official docs, there are three addressing style available, so I think a boolean switch may be insufficient.
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
As documented in the official docs, there are three addressing styles available, so I think a boolean switch may be insufficient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo merged PR #6504:
URL: https://github.com/apache/opendal/pull/6504
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2277128524
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
Thank you for the explanation. The PR looks good to me. The only thing we need is to add a private enum, parse it, return meaningful errors to users, and then use it in the OSS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "howardlau1999 (via GitHub)" <gi...@apache.org>.
howardlau1999 commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2275785600
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
> What's the difference of `CNAME` compared to `path-style`?
For path style, the bucket name will appear in the URL path, i.e. https://oss-$region.aliyuncs.com/mybucket/objectname. However, this is normally forbidden. But for some historical reasons, some buckets are allowed to be accessed using this method.
For cname, the user should bind a bucket to a custom domain and point the domain to official endpoints using CNAME records. For example, if the user binds a bucket named `mybucket` to a domain `storage.foo.com` and sets up a CNAME record to point `storage.foo.com` to an official endpoint (e.g. `oss-us-east-1.aliyuncs.com`), then no bucket name will be needed to appear when using this domain to access objects. The URL is just https://storage.foo.com/objectname.
Technically, this is all about the `Host` header in the request. If a custom domain is not bound to a bucket in OSS, the access will be denied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] WIP: feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "asukaminato0721 (via GitHub)" <gi...@apache.org>.
asukaminato0721 commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2269718193
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
How about an enum?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #6504:
URL: https://github.com/apache/opendal/pull/6504#discussion_r2275751314
##########
core/src/services/oss/backend.rs:
##########
@@ -104,6 +104,13 @@ impl OssBuilder {
self
}
+ /// Set addressing style for the endpoint.
+ pub fn addressing_style(mut self, addressing_style: &str) -> Self {
Review Comment:
I see, I understand.
---
Your existing implementation is sufficient. We can accept `&str` and convert it to a private enum (note this should not be part of the public API). Others LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(services/oss): allow users to specify endpoint addressing style [opendal]
Posted by "howardlau1999 (via GitHub)" <gi...@apache.org>.
howardlau1999 commented on PR #6504:
URL: https://github.com/apache/opendal/pull/6504#issuecomment-3245385145
I am quite occupied these days. I will address your comments this weekend.
获取Outlook for Android<https://aka.ms/AAb9ysg>
________________________________
From: Xuanwo ***@***.***>
Sent: Tuesday, September 2, 2025 5:46:28 PM
To: apache/opendal ***@***.***>
Cc: Howard Lau ***@***.***>; Mention ***@***.***>
Subject: Re: [apache/opendal] feat(services/oss): allow users to specify endpoint addressing style (PR #6504)
[https://avatars.githubusercontent.com/u/5351546?s=20&v=4]Xuanwo left a comment (apache/opendal#6504)<https://github.com/apache/opendal/pull/6504#issuecomment-3244594169>
Thank you for the explanation. The PR looks good to me. The only thing we need is to add a private enum, parse it, return meaningful errors to users, and then use it in the OSS.
Hi, @howardlau1999<https://github.com/howardlau1999> the only thing left for this PR to get merged.
―
Reply to this email directly, view it on GitHub<https://github.com/apache/opendal/pull/6504#issuecomment-3244594169>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABIB3ORV7X2BE4TC4QUPPDT3QVRPJAVCNFSM6AAAAACDV44TO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTENBUGU4TIMJWHE>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org