You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "flaneur2020 (via GitHub)" <gi...@apache.org> on 2024/08/05 10:30:09 UTC
[I] new feature: allow inject a customized signer for aws s3 [opendal]
flaneur2020 opened a new issue, #4960:
URL: https://github.com/apache/opendal/issues/4960
### Feature Description
background: https://github.com/apache/iceberg-rust/issues/506
> It seems we need to implement this signer in iceberg-rust and pass it as a customized signer to opendal.
### Problem and Solution
but currently the signer is a struct `AwsV4Signer`.
we need make the signer in `S3Core` a trait before allowing it customizable.
maybe we can defined a trait called `S3Signer` in opendal, and make `AwsV4Signer` as it's implementation by default.
### Additional Context
_No response_
### Are you willing to contribute to the development of this feature?
- [X] Yes, I am willing to contribute to the development of this feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] new feature: allow inject a customized signer for aws s3 [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4960:
URL: https://github.com/apache/opendal/issues/4960#issuecomment-2268892473
Hi, [`S3::customized_credential_load`](https://docs.rs/opendal/latest/opendal/services/struct.S3.html#method.customized_credential_load) should allow you to implement your own logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] new feature: allow inject a customized signer for aws s3 [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4960:
URL: https://github.com/apache/opendal/issues/4960#issuecomment-2269425885
Thank you for the clarification. I now understand what the problem is.
My current ideas:
- Add an `Sign` trait for entire reqsign
```rust
use http;
#[async_trait]
trait Sign: 'static {
type Credential;
async fn sign(&self, req: &mut http::request::Parts, cred: &Self::Credential) -> Result<()>
}
```
- Expose a `Signer` from the reqsign
reqsign will expose a `type AwsSigner = Arc<dyn Sign<Credential=AwsCredential>>`.
- OpenDAL can accept any `impl Sign<Credential=AwsCredential>` with AwsV4Signer as default impl.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] new feature: allow inject a customized signer for aws s3 [opendal]
Posted by "meteorgan (via GitHub)" <gi...@apache.org>.
meteorgan commented on issue #4960:
URL: https://github.com/apache/opendal/issues/4960#issuecomment-2561095815
@rshkv The code in v0.16.x hasn't been merged into `main`. Maybe that's the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] new feature: allow inject a customized signer for aws s3 [opendal]
Posted by "rshkv (via GitHub)" <gi...@apache.org>.
rshkv commented on issue #4960:
URL: https://github.com/apache/opendal/issues/4960#issuecomment-2561077963
@Xuanwo, I'm slightly confused about the the state of `reqsign` with respect to configurable signers.
The new `Sign` trait was introduced in https://github.com/Xuanwo/reqsign/pull/459 and then replaced in https://github.com/Xuanwo/reqsign/pull/482 with the `Load` and `Build` traits which should achieve the same.
I'm confused about how `reqsign` is published (new to Rust). The 0.16.1 crate docs have an [`AwsV4Signer`](https://docs.rs/reqsign/0.16.1/reqsign/struct.AwsV4Signer.html) which I don't find in the code. And the 0.16.1 crate docs don't have `Load` or `Build` traits. (But I also can't find the `reqsign_core` and the `reqsign_aws_v4` crates published so I think I'm missing something.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] new feature: allow inject a customized signer for aws s3 [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4960:
URL: https://github.com/apache/opendal/issues/4960#issuecomment-2561110372
> The new `Sign` trait was introduced in [Xuanwo/reqsign#459](https://github.com/Xuanwo/reqsign/pull/459) and then replaced in [Xuanwo/reqsign#482](https://github.com/Xuanwo/reqsign/pull/482) with the `Load` and `Build` traits which should achieve the same.
Hi, that will be included in the `0.17` release.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] new feature: allow inject a customized signer for aws s3 [opendal]
Posted by "rshkv (via GitHub)" <gi...@apache.org>.
rshkv commented on issue #4960:
URL: https://github.com/apache/opendal/issues/4960#issuecomment-2561834751
Ah, gotcha. Thank you for the quick reply.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] new feature: allow inject a customized signer for aws s3 [opendal]
Posted by "flaneur2020 (via GitHub)" <gi...@apache.org>.
flaneur2020 commented on issue #4960:
URL: https://github.com/apache/opendal/issues/4960#issuecomment-2268880929
to make the `Signer` customizable, i've got some difficulties to solely add it in the opendal part.
adding a trait called `S3Signer` in opendal requires a signature to be object safe, but a signature like `fn sign<T>(&self, req: &mut Request<T>, cred: &AwsCredential) -> Result<()>` is
not object safe because it contains a generic parameter inside it.
so i think there might have two ways to make it possible:
1. add a `GenericSigner` trait with `fn sign(&self, req: &mut dyn SignableRequest, cred: &AwsCredential) -> Result<()>` in `reqsign`.
but as `Signer` accepts `&mut impl SignableRequest`, we may have to change the api of `Signer` to allow it accept trait objects.
3. change `Signer` into an `enum` like:
```
pub enum Signer {
V4(V4Signer),
Boxed(Box<dyn CustomSigner>)
}
```
this approach can make the api of `Signer` unchanged, but would introduce a small dispatch cost on every sign.
@Xuanwo any suggestion about this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] new feature: allow inject a customized signer for aws s3 [opendal]
Posted by "flaneur2020 (via GitHub)" <gi...@apache.org>.
flaneur2020 commented on issue #4960:
URL: https://github.com/apache/opendal/issues/4960#issuecomment-2269108002
> Hi, [`S3::customized_credential_load`](https://docs.rs/opendal/latest/opendal/services/struct.S3.html#method.customized_credential_load) should allow you to implement your own logic.
imho `customized_credential_load` is to customize the credential logic, but here we need customize is the signing logic.
the `Signer` currently is not a trait like `dyn AwsCredentialLoad` yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org