You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "zooeywm (via GitHub)" <gi...@apache.org> on 2023/09/04 08:39:04 UTC
[GitHub] [incubator-opendal] zooeywm created a discussion: S3 service root and region not effective
GitHub user zooeywm created a discussion: S3 service root and region not effective
1. Operator create:
```rust
fn create_s3_operator(
storage_server_id: Uuid,
options: &ObjectServerOption,
) -> anyhow::Result<Operator> {
let endpoint = &options.endpoint;
let access_key_id = &options.access_key_id;
let secret_access_key = &options.secret_access_key;
let region = &options.region;
let bucket = &options.default_bucket;
let mut builder = S3::default();
builder
.endpoint(endpoint)
.root(&format!("storage-{}", storage_server_id))
.bucket(bucket)
.region(region)
.access_key_id(access_key_id)
.secret_access_key(secret_access_key)
.allow_anonymous()
.build()?;
Ok(Operator::new(builder)?.finish())
}
```
2. Use operator:
```rust
async fn upload(
&self,
storage_server: &StorageServer,
meta_id: Uuid,
content: &[u8],
) -> ... {
...
let operator = create_s3_operator(storage_server.id, options)?;
operator.write(&meta_id.to_string(), content.to_owned()).await?;
...
}
```
in my expectation, this operator will create a file named `{meta_id}` in the path `storage-{storage_server_id}`, but the result is that it creates the file named {meta_id}` in the root directory.
And I find that the build info has logged twice:
```
2023-09-04T08:02:06.663003Z DEBUG ThreadId(19) opendal::services::s3::backend: xx\opendal-0.39.0\src\services\s3\backend.rs:692: backend build started: Builder { root: Some("storage-4f600062-0105-40a4-aa7d-6f373cc51358"), bucket: "xx", endpoint: Some("xx"), region: Some("xx"), .. }
2023-09-04T08:02:06.663295Z DEBUG ThreadId(19) opendal::services::s3::backend: xx\opendal-0.39.0\src\services\s3\backend.rs:695: backend use root /storage-4f600062-0105-40a4-aa7d-6f373cc51358/
...
2023-09-04T08:02:06.669007Z DEBUG ThreadId(19) opendal::services::s3::backend: xx\opendal-0.39.0\src\services\s3\backend.rs:692: backend build started: Builder { root: None, bucket: "xx", endpoint: Some("xx"), region: None, .. }
2023-09-04T08:02:06.669186Z DEBUG ThreadId(19) opendal::services::s3::backend: xx\opendal-0.39.0\src\services\s3\backend.rs:695: backend use root /
```
GitHub link: https://github.com/apache/incubator-opendal/discussions/3006
----
This is an automatically sent email for commits@opendal.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@opendal.apache.org
[GitHub] [incubator-opendal] Xuanwo edited a comment on the discussion: S3 service root and region not effective
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
GitHub user Xuanwo edited a comment on the discussion: S3 service root and region not effective
Please don't call `builder.build?` directly, it will take and consume `builder` itself. Instead:
```rust
let mut builder = S3::default();
builder
.endpoint(endpoint)
.root(&format!("storage-{}", storage_server_id))
.bucket(bucket)
.region(region)
.access_key_id(access_key_id)
.secret_access_key(secret_access_key)
.allow_anonymous();
Ok(Operator::new(builder)?.finish())
```
GitHub link: https://github.com/apache/incubator-opendal/discussions/3006#discussioncomment-6902460
----
This is an automatically sent email for commits@opendal.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@opendal.apache.org
[GitHub] [incubator-opendal] Xuanwo added a comment to the discussion: S3 service root and region not effective
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
GitHub user Xuanwo added a comment to the discussion: S3 service root and region not effective
```rust
let mut builder = S3::default();
builder
.endpoint(endpoint)
.root(&format!("storage-{}", storage_server_id))
.bucket(bucket)
.region(region)
.access_key_id(access_key_id)
.secret_access_key(secret_access_key)
.allow_anonymous()
.build()?;
Ok(Operator::new(builder)?.finish())
```
Please don't call `builder.build?` directly, it will take and consume `builder` itself. Instead:
```rust
let mut builder = S3::default();
builder
.endpoint(endpoint)
.root(&format!("storage-{}", storage_server_id))
.bucket(bucket)
.region(region)
.access_key_id(access_key_id)
.secret_access_key(secret_access_key)
.allow_anonymous();
Ok(Operator::new(builder)?.finish())
```
GitHub link: https://github.com/apache/incubator-opendal/discussions/3006#discussioncomment-6902460
----
This is an automatically sent email for commits@opendal.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@opendal.apache.org
[GitHub] [incubator-opendal] zooeywm added a comment to the discussion: S3 service root and region not effective
Posted by "zooeywm (via GitHub)" <gi...@apache.org>.
GitHub user zooeywm added a comment to the discussion: S3 service root and region not effective
Thanks for your answer! It solved my problem. I have misunderstood the meaning of builder.
GitHub link: https://github.com/apache/incubator-opendal/discussions/3006#discussioncomment-6902589
----
This is an automatically sent email for commits@opendal.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@opendal.apache.org
[GitHub] [incubator-opendal] zooeywm closed a discussion: S3 service root and region not effective
Posted by "zooeywm (via GitHub)" <gi...@apache.org>.
GitHub user zooeywm closed a discussion: S3 service root and region not effective
1. Operator create:
```rust
fn create_s3_operator(
storage_server_id: Uuid,
options: &ObjectServerOption,
) -> anyhow::Result<Operator> {
let endpoint = &options.endpoint;
let access_key_id = &options.access_key_id;
let secret_access_key = &options.secret_access_key;
let region = &options.region;
let bucket = &options.default_bucket;
let mut builder = S3::default();
builder
.endpoint(endpoint)
.root(&format!("storage-{}", storage_server_id))
.bucket(bucket)
.region(region)
.access_key_id(access_key_id)
.secret_access_key(secret_access_key)
.allow_anonymous()
.build()?;
Ok(Operator::new(builder)?.finish())
}
```
2. Use operator:
```rust
async fn upload(
&self,
storage_server: &StorageServer,
meta_id: Uuid,
content: &[u8],
) -> ... {
...
let operator = create_s3_operator(storage_server.id, options)?;
operator.write(&meta_id.to_string(), content.to_owned()).await?;
...
}
```
in my expectation, this operator will create a file named `{meta_id}` in the path `storage-{storage_server_id}`, but the result is that it creates the file named {meta_id}` in the root directory.
And I find that the build info has logged twice:
```
2023-09-04T08:02:06.663003Z DEBUG ThreadId(19) opendal::services::s3::backend: xx\opendal-0.39.0\src\services\s3\backend.rs:692: backend build started: Builder { root: Some("storage-4f600062-0105-40a4-aa7d-6f373cc51358"), bucket: "xx", endpoint: Some("xx"), region: Some("xx"), .. }
2023-09-04T08:02:06.663295Z DEBUG ThreadId(19) opendal::services::s3::backend: xx\opendal-0.39.0\src\services\s3\backend.rs:695: backend use root /storage-4f600062-0105-40a4-aa7d-6f373cc51358/
...
2023-09-04T08:02:06.669007Z DEBUG ThreadId(19) opendal::services::s3::backend: xx\opendal-0.39.0\src\services\s3\backend.rs:692: backend build started: Builder { root: None, bucket: "xx", endpoint: Some("xx"), region: None, .. }
2023-09-04T08:02:06.669186Z DEBUG ThreadId(19) opendal::services::s3::backend: xx\opendal-0.39.0\src\services\s3\backend.rs:695: backend use root /
```
GitHub link: https://github.com/apache/incubator-opendal/discussions/3006
----
This is an automatically sent email for commits@opendal.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@opendal.apache.org