You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "waynexia (via GitHub)" <gi...@apache.org> on 2024/02/21 15:45:46 UTC
[PR] feat: object store adapter based on v0.9 [opendal]
waynexia opened a new pull request, #4233:
URL: https://github.com/apache/opendal/pull/4233
Bump the `object_store` version in `object_store_opendal` to `v0.9`
Changed API:
- `put`: request file meta in return value
- `put_opts`: new API
- `list` not an async fn
- `list_with_offset`: not an async fn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: object store adapter based on v0.9 [opendal]
Posted by "waynexia (via GitHub)" <gi...@apache.org>.
waynexia commented on code in PR #4233:
URL: https://github.com/apache/opendal/pull/4233#discussion_r1498714272
##########
integrations/object_store/src/lib.rs:
##########
@@ -62,12 +65,34 @@ impl std::fmt::Display for OpendalStore {
#[async_trait]
impl ObjectStore for OpendalStore {
- async fn put(&self, location: &Path, bytes: Bytes) -> Result<()> {
- Ok(self
- .inner
+ async fn put(&self, location: &Path, bytes: Bytes) -> Result<PutResult> {
+ self.inner
.write(location.as_ref(), bytes)
.await
- .map_err(|err| format_object_store_error(err, location.as_ref()))?)
+ .map_err(|err| format_object_store_error(err, location.as_ref()))?;
+ let meta = self
+ .inner
+ .stat(location.as_ref())
+ .await
+ .map_err(|err| format_object_store_error(err, location.as_ref()))?;
Review Comment:
Fixed https://github.com/apache/opendal/pull/4233/commits/877b72a2d7f99254660bc8e1c4732fa3c5fa35b9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: object store adapter based on v0.9 [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #4233:
URL: https://github.com/apache/opendal/pull/4233#discussion_r1498619666
##########
integrations/object_store/src/lib.rs:
##########
@@ -62,12 +65,34 @@ impl std::fmt::Display for OpendalStore {
#[async_trait]
impl ObjectStore for OpendalStore {
- async fn put(&self, location: &Path, bytes: Bytes) -> Result<()> {
- Ok(self
- .inner
+ async fn put(&self, location: &Path, bytes: Bytes) -> Result<PutResult> {
+ self.inner
.write(location.as_ref(), bytes)
.await
- .map_err(|err| format_object_store_error(err, location.as_ref()))?)
+ .map_err(|err| format_object_store_error(err, location.as_ref()))?;
+ let meta = self
+ .inner
+ .stat(location.as_ref())
+ .await
+ .map_err(|err| format_object_store_error(err, location.as_ref()))?;
Review Comment:
It's possible for us to introduce the same thing. Before we implement it, we can set those field to `None`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: object store adapter based on v0.9 [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo merged PR #4233:
URL: https://github.com/apache/opendal/pull/4233
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: object store adapter based on v0.9 [opendal]
Posted by "waynexia (via GitHub)" <gi...@apache.org>.
waynexia commented on code in PR #4233:
URL: https://github.com/apache/opendal/pull/4233#discussion_r1497819107
##########
integrations/object_store/Cargo.toml:
##########
@@ -31,9 +31,10 @@ version = "0.42.0+core.0.45.0"
async-trait = "0.1"
bytes = "1"
futures = "0.3"
-object_store = "0.7"
+futures-util = "0.3"
+object_store = "0.9"
opendal = { version = "0.45.0", path = "../../core" }
-tokio = "1"
+tokio = { version = "1", default-features = false }
Review Comment:
Limit the feature gates of `tokio`
##########
integrations/object_store/src/lib.rs:
##########
@@ -62,12 +65,34 @@ impl std::fmt::Display for OpendalStore {
#[async_trait]
impl ObjectStore for OpendalStore {
- async fn put(&self, location: &Path, bytes: Bytes) -> Result<()> {
- Ok(self
- .inner
+ async fn put(&self, location: &Path, bytes: Bytes) -> Result<PutResult> {
+ self.inner
.write(location.as_ref(), bytes)
.await
- .map_err(|err| format_object_store_error(err, location.as_ref()))?)
+ .map_err(|err| format_object_store_error(err, location.as_ref()))?;
+ let meta = self
+ .inner
+ .stat(location.as_ref())
+ .await
+ .map_err(|err| format_object_store_error(err, location.as_ref()))?;
+ Ok(PutResult {
+ e_tag: meta.etag().map(|x| x.to_string()),
+ version: meta.version().map(|x| x.to_string()),
+ })
+ }
+
+ async fn put_opts(
+ &self,
+ _location: &Path,
+ _bytes: Bytes,
+ _opts: PutOptions,
Review Comment:
Looks like `Operator` doesn't have a similar thing. So I left it unimplemented.
##########
integrations/object_store/src/lib.rs:
##########
@@ -62,12 +65,34 @@ impl std::fmt::Display for OpendalStore {
#[async_trait]
impl ObjectStore for OpendalStore {
- async fn put(&self, location: &Path, bytes: Bytes) -> Result<()> {
- Ok(self
- .inner
+ async fn put(&self, location: &Path, bytes: Bytes) -> Result<PutResult> {
+ self.inner
.write(location.as_ref(), bytes)
.await
- .map_err(|err| format_object_store_error(err, location.as_ref()))?)
+ .map_err(|err| format_object_store_error(err, location.as_ref()))?;
+ let meta = self
+ .inner
+ .stat(location.as_ref())
+ .await
+ .map_err(|err| format_object_store_error(err, location.as_ref()))?;
Review Comment:
Is there any way to avoid this extra `stat` call?
##########
integrations/object_store/Cargo.toml:
##########
@@ -31,9 +31,10 @@ version = "0.42.0+core.0.45.0"
async-trait = "0.1"
bytes = "1"
futures = "0.3"
-object_store = "0.7"
+futures-util = "0.3"
Review Comment:
for adapting future to stream. See `list`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org