You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "Xuanwo (via GitHub)" <gi...@apache.org> on 2023/04/28 09:30:08 UTC
[GitHub] [incubator-opendal] Xuanwo opened a new issue, #2158: bug: Retry for batch operation seems not work as expected
Xuanwo opened a new issue, #2158:
URL: https://github.com/apache/incubator-opendal/issues/2158
The following batch operation is not retried:
```rust
[2023-04-28T09:26:16Z DEBUG opendal::services] service=s3 operati***=batch-delete count=1000 -> finished: 1000, succeed: 999, failed: 1
[2023-04-28T09:26:16Z DEBUG opendal::services] service=s3 operati***=scan path=test_list_rich_dir/ -> partial entries read finished
[2023-04-28T09:26:16Z DEBUG opendal::services] service=s3 operati***=list path=test_list_rich_dir/ -> all entries read finished
Error: Unexpected (permanent) at delete, c***text: { service: s3, path: test_list_rich_dir/file-468 } => DeleteObjectsResultError { code: "InternalError", key: "***28fbef7d-d9ce-44a3-8d17-1da46778a322/test_list_rich_dir/file-468", message: "We encountered an internal error. Please try again." }
failures:
services_s3::list_test_list_rich_dir
```
https://github.com/apache/incubator-opendal/actions/runs/4829106954/jobs/8603738353?pr=2157
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Ji-Xinyou commented on issue #2158: bug: Retry for batch operation seems not work as expected
Posted by "Ji-Xinyou (via GitHub)" <gi...@apache.org>.
Ji-Xinyou commented on issue #2158:
URL: https://github.com/apache/incubator-opendal/issues/2158#issuecomment-1528959057
Hi @Xuanwo, you could assign this to me if possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Ji-Xinyou commented on issue #2158: bug: Retry for batch operation seems not work as expected
Posted by "Ji-Xinyou (via GitHub)" <gi...@apache.org>.
Ji-Xinyou commented on issue #2158:
URL: https://github.com/apache/incubator-opendal/issues/2158#issuecomment-1528723256
> > This makes the error `Permanent`.
>
> I got it. We aren't parse the errors returned in batch correctly.
Yup, I think some errors should be permanent while the others could be temporary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo closed issue #2158: bug: Retry for batch operation seems not work as expected
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo closed issue #2158: bug: Retry for batch operation seems not work as expected
URL: https://github.com/apache/incubator-opendal/issues/2158
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Ji-Xinyou commented on issue #2158: bug: Retry for batch operation seems not work as expected
Posted by "Ji-Xinyou (via GitHub)" <gi...@apache.org>.
Ji-Xinyou commented on issue #2158:
URL: https://github.com/apache/incubator-opendal/issues/2158#issuecomment-1528719343
The bug seems is caused by
```Rust
for i in result.error {
let path = build_rel_path(&self.core.root, &i.key);
batched_result.push((
path,
Err(Error::new(ErrorKind::Unexpected, &format!("{i:?}"))),
));
}
```
This makes the error `Permanent`.
But in our retry layer, we have
```Rust
Ok(rp) => rp.results().iter().any(|(_, v)| {
if let Err(e) = v {
e.is_temporary()
} else {
false
}
}),
Err(err) => err.is_temporary(),
````
Which only retries temporary errors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on issue #2158: bug: Retry for batch operation seems not work as expected
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #2158:
URL: https://github.com/apache/incubator-opendal/issues/2158#issuecomment-1528720880
> This makes the error `Permanent`.
I got it. We aren't parse the errors returned in batch correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on issue #2158: bug: Retry for batch operation seems not work as expected
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #2158:
URL: https://github.com/apache/incubator-opendal/issues/2158#issuecomment-1528961071
Thanks a lot! Have fun!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Ji-Xinyou commented on issue #2158: bug: Retry for batch operation seems not work as expected
Posted by "Ji-Xinyou (via GitHub)" <gi...@apache.org>.
Ji-Xinyou commented on issue #2158:
URL: https://github.com/apache/incubator-opendal/issues/2158#issuecomment-1528723643
To implement this, we should take [this](https://docs.aws.amazon.com/AmazonS3/latest/API/API_Error.html) as a reference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org