You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "sundy-li (via GitHub)" <gi...@apache.org> on 2024/01/23 14:38:22 UTC
[I] `oio::Write::poll_close` can't be retried multiple times when error occurs [opendal]
sundy-li opened a new issue, #4058:
URL: https://github.com/apache/opendal/issues/4058
https://github.com/apache/opendal/blob/3c4e2f1c42b88323ed13c4eb56b95e75b98b75a9/core/src/services/fs/writer.rs#L67C8-L91
```
fn poll_close(&mut self, cx: &mut Context<'_>) -> Poll<Result<()>> {
loop {
if let Some(fut) = self.fut.as_mut() {
let res = ready!(fut.poll_unpin(cx));
self.fut = None;
return Poll::Ready(res);
}
let mut f = self.f.take().expect("FsWriter must be initialized");
...
}
```
If any error happens(eg: disk full), retry layer will retry to call `poll_close` function, but the `self.f` is taken, so panic happens.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `fs::Writer::poll_close` can't be retried multiple times when error occurs [opendal]
Posted by "hoslo (via GitHub)" <gi...@apache.org>.
hoslo commented on issue #4058:
URL: https://github.com/apache/opendal/issues/4058#issuecomment-1925296019
> > @Xuanwo重试期间似乎运行良好。您认为这有什么问题吗？https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=b7bb0b4ad4ddc48cc08566694b41adb8
>
> 很抱歉没有向您通报进展情况。主要问题是发生错误时`poll_close`接受`F`但不返回它。我们可以调整 future 以返回`(F, Result<()>)`.
My link above does this, can I create a new PR for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `fs::Writer::poll_close` can't be retried multiple times when error occurs [opendal]
Posted by "hoslo (via GitHub)" <gi...@apache.org>.
hoslo commented on issue #4058:
URL: https://github.com/apache/opendal/issues/4058#issuecomment-1925197038
@Xuanwo It seems to be running fine during the retry. Do you see any issues with this?
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=b7bb0b4ad4ddc48cc08566694b41adb8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `fs::Writer::poll_close` can't be retried multiple times when error occurs [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4058:
URL: https://github.com/apache/opendal/issues/4058#issuecomment-5244843566
Closing as completed. The poll-based FsWriter::poll_close implementation from this report no longer exists. The current oio::Write::close API borrows &mut self, and FsWriter keeps the file on the writer across close errors, so retrying close no longer hits the taken-file panic described here. #6325 remains open for the separate disk-full flush/data-loss case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `fs::Writer::poll_close` can't be retried multiple times when error occurs [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4058:
URL: https://github.com/apache/opendal/issues/4058#issuecomment-1925284374
> @Xuanwo It seems to be running fine during the retry. Do you see any issues with this? https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=b7bb0b4ad4ddc48cc08566694b41adb8
Apologies for not updating you on the progress. The main issue is that `poll_close` takes `F` and doesn't return it when an error occurs. We could adjust the future to instead return a `(F, Result<()>)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `fs::Writer::poll_close` can't be retried multiple times when error occurs [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo closed issue #4058: `fs::Writer::poll_close` can't be retried multiple times when error occurs
URL: https://github.com/apache/opendal/issues/4058
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `fs::Writer::poll_close` can't be retried multiple times when error occurs [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4058:
URL: https://github.com/apache/opendal/issues/4058#issuecomment-1925299084
> My link above does this, can I create a new PR for this?
Welcome! Thanks in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `fs::Writer::poll_close` can't be retried multiple times when error occurs [opendal]
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
tisonkun commented on issue #4058:
URL: https://github.com/apache/opendal/issues/4058#issuecomment-2029269950
@Xuanwo what is the progress of this issue?
I saw #4141 has been superseded by #4309, but perhaps the original issue is yet to resolved?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `fs::Writer::poll_close` can't be retried multiple times when error occurs [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4058:
URL: https://github.com/apache/opendal/issues/4058#issuecomment-2029351887
> I saw #4141 has been superseded by #4309, but perhaps the original issue is yet to resolved?
Yes, I'm working with the tokio team on this https://github.com/tokio-rs/tokio/pull/6330
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] `fs::Writer::poll_close` can't be retried multiple times when error occurs [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4058:
URL: https://github.com/apache/opendal/issues/4058#issuecomment-1906201418
Also affects `hdfs`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org