You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2023/09/13 11:51:54 UTC
[GitHub] [arrow-datafusion] alamb opened a new issue, #7546: DiskManager's temp files can be deleted too early
alamb opened a new issue, #7546:
URL: https://github.com/apache/arrow-datafusion/issues/7546
### Describe the bug
After https://github.com/apache/arrow-datafusion/pull/7510, we saw intermittent errors in a test reading spill files: https://github.com/apache/arrow-datafusion/issues/7537 and https://github.com/apache/arrow-datafusion/issues/7523
Some great debugging by @sarutak and @viirya (see description on https://github.com/apache/arrow-datafusion/pull/7538#issue-1893761145) found that the `NamedTempFiles` returned by [`DiskManager::create_tmp_file`](https://docs.rs/datafusion/latest/datafusion/execution/struct.DiskManager.html#method.create_tmp_file) can be deleted too soon, if the DiskManager is dropped while the plan is still outstanding
### To Reproduce
Run a plan and drop the disk manager
### Expected behavior
It should not be possible to have the NamedtempFiles deleted out from under a running plan.
There are a bunch of various proposals on https://github.com/apache/arrow-datafusion/pull/7538
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #7546: DiskManager's temp files can be deleted too early
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #7546: DiskManager's temp files can be deleted too early
URL: https://github.com/apache/arrow-datafusion/issues/7546
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org