You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2023/09/13 11:51:54 UTC

[GitHub] [arrow-datafusion] alamb opened a new issue, #7546: DiskManager's temp files can be deleted too early

alamb opened a new issue, #7546:
URL: https://github.com/apache/arrow-datafusion/issues/7546

   ### Describe the bug
   
   After https://github.com/apache/arrow-datafusion/pull/7510, we saw intermittent errors in a test reading spill files:  https://github.com/apache/arrow-datafusion/issues/7537 and  https://github.com/apache/arrow-datafusion/issues/7523
   
   Some great debugging by @sarutak  and @viirya  (see description on https://github.com/apache/arrow-datafusion/pull/7538#issue-1893761145) found that the `NamedTempFiles` returned by [`DiskManager::create_tmp_file`](https://docs.rs/datafusion/latest/datafusion/execution/struct.DiskManager.html#method.create_tmp_file) can be deleted too soon, if the DiskManager is dropped while the plan is still outstanding
   
   ### To Reproduce
   
   Run a plan and drop the disk manager
   
   ### Expected behavior
   
   It should not be possible to have the NamedtempFiles deleted out from under a running plan.
   
   There are a bunch of various proposals on https://github.com/apache/arrow-datafusion/pull/7538
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-datafusion] alamb closed issue #7546: DiskManager's temp files can be deleted too early

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #7546: DiskManager's temp files can be deleted too early
URL: https://github.com/apache/arrow-datafusion/issues/7546


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org