[jira] [Created] (ARROW-18313) Issues with open_dataset()

["N Gautam Animesh (Jira)" <ji...@apache.org>]

N Gautam Animesh created ARROW-18313:
----------------------------------------

             Summary: Issues with open_dataset()
                 Key: ARROW-18313
                 URL: https://issues.apache.org/jira/browse/ARROW-18313
             Project: Apache Arrow
          Issue Type: Bug
            Reporter: N Gautam Animesh
         Attachments: image-2022-11-11-09-19-16-065.png

On using open_dataset, it creates a connection due to which the files in the directory get blocked and we cannot perform other operations on the file like replace!

Actual issue:
 # We are running an atomic operation on a bunch of files, which replaces the temp file names to the target file names.
 # But while this is happening, if we try to run open_dataset() on that particular directory, the atomic operation is failing and there are both target files and temp files in the directory.
 # It is blocking the files that have been read through open_dataset().
 # Please, provide me with more about how we can handle such problems.
 # Snapshot: !image-2022-11-11-09-19-16-065.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)