You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "carols10cents (via GitHub)" <gi...@apache.org> on 2024/03/13 16:44:45 UTC

[I] Local object store copy/rename with nonexistent `from` file loops forever instead of erroring [arrow-rs]

carols10cents opened a new issue, #5503:
URL: https://github.com/apache/arrow-rs/issues/5503

   ## What happened
   
   Using a `Local` file object store, I called `copy` accidentally with a `from` path that was not an existing file. 
   
   ## What I expected to happen
   
   I expected the `copy` to succeed before I realized my mistake 😂 But once I realized the `from` file didn't exist, I expected `copy` to return an error.
   
   ## What happened instead
   
   The operation I was doing that called `copy` timed out.
   
   ## To reproduce
   
   If I add these lines to the `rename_and_copy` test function:
   
   ```rust
           let nonexistent = Path::from("test3");
           storage.copy(&nonexistent, &path2).await.unwrap_err();
   ```
   
   and run the local tests, `file_test` will run forever.
   
   If I change this line: https://github.com/apache/arrow-rs/blob/ad3b4c92508a17ea679c9695a89f85c2cdcc4fcf/object_store/src/local.rs#L620
   
   to be `Err(source) => match dbg!(source.kind()) {` and run the `file_test` with `--nocapture`, I get infinite printouts of:
   
   ```
   [src/local.rs:622] source.kind() = NotFound
   ```
   
   When the `match` sees `NotFound`, it creates the parent directories and then loops again: https://github.com/apache/arrow-rs/blob/ad3b4c92508a17ea679c9695a89f85c2cdcc4fcf/object_store/src/local.rs#L622
   
   ... which won't fix the problem of `from` not being found.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Local object store copy/rename with nonexistent `from` file loops forever instead of erroring [arrow-rs]

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #5503:
URL: https://github.com/apache/arrow-rs/issues/5503#issuecomment-2061308538

   `label_issue.py` automatically added labels {'object-store'} from #5528


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Local object store copy/rename with nonexistent `from` file loops forever instead of erroring [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5503:
URL: https://github.com/apache/arrow-rs/issues/5503#issuecomment-2741681898

   Migrating from arrow-rs issue [#5503](https://github.com/apache/arrow-rs/issues/5503)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Local object store copy/rename with nonexistent `from` file loops forever instead of erroring [arrow-rs]

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold closed issue #5503: Local object store copy/rename with nonexistent `from` file loops forever instead of erroring
URL: https://github.com/apache/arrow-rs/issues/5503


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org