You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "Christian (Jira)" <ji...@apache.org> on 2022/02/18 17:27:00 UTC

[jira] [Created] (ARROW-15729) Reading large files randomly freezes

Christian created ARROW-15729:
---------------------------------

             Summary: Reading large files randomly freezes
                 Key: ARROW-15729
                 URL: https://issues.apache.org/jira/browse/ARROW-15729
             Project: Apache Arrow
          Issue Type: Bug
          Components: R
            Reporter: Christian
             Fix For: 6.0.1


Hi -

I recently upgraded to Arrow 6.0.1 and am using it in R.

Whenever reading a large file (~10gb) in Windows it randomly freezes sometimes. I can see the memory being allocated in the first 10-20 seconds, but then nothing happens and R just doesn't respond.

I'm using the option options(arrow.use_threads=FALSE).

I didn't have this issue with the previous version I was using. And the file reads fine under Linux.

I would post a reproducible example but it happens randomly. I even thought I would just read large files in pieces by first getting all the distinct sections of a specific column (with compute>collect) but that hangs too.

Any ideas would be appreciated.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)