You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "cyrankiewicz (via GitHub)" <gi...@apache.org> on 2025/11/12 14:37:03 UTC

[I] [Python] scanner(fragment_readahead=0).to_table() hangs indefinitely. [arrow]

cyrankiewicz opened a new issue, #48111:
URL: https://github.com/apache/arrow/issues/48111

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   I've encountered a situation where calling .to_table() on a pyarrow.dataset.Scanner causes the process to hang indefinitely.
   
   ```
   import pyarrow as pa
   import pyarrow.dataset as pd
   import pyarrow.parquet as pq
   t = pa.table({"col": 'pyarrow'})
   pq.write_table(t, 'table.parquet')
   pd.dataset('table.parquet').scanner(fragment_readahead=0).to_table()
   # never returns
   ```
   
   Note, that the data to read is minimal and the following works:
   
   ```
   import pyarrow as pa
   import pyarrow.dataset as pd
   import pyarrow.parquet as pq
   t = pa.table({"col": 'pyarrow'})
   pq.write_table(t, 'table.parquet')
   pd.dataset('table.parquet').scanner(fragment_readahead=0).head(100)
   pyarrow.Table
   col: string
   ----
   col: [["p","y","a","r","r","o","w"]]
   ```
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Python] scanner(fragment_readahead=0).to_table() hangs indefinitely. [arrow]

Posted by "wojiaodoubao (via GitHub)" <gi...@apache.org>.
wojiaodoubao commented on issue #48111:
URL: https://github.com/apache/arrow/issues/48111#issuecomment-3524867056

   Hi @cyrankiewicz , which version do you use?  I'm not sure if I'm encountering the same issue, my version is 18.1.0.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Python][C++] scanner(fragment_readahead=0).to_table() hangs indefinitely. [arrow]

Posted by "lhoestq (via GitHub)" <gi...@apache.org>.
lhoestq commented on issue #48111:
URL: https://github.com/apache/arrow/issues/48111#issuecomment-3577082406

   Not sure if related, but fyi there is also a similar issue here: https://github.com/apache/arrow/issues/45214


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Python] scanner(fragment_readahead=0).to_table() hangs indefinitely. [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #48111:
URL: https://github.com/apache/arrow/issues/48111#issuecomment-3528803153

   I've been able to reproduce on PyArrow 22.00 with Python 3.14. I'll try to see when was this introduced. I confirm, there seems to be deadlock on the dataset scanner when `fragment_readahead=0`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org