You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/26 02:02:40 UTC

[GitHub] [beam] rviscomi commented on issue #20137: textio (and fileio in general) takes too long to estimate sizes of large globs

rviscomi commented on issue #20137:
URL: https://github.com/apache/beam/issues/20137#issuecomment-1166396704

   /sub
   
   I filed the question originally on Stack Overflow and I'm still encountering this issue with `ReadAllFromText`. It estimates the size of 10M files on GCS **four times**, taking hours before anything can even be processed. I'd love to see this feature gap closed between Java and Python SDKs!
   
   <img width="513" alt="image" src="https://user-images.githubusercontent.com/1120896/175796221-5e7de5b2-c3b6-444a-a102-f16a9950a418.png">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org