[GitHub] [arrow] jorisvandenbossche commented on issue #37802: ds.dataset cannot filter on hive-style partitions created with org.apache.spark.version: '3.4.1'

["jorisvandenbossche (via GitHub)" <gi...@apache.org>]

jorisvandenbossche commented on issue #37802:
URL: https://github.com/apache/arrow/issues/37802#issuecomment-1740857100

   If you leave out the `to_polars().head()` (only run the `to_table()`) in the example above, do you then have the same issue? The `to_table` method already reads everything into memory, so it would be strange that this would not yet fail when conversion to polars (which should be mostly zero-copy) fails. 
   
   Could you show the schema of the dataset?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org