[GitHub] [arrow] johnseekins commented on issue #34688: Dataset -> Table -> Dataset loses partitions?

["johnseekins (via GitHub)" <gi...@apache.org>]

johnseekins commented on issue #34688:
URL: https://github.com/apache/arrow/issues/34688#issuecomment-1489473730

   As far as I've been able to find, the correct answer is to basically not use datasets to try and load large amounts of data. And generally, using partitions with pyarrow seems to simply remove those columns from the actual data, which somewhat defeats the purpose of them. Closing this, but maybe some day we'll come back to it with renewed vigor!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org