Re: [I] [Python] Add compression option to pyarrow.csv.WriteOptions [arrow]

["jorisvandenbossche (via GitHub)" <gi...@apache.org>]

jorisvandenbossche commented on issue #40658:
URL: https://github.com/apache/arrow/issues/40658#issuecomment-2009120646

   It's not built in into the csv writer itself (for the reading side it is), but you can let it compress on the fly using `CompressedOutputStream`. See the usage example at https://arrow.apache.org/docs/python/csv.html#usage:
   
   ```python
   import pyarrow as pa
   import pyarrow.csv as csv
   
   with pa.CompressedOutputStream("tips.csv.gz", "gzip") as out:
       csv.write_csv(table, out)
   ```
   
   That's only the direct CSV writer, and doesn't work through `pyarrow.dataset`. For compressed CSV in writing datasets we have an open feature request: https://github.com/apache/arrow/issues/34085


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org