[I] pq.write_table fails with OSError: Lz4 compression failure. [arrow]

["skaae (via GitHub)" <gi...@apache.org>]

skaae opened a new issue, #50613:
URL: https://github.com/apache/arrow/issues/50613

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   I encountered a table with where pq.write_table throws compression error:
   
   ```
   pq.write_table(table, "foo", compression='lz4')
   *** OSError: Lz4 compression failure.
   ```
   
   using compression='gzip' works. 
   
   The minimal table I was able to find contains 42 rows with a single columns of strings. Each string is 30-60 mb.
   The strings in the table are xml and can be compressed to ~100mb in total. 
   
   I can share the table, but would like to not do it publicly. 
   
   
   
   ### Component(s)
   
   Parquet


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Parquet] pq.write_table fails with OSError: Lz4 compression failure. [arrow]

["thisisnic (via GitHub)" <gi...@apache.org>]

thisisnic commented on issue #50613:
URL: https://github.com/apache/arrow/issues/50613#issuecomment-6428784306

   Thanks for reporting this @skaae! Can conform this reproduced on 26.0.0.
   
   Did a brief Claude analysis, will post explanation below.
   
   > :robot: :


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org