You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by "Keith Gable (Jira)" <ji...@apache.org> on 2020/01/21 00:33:00 UTC

[jira] [Created] (ARROW-7625) Parquet GLib and Red Parquet (Ruby) do not allow specifying compression type

Keith Gable created ARROW-7625:
----------------------------------

             Summary: Parquet GLib and Red Parquet (Ruby) do not allow specifying compression type
                 Key: ARROW-7625
                 URL: https://issues.apache.org/jira/browse/ARROW-7625
             Project: Apache Arrow
          Issue Type: Bug
         Environment: red-arrow 0.15.1
red-parquet 0.15.1
libarrow 0.15.1
libparquet 0.15.1
            Reporter: Keith Gable


It seems that the ArrowFileWriter being used by parquet-glib just uses the default writer properties ([https://github.com/apache/arrow/blob/master/c_glib/parquet-glib/arrow-file-writer.cpp#L184),] and does not offer the user the ability to override this. As a consumer of the GLib API in Ruby (red-parquet), I therefore have no way of compressing Parquet columns. Of course, I can compress the entire file by doing something like {{t.save('...', format: 'parquet', compression: 'GZIP')}}, but this is not compatible with most tools and isn't the correct way of compressing a Parquet file.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)