You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/12/30 22:43:27 UTC
[GitHub] [arrow-cookbook] davisusanibar opened a new pull request, #289: [Java] Adding java read compression files with LZ4 codec
davisusanibar opened a new pull request, #289:
URL: https://github.com/apache/arrow-cookbook/pull/289
Related to #288
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-cookbook] lidavidm commented on a diff in pull request #289: [Java] Adding java read compression files with LZ4 codec
Posted by GitBox <gi...@apache.org>.
lidavidm commented on code in PR #289:
URL: https://github.com/apache/arrow-cookbook/pull/289#discussion_r1062465382
##########
java/source/io.rst:
##########
@@ -320,6 +320,65 @@ We are providing a path with auto generated arrow files for testing purposes, ch
Jhon 29
Thomy 33
+Read - From Compressed File
+---------------------------
+
+We are providing a path with auto generated arrow files for testing purposes, change that at your convenience.
+
+Compressed file generated through this code example:
+
+.. code:: python
+
+ import pandas as pd
+ import pyarrow as pa
+
+ pd.DataFrame({'key': range(4)}).to_feather('lz4.arrow', compression='lz4')
+ pd.DataFrame({'key': range(4)}).to_feather('zstd.arrow', compression='zstd')
+
+.. note::
+
+ Java Vector module offer read files without compression codec, in case reading
+ compressed files is required consider to also add Java Compression module
+ dependency.
Review Comment:
```suggestion
The ``arrow-vector`` module can only read uncompressed files by itself. Add
a dependency on ``arrow-compression`` to also be able to read compressed files.
```
##########
java/source/io.rst:
##########
@@ -320,6 +320,65 @@ We are providing a path with auto generated arrow files for testing purposes, ch
Jhon 29
Thomy 33
+Read - From Compressed File
+---------------------------
+
+We are providing a path with auto generated arrow files for testing purposes, change that at your convenience.
+
+Compressed file generated through this code example:
+
+.. code:: python
+
+ import pandas as pd
+ import pyarrow as pa
+
+ pd.DataFrame({'key': range(4)}).to_feather('lz4.arrow', compression='lz4')
+ pd.DataFrame({'key': range(4)}).to_feather('zstd.arrow', compression='zstd')
Review Comment:
This isn't checked in?
##########
java/source/io.rst:
##########
@@ -320,6 +320,65 @@ We are providing a path with auto generated arrow files for testing purposes, ch
Jhon 29
Thomy 33
+Read - From Compressed File
+---------------------------
+
+We are providing a path with auto generated arrow files for testing purposes, change that at your convenience.
Review Comment:
Is it possible to provide a write-compressed-file example too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-cookbook] lidavidm commented on a diff in pull request #289: [Java] Adding java read compression files with LZ4 codec
Posted by GitBox <gi...@apache.org>.
lidavidm commented on code in PR #289:
URL: https://github.com/apache/arrow-cookbook/pull/289#discussion_r1062487287
##########
java/source/io.rst:
##########
@@ -320,6 +320,65 @@ We are providing a path with auto generated arrow files for testing purposes, ch
Jhon 29
Thomy 33
+Read - From Compressed File
+---------------------------
+
+We are providing a path with auto generated arrow files for testing purposes, change that at your convenience.
Review Comment:
Hmm. There's no practical way to write a compressed file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org