[GitHub] [iceberg] rdblue commented on a diff in pull request #6501: Python: Use PyArrow buffer

[GitBox <gi...@apache.org>]

rdblue commented on code in PR #6501:
URL: https://github.com/apache/iceberg/pull/6501#discussion_r1059492469


##########
python/pyiceberg/io/pyarrow.py:
##########
@@ -151,7 +160,7 @@ def open(self) -> InputStream:
                 an AWS error code 15
         """
         try:
-            input_file = self._filesystem.open_input_file(self._path)
+            input_file = self._filesystem.open_input_stream(self._path, buffer_size=self._buffer_size)

Review Comment:
   I think that we need to have a flag to signal when `seek` will not be called. The protocol includes seek and we need to support it for Parquet files. However, we can hint to Arrow that we don't need to seek. How about adding `seekable=True` to `open` and allowing the caller to use this by setting `seekable=False`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org