[GitHub] [iceberg] JonasJ-ap commented on a diff in pull request #6822: Python: Set PyArrow as the default FileIO

["JonasJ-ap (via GitHub)" <gi...@apache.org>]

JonasJ-ap commented on code in PR #6822:
URL: https://github.com/apache/iceberg/pull/6822#discussion_r1104910238


##########
python/tests/catalog/test_glue.py:
##########
@@ -52,7 +52,7 @@ def test_create_table_with_database_location(
     _bucket_initialize: None, _patch_aiobotocore: None, table_schema_nested: Schema, database_name: str, table_name: str
 ) -> None:
     identifier = (database_name, table_name)
-    test_catalog = GlueCatalog("glue")
+    test_catalog = GlueCatalog("glue", **{"py-io-impl": "pyiceberg.io.fsspec.FsspecFileIO"})

Review Comment:
   I think `PyArrowFileIO` is not compatible with the mocked S3 FIleSystem (handled by `moto` and `_patch_aiobotocore`) we used in the unit test. I received 
   ```
   OSError: When getting information for key '...' in bucket 'test_bucket': AWS Error ACCESS_DENIED during HeadObject operation: No response body.
   ```
   when using `PyArrowFileIO` in the unit test. Seems it still trying to interact with the real s3 bucket not the mocked one.
   
   I also rerun the `integration_test_glue.py` with the new changes and verify that everything works fine with io default to `PyArrowFileIO`
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org