You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/08/13 04:58:14 UTC
[GitHub] [arrow] kshitij12345 commented on a diff in pull request #13629: ARROW-16719: [Python] Add path/URI /+ filesystem handling to parquet.read_metadata
kshitij12345 commented on code in PR #13629:
URL: https://github.com/apache/arrow/pull/13629#discussion_r945076750
##########
python/pyarrow/tests/parquet/test_metadata.py:
##########
@@ -533,6 +536,44 @@ def test_metadata_exceeds_message_size():
metadata = pq.read_metadata(pa.BufferReader(buf))
+def test_metadata_schema_filesystem(tmpdir):
+ table = pa.table({"a": [1, 2, 3]})
+
+ # URI writing to local file.
+ fname = "data.parquet"
+ file_path = 'file:///' + os.path.join(str(tmpdir), fname)
+
+ pq.write_table(table, file_path)
+
+ # Get expected `metadata` from path.
+ metadata = pq.read_metadata(tmpdir / fname)
+ schema = table.schema
+
+ assert pq.read_metadata(file_path).equals(metadata)
+ assert pq.read_metadata(
+ fname, filesystem=f'file:///{tmpdir}').equals(metadata)
+
+ assert pq.read_schema(file_path).equals(schema)
+ assert pq.read_schema(fname, filesystem=f'file:///{tmpdir}').equals(schema)
+
+ with util.change_cwd(tmpdir):
+ # Pass `filesystem` arg
+ assert pq.read_metadata(
+ fname, filesystem=LocalFileSystem()).equals(metadata)
+ assert pq.read_metadata(
+ fname, filesystem=LocalFileSystem.get_instance()).equals(metadata)
Review Comment:
Thanks! I was away from keyboard for the week. Will fix it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org