You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "jiakai-li (via GitHub)" <gi...@apache.org> on 2024/12/07 21:04:17 UTC
Re: [I] [Investigate] Whether `data_files` metadata table requires both pyarrow and s3fs [iceberg-python]
jiakai-li commented on issue #1317:
URL: https://github.com/apache/iceberg-python/issues/1317#issuecomment-2525310075
I was trying to identify the issue, but I can't seem to reproduce it, not sure if I've done it wrong (sorry that I'm very new to the community and this is the second issue I tried to work with). Below is the code I tried in order to reproduce the issue. It was run with the `s3fs` module removed from the environment, which runs ok:
```python
from pyiceberg.catalog.sql import SqlCatalog
warehouse_path = r"/tmp/warehouse"
catalog = SqlCatalog(
"default",
**{
"uri": f"sqlite:///{warehouse_path}/pyiceberg_catalog.db",
"warehouse": f"file://{warehouse_path}",
},
)
import pyarrow.parquet as pq
df = pq.read_table(r"/tmp/warehouse/yellow_tripdata_2024-01.parquet")
table = catalog.load_table('default.taxi_dataset')
table.append(df)
from pyiceberg.table import InspectTable
inspect = InspectTable(table)
inspect.data_files().to_pandas()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org