You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Fokko (via GitHub)" <gi...@apache.org> on 2023/03/01 12:00:57 UTC
[GitHub] [iceberg] Fokko opened a new pull request, #6971: Python: Pass location when recreating the FileIO
Fokko opened a new pull request, #6971:
URL: https://github.com/apache/iceberg/pull/6971
We need to pass in the metadata location when we recreate the file io for the table. We re-create the table because there can be new configuration, but we don't supply the location.
This can cause issues when you have both PyArrow and FSspec installed, you don't have the FileIO set explicitly in your config:
```yaml
catalog:
default:
...
py-io-impl: pyiceberg.io.fsspec.FsspecFileIO
```
When you fetch the metadata from ADLS using FSSpec, but then the re-created FileIO is PyArrow.
When we pass in the url, the schema gets taken into account and PyArrow won't be considered for ADLS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #6971: Python: Pass location when recreating the FileIO
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #6971:
URL: https://github.com/apache/iceberg/pull/6971#issuecomment-1454182193
Is it possible to test this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko merged pull request #6971: Python: Pass location when recreating the FileIO
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko merged PR #6971:
URL: https://github.com/apache/iceberg/pull/6971
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on pull request #6971: Python: Pass location when recreating the FileIO
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #6971:
URL: https://github.com/apache/iceberg/pull/6971#issuecomment-1454676021
@rdblue this one is a bit tricky to test since it was just missing the argument.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org