You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "sebpretzer (via GitHub)" <gi...@apache.org> on 2023/12/18 17:50:07 UTC
[I] Remove MockAWSResponse and pyiceberg.io.fsspec.FsspecFileIO from testing [iceberg-python]
sebpretzer opened a new issue, #222:
URL: https://github.com/apache/iceberg-python/issues/222
### Feature Request / Improvement
Hi Team,
I was attempting to mess around with `PyIceberg` for my data pipelines. I was struggling implementing the same test framework due to dependency mismatch in regards to how the `MockAWSResponse` was implemented. After a few attempts at fixing the patch itself, I managed to circumvent the issue completely using a [moto server](http://docs.getmoto.org/en/latest/docs/server_mode.html#start-within-python).
I have implemented the change in a [fork of the repo](https://github.com/sebpretzer/iceberg-python/tree/moto_server_implementation), to show you how it works. This removes the need to patch `aiobotocore` and use `FsspecFileIO` ([relevant old convo](https://github.com/apache/iceberg/pull/6822#discussion_r1104878428)). I have made sure this passes tests (at least on my machine).
Would it be ok for me to open a PR and carry these changes though? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Remove MockAWSResponse and pyiceberg.io.fsspec.FsspecFileIO from testing [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #222:
URL: https://github.com/apache/iceberg-python/issues/222#issuecomment-1861185357
Hey @sebpretzer thanks for raising this PR, and I'd would love to get rid of those patches since they are hard to interpret on what's going on. Looking forward to the PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Remove MockAWSResponse and pyiceberg.io.fsspec.FsspecFileIO from testing [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #222:
URL: https://github.com/apache/iceberg-python/issues/222#issuecomment-1863126161
Closing this one since the PR is in, thanks for working on this @sebpretzer 🙌
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Remove MockAWSResponse and pyiceberg.io.fsspec.FsspecFileIO from testing [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko closed issue #222: Remove MockAWSResponse and pyiceberg.io.fsspec.FsspecFileIO from testing
URL: https://github.com/apache/iceberg-python/issues/222
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org