You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "srilman (via GitHub)" <gi...@apache.org> on 2025/03/24 00:08:44 UTC
[PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]
srilman opened a new pull request, #1834:
URL: https://github.com/apache/iceberg-python/pull/1834
Closes #685.
# Rationale for this change
Reopens PR #687 that adds a Snowflake Catalog that was closed. I addressed comments and applied some additional changes based on errors found when used with the Bodo data processing library.
One way Snowflake supports Iceberg is via managed tables, where Snowflake has both read and write access to these tables. They are basically regular Snowflake tables with an Iceberg backend. Outside of Snowflake, these tables are read-only. To work with them, we wrap some SQL calls in a Catalog API.
I skipped some of the less-commonly used APIs that can be filled in later.
# Are these changes tested?
Tested manually by itself and with the Bodo library on both AWS and Azure. Some of the Azure tests don't translate well because . Also copied the mock tests from the original PR.
# Are there any user-facing changes?
Users can read and query Snowflake-managed Iceberg tables, with minimal write operations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1834:
URL: https://github.com/apache/iceberg-python/pull/1834#issuecomment-4264397204
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #1834:
URL: https://github.com/apache/iceberg-python/pull/1834#issuecomment-2779545731
@srilman From what I understand from Snowflake, it is also transitioning to the Rest Catalog protocol. I much rather use that since it is properly tested. As you mentioned, there are some unsolved issues with the Snowflake catalog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]
Posted by "monti-python (via GitHub)" <gi...@apache.org>.
monti-python commented on PR #1834:
URL: https://github.com/apache/iceberg-python/pull/1834#issuecomment-3313941885
@iamontheinet @jdanielmyers What's your take on this?
At my org, we'd love to use Snowflake-managed Iceberg tables but the lack of support for them in pyiceberg has been blocking us for months...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1834:
URL: https://github.com/apache/iceberg-python/pull/1834#issuecomment-4309592074
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1834:
URL: https://github.com/apache/iceberg-python/pull/1834#issuecomment-4071510760
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]
Posted by "corleyma (via GitHub)" <gi...@apache.org>.
corleyma commented on PR #1834:
URL: https://github.com/apache/iceberg-python/pull/1834#issuecomment-3314140342
@monti-python I think the best way to unblock yourself for Snowflake is to do like this article suggests, and sync the Snowflake-managed Iceberg table with an "Open Catalog" (aka snowflake-managed polaris) and then query using pyiceberg/Spark/etc, since the open catalog/Polaris implements REST catalog spec.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]
Posted by "monti-python (via GitHub)" <gi...@apache.org>.
monti-python commented on PR #1834:
URL: https://github.com/apache/iceberg-python/pull/1834#issuecomment-3192992339
Hi @Fokko,
Are you aware of any public announcement from Snowflake that they are transitioning to REST catalog for Snowflake-manages tables?
AFAIU this is not in their roadmap
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]
Posted by "monti-python (via GitHub)" <gi...@apache.org>.
monti-python commented on PR #1834:
URL: https://github.com/apache/iceberg-python/pull/1834#issuecomment-3452336060
Hi @corleyma,
Thanks for the tip!
Unfortunately that would imply setting the catalog as "external", which doesn't support table writes from 3rd parties (PyIceberg, Spark, etc). Therefore it wouldn't be a feasible solution I'm afraid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog [iceberg-python]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #1834: Support Snowflake-Managed Iceberg Tables via SnowflakeCatalog
URL: https://github.com/apache/iceberg-python/pull/1834
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org