You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "eickler (via GitHub)" <gi...@apache.org> on 2025/12/31 11:04:08 UTC
[PR] Hierarchical namespace support in Datafusion integration [iceberg-rust]
eickler opened a new pull request, #1977:
URL: https://github.com/apache/iceberg-rust/pull/1977
## Which issue does this PR close?
- Closes #1976 .
## What changes are included in this PR?
In the PR, instead of just listing the top-level namespaces in the IcebergCatalogProvider, the namespaces are hierarchically traversed and made available using a dot notation ("parent.child").
## Are these changes tested?
The first commit contains only a) unit test cases that safeguard existing functionality and b) a unit test to demonstrate the gap with hierarchical namespaces (i.e., fails before the fix). Besides the unit tests, there is an additional integration test cases that demonstrates that CRUD on tables in child namespaces works with Datafusion using the dot notation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(datafusion): Hierarchical namespace support [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1977:
URL: https://github.com/apache/iceberg-rust/pull/1977#issuecomment-4436011771
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(datafusion): Hierarchical namespace support [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1977:
URL: https://github.com/apache/iceberg-rust/pull/1977#issuecomment-4158987612
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(datafusion): Hierarchical namespace support [iceberg-rust]
Posted by "eickler (via GitHub)" <gi...@apache.org>.
eickler commented on code in PR #1977:
URL: https://github.com/apache/iceberg-rust/pull/1977#discussion_r2725637335
##########
crates/integrations/datafusion/tests/integration_datafusion_test.rs:
##########
@@ -943,3 +943,64 @@ async fn test_insert_into_partitioned() -> Result<()> {
Ok(())
}
+
+#[tokio::test]
+async fn test_child_namespace_crud() -> Result<()> {
Review Comment:
Is it ok?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Hierarchical namespace support in Datafusion integration [iceberg-rust]
Posted by "eickler (via GitHub)" <gi...@apache.org>.
eickler commented on code in PR #1977:
URL: https://github.com/apache/iceberg-rust/pull/1977#discussion_r2658874011
##########
crates/integrations/datafusion/tests/integration_datafusion_test.rs:
##########
@@ -943,3 +943,64 @@ async fn test_insert_into_partitioned() -> Result<()> {
Ok(())
}
+
+#[tokio::test]
+async fn test_child_namespace_crud() -> Result<()> {
Review Comment:
Cool, I haven't heard about that one. I changed it, please let me know which one you like better.
Side comment: I see the benefits of sqllogictest, but while writing the test, I was wondering how far sqllogictest gets you in interoperability testing SQL/Datafusion and Iceberg. In interoperability tests, you often verify fine points on each side against each other, which would not be visible through a test framework for one side only. As a result, you split tests between what you can do in the framework and what you can't do (see the PR). There may be a case for both "classic" and sqllogictest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(datafusion): Hierarchical namespace support [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1977:
URL: https://github.com/apache/iceberg-rust/pull/1977#issuecomment-4384247755
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Hierarchical namespace support in Datafusion integration [iceberg-rust]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on code in PR #1977:
URL: https://github.com/apache/iceberg-rust/pull/1977#discussion_r2658513197
##########
crates/integrations/datafusion/tests/integration_datafusion_test.rs:
##########
@@ -943,3 +943,64 @@ async fn test_insert_into_partitioned() -> Result<()> {
Ok(())
}
+
+#[tokio::test]
+async fn test_child_namespace_crud() -> Result<()> {
Review Comment:
We are trying to migrate off the rust datafusion integration tests to sqllogictests. Do you think we can rewrite this test in slt? Some examples can be found here: https://github.com/apache/iceberg-rust/tree/main/crates/sqllogictest/testdata/slts/df_test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(datafusion): Hierarchical namespace support [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #1977: feat(datafusion): Hierarchical namespace support
URL: https://github.com/apache/iceberg-rust/pull/1977
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org