You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/07/24 18:00:30 UTC
[I] Add Iceberg integration [datafusion-ballista]
andygrove opened a new issue, #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
There is a PR to add Ballista integration to iceberg-rust https://github.com/apache/iceberg-rust/pull/2613 and there is a discussion about moving it to a new location.
**Describe the solution you'd like**
Let's discuss bringing this into Ballista repo.
**Describe alternatives you've considered**
**Additional context**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Iceberg integration [datafusion-ballista]
Posted by "NoahKusaba (via GitHub)" <gi...@apache.org>.
NoahKusaba commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5079120888
Sounds good to me. I can close my PR on iceberg-rust and re-open it here if that works with everyone?
We can merge it in once I get the necessary iceberg-datafusion changes in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Iceberg integration [datafusion-ballista]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5073717041
Same like python support, good idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Iceberg integration [datafusion-ballista]
Posted by "sandugood (via GitHub)" <gi...@apache.org>.
sandugood commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5083583879
+1
Would like to contribute and then test Ballista on my current workloads :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Iceberg integration [datafusion-ballista]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5271593538
https://github.com/apache/iceberg-rust/discussions/2992
Discussion you might be interested in
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Iceberg integration [datafusion-ballista]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5072944934
@milenkovicm I was thinking we could add a separate crate (not part of main workspace) and have independent release cycle so that it doesn't impact current Ballista release cycle
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Iceberg integration [datafusion-ballista]
Posted by "NoahKusaba (via GitHub)" <gi...@apache.org>.
NoahKusaba commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5239385001
@andygrove
datafusion-distributed is also looking to get iceberg-datafusion out of iceberg-rust, or to fork and maintain their own copy.
https://github.com/datafusion-contrib/datafusion-distributed/issues/533
I think it's extremely important that we move it to datafusion-contrib, so we don't end up with a bunch of forks of this library.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Iceberg integration [datafusion-ballista]
Posted by "phillipleblanc (via GitHub)" <gi...@apache.org>.
phillipleblanc commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5086106227
This makes sense to me!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Iceberg integration [datafusion-ballista]
Posted by "NoahKusaba (via GitHub)" <gi...@apache.org>.
NoahKusaba commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5195078319
@andygrove can we also look to moving the iceberg-datafusion crate out of iceberg-rust? It seems their team is interested in doing so as well. Currently getting PR's merged in is an extremely slow process due to lack of reviewers.
Not sure where it would end up living.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [EPIC] Production-grade Apache Iceberg scans [datafusion-ballista]
Posted by "NoahKusaba (via GitHub)" <gi...@apache.org>.
NoahKusaba commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-6093685406
Off the top of my head, with https://github.com/apache/datafusion-iceberg/pull/20 merging, nothing is blocking standalone table scans now. I'm opting to wait for https://github.com/apache/datafusion-iceberg/pull/48, which is waiting on some other repo-wide refactors that are in progress, as I'll have to refactor after the fact anyway.
I'll look to split out the standalone table-scan PR for iceberg in the coming weeks, which also sets the skeleton for table appends too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Iceberg integration [datafusion-ballista]
Posted by "martin-g (via GitHub)" <gi...@apache.org>.
martin-g commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5080500728
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add Iceberg integration [datafusion-ballista]
Posted by "NoahKusaba (via GitHub)" <gi...@apache.org>.
NoahKusaba commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5153600228
Opened the PR at
https://github.com/apache/datafusion-ballista/pull/2217
It's slow going getting the iceberg-datafusion changes merged in, so currently it's pointed to my iceberg-rust branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org