You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/07/24 18:00:30 UTC

[I] Add Iceberg integration [datafusion-ballista]

andygrove opened a new issue, #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174

   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   There is a PR to add Ballista integration to iceberg-rust https://github.com/apache/iceberg-rust/pull/2613 and there is a discussion about moving it to a new location.
   
   **Describe the solution you'd like**
   Let's discuss bringing this into Ballista repo.
   
   **Describe alternatives you've considered**
   
   **Additional context**
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Add Iceberg integration [datafusion-ballista]

Posted by "NoahKusaba (via GitHub)" <gi...@apache.org>.
NoahKusaba commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5079120888

   Sounds good to me. I can close my PR on iceberg-rust and re-open it here if that works with everyone?
   We can merge it in once I get the necessary iceberg-datafusion changes in. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Add Iceberg integration [datafusion-ballista]

Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5073717041

   Same like python support, good idea 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Add Iceberg integration [datafusion-ballista]

Posted by "sandugood (via GitHub)" <gi...@apache.org>.
sandugood commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5083583879

   +1
   Would like to contribute and then test Ballista on my current workloads :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Add Iceberg integration [datafusion-ballista]

Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5271593538

   https://github.com/apache/iceberg-rust/discussions/2992
   
   Discussion you might be interested in


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Add Iceberg integration [datafusion-ballista]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5072944934

   @milenkovicm I was thinking we could add a separate crate (not part of main workspace) and have independent release cycle so that it doesn't impact current Ballista release cycle


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Add Iceberg integration [datafusion-ballista]

Posted by "NoahKusaba (via GitHub)" <gi...@apache.org>.
NoahKusaba commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5239385001

   @andygrove 
   datafusion-distributed is also looking to get iceberg-datafusion out of iceberg-rust, or to fork and maintain their own copy.
   https://github.com/datafusion-contrib/datafusion-distributed/issues/533
   I think it's extremely important that we move it to datafusion-contrib, so we don't end up with a bunch of forks of this library. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Add Iceberg integration [datafusion-ballista]

Posted by "phillipleblanc (via GitHub)" <gi...@apache.org>.
phillipleblanc commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5086106227

   This makes sense to me!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Add Iceberg integration [datafusion-ballista]

Posted by "NoahKusaba (via GitHub)" <gi...@apache.org>.
NoahKusaba commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5195078319

   @andygrove can we also look to moving the iceberg-datafusion crate out of iceberg-rust? It seems their team is interested in doing so as well. Currently getting PR's merged in is an extremely slow process due to lack of reviewers. 
   Not sure where it would end up living. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Production-grade Apache Iceberg scans [datafusion-ballista]

Posted by "NoahKusaba (via GitHub)" <gi...@apache.org>.
NoahKusaba commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-6093685406

   Off the top of my head, with https://github.com/apache/datafusion-iceberg/pull/20 merging, nothing is blocking standalone table scans now. I'm opting to wait for https://github.com/apache/datafusion-iceberg/pull/48, which is waiting on some other repo-wide refactors that are in progress, as I'll have to refactor after the fact anyway.
   
   I'll look to split out the standalone table-scan PR for iceberg in the coming weeks, which also sets the skeleton for table appends too. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Add Iceberg integration [datafusion-ballista]

Posted by "martin-g (via GitHub)" <gi...@apache.org>.
martin-g commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5080500728

   +1


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Add Iceberg integration [datafusion-ballista]

Posted by "NoahKusaba (via GitHub)" <gi...@apache.org>.
NoahKusaba commented on issue #2174:
URL: https://github.com/apache/datafusion-ballista/issues/2174#issuecomment-5153600228

   Opened the PR at
   https://github.com/apache/datafusion-ballista/pull/2217
   It's slow going getting the iceberg-datafusion changes merged in, so currently it's pointed to my iceberg-rust branch. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org