You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datafusion.apache.org by "gabotechs (via GitHub)" <gi...@apache.org> on 2026/09/09 15:58:11 UTC
[PR] Port iceberg-rust history [datafusion-iceberg]
gabotechs opened a new pull request, #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1
Brings all the commit history from https://github.com/apache/iceberg-rust that has ever contributed to https://github.com/apache/iceberg-rust/tree/main/crates/integrations/datafusion.
Each individual commit maintains the original author, and only the changes to files inside the `crates/integrations/datafusion` folder have been kept.
The commits are taken "as-is", so the project still does not compile, as it still points to a non-existing Rust workspace. This will be address in a follow up PR, leaving the project ready for contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5605055826
> Thanks @kumarUjjawal! I'll leave this PR open for a day to give iceberg-rust people to chime in (in case I overlooked something), but otherwise will merge it as-is
I think @alamb was planning on having a formal vote before this gets merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5606307137
> I think @alamb was planning on having a formal vote before this gets merged
yes, please wait for formal vote from both iceberg and datafusion before merging this 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "gabotechs (via GitHub)" <gi...@apache.org>.
gabotechs commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5622224048
Nice, I’ll wait for the green light from @andygrove and @alamb and I’ll pull this in then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5606681887
Vote thread on datafusion maling list: https://lists.apache.org/thread/6k5lws31c61y1lx2s34pxo4lmc7334lr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "gabotechs (via GitHub)" <gi...@apache.org>.
gabotechs commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5605035972
Thanks @kumarUjjawal! I'll leave this PR open for a day to give iceberg-rust people to chime in (in case I overlooked something), but otherwise will merge it as-is
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5622180303
BTW both threads have enough binding votes (3+):
- Datafusion mailing list: https://lists.apache.org/thread/6k5lws31c61y1lx2s34pxo4lmc7334lr
- Iceberg mailing list: https://lists.apache.org/thread/ytb9x7x8ff3b7031kcrc05nzjtmh091s
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "gabotechs (via GitHub)" <gi...@apache.org>.
gabotechs commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5635557150
Done, the folder structure is the following one:
- `src/`: the main crate with the DataFusion integration, what previously was `crates/integrations/datafusion/`
- `sqllogictest/`: crate with the sqllogictests, what previously was `crates/sqllogictest/`
- `playground/`: crate with the iceberg-playground tool, what previously was `crates/integrations/playground/`
This means that even if there's a workspace with attached crates, there's still one main central crate that is DataFusion integration. It's not exactly the same folder structure we had before, but IMO this is a good solution. Happy to hear other opinions if any.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5622273141
> Nice, I’ll wait for the green light from @andygrove and @alamb and I’ll pull this in then.
My vote thread says it will be open for 7 days -- so I expect that to be complete next week
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "CTTY (via GitHub)" <gi...@apache.org>.
CTTY commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5627403468
An open question: should we move iceberg-playground to here as well?
For the context: https://github.com/apache/iceberg-rust/blob/main/crates/integrations/playground/README.md
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "blackmwk (via GitHub)" <gi...@apache.org>.
blackmwk commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5660949318
> Done, the folder structure is the following one:
>
> * `src/`: the main crate with the DataFusion integration, what previously was `crates/integrations/datafusion/`
> * `sqllogictest/`: crate with the sqllogictests, what previously was `crates/sqllogictest/`
> * `playground/`: crate with the iceberg-playground tool, what previously was `crates/integrations/playground/`
>
> This means that even if there's a workspace with attached crates, there's still one main central crate that is DataFusion integration. It's not exactly the same folder structure we had before, but IMO this is a good solution. Happy to hear other opinions if any.
Thanks @gabotechs for porting the changes. I would sugges to keep the origin crates dir, and put each crates under them for serveral reasons:
1. A repo will not only contain code, but also build scripts, license file, doc file etc. If we put the main crate in the root dir, it will make things a little difficult to maintain and read.
2. Sometimes we may need discover all crates dynamically in scripts, and putting all crates under a `crates` dir makes this easier to do.
It's not a blocker, but it's a convention followed by most multi crate projects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "gabotechs (via GitHub)" <gi...@apache.org>.
gabotechs merged PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "gabotechs (via GitHub)" <gi...@apache.org>.
gabotechs commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5635125479
Taking a look at both the `sqllogictest` and `iceberg-playground` crates. It does sound like it makes sense to host them here. Working on that now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5607147100
Iceberg mailing list vote thread: https://lists.apache.org/thread/ytb9x7x8ff3b7031kcrc05nzjtmh091s
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "dannycjones (via GitHub)" <gi...@apache.org>.
dannycjones commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5616924561
OK I see your point @Xuanwo, we'd need to start moving parts of the repo around. This LGTM, we can certainly move later.
> > > but do we want to put the crate inside a crates/ directory
> >
> >
> > I believe it's better to move AS IS and then we can do any refactor we want.
>
> We're already moving directory here from `crates/integrations` to root. I don't disagree more broadly though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5712442619
🚀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5616859933
> but do we want to put the crate inside a crates/ directory
I believe it's better to move AS IS and than we can do any refactor we want.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "dannycjones (via GitHub)" <gi...@apache.org>.
dannycjones commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5616905731
> > but do we want to put the crate inside a crates/ directory
>
> I believe it's better to move AS IS and then we can do any refactor we want.
We're already moving directory here from `crates/integrations` to root. I don't disagree more broadly though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "blackmwk (via GitHub)" <gi...@apache.org>.
blackmwk commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5627213568
Hi, @gabotechs I think you also need to port the sqllogictest dir?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5606346840
btw i spot checked this pr with https://github.com/apache/iceberg-rust/commits/main/crates/integrations/datafusion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5606638843
Relevant mailing list discussion: https://lists.apache.org/thread/53lqjw7371myww42l19vx923cjzrgg68
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org
Re: [PR] Port iceberg-rust history [datafusion-iceberg]
Posted by "gabotechs (via GitHub)" <gi...@apache.org>.
gabotechs commented on PR #1:
URL: https://github.com/apache/datafusion-iceberg/pull/1#issuecomment-5710418028
🫡 It was a pleasure to have you all here, thanks everyone!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datafusion.apache.org
For additional commands, e-mail: dev-help@datafusion.apache.org