You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Amogh-2404 (via GitHub)" <gi...@apache.org> on 2026/06/06 11:41:14 UTC
[PR] Add blog post: The Arrow C Data Interface: Zero-Copy Between Rust and the JVM in DataFusion Comet [datafusion-site]
Amogh-2404 opened a new pull request, #195:
URL: https://github.com/apache/datafusion-site/pull/195
This adds a tutorial blog post on the Arrow C Data Interface, using DataFusion Comet's real Rust↔JVM handoff as the worked example. It addresses apache/datafusion-comet#3678.
Closes apache/datafusion-comet#3678
- New post: `content/blog/2026-06-06-arrow-c-data-interface.md`
- 5 diagrams under `content/images/arrow-c-data-interface/`
- Category: tutorial
The post separates the two questions the interface answers — memory layout vs. ownership — and traces a single `Int32` column `[1, 2, NULL, 4]` field by field across the boundary. The Comet section (`move_to_spark`, `from_spark`, `executePlan`, `CometPlainVector`, the `arrow_ffi_safe` copy fallbacks) is pinned to source at a fixed commit (`e79183ee`).
Happy to adjust scope, framing, or length before merge — flag anything you'd like covered or cut.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add blog post: The Arrow C Data Interface: Zero-Copy Between Rust and the JVM in DataFusion Comet [datafusion-site]
Posted by "Amogh-2404 (via GitHub)" <gi...@apache.org>.
Amogh-2404 commented on PR #195:
URL: https://github.com/apache/datafusion-site/pull/195#issuecomment-4880838367
Thanks, that makes sense. I’ll take another pass from the actual Comet PRs and see if there’s a real post there. If not, I’ll close this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add blog post: The Arrow C Data Interface: Zero-Copy Between Rust and the JVM in DataFusion Comet [datafusion-site]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on PR #195:
URL: https://github.com/apache/datafusion-site/pull/195#issuecomment-4671492574
This @Amogh-2404. I was really assuming that the blog post would draw on all the lessons learned by the contributors that actually put in the hours implementing and optimizing Comet's use of FFI over the past two years rather than just a generic AI generated blog post. I guess I should have made that clear in the issue. Let's see what other contributors think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add blog post: The Arrow C Data Interface: Zero-Copy Between Rust and the JVM in DataFusion Comet [datafusion-site]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed pull request #195: Add blog post: The Arrow C Data Interface: Zero-Copy Between Rust and the JVM in DataFusion Comet
URL: https://github.com/apache/datafusion-site/pull/195
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add blog post: The Arrow C Data Interface: Zero-Copy Between Rust and the JVM in DataFusion Comet [datafusion-site]
Posted by "Amogh-2404 (via GitHub)" <gi...@apache.org>.
Amogh-2404 commented on PR #195:
URL: https://github.com/apache/datafusion-site/pull/195#issuecomment-4671885333
Yeah, that's fair @andygrove. I'll leave it to you and the others on whether any version of this worth keeping.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add blog post: The Arrow C Data Interface: Zero-Copy Between Rust and the JVM in DataFusion Comet [datafusion-site]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #195:
URL: https://github.com/apache/datafusion-site/pull/195#issuecomment-4878519744
> Yeah, that's fair @andygrove. I'll leave it to you and the others on whether any version of this worth keeping.
One thing you could do is use AI to research the releated PRs in datafusion-comet and then try and summarize the lessions /et from there.
Then at least it wouldn't be generic, but it would still probably need review by the authors themselves
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org