You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "singhpratech (via GitHub)" <gi...@apache.org> on 2026/09/08 12:36:10 UTC
[PR] GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() [arrow]
singhpratech opened a new pull request, #51236:
URL: https://github.com/apache/arrow/pull/51236
### Rationale for this change
Follows up on #39793. `Array$export_to_c()` and `Array$import_from_c()` work, but the functions that
produce the pointers they take, `allocate_arrow_schema()` and friends, are not exported, and none of
it has a help page, so a package that hands a column to another library over the C Data Interface
has to use `arrow:::`, which `R CMD check` flags.
### What changes are included in this PR?
The six allocate/delete functions from the issue are exported and documented on one help page
(`?c-data-interface`) with a round-trip example; the `Array` methods list names `$export_to_c()`
and `$import_from_c()`; a NEWS entry; two tests in test-bridge.R. NAMESPACE and the man pages were
regenerated with roxygen2 8.0.0, the version in `Config/roxygen2/version`, so the diff is only the
additions. Per @thisisnic's note on #51227, the alignment behaviour of `Array$create()` is a code
comment in the new file, not user documentation.
### Are these changes tested?
Yes: `test-bridge.R` gains a test that the six names are exported and a test that an `Array` with
a null round-trips through the exported functions. The example on the help page runs under
`R CMD check` as well.
### Are there any user-facing changes?
Yes: six newly exported functions and one new help page. No existing behaviour changes.
Thank you for looking at this. Happy to adjust the wording of the help page or move the
functions' documentation wherever fits the package's conventions better.
Closes #39793.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on PR #51236:
URL: https://github.com/apache/arrow/pull/51236#issuecomment-5912119402
I think it makes more sense to keep nanoarrow as the canonical way to do this, yeah, thank.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on code in PR #51236:
URL: https://github.com/apache/arrow/pull/51236#discussion_r4030893454
##########
r/tests/testthat/test-bridge.R:
##########
@@ -81,3 +81,29 @@ test_that("Pointer wrapper errors for unknown object", {
"Can't parse 'this is not an integer'"
)
})
+
+test_that("the C Data Interface allocators are exported", {
+ exported <- getNamespaceExports("arrow")
+ for (f in c(
+ "allocate_arrow_schema",
+ "delete_arrow_schema",
+ "allocate_arrow_array",
+ "delete_arrow_array",
+ "allocate_arrow_array_stream",
+ "delete_arrow_array_stream"
+ )) {
+ expect_true(f %in% exported, label = paste(f, "is exported"))
+ }
+})
Review Comment:
What's the reason for this test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() [arrow]
Posted by "singhpratech (via GitHub)" <gi...@apache.org>.
singhpratech commented on code in PR #51236:
URL: https://github.com/apache/arrow/pull/51236#discussion_r4030948195
##########
r/tests/testthat/test-bridge.R:
##########
@@ -81,3 +81,29 @@ test_that("Pointer wrapper errors for unknown object", {
"Can't parse 'this is not an integer'"
)
})
+
+test_that("the C Data Interface allocators are exported", {
+ exported <- getNamespaceExports("arrow")
+ for (f in c(
+ "allocate_arrow_schema",
+ "delete_arrow_schema",
+ "allocate_arrow_array",
+ "delete_arrow_array",
+ "allocate_arrow_array_stream",
+ "delete_arrow_array_stream"
+ )) {
+ expect_true(f %in% exported, label = paste(f, "is exported"))
+ }
+})
Review Comment:
It pins the six names as part of the exported interface, so a later roxygen run that loses an `@export` fails here, naming the function, rather than in a downstream package's check. The round-trip test below reaches them through `arrow::` and would fail too, so if one test is enough I'll drop this one and keep the round trip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51236:
URL: https://github.com/apache/arrow/pull/51236#issuecomment-5585232389
:warning: GitHub issue #39793 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() [arrow]
Posted by "singhpratech (via GitHub)" <gi...@apache.org>.
singhpratech commented on PR #51236:
URL: https://github.com/apache/arrow/pull/51236#issuecomment-5925061337
Thank you both for taking the time, and Dewey, thanks for the `nanoarrow/r.h` pointer — inlining the
allocators that way is a clean answer for a package that cannot take a dependency, and I will use it.
I am glad to go whichever way the project prefers. I would just like to put a few facts next to the
decision, since they shaped this PR and may be useful whoever reads #39793 next:
- The project's own Python ↔ R integration guide (`docs/source/python/integration/python_r.rst`)
shows R code calling `Array$import_from_c(array_ptr, schema_ptr)` and `Array$export_to_c(...)`.
Today the R package has no help page for either method, and `?Array` does not list them.
- The six allocators are already in the package: `r/R/python.R` uses them for the reticulate bridge.
Exporting them is +170 / −0 across eight files and changes no behaviour.
- #39793 has been open since January 2024 and was un-staled in November 2025, when a maintainer asked
on the issue whether the reporter would open a PR; this PR is that follow-through.
- nanoarrow's own R package calls `Array$import_from_c()` and `$export_to_c()`, so those two methods
are public API in practice already.
Where I agree completely: for working with the C structures, nanoarrow is the right tool and the
right recommendation. The case this PR was aimed at is narrower — a package that already depends on
arrow and only needs to pass one array across the interface — and for that case the choice today is
`arrow:::` (which `R CMD check` flags) or a second dependency to allocate two small structs.
Two ways to close this out, and I am happy with either:
1. **Documentation only.** I trim the PR to documenting `$export_to_c()` and `$import_from_c()` on
`?Array`, with the example allocating through nanoarrow and a line pointing readers to nanoarrow
for everything beyond passing a pointer; the exports, the test and the NEWS entry go. If that is
the decision, closing #39793 with that reason would spare the next person the same loop.
2. **As filed.** Export the six functions as the issue asks.
Tell me which, and I will have the branch updated the same day. Thanks again for the reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() [arrow]
Posted by "singhpratech (via GitHub)" <gi...@apache.org>.
singhpratech commented on PR #51236:
URL: https://github.com/apache/arrow/pull/51236#issuecomment-5704941591
The methods are the public part already: `$export_to_c()` and `$import_from_c()` are what nanoarrow itself calls (`as_nanoarrow_array.Array` hands `x$export_to_c()` a `nanoarrow_allocate_array()`, and `as_arrow_array.nanoarrow_array` calls `arrow::Array$import_from_c()`), and 25.0.0 has no help page for either. The six allocators already exist in the package for the Python bridge. The change exports them and documents the methods so a package that already depends on arrow can call arrow's own methods without `arrow:::` (which R CMD check flags) or a second dependency for a struct allocation. That is the whole case, and it is a small one.
The case for nanoarrow instead is that it is the project's answer for the C Data Interface in R, one route is better than two, and the use case on the issue (r-polars) has since gone through nanoarrow. If that is where you would rather keep it, I will cut this to the documentation: the two methods on `?Array`, with the example allocating through nanoarrow, and drop the exports, the new test and the NEWS entry. Your call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on PR #51236:
URL: https://github.com/apache/arrow/pull/51236#issuecomment-5923673502
Thanks for the ping!
I'm an obvious fan of nanoarrow...the pointer functions there are hopefully well documented and help you get/work with the addresses in various ways that aren't exposed here. If for some reason you don't want that as a runtime dependency, you can also inline the allocators and deleters (by copying this header or adding nanoarrow to `LinkingTo` and using it directly): https://github.com/apache/arrow-nanoarrow/blob/main/r/inst/include/nanoarrow/r.h ).
That said, documenting `import_from_c()` and `export_to_c()` I think is reasonable. They have been around for a long time and aren't likely to change as far as I know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on code in PR #51236:
URL: https://github.com/apache/arrow/pull/51236#discussion_r4030897673
##########
r/R/array.R:
##########
@@ -82,6 +82,11 @@
#' - `$View(type)`: Construct a zero-copy view of this array with the given type.
#' - `$Validate()` : Perform any validation checks to determine obvious inconsistencies
#' within the array's internal data. This can be an expensive check, potentially `O(length)`
+#' - `$export_to_c(array_ptr, schema_ptr)`: Fill an `ArrowArray` and an `ArrowSchema` struct
+#' from this array for the Arrow C Data Interface; the pointers come from
+#' [allocate_arrow_array()] and [allocate_arrow_schema()].
+#' - `Array$import_from_c(array_ptr, schema_ptr)`: Build an `Array` from structs
Review Comment:
This last one looks different
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on PR #51236:
URL: https://github.com/apache/arrow/pull/51236#issuecomment-5704799245
Before we consider merging this, I think it'd be good to understand why we would want to do this instead of recommending folk use nanoarrow for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org