You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "thisisnic (via GitHub)" <gi...@apache.org> on 2026/09/30 12:52:12 UTC
[PR] GH-51655: [R] Advance long-standing deprecations in map_batches() and pull() [arrow]
thisisnic opened a new pull request, #51658:
URL: https://github.com/apache/arrow/pull/51658
### Rationale for this change
Old deprecation warnings hadn't been dealt with
### What changes are included in this PR?
I've deprecated the `.data.frame` argument in `map_batches()` but I think we actually may be better off leaving `pull()` as-is given it's been like that for such a long time.
### Are these changes tested?
Existing tests
### Are there any user-facing changes?
Yes
`pull()` on Arrow data no longer warns about a future change of default; it will keep returning an R vector by default, with `as_vector = FALSE` or `options(arrow.pull_as_vector = FALSE)` to get a `ChunkedArray`.
The deprecated `.data.frame` argument to `map_batches()` has been removed.
### Was AI used for this PR?
In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.
**PR code and description written by:**
- [x] Human
- [x] AI
**Reviewed before submission by:**
- [x] Human
- [ ] AI
- [ ] Not reviewed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-51655: [R] Advance long-standing deprecations in map_batches() and pull() [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic merged PR #51658:
URL: https://github.com/apache/arrow/pull/51658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-51655: [R] Advance long-standing deprecations in map_batches() and pull() [arrow]
Posted by "jonkeane (via GitHub)" <gi...@apache.org>.
jonkeane commented on code in PR #51658:
URL: https://github.com/apache/arrow/pull/51658#discussion_r4144955960
##########
r/R/dplyr-collect.R:
##########
@@ -65,32 +65,12 @@ pull.Dataset <- function(.data, var = -1, ..., as_vector = getOption("arrow.pull
}
pull.RecordBatchReader <- pull.arrow_dplyr_query <- pull.Dataset
-pull.ArrowTabular <- function(x, var = -1, ..., as_vector = getOption("arrow.pull_as_vector")) {
+pull.ArrowTabular <- function(x, var = -1, ..., as_vector = getOption("arrow.pull_as_vector", TRUE)) {
out <- x[[vars_pull(names(x), !!enquo(var))]]
handle_pull_as_vector(out, as_vector)
}
handle_pull_as_vector <- function(out, as_vector) {
- if (is.null(as_vector)) {
Review Comment:
I did some digging (well tasked a clanker to do the digging...) to see why we did this in the first place. Here it is for posterity (or some future clanker to dig it up). We originally proposed to do this since there wasn't another ergonomic way to get a chunked array, and unlike `dbplyr` where this was originally inspired by, Arrow has an in-memory representation, so there wasn't a _requirement_ to turn it into an R vector.
It turns out that broke dependencies so we made it opt in at the time, though there has been consistent feedback that other engines like `data.table` also return just a vector (and conceptually `dplyr` does this with `data.frame`s too, it strips the outer `data.frame`ness).
I'm good with undeprecating this. Though I am slightly curious (and we should do this in a follow on if we do), do we now have a (more) ergonomic way to get a chunked array? is it `pull(..., as_vector=FALSE)`? or something else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-51655: [R] Advance long-standing deprecations in map_batches() and pull() [arrow]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #51658:
URL: https://github.com/apache/arrow/pull/51658#discussion_r4144779261
##########
r/R/dataset-scan.R:
##########
@@ -220,17 +220,9 @@ tail_from_batches <- function(batches, n) {
#' the result; use `FALSE` to evaluate `FUN` on all batches before returning
#' the reader.
#' @param ... Additional arguments passed to `FUN`
-#' @param .data.frame Deprecated argument, ignored
#' @return An `arrow_dplyr_query`.
#' @export
-map_batches <- function(X, FUN, ..., .schema = NULL, .lazy = TRUE, .data.frame = NULL) {
- if (!is.null(.data.frame)) {
- warning(
- "The .data.frame argument is deprecated. ",
- "Call collect() on the result to get a data.frame.",
- call. = FALSE
- )
- }
+map_batches <- function(X, FUN, ..., .schema = NULL, .lazy = TRUE) {
Review Comment:
This removes a public argument after a multi-release deprecation, but the current `r/NEWS.md` development section has no release note for it (or for the finalized `pull()` behavior). Please document these user-facing changes, listing the argument removal under breaking changes so users can prepare before upgrading.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org