You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "kou (via GitHub)" <gi...@apache.org> on 2024/02/04 04:33:52 UTC
[I] [CI][R][Python] R and Python integration's tests failed [arrow]
kou opened a new issue, #39933:
URL: https://github.com/apache/arrow/issues/39933
### Describe the bug, including details regarding any error messages, version, and platform.
https://github.com/apache/arrow/actions/runs/7760862085/job/21168058007#step:6:32228
<details>
```text
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test-python-flight.R:32:5'): flight_put ─────────────────────────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. └─arrow::flight_put(client, example_data, path = flight_obj) at test-python-flight.R:32:5
2. ├─reticulate::r_to_py(data)
3. └─arrow:::r_to_py.Table(data)
4. ├─reticulate::r_to_py(as_record_batch_reader(x), convert = FALSE)
5. └─arrow:::r_to_py.RecordBatchReader(...)
6. └─pa$lib$RecordBatchReader$`_import_from_c`(pyarrow_compatible_pointer(stream_ptr))
7. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error ('test-python-flight.R:42:5'): flight_put with max_chunksize ──────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. └─arrow::flight_put(client, example_data, path = flight_obj, max_chunksize = 1) at test-python-flight.R:42:5
2. ├─reticulate::r_to_py(data)
3. └─arrow:::r_to_py.Table(data)
3. └─pa$Field$`_import_from_c`(pyarrow_compatible_pointer(schema_ptr))
4. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error ('test-python.R:157:3'): RecordBatchReader to python ──────────────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─reticulate::r_to_py(reader) at test-python.R:157:3
2. └─arrow:::r_to_py.RecordBatchReader(reader)
3. └─pa$lib$RecordBatchReader$`_import_from_c`(pyarrow_compatible_pointer(stream_ptr))
4. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error ('test-python.R:177:3'): RecordBatchReader from python ────────────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─reticulate::r_to_py(reader) at test-python.R:177:3
2. └─arrow:::r_to_py.RecordBatchReader(reader)
3. └─pa$lib$RecordBatchReader$`_import_from_c`(pyarrow_compatible_pointer(stream_ptr))
4. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
```
</details>
The last success commit: https://github.com/apache/arrow/commit/2721134715b7dedfa2715bcf47548728ff702d5a
The first failed commit: https://github.com/apache/arrow/commit/b684028dfbeeed85d132a1249449a85877d796b1
Commits between them:
* https://github.com/apache/arrow/commit/44d5597a0e8a4d635f1aec82ba885f61b5c17829
* https://github.com/apache/arrow/commit/3d45ac96534fc76b820b488aa02182e6b93a388f
* https://github.com/apache/arrow/commit/a1c1773b724e4d78faf9a097247c7e976cd2cbfa
* https://github.com/apache/arrow/commit/4ceb66101382d74c6ef73ff546fad10183ab58d8
@jorisvandenbossche Could you take a look at this PyCapsule related error messages?
### Component(s)
Continuous Integration, Python, R
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1929464756
Got it!
It looks like capsules with a null name are accepted...perhaps it should be updated to accept a capsule with NULL or r_extptr?
https://github.com/apache/arrow/blob/cd5faafb0c811d5985156c1fd1aecd1aa7130e9f/python/pyarrow/types.pxi#L104-L105
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1929654900
> In [rstudio/reticulate@9b2c8d4](https://github.com/rstudio/reticulate/commit/9b2c8d484edb5e9225a2bf0f65e6bebfeb7dc4ad) the name changed from "NULL" to "r_extptr" and code was added to validate against the name (by calling `PyCapsule_IsValid`).
This seems to be about storing R objects inside capsules. This is not what the R-PyArrow bridge is doing, is it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot closed issue #39933: [CI][R][Python] R and Python integration's tests failed
URL: https://github.com/apache/arrow/issues/39933
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "amoeba (via GitHub)" <gi...@apache.org>.
amoeba commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1928819651
Thanks @paleolimbot. I just checked [the diff between 1.34.0 and 1.35.0](https://github.com/rstudio/reticulate/compare/v1.34.0...v1.35.0) and I think you're right on both counts. In https://github.com/rstudio/reticulate/commit/9b2c8d484edb5e9225a2bf0f65e6bebfeb7dc4ad the name changed from "NULL" to "r_extptr" and code was added to validate against the name (by calling `PyCapsule_IsValid`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1930050407
I will also submit a fix on the R side...it didn't occur to me at the time to convert to a Python integer using reticulate to work around the fact that R doesn't have 64-bit integers. If it's easy, it would probably also be a good idea to fix it in Python (or remove the ability to pass a pointer in that way, or fix the comment to reflect the new nature of how an external pointer gets converted using reticulate).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1929671791
My understanding is that the `r_to_py` will convert any extptr to such a capsule:
https://github.com/rstudio/reticulate/blob/950169382a92328bde39bff3b68f5014ac582986/src/python.cpp#L1971-L1974
And I suppose the array/schema pointers in the R package are "external pointers", and so reticulate automatically puts them in a capsule before passing to `_import_from_c` (since this is a python call, it converts any argument to it with `r_to_py`?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1928520043
#39942 may be happen by the same cause.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1929706412
Ahah, I see. Then I guess it makes sense to make the check more tolerant on the PyArrow side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1929015392
> so perhaps the name check was made stricter on the Python side when capsules started being used there.
FWIW, the python.R code is not yet using capsules (it's still using `_import_from_c(..)` with raw pointers, as can also be seen in the error traceback), so this "name check" is happening somewhere in reticulate itself?
(just want to say that AFAIU this is not related to the capsule work on the python side)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1929469771
Or, alternatively, we could fix in R by generating the Python integer there:
``` r
py <- reticulate::import_builtins(convert = FALSE)
# addr <- external_pointer_addr_double(ptr)
addr <- "12345"
class(py$int(addr))
#> [1] "python.builtin.int" "python.builtin.object"
py$int(addr)
#> 12345
```
<sup>Created on 2024-02-06 with [reprex v2.0.2](https://reprex.tidyverse.org)</sup>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "amoeba (via GitHub)" <gi...@apache.org>.
amoeba commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1928584583
The timing of this failure starting to show up is around the time [version 1.35.0 of reticulate came out](https://github.com/rstudio/reticulate/releases/tag/v1.35.0) and it looks like 1.35.0 was used in during the above failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "amoeba (via GitHub)" <gi...@apache.org>.
amoeba commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1928652618
I ran the tests in ./r/tests/testthat/test-python-flight.R with reticulate 1.34.0 and 1.35.0 and get _different_ failures so I think this needs a deeper investigation.
<details>
<summary>reticulate 1.34.0</summary>
```r
==> Testing R file using 'testthat'
ℹ Loading arrow
[ FAIL 6 | WARN 0 | SKIP 1 | PASS 1 ]
── Error (test-python-flight.R:28:5): flight_path_exists ───────────────────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_iterate(it, f)`: pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail: Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─testthat::expect_false(flight_obj %in% list_flights(client)) at test-python-flight.R:28:5
2. │ └─testthat::quasi_label(enquo(object), label, arg = "object")
3. │ └─rlang::eval_bare(expr, quo_get_env(quo))
4. ├─flight_obj %in% list_flights(client)
5. └─arrow::list_flights(client)
6. └─reticulate::iterate(generator, function(x) as.character(x$descriptor$path[[1]])) at r/R/flight.R:121:3
7. └─reticulate:::py_iterate(it, f)
── Error (test-python-flight.R:33:5): flight_put ───────────────────────────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail: Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─testthat::expect_true(flight_path_exists(client, flight_obj)) at test-python-flight.R:33:5
2. │ └─testthat::quasi_label(enquo(object), label, arg = "object")
3. │ └─rlang::eval_bare(expr, quo_get_env(quo))
4. └─arrow::flight_path_exists(client, flight_obj)
5. ├─base::tryCatch(...) at r/R/flight.R:128:3
6. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
7. │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
8. │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
9. └─client$get_flight_info(descriptor_for_path(path)) at r/R/flight.R:130:7
10. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error (test-python-flight.R:43:5): flight_put with max_chunksize ────────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail: Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─testthat::expect_true(flight_path_exists(client, flight_obj)) at test-python-flight.R:43:5
2. │ └─testthat::quasi_label(enquo(object), label, arg = "object")
3. │ └─rlang::eval_bare(expr, quo_get_env(quo))
4. └─arrow::flight_path_exists(client, flight_obj)
5. ├─base::tryCatch(...) at r/R/flight.R:128:3
6. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
7. │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
8. │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
9. └─client$get_flight_info(descriptor_for_path(path)) at r/R/flight.R:130:7
10. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error (test-python-flight.R:56:5): flight_get ───────────────────────────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail: Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─expect_equal_data_frame(flight_get(client, flight_obj), example_data) at test-python-flight.R:56:5
2. │ ├─expect_equal(as.data.frame(x), as.data.frame(y), ...) at tests/testthat/helper-expectation.R:24:3
3. │ │ └─base::inherits(object, "ArrowObject") at tests/testthat/helper-expectation.R:35:3
4. │ └─base::as.data.frame(x)
5. └─arrow::flight_get(client, flight_obj)
6. └─arrow:::flight_reader(client, path) at r/R/flight.R:96:3
7. └─client$get_flight_info(descriptor_for_path(path)) at r/R/flight.R:102:3
8. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error (test-python-flight.R:62:5): flight_put with RecordBatch ──────────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail: Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─expect_equal_data_frame(flight_get(client, flight_obj2), example_data) at test-python-flight.R:62:5
2. │ ├─expect_equal(as.data.frame(x), as.data.frame(y), ...) at tests/testthat/helper-expectation.R:24:3
3. │ │ └─base::inherits(object, "ArrowObject") at tests/testthat/helper-expectation.R:35:3
4. │ └─base::as.data.frame(x)
5. └─arrow::flight_get(client, flight_obj2)
6. └─arrow:::flight_reader(client, path) at r/R/flight.R:96:3
7. └─client$get_flight_info(descriptor_for_path(path)) at r/R/flight.R:102:3
8. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error (test-python-flight.R:66:5): flight_put with overwrite = FALSE ────────
<pyarrow.lib.ArrowInvalid/python.builtin.ValueError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: pyarrow.lib.ArrowInvalid: invalid literal for int() with base 10: ''. Detail: Python exception: ValueError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─testthat::expect_error(...) at test-python-flight.R:66:5
2. │ └─testthat:::expect_condition_matching(...)
3. │ └─testthat:::quasi_capture(...)
4. │ ├─testthat (local) .capture(...)
5. │ │ └─base::withCallingHandlers(...)
6. │ └─rlang::eval_bare(quo_get_expr(.quo), quo_get_env(.quo))
7. └─arrow::flight_put(...)
8. └─arrow::flight_path_exists(client, path) at r/R/flight.R:66:3
9. ├─base::tryCatch(...) at r/R/flight.R:128:3
10. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
11. │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
12. │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
13. └─client$get_flight_info(descriptor_for_path(path)) at r/R/flight.R:130:7
14. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Skipped tests (1) ───────────────────────────────────────────────────────────────────────────────────────────────
• empty test (1): test-python-flight.R:75:3
[ FAIL 6 | WARN 0 | SKIP 1 | PASS 1 ]
Test complete
```
</details>
<details>
<summary>reticulate 1.34.0</summary>
```r
==> Testing R file using 'testthat'
ℹ Loading arrow
[ FAIL 5 | WARN 0 | SKIP 1 | PASS 2 ]
── Error (test-python-flight.R:32:5): flight_put ───────────────────────────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. └─arrow::flight_put(client, example_data, path = flight_obj) at test-python-flight.R:32:5
2. ├─reticulate::r_to_py(data) at r/R/flight.R:73:3
3. └─arrow:::r_to_py.Table(data)
4. ├─reticulate::r_to_py(as_record_batch_reader(x), convert = FALSE) at r/R/python.R:111:3
5. └─arrow:::r_to_py.RecordBatchReader(...)
6. └─pa$lib$RecordBatchReader$`_import_from_c`(pyarrow_compatible_pointer(stream_ptr)) at r/R/python.R:227:3
7. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error (test-python-flight.R:42:5): flight_put with max_chunksize ────────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. └─arrow::flight_put(client, example_data, path = flight_obj, max_chunksize = 1) at test-python-flight.R:42:5
2. ├─reticulate::r_to_py(data) at r/R/flight.R:73:3
3. └─arrow:::r_to_py.Table(data)
4. ├─reticulate::r_to_py(as_record_batch_reader(x), convert = FALSE) at r/R/python.R:111:3
5. └─arrow:::r_to_py.RecordBatchReader(...)
6. └─pa$lib$RecordBatchReader$`_import_from_c`(pyarrow_compatible_pointer(stream_ptr)) at r/R/python.R:227:3
7. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error (test-python-flight.R:56:5): flight_get ───────────────────────────────
<pyarrow.lib.ArrowKeyError/python.builtin.KeyError/python.builtin.LookupError/pyarrow.lib.ArrowException/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: pyarrow.lib.ArrowKeyError: 'Flight not found.'. Detail: Python exception: KeyError
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─expect_equal_data_frame(flight_get(client, flight_obj), example_data) at test-python-flight.R:56:5
2. │ ├─expect_equal(as.data.frame(x), as.data.frame(y), ...) at tests/testthat/helper-expectation.R:24:3
3. │ │ └─base::inherits(object, "ArrowObject") at tests/testthat/helper-expectation.R:35:3
4. │ └─base::as.data.frame(x)
5. └─arrow::flight_get(client, flight_obj)
6. └─arrow:::flight_reader(client, path) at r/R/flight.R:96:3
7. └─client$get_flight_info(descriptor_for_path(path)) at r/R/flight.R:102:3
8. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error (test-python-flight.R:61:5): flight_put with RecordBatch ──────────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. └─arrow::flight_put(client, RecordBatch$create(example_data), path = flight_obj2) at test-python-flight.R:61:5
2. ├─reticulate::r_to_py(data) at r/R/flight.R:73:3
3. └─arrow:::r_to_py.RecordBatch(data)
4. └─pa$RecordBatch$`_import_from_c`(...) at r/R/python.R:84:3
5. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Error (test-python-flight.R:66:5): flight_put with overwrite = FALSE ────────
<python.builtin.ValueError/python.builtin.Exception/python.builtin.BaseException/python.builtin.object/error/condition>
Error in `py_call_impl(callable, call_args$unnamed, call_args$named)`: ValueError: PyCapsule_GetPointer called with incorrect name
Run `reticulate::py_last_error()` for details.
Backtrace:
▆
1. ├─testthat::expect_error(...) at test-python-flight.R:66:5
2. │ └─testthat:::expect_condition_matching(...)
3. │ └─testthat:::quasi_capture(...)
4. │ ├─testthat (local) .capture(...)
5. │ │ └─base::withCallingHandlers(...)
6. │ └─rlang::eval_bare(quo_get_expr(.quo), quo_get_env(.quo))
7. └─arrow::flight_put(...)
8. ├─reticulate::r_to_py(data) at r/R/flight.R:73:3
9. └─arrow:::r_to_py.Table(data)
10. ├─reticulate::r_to_py(as_record_batch_reader(x), convert = FALSE) at r/R/python.R:111:3
11. └─arrow:::r_to_py.RecordBatchReader(...)
12. └─pa$lib$RecordBatchReader$`_import_from_c`(pyarrow_compatible_pointer(stream_ptr)) at r/R/python.R:227:3
13. └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)
── Skipped tests (1) ───────────────────────────────────────────────────────────────────────────────────────────────
• empty test (1): test-python-flight.R:75:3
[ FAIL 5 | WARN 0 | SKIP 1 | PASS 2 ]
Test complete
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "amoeba (via GitHub)" <gi...@apache.org>.
amoeba commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1928621465
Let me verify my guess, I'll report back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1928614529
Can we pin reticulate for now?
Could you report this to reticulate?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1928657729
I believe reticulate 1.35.0 changed the "name" of the capsule. I am not sure what it was before and I am not sure what it is now...we started using external pointers on the R side in Arrow 7.0.0, which converted to Python capsules automatically, because it felt better than converting the pointer to an R `double()`, sending that to Python, then static casting that to a `uintptr_t` and calling it a pointer again.
One of the errors is along the lines of `PyCapsule_GetPointer() called with incorrect name`, so perhaps the name check was made stricter on the Python side when capsules started being used there.
On the R side, we could go back to calling `_import_from_c` with a floating-point value (although I would rather not). We also have the ability to call it with a string representation of the address, which is also lossless, but this is rejected by pyarrow (I just checked). I also just checked passing a floating point value and that results in the test passing for me.
The code on the R side that could change is here:
https://github.com/apache/arrow/blob/672238ff6352fa388b54182d8ae1667f9e99c327/r/R/python.R#L341-L353
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [CI][R][Python] R and Python integration's tests failed [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #39933:
URL: https://github.com/apache/arrow/issues/39933#issuecomment-1930084137
We should do both, because ideally one can use an older R-Arrow with a newer PyArrow and vice-versa.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org