You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/09/29 07:31:34 UTC
[GitHub] [arrow-nanoarrow] krlmlr opened a new issue, #55: [R] Representation of BLOBs
krlmlr opened a new issue, #55:
URL: https://github.com/apache/arrow-nanoarrow/issues/55
Arrow issue: https://issues.apache.org/jira/projects/ARROW/issues/ARROW-17885
BLOBs come back as lists of integers, where a list of raw is expected.
``` r
library(nanoarrow)
data <- data.frame(
a = 1:3,
b = 2.5,
c = "three",
stringsAsFactors = FALSE
)
data$d <- blob::blob(as.raw(1:10))
tbl <- as_nanoarrow_array(data)
tbl
#> <nanoarrow_array +s[3]>
#> $ length : int 3
#> $ null_count: int 0
#> $ offset : int 0
#> $ buffers :List of 1
#> ..$ :<nanoarrow_buffer_validity[0 b] at 0x0>
#> $ children :List of 4
#> ..$ a:<nanoarrow_array i[3]>
#> .. ..$ length : int 3
#> .. ..$ null_count: int 0
#> .. ..$ offset : int 0
#> .. ..$ buffers :List of 2
#> .. .. ..$ :<nanoarrow_buffer_validity[0 b] at 0x0>
#> .. .. ..$ :<nanoarrow_buffer_data_int32[12 b] at 0x110fb94f8>
#> .. ..$ dictionary: NULL
#> .. ..$ children : list()
#> ..$ b:<nanoarrow_array g[3]>
#> .. ..$ length : int 3
#> .. ..$ null_count: int 0
#> .. ..$ offset : int 0
#> .. ..$ buffers :List of 2
#> .. .. ..$ :<nanoarrow_buffer_validity[0 b] at 0x0>
#> .. .. ..$ :<nanoarrow_buffer_data_double[24 b] at 0x1312bebe8>
#> .. ..$ dictionary: NULL
#> .. ..$ children : list()
#> ..$ c:<nanoarrow_array u[3]>
#> .. ..$ length : int 3
#> .. ..$ null_count: int 0
#> .. ..$ offset : int 0
#> .. ..$ buffers :List of 3
#> .. .. ..$ :<nanoarrow_buffer_validity[0 b] at 0x0>
#> .. .. ..$ :<nanoarrow_buffer_data_offset32[16 b] at 0x1002a8000>
#> .. .. ..$ :<nanoarrow_buffer_data_utf8[15 b] at 0x1002a8080>
#> .. ..$ dictionary: NULL
#> .. ..$ children : list()
#> ..$ d:<nanoarrow_array +l[3]>
#> .. ..$ length : int 3
#> .. ..$ null_count: int 0
#> .. ..$ offset : int 0
#> .. ..$ buffers :List of 2
#> .. .. ..$ :<nanoarrow_buffer_validity[0 b] at 0x0>
#> .. .. ..$ :<nanoarrow_buffer_data_offset32[16 b] at 0x1002a8040>
#> .. ..$ children :List of 1
#> .. .. ..$ item:<nanoarrow_array C[30]>
#> .. .. .. ..$ length : int 30
#> .. .. .. ..$ null_count: int 0
#> .. .. .. ..$ offset : int 0
#> .. .. .. ..$ buffers :List of 2
#> .. .. .. .. ..$ :<nanoarrow_buffer_validity[0 b] at 0x0>
#> .. .. .. .. ..$ :<nanoarrow_buffer_data_uint8[30 b] at 0x1002a8100>
#> .. .. .. ..$ dictionary: NULL
#> .. .. .. ..$ children : list()
#> .. ..$ dictionary: NULL
#> $ dictionary: NULL
waldo::compare(as.data.frame(tbl), data)
#> `old$d[[1]]` is an integer vector (1, 2, 3, 4, 5, ...)
#> `new$d[[1]]` is a raw vector (01, 02, 03, 04, 05, ...)
#>
#> `old$d[[2]]` is an integer vector (1, 2, 3, 4, 5, ...)
#> `new$d[[2]]` is a raw vector (01, 02, 03, 04, 05, ...)
#>
#> `old$d[[3]]` is an integer vector (1, 2, 3, 4, 5, ...)
#> `new$d[[3]]` is a raw vector (01, 02, 03, 04, 05, ...)
```
<sup>Created on 2022-09-29 with [reprex v2.0.2](https://reprex.tidyverse.org)</sup>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-nanoarrow] paleolimbot commented on issue #55: [R] Representation of BLOBs
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #55:
URL: https://github.com/apache/arrow-nanoarrow/issues/55#issuecomment-1582961904
I forget if this was solved in 0.1 or 0.2, but I believe this is solved now:
``` r
library(nanoarrow)
data <- data.frame(
a = 1:3,
b = 2.5,
c = "three",
stringsAsFactors = FALSE
)
data$d <- blob::blob(as.raw(1:10))
array <- as_nanoarrow_array(data)
array$children$d
#> <nanoarrow_array binary[3]>
#> $ length : int 3
#> $ null_count: int 0
#> $ offset : int 0
#> $ buffers :List of 3
#> ..$ :<nanoarrow_buffer validity<bool>[0][0 b]> ``
#> ..$ :<nanoarrow_buffer data_offset<int32>[4][16 b]> `0 10 20 30`
#> ..$ :<nanoarrow_buffer data<binary>[30 b]> `01 02 03 04 05 06 07 08 09 0a ...`
#> $ dictionary: NULL
#> $ children : list()
convert_array(array$children$d)
#> <blob[3]>
#> [1] blob[10 B] blob[10 B] blob[10 B]
```
<sup>Created on 2023-06-08 with [reprex v2.0.2](https://reprex.tidyverse.org)</sup>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-nanoarrow] paleolimbot commented on issue #55: [R] Representation of BLOBs
Posted by GitBox <gi...@apache.org>.
paleolimbot commented on issue #55:
URL: https://github.com/apache/arrow-nanoarrow/issues/55#issuecomment-1262259300
I'll add list-of-raw conversions to #50. Right now nanoarrow uses arrow to convert *to* a nanoarrow array; #50 defines the conversion framework back to R land and conversions for the basic types that don't depend on arrow (which I was going to merge later today).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-nanoarrow] paleolimbot commented on issue #55: [R] Representation of BLOBs
Posted by GitBox <gi...@apache.org>.
paleolimbot commented on issue #55:
URL: https://github.com/apache/arrow-nanoarrow/issues/55#issuecomment-1262260225
(I added a note on the JIRA thread...that one is pretty solveable and I don't think the blob package was considered at the time the conversions were written).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-nanoarrow] paleolimbot closed issue #55: [R] Representation of BLOBs
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot closed issue #55: [R] Representation of BLOBs
URL: https://github.com/apache/arrow-nanoarrow/issues/55
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org