You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Sascha (Jira)" <ji...@apache.org> on 2019/10/30 22:17:00 UTC

[jira] [Commented] (ARROW-7028) [R] Date roundtrip results in different R storage mode

    [ https://issues.apache.org/jira/browse/ARROW-7028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963474#comment-16963474 ] 

Sascha commented on ARROW-7028:
-------------------------------

It seems that R itself can handle numeric values as Dates, but also in a slightly inconsistent way.

``` r
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#> filter, lag
#> The following objects are masked from 'package:base':
#> 
#> intersect, setdiff, setequal, union

tmp = tempdir()

dat1 = tibble(dt = as.Date(1.5, origin = "1970-01-01"))
dat2 = tibble(dt = as.Date(1, origin = "1970-01-01"))

dat1
#> # A tibble: 1 x 1
#> dt 
#> <date> 
#> 1 1970-01-02
dat2
#> # A tibble: 1 x 1
#> dt 
#> <date> 
#> 1 1970-01-02

all.equal(dat1$dt, dat2$dt)
#> [1] "Mean relative difference: 0.3333333"

as.POSIXct(dat1$dt)
#> [1] "1970-01-02 13:00:00 CET"
as.POSIXct(dat2$dt)
#> [1] "1970-01-02 01:00:00 CET"

all.equal(as.POSIXct(dat1$dt), dat1$dt)
#> [1] TRUE

as.Date(dat1$dt) %>% as.numeric()
#> [1] 1.5
as.Date(as.POSIXct(dat1$dt)) %>% as.numeric()
#> [1] 1

arrow::write_parquet(dat1, file.path(tmp, "dat1.parquet"))
dat1pq = arrow::read_parquet(file.path(tmp, "dat1.parquet"))

all.equal(dat1$dt, dat1pq$dt)
#> [1] "Mean relative difference: 0.3333333"
```

<sup>Created on 2019-10-30 by the [reprex package](https://reprex.tidyverse.org) (v0.3.0)</sup>

 

> [R] Date roundtrip results in different R storage mode
> ------------------------------------------------------
>
>                 Key: ARROW-7028
>                 URL: https://issues.apache.org/jira/browse/ARROW-7028
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: R
>    Affects Versions: 0.15.0
>            Reporter: Sascha
>            Priority: Major
>         Attachments: image-2019-10-30-23-08-17-296.png
>
>
> When saving R-dataframes with parquet and loading them again, the internal representation of Dates changes, leading e.g. to errors when comparing them in dplyr::if_else.
> {code}
> library(dplyr)
> #> 
> #> Attaching package: 'dplyr'
> #> The following objects are masked from 'package:stats':
> #> 
> #> filter, lag
> #> The following objects are masked from 'package:base':
> #> 
> #> intersect, setdiff, setequal, union
> tmp = tempdir()
> dat = tibble(tag = as.Date("2018-01-01"))
> dat2 = tibble(tag2 = as.Date("2019-01-01"))
> arrow::write_parquet(dat, file.path(tmp, "dat.parquet"))
> dat = arrow::read_parquet(file.path(tmp, "dat.parquet"))
> typeof(dat$tag)
> #> [1] "integer"
> typeof(dat2$tag2)
> #> [1] "double"
> bind_cols(dat, dat2) %>%
>  mutate(comparison = if_else(TRUE, tag, tag2))
> #> `false` must be a `Date` object, not a `Date` object
> {code}
> Created on 2019-10-30 by the [reprex package](https://reprex.tidyverse.org) (v0.3.0)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)