You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Uwe Korn (Jira)" <ji...@apache.org> on 2019/10/30 09:45:01 UTC
[jira] [Commented] (ARROW-7028) Dates in R are different when saved
and loaded with parquet
[ https://issues.apache.org/jira/browse/ARROW-7028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16962860#comment-16962860 ]
Uwe Korn commented on ARROW-7028:
---------------------------------
cc [~npr] This is the issue I talked about on Zulip yesterday. This time with an argument why we should ensure better reconstruction.
> Dates in R are different when saved and loaded with parquet
> -----------------------------------------------------------
>
> Key: ARROW-7028
> URL: https://issues.apache.org/jira/browse/ARROW-7028
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
> Affects Versions: 0.15.0
> Reporter: Sascha
> Priority: Major
>
> When saving R-dataframes with parquet and loading them again, the internal representation of Dates changes, leading e.g. to errors when comparing them in dplyr::if_else.
> {code}
> library(dplyr)
> #>
> #> Attaching package: 'dplyr'
> #> The following objects are masked from 'package:stats':
> #>
> #> filter, lag
> #> The following objects are masked from 'package:base':
> #>
> #> intersect, setdiff, setequal, union
> tmp = tempdir()
> dat = tibble(tag = as.Date("2018-01-01"))
> dat2 = tibble(tag2 = as.Date("2019-01-01"))
> arrow::write_parquet(dat, file.path(tmp, "dat.parquet"))
> dat = arrow::read_parquet(file.path(tmp, "dat.parquet"))
> typeof(dat$tag)
> #> [1] "integer"
> typeof(dat2$tag2)
> #> [1] "double"
> bind_cols(dat, dat2) %>%
> mutate(comparison = if_else(TRUE, tag, tag2))
> #> `false` must be a `Date` object, not a `Date` object
> {code}
> Created on 2019-10-30 by the [reprex package](https://reprex.tidyverse.org) (v0.3.0)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)