You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Matthew Summersgill (Jira)" <ji...@apache.org> on 2021/03/04 17:53:00 UTC
[jira] [Commented] (ARROW-3783) [R] Incorrect collection of float
type
[ https://issues.apache.org/jira/browse/ARROW-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17295474#comment-17295474 ]
Matthew Summersgill commented on ARROW-3783:
--------------------------------------------
[~npr] – Just wanted to check and see what the status on this one was. Issue shows closed, and I see there have been some PR's in the past, but it wasn't clear if anything was merged. Regardless, when I attempt to collect a float32 value from Spark to R, I get the following warning. Is something that would make sense for me to open a new issue?
{code:r}
str(collect(SparkR::sql("SELECT float('1') AS x;"))$x[[1]])
## num 1
## Warning message:In value[[3L]](cond) : The conversion from Spark DataFrame to R DataFrame was attempted with Arrow optimization because 'spark.sql.execution.arrow.sparkr.enabled' is set to true; however, failed, attempting non-optimization. Reason: Error in checkSchemaInArrow(schema(x)): Arrow optimization in R does not support float type yet.
{code}
> [R] Incorrect collection of float type
> --------------------------------------
>
> Key: ARROW-3783
> URL: https://issues.apache.org/jira/browse/ARROW-3783
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
> Reporter: Javier Luraschi
> Assignee: Neal Richardson
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.16.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Repro from `sparklyr`:
>
> {code:java}
> library(sparklyr)
> library(arrow)
> sc <- spark_connect(master = "local")
> DBI::dbGetQuery(sc, "SELECT cast(1 as float)"){code}
>
> Actual:
> {code:java}
> CAST(1 AS FLOAT)
> 1 1065353216{code}
> Expected:
>
> {code:java}
> CAST(1 AS FLOAT)
> 1 1{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)