You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Matthew Summersgill (Jira)" <ji...@apache.org> on 2021/03/04 17:53:00 UTC

[jira] [Commented] (ARROW-3783) [R] Incorrect collection of float type

    [ https://issues.apache.org/jira/browse/ARROW-3783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17295474#comment-17295474 ] 

Matthew Summersgill commented on ARROW-3783:
--------------------------------------------

[~npr]  – Just wanted to check and see what the status on this one was. Issue shows closed, and I see there have been some PR's in the past, but it wasn't clear if anything was merged. Regardless, when I attempt to collect a float32 value from Spark to R, I get the following warning. Is something that would make sense for me to open a new issue?
{code:r}
str(collect(SparkR::sql("SELECT float('1') AS x;"))$x[[1]])
## num 1

## Warning message:In value[[3L]](cond) :  The conversion from Spark DataFrame to R DataFrame was attempted with Arrow optimization because 'spark.sql.execution.arrow.sparkr.enabled' is set to true; however, failed, attempting non-optimization. Reason: Error in checkSchemaInArrow(schema(x)): Arrow optimization in R does not support float type yet.
{code}

> [R] Incorrect collection of float type
> --------------------------------------
>
>                 Key: ARROW-3783
>                 URL: https://issues.apache.org/jira/browse/ARROW-3783
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: R
>            Reporter: Javier Luraschi
>            Assignee: Neal Richardson
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.16.0
>
>          Time Spent: 2h
>  Remaining Estimate: 0h
>
> Repro from `sparklyr`:
>  
> {code:java}
> library(sparklyr)
> library(arrow)
> sc <- spark_connect(master = "local")
> DBI::dbGetQuery(sc, "SELECT cast(1 as float)"){code}
>  
> Actual:
> {code:java}
>   CAST(1 AS FLOAT)
> 1       1065353216{code}
> Expected:
>  
> {code:java}
>   CAST(1 AS FLOAT)
> 1                1{code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)