You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2020/12/17 08:23:02 UTC
[jira] [Commented] (SPARK-31517) SparkR::orderBy with multiple
columns descending produces error
[ https://issues.apache.org/jira/browse/SPARK-31517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17250892#comment-17250892 ]
Hyukjin Kwon commented on SPARK-31517:
--------------------------------------
Fixed in https://github.com/apache/spark/pull/28386
> SparkR::orderBy with multiple columns descending produces error
> ---------------------------------------------------------------
>
> Key: SPARK-31517
> URL: https://issues.apache.org/jira/browse/SPARK-31517
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Affects Versions: 2.4.5
> Environment: Databricks Runtime 6.5
> Reporter: Ross Bowen
> Priority: Major
>
> When specifying two columns within an `orderBy()` function, to attempt to get an ordering by two columns in descending order, an error is returned.
> {code:java}
> library(magrittr)
> library(SparkR)
> cars <- cbind(model = rownames(mtcars), mtcars)
> carsDF <- createDataFrame(cars)
> carsDF %>%
> mutate(rank = over(rank(), orderBy(windowPartitionBy(column("cyl")), desc(column("mpg")), desc(column("disp"))))) %>%
> head() {code}
> This returns an error:
> {code:java}
> Error in ns[[i]] : subscript out of bounds{code}
> This seems to be related to the more general issue that the following code, excluding the use of the `desc()` function also fails:
> {code:java}
> carsDF %>%
> mutate(rank = over(rank(), orderBy(windowPartitionBy(column("cyl")), column("mpg"), column("disp")))) %>%
> head(){code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org