You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (JIRA)" <ji...@apache.org> on 2016/06/18 20:09:05 UTC
[jira] [Commented] (SPARK-6814) Support sorting for any data type
in SparkR
[ https://issues.apache.org/jira/browse/SPARK-6814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15338147#comment-15338147 ]
Dongjoon Hyun commented on SPARK-6814:
--------------------------------------
Hi, [~shivaram].
Since SparkR RDD is hiding from users now, can we simply close this issue?
> Support sorting for any data type in SparkR
> -------------------------------------------
>
> Key: SPARK-6814
> URL: https://issues.apache.org/jira/browse/SPARK-6814
> Project: Spark
> Issue Type: New Feature
> Components: SparkR
> Reporter: Shivaram Venkataraman
> Priority: Critical
>
> I get various "return status == 0 is false" and "unimplemented type" errors trying to get data out of any rdd with top() or collect(). The errors are not consistent. I think spark is installed properly because some operations do work. I apologize if I'm missing something easy or not providing the right diagnostic info – I'm new to SparkR, and this seems to be the only resource for SparkR issues.
> Some logs:
> {code}
> Browse[1]> top(estep.rdd, 1L)
> Error in order(unlist(part, recursive = FALSE), decreasing = !ascending) :
> unimplemented type 'list' in 'orderVector1'
> Calls: do.call ... Reduce -> <Anonymous> -> func -> FUN -> FUN -> order
> Execution halted
> 15/02/13 19:11:57 ERROR Executor: Exception in task 0.0 in stage 14.0 (TID 14)
> org.apache.spark.SparkException: R computation failed with
> Error in order(unlist(part, recursive = FALSE), decreasing = !ascending) :
> unimplemented type 'list' in 'orderVector1'
> Calls: do.call ... Reduce -> <Anonymous> -> func -> FUN -> FUN -> order
> Execution halted
> at edu.berkeley.cs.amplab.sparkr.BaseRRDD.compute(RRDD.scala:69)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62)
> at org.apache.spark.scheduler.Task.run(Task.scala:54)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 15/02/13 19:11:57 WARN TaskSetManager: Lost task 0.0 in stage 14.0 (TID 14, localhost): org.apache.spark.SparkException: R computation failed with
> Error in order(unlist(part, recursive = FALSE), decreasing = !ascending) :
> unimplemented type 'list' in 'orderVector1'
> Calls: do.call ... Reduce -> <Anonymous> -> func -> FUN -> FUN -> order
> Execution halted
> edu.berkeley.cs.amplab.sparkr.BaseRRDD.compute(RRDD.scala:69)
> org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
> org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
> org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62)
> org.apache.spark.scheduler.Task.run(Task.scala:54)
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org