You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Praveen Rachabattuni (JIRA)" <ji...@apache.org> on 2015/02/24 04:52:10 UTC
[jira] [Updated] (PIG-4234) Order By error after Group By in Spark
[ https://issues.apache.org/jira/browse/PIG-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Praveen Rachabattuni updated PIG-4234:
--------------------------------------
Fix Version/s: spark-branch
> Order By error after Group By in Spark
> --------------------------------------
>
> Key: PIG-4234
> URL: https://issues.apache.org/jira/browse/PIG-4234
> Project: Pig
> Issue Type: Bug
> Components: spark
> Reporter: Carlos Balduz
> Labels: spork
> Fix For: spark-branch
>
>
> Trying to sort after a Group By produces the following error:
> 2014-10-14 16:04:55,189 [Executor task launch worker-0] ERROR org.apache.spark.executor.Executor - Exception in task 3.0 in stage 0.0 (TID 4)
> java.io.NotSerializableException: org.apache.pig.data.SelfSpillBag$MemoryLimits
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
> at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
> at java.util.ArrayList.writeObject(ArrayList.java:742)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495)
> at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1377)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1173)
> at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1377)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1173)
> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
> at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:42)
> at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:73)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:187)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Operations like for instance Rank By are not possible with this error, since it needs to sort right after grouping the data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)