[jira] Updated: (PIG-26) distinct does not work on Bags that have spilled to disk.

["Benjamin Reed (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-26?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Benjamin Reed updated PIG-26:
-----------------------------

    Attachment: distinct-test.patch

The patch tweaks the bag test to manifest the bug.

> distinct does not work on Bags that have spilled to disk.
> ---------------------------------------------------------
>
>                 Key: PIG-26
>                 URL: https://issues.apache.org/jira/browse/PIG-26
>             Project: Pig
>          Issue Type: Bug
>          Components: data
>    Affects Versions: 0.0.0, 0.1.0, site
>            Reporter: Benjamin Reed
>            Assignee: Benjamin Reed
>         Attachments: distinct-test.patch
>
>
> If you call distinct on a bag that has spilled to disk, you get the following error:
> java.lang.NullPointerException
>         at org.apache.pig.data.BigDataBag$FileMerger$1.compare(BigDataBag.java:288)
>         at org.apache.pig.data.BigDataBag$FileMerger$1.compare(BigDataBag.java:280)
>         at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:594)
>         at java.util.PriorityQueue.siftUp(PriorityQueue.java:572)
>         at java.util.PriorityQueue.offer(PriorityQueue.java:274)
>         at java.util.PriorityQueue.add(PriorityQueue.java:251)
>         at org.apache.pig.data.BigDataBag$FileMerger.<init>(BigDataBag.java:304)
>         at org.apache.pig.data.BigDataBag.doSorting(BigDataBag.java:167)
>         at org.apache.pig.data.BigDataBag.content(BigDataBag.java:211)
>         at org.apache.pig.test.TestDataModel.testBigDataBag(TestDataModel.java:343)
>         at org.apache.pig.test.TestDataModel.testBigDataBagOnDisk(TestDataModel.java:210)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.