[jira] [Updated] (PIG-5380) SortedDataBag hitting ConcurrentModificationException or producing incorrect output in a corner-case

["Koji Noguchi (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-5380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Koji Noguchi updated PIG-5380:
------------------------------
    Attachment: pig-5380-v02.patch

{quote}
Looking back on changes, I think I understand that this is a bigger bug than this jira given all these self-spilling bags are designed on the premise that no other threads would touch them (and thus lockings are dropped).
{quote}
This turned out to be false. PIG-5390 has the details. 

bq. I think same change is required in InternalSortedBag as well as code is exactly same and it can spill too
Made the same changes to InternalSortedBag.
Also while comparing the two source codes,  I think I found one section that was missing locking while reading from memory.  Added.


> SortedDataBag hitting ConcurrentModificationException or producing incorrect output in a corner-case 
> -----------------------------------------------------------------------------------------------------
>
>                 Key: PIG-5380
>                 URL: https://issues.apache.org/jira/browse/PIG-5380
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Koji Noguchi
>            Assignee: Koji Noguchi
>            Priority: Major
>         Attachments: pig-5380-v01.patch, pig-5380-v02.patch
>
>
> User had a UDF that created large SortedDataBag.  This UDF was failing with 
> {noformat}
> java.util.ConcurrentModificationException
>   at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
>   at java.util.ArrayList$Itr.next(ArrayList.java:851)
>   at org.apache.pig.data.SortedDataBag$SortedDataBagIterator.readFromPriorityQ(SortedDataBag.java:346)
>   at org.apache.pig.data.SortedDataBag$SortedDataBagIterator.next(SortedDataBag.java:322)
>   at org.apache.pig.data.SortedDataBag$SortedDataBagIterator.hasNext(SortedDataBag.java:235)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)