[jira] [Created] (PIG-4012) java.lang.IllegalArgumentException: Comparison method violates its general contract! SpillableMemoryManager

["David Dreyfus (JIRA)" <ji...@apache.org>]

David Dreyfus created PIG-4012:
----------------------------------

             Summary: java.lang.IllegalArgumentException: Comparison method violates its general contract! SpillableMemoryManager
                 Key: PIG-4012
                 URL: https://issues.apache.org/jira/browse/PIG-4012
             Project: Pig
          Issue Type: Bug
          Components: impl
    Affects Versions: 0.12.0
         Environment: java version "1.7.0_60-ea"
Java(TM) SE Runtime Environment (build 1.7.0_60-ea-b02)
Java HotSpot(TM) 64-Bit Server VM (build 24.60-b04, mixed mode)

            Reporter: David Dreyfus


java.lang.IllegalArgumentException: Comparison method violates its general contract!
        at java.util.TimSort.mergeHi(TimSort.java:868)
        at java.util.TimSort.mergeAt(TimSort.java:485)
        at java.util.TimSort.mergeForceCollapse(TimSort.java:426)
        at java.util.TimSort.sort(TimSort.java:223)
        at java.util.TimSort.sort(TimSort.java:173)
        at java.util.Arrays.sort(Arrays.java:659)
        at java.util.Collections.sort(Collections.java:217)
        at org.apache.pig.impl.util.SpillableMemoryManager.handleNotification(SpillableMemoryManager.java:199)
        at sun.management.NotificationEmitterSupport.sendNotification(NotificationEmitterSupport.java:156)
        at sun.management.MemoryImpl.createNotification(MemoryImpl.java:168)
        at sun.management.MemoryPoolImpl$PoolSensor.triggerAction(MemoryPoolImpl.java:301)
        at sun.management.Sensor.trigger(Sensor.java:137)

>From SpillableMemoryManager.java:
                /**
                 * We don't lock anything, so this sort may not be stable if a WeakReference suddenly
                 * becomes null, but it will be close enough.
                 * Also between the time we sort and we use these spillables, they
                 * may actually change in size - so this is just best effort
                 */
Issue may be due to Java 7 and reporting vs ignoring the exception.
Trying      
-Djava.util.Arrays.useLegacyMergeSort=true

http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6804124
suggests the newer MergeSort is much faster.

Someone may want to make the sorting stable in SpillableMemoryManager so that the new merge sort can be used without failure.



--
This message was sent by Atlassian JIRA
(v6.2#6252)