[jira] [Created] (FLINK-16387) Use LinkedHashMap for deterministic order in HeapMapState.java

["cpugputpu (Jira)" <ji...@apache.org>]

cpugputpu created FLINK-16387:
---------------------------------

             Summary: Use LinkedHashMap for deterministic order in HeapMapState.java
                 Key: FLINK-16387
                 URL: https://issues.apache.org/jira/browse/FLINK-16387
             Project: Flink
          Issue Type: Bug
            Reporter: cpugputpu


The tests in _org.apache.flink.table.runtime.operators.join.RowTimeBoundedStreamJoinTest#testRowTimeInnerJoinWithCommonBounds_ and _org.apache.flink.table.runtime.operators.join.ProcTimeBoundedStreamJoinTest#testProcTimeInnerJoinWithCommonBounds_ fail due to a different iteration order of a HashMap. The failure is presented as follows.

_org.junit.internal.ArrayComparisonFailure: output wrong.: arrays first differed at element [0]; 
expected:<(+|1,1a1,1,1b3)> 
but was:<(+|1,1a3,1,1b3)>
at org.apache.flink.table.runtime.operators.join.ProcTimeBoundedStreamJoinTest.testProcTimeInnerJoinWithCommonBounds(ProcTimeBoundedStreamJoinTest.java:101_

 

The root cause of this failure lies in a HashMap's iterator, which makes no guarantee about the iteration order. A brief stack trace is for your reference:

ava.util.HashMap$EntrySet.iterator(HashMap.java:1014)
org.apache.flink.runtime.state.heap.HeapMapState.iterator(HeapMapState.java:161)
org.apache.flink.runtime.state.UserFacingMapState.iterator(UserFacingMapState.java:95)
org.apache.flink.table.runtime.operators.join.TimeBoundedStreamJoin.processElement2(TimeBoundedStreamJoin.java:246)
org.apache.flink.table.runtime.operators.join.ProcTimeBoundedStreamJoin.processElement2(ProcTimeBoundedStreamJoin.java:29)
org.apache.flink.table.runtime.operators.join.TimeBoundedStreamJoin.processElement2(TimeBoundedStreamJoin.java:52)
org.apache.flink.streaming.api.operators.co.LegacyKeyedCoProcessOperator.processElement2(LegacyKeyedCoProcessOperator.java:89)
org.apache.flink.streaming.util.TwoInputStreamOperatorTestHarness.processElement2(TwoInputStreamOperatorTestHarness.java:57)
org.apache.flink.table.runtime.operators.join.ProcTimeBoundedStreamJoinTest.testProcTimeInnerJoinWithCommonBounds(ProcTimeBoundedStreamJoinTest.java:70)

The specification about HashMap says that "this class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time". The documentation is here for your reference: [https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html]

 

The fix is to use LinkedHashMap instead of HashMap. In this way, the test will not suffer from failure any more and the code will be more stable, free of this non-deterministic behaviour.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)