[jira] [Commented] (FLINK-12693) Store state per key-group in CopyOnWriteStateTable

["Lisheng Sun (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120452#comment-17120452 ] 

Lisheng Sun commented on FLINK-12693:
-------------------------------------

hi [~banmoy] 

According to  test's result,  the performance of calculation hash in CopyOnWriteStateMap is much worse than JDK HashMap.

 Could you tell what the new hash algorithm is for.  Thank you.

CopyOnWriteStateMap#computeHashForOperationAndDoIncrementalRehash#compositeHash#bitMix

 
{code:java}
public static int bitMix(int in) {
   in ^= in >>> 16;
   in *= 0x85ebca6b;
   in ^= in >>> 13;
   in *= 0xc2b2ae35;
   in ^= in >>> 16;
   return in;
}
{code}
HashMap#hash
{code:java}
static final int hash(Object key) {
    int h;
    return (key == null) ? 0 : (h = key.hashCode()) ^ (h >>> 16);
}{code}
 

> Store state per key-group in CopyOnWriteStateTable
> --------------------------------------------------
>
>                 Key: FLINK-12693
>                 URL: https://issues.apache.org/jira/browse/FLINK-12693
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / State Backends
>            Reporter: Yu Li
>            Assignee: PengFei Li
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Since we propose to use KeyGroup as the unit of spilling/loading, the first step is to store state per key-groups. Currently {{NestedMapsStateTable}} natively supports this, so we only need to refine {{CopyOnWriteStateTable}}
> The main efforts required here is to extract the customized hash-map out of {{CopyOnWriteStateTable}} then use such a hash-map as the state holder for each KeyGroup. Whereafter we could extract some common logic out into {{StateTable}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)