[jira] [Commented] (FLINK-17302) Add multiget() interface for state of rocksdb statebackend

["guxiang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-17302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511070#comment-17511070 ] 

guxiang commented on FLINK-17302:
---------------------------------

Hi, I have written the following demo, do you have any suggestions? 

 

in   RocksDBMapState.java   add a  function
{code:java}
//代码占位符
public Map<UK, UV> multiGet(List<UK> userKey) throws Exception{
    HashMap<UK, UV> ukUvHashMap = new HashMap<>();
    List<byte[]> rawKeyBytesList = new LinkedList<>();
    List<ColumnFamilyHandle> columnFamilyHandleList = new LinkedList<>();
    Map<byte[], UK> ukBytesMap = new HashMap<>();
    for (UK uk : userKey) {
        byte[] rawKeyBytes =
                serializeCurrentKeyWithGroupAndNamespacePlusUserKey(uk, userKeySerializer);
        rawKeyBytesList.add(rawKeyBytes);
        columnFamilyHandleList.add(columnFamily);
        ukBytesMap.put(rawKeyBytes, uk);
    }

    List<byte[]> resultBytes = backend.db.multiGetAsList(
            columnFamilyHandleList,
            rawKeyBytesList);


    for (int i = 0; i < userKey.size(); i++) {
        byte[] rawValueBytes = resultBytes.get(i);
        UV tempEntry = rawValueBytes == null
                ? null
                : deserializeUserValue(dataInputView, rawValueBytes, userValueSerializer);
        ukUvHashMap.put(userKey.get(i), tempEntry);
    }
    return ukUvHashMap;
} {code}

> Add multiget() interface for  state of rocksdb statebackend
> -----------------------------------------------------------
>
>                 Key: FLINK-17302
>                 URL: https://issues.apache.org/jira/browse/FLINK-17302
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / State Backends
>    Affects Versions: 1.10.0
>            Reporter: Peidian Li
>            Priority: Not a Priority
>              Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> Users that need to lookup many keys in order to process an application level request end up calling Get() in a loop to read the required KVs. Rocksdb provides multiget() interface to improve reading efficiency.Do we consider providing multiget() interface for rocksdb statebackend ？
> See also [https://github.com/facebook/rocksdb/wiki/MultiGet-Performance].
> For other statebackend, we can call the get () interface in a loop  to implement multiget().
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)