[jira] [Commented] (SAMZA-545) Make in-memory key-value store skip serde

["Yi Pan (Data Infrastructure) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304169#comment-14304169 ] 

Yi Pan (Data Infrastructure) commented on SAMZA-545:
----------------------------------------------------

Can we let the KV-store handle the raw object serialization in write if changelog is attached? I.e. not requiring the caller to serialize the object to bytes and then pass to put()? 

That would let me to write the following code w/o caring whether the underlying store has a changelog or not:
{code}
KeyValueStore<Object, Object> store = new InMemoryKeyValueStore();
store.get(key);
store.put(key, value);
{code}

> Make in-memory key-value store skip serde
> -----------------------------------------
>
>                 Key: SAMZA-545
>                 URL: https://issues.apache.org/jira/browse/SAMZA-545
>             Project: Samza
>          Issue Type: Bug
>          Components: kv
>    Affects Versions: 0.9.0
>            Reporter: Chris Riccomini
>
> SAMZA-256 added an in-memory implementation of the samza-kv store. Due to the layering in Samza's KV-store APIs, the in-memory store still holds raw bytes, and the Serde is used to transform objects back into POJOs. On the read-side, it is unnecessary to deserialize the bytes back into an object. The in-memory KV store should just hold the raw object.
> Semantically, this does change the behavior of the KV-store a bit, when using in-memory stores. If an object is mutated after it's been written to an in-memory store, and then store.get is called, the mutated object will be returned. This is not the case with regular (LevelDB/RocksDB) KV-stores.
> Writes will still require serializing the object if a changelog is attached. If a changelog is not attached, then I'd argue that the in-memory KV store should not be used at all, and a simple HashMap should be used instead.
> When updating the code, we should be mindful to keep the API as clean as possible, while shifting the layers around.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)