[jira] [Commented] (SAMZA-2075) Allow Serde in InMemorySystem

["Xinyu Liu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745328#comment-16745328 ] 

Xinyu Liu commented on SAMZA-2075:
----------------------------------

Problem description from Tom Davis:

"I am in the process of updating a project to 1.0 and spent today debugging a
rather odd test failure. When using input/output streams with IntegerSerde,
things worked fine -- however, using LongSerde, every message value was 0! I
eventually found that InMemorySystemDescriptor#getInputDescriptor ignores the
serde passed to it. However, I had still specified in my config:

streams.in-0.samza.msg.serde=integer

Apparently that *was* respected by some part of the system because integers were
deserialized properly! Removing this configuration value results in my operator
receiving a byte array since the in-memory system only uses NoOpSerde.

This behavior appears inconsistent with the previous version of Samza. The old
`getInputStream` was passed a serde that was always used, but since the new
version receives a Descriptor that has already discarded the serde, I am forced
into assuming NoOpSerde everywhere, at least for testing purposes.

Not the end of the world, but it does introduce an inconsistency between the
in-memory system and any other -- one that requires a fair bit of domain
knowledge to avoid."


> Allow Serde in InMemorySystem
> -----------------------------
>
>                 Key: SAMZA-2075
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2075
>             Project: Samza
>          Issue Type: Improvement
>    Affects Versions: 1.1
>            Reporter: Xinyu Liu
>            Priority: Major
>
> Currently we only allow in-memory system to contain deserialized objects. It will be good to add serde support to it so it can be used the same as normal input streams. This is good for prototyping and testing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)