You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Antony Stubbs (Jira)" <ji...@apache.org> on 2020/02/19 22:20:00 UTC

[jira] [Updated] (KAFKA-9259) suppress() for windowed-Serdes does not work with default serdes

     [ https://issues.apache.org/jira/browse/KAFKA-9259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Antony Stubbs updated KAFKA-9259:
---------------------------------
    Description: 
The suppress() operator either inherits serdes from its upstream operator or falls back to default serdes from the config.

If the upstream operator is an windowed aggregation, the window-aggregation operator wraps the user passed-in serde with a window-serde and pushed it into suppress() – however, if default serdes are used, the window-aggregation operator cannot push anything into suppress(). At runtime, it just creates a default serde and wraps it according. For this case, suppress() also falls back to default serdes; however, it does not wrap the serde and thus a ClassCastException is thrown when the serde is used later.

suppress() is already aware if the upstream aggregation is time/session windowed or not and thus should use this information to wrap default serdes accordingly.

The current workaround for windowed-suppress is to overwrite the default serde upstream to suppress(), such that suppress() inherits serdes and does not fall back to default serdes.

 

For the search engines, when using Avro, this results in a ClassCastException:

{{Caused by: java.lang.ClassCastException: class org.apache.kafka.streams.kstream.Windowed cannot be cast to class org.apache.avro.specific.SpecificRecord (org.apache.kafka.streams.kstream.Windowed and org.apache.avro.specific.SpecificRecord are in unnamed module of loader 'app') at io.confluent.kafka.streams.serdes.avro.SpecificAvroSerializer.serialize(SpecificAvroSerializer.java:38)}}

  was:
The suppress() operator either inherits serdes from its upstream operator or falls back to default serdes from the config.

If the upstream operator is an windowed aggregation, the window-aggregation operator wraps the user passed-in serde with a window-serde and pushed it into suppress() – however, if default serdes are used, the window-aggregation operator cannot push anything into suppress(). At runtime, it just creates a default serde and wraps it according. For this case, suppress() also falls back to default serdes; however, it does not wrap the serde and thus a ClassCastException is thrown when the serde is used later.

suppress() is already aware if the upstream aggregation is time/session windowed or not and thus should use this information to wrap default serdes accordingly.

The current workaround for windowed-suppress is to overwrite the default serde upstream to suppress(), such that suppress() inherits serdes and does not fall back to default serdes.


> suppress() for windowed-Serdes does not work with default serdes
> ----------------------------------------------------------------
>
>                 Key: KAFKA-9259
>                 URL: https://issues.apache.org/jira/browse/KAFKA-9259
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 2.1.0
>            Reporter: Matthias J. Sax
>            Assignee: Omkar Mestry
>            Priority: Major
>              Labels: newbie
>
> The suppress() operator either inherits serdes from its upstream operator or falls back to default serdes from the config.
> If the upstream operator is an windowed aggregation, the window-aggregation operator wraps the user passed-in serde with a window-serde and pushed it into suppress() – however, if default serdes are used, the window-aggregation operator cannot push anything into suppress(). At runtime, it just creates a default serde and wraps it according. For this case, suppress() also falls back to default serdes; however, it does not wrap the serde and thus a ClassCastException is thrown when the serde is used later.
> suppress() is already aware if the upstream aggregation is time/session windowed or not and thus should use this information to wrap default serdes accordingly.
> The current workaround for windowed-suppress is to overwrite the default serde upstream to suppress(), such that suppress() inherits serdes and does not fall back to default serdes.
>  
> For the search engines, when using Avro, this results in a ClassCastException:
> {{Caused by: java.lang.ClassCastException: class org.apache.kafka.streams.kstream.Windowed cannot be cast to class org.apache.avro.specific.SpecificRecord (org.apache.kafka.streams.kstream.Windowed and org.apache.avro.specific.SpecificRecord are in unnamed module of loader 'app') at io.confluent.kafka.streams.serdes.avro.SpecificAvroSerializer.serialize(SpecificAvroSerializer.java:38)}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)