what is the key used for change log topic backed by windowed store

[Sachin Mittal <sj...@gmail.com>]

Hi All,
Our stream is something like

builder.stream()
        .groupByKey()
        .aggregate(Initializer, Aggregator, TimeWindows, valueSerde,
"table-name')

So This creates a changelog topic.
I was wondering what would be the key used for this topic.

Would it be they key we use to group by or a compounded key of (our key,
window key).

I am asking this because what is observed that when stream runs for few
days we start getting the exception where message size is greater than the
allowable message size.

However our messages are windowed on hourly windows, so size of a message
should be far less than the max message size.

Thanks
Sachin

Re: what is the key used for change log topic backed by windowed store

[Damian Guy <da...@gmail.com>]

Hi Sachin,

The key is a combination of the record key + window start time + a sequence
number. The timestamp is 8 bytes and the sequence number is 4 bytes.

Thanks,
Damian

On Thu, 22 Dec 2016 at 15:26 Sachin Mittal <sj...@gmail.com> wrote:

> Hi All,
> Our stream is something like
>
> builder.stream()
>         .groupByKey()
>         .aggregate(Initializer, Aggregator, TimeWindows, valueSerde,
> "table-name')
>
> So This creates a changelog topic.
> I was wondering what would be the key used for this topic.
>
> Would it be they key we use to group by or a compounded key of (our key,
> window key).
>
> I am asking this because what is observed that when stream runs for few
> days we start getting the exception where message size is greater than the
> allowable message size.
>
> However our messages are windowed on hourly windows, so size of a message
> should be far less than the max message size.
>
> Thanks
> Sachin
>