is exactly-once supported with kafka streams application with external state store like redis

[Pushkar Deole <pd...@gmail.com>]

Hi All,

We are using streams application with redis for state store.
Redis was mainly considered instead of kafka state stores because of the
reason that global state store once updated by one application instance was
taking few milliseconds to reflect updated global state to another
application instance.
Now, we may need to enable exactly-once semantics, however wondering if it
would work with redis state store, or rollbacks would still have stale
state left in redis?

Re: is exactly-once supported with kafka streams application with external state store like redis

[Guozhang Wang <gu...@gmail.com>]

Hello Pushkar,

Unfortunately it is not supported at the moment, see
https://issues.apache.org/jira/browse/KAFKA-12475 for some details.
Currently the community is working on KAFKA-12549 to support
transactional state stores, at that time customized remote stores
would be able to provide additional API implementations in order to
support EOS.

Guozhang

On Mon, Apr 3, 2023 at 5:06 AM Pushkar Deole <pd...@gmail.com> wrote:
>
> Hi All,
>
> We are using streams application with redis for state store.
> Redis was mainly considered instead of kafka state stores because of the
> reason that global state store once updated by one application instance was
> taking few milliseconds to reflect updated global state to another
> application instance.
> Now, we may need to enable exactly-once semantics, however wondering if it
> would work with redis state store, or rollbacks would still have stale
> state left in redis?