You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Tomer Cohen <il...@gmail.com> on 2021/10/27 13:47:48 UTC
Kafka streams - message not materialized in intermediate topics
Hello Kafka team,
I am seeing an odd behavior when using kafka streams.
During periods of heavier volumes, there are messages coming in. However,
they do look like they are processed by the adder/subtractor and are not
materialized in the intermediate topics which causes them not to be
outputted in the final topic.
Is there any way to debug this or log out when a message is dropped in the
stream and not processed for whatever reason?
Thanks,
Tomer
Re: Kafka streams - message not materialized in intermediate topics
Posted by Bill Bejeck <bi...@confluent.io.INVALID>.
Hi Tomer,
To dump the topology you can do
`System.out.println(topology.describe().toString())`.
But if you can post just the code that would be fine as well.
I understand about the logs, one thing to do is grep out any sensitive
information, but I get it if you can't do that.
Thanks,
Bill
On Thu, Oct 28, 2021 at 1:23 PM Tomer Cohen <il...@gmail.com> wrote:
> Hi Bill,
>
> Is there an easy way to dump the topology to share?
>
> The logs contain sensitive information, is there something else that can be
> provided?
>
> Thanks,
>
> Tomer
>
> On Thu, Oct 28, 2021 at 12:23 PM Bill Bejeck <bi...@confluent.io.invalid>
> wrote:
>
> > Hi Tomer,
> >
> > Can you share your topology and any log files?
> >
> > Thanks,
> > Bill
> >
> > On Thu, Oct 28, 2021 at 12:07 PM Tomer Cohen <il...@gmail.com> wrote:
> >
> > > Hi Bill/Matthias,
> > >
> > > Thanks for the replies.
> > >
> > > The issue is I never see a result, I have a log that shows the message
> > > coming in, but the adder/subtractor is never invoked for it even though
> > it
> > > should. So no result gets published to the intermediate topic I have.
> > >
> > > Thanks,
> > >
> > > Tomer
> > >
> > > On Thu, Oct 28, 2021 at 11:57 AM Bill Bejeck <bill@confluent.io.invalid
> >
> > > wrote:
> > >
> > > > Tomer,
> > > >
> > > > As Matthias pointed out for a single, final result you need to use
> the
> > > > `suppress()` operator.
> > > >
> > > > But back to your original question,
> > > >
> > > > they are processed by the adder/subtractor and are not
> > > > > materialized in the intermediate topics which causes them not to be
> > > > > outputted in the final topic
> > > > >
> > > >
> > > > Is the issue you never see a result or were you curious about the
> > > > intermediate calculations?
> > > > HTH,
> > > > Bill
> > > >
> > > > On Thu, Oct 28, 2021 at 1:05 AM Matthias J. Sax <mj...@apache.org>
> > > wrote:
> > > >
> > > > > For this case, you can call `aggregate(...).suppress()`.
> > > > >
> > > > > -Matthias
> > > > >
> > > > > On 10/27/21 12:42 PM, Tomer Cohen wrote:
> > > > > > Hi Bill,
> > > > > >
> > > > > > Thanks for the prompt reply.
> > > > > >
> > > > > > Setting to 0 forces a no collection window, so if I get 10
> messages
> > > to
> > > > > > aggregate for example, it will send 10 updates. But I only want
> to
> > > > > publish
> > > > > > the final state only.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Tomer
> > > > > >
> > > > > > On Wed, Oct 27, 2021 at 2:10 PM Bill Bejeck
> > > <bill@confluent.io.invalid
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Tomer,
> > > > > >>
> > > > > >> From the description you've provided, it sounds to me like you
> > > have a
> > > > > >> stateful operation.
> > > > > >>
> > > > > >> The thing to keep in mind with stateful operations in Kafka
> > Streams
> > > is
> > > > > that
> > > > > >> every result is not written to the changelog and forwarded
> > > downstream.
> > > > > >> Kafka Streams uses a cache for stateful operations and it's only
> > on
> > > > > cache
> > > > > >> flush either when it's full or when Kafka Streams commits (every
> > 30
> > > > > seconds
> > > > > >> by default) that Kafka Streams writes the results of the
> stateful
> > > > > >> operations to the changelog and forwards the records downstream
> to
> > > > other
> > > > > >> processors.
> > > > > >>
> > > > > >> If you want every Kafka Streams to forward every record you'll
> > need
> > > to
> > > > > set
> > > > > >> the `StreamConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG` to 0.
> > > > > >>
> > > > > >> If I haven't understood your experience accurately can you
> > provide a
> > > > few
> > > > > >> more details?
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Bill
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Oct 27, 2021 at 9:48 AM Tomer Cohen <il...@gmail.com>
> > > > wrote:
> > > > > >>
> > > > > >>> Hello Kafka team,
> > > > > >>>
> > > > > >>> I am seeing an odd behavior when using kafka streams.
> > > > > >>>
> > > > > >>> During periods of heavier volumes, there are messages coming
> in.
> > > > > However,
> > > > > >>> they do look like they are processed by the adder/subtractor
> and
> > > are
> > > > > not
> > > > > >>> materialized in the intermediate topics which causes them not
> to
> > be
> > > > > >>> outputted in the final topic.
> > > > > >>>
> > > > > >>> Is there any way to debug this or log out when a message is
> > dropped
> > > > in
> > > > > >> the
> > > > > >>> stream and not processed for whatever reason?
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>>
> > > > > >>> Tomer
> > > > > >>>
> > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Kafka streams - message not materialized in intermediate topics
Posted by Tomer Cohen <il...@gmail.com>.
Hi Bill,
Is there an easy way to dump the topology to share?
The logs contain sensitive information, is there something else that can be
provided?
Thanks,
Tomer
On Thu, Oct 28, 2021 at 12:23 PM Bill Bejeck <bi...@confluent.io.invalid>
wrote:
> Hi Tomer,
>
> Can you share your topology and any log files?
>
> Thanks,
> Bill
>
> On Thu, Oct 28, 2021 at 12:07 PM Tomer Cohen <il...@gmail.com> wrote:
>
> > Hi Bill/Matthias,
> >
> > Thanks for the replies.
> >
> > The issue is I never see a result, I have a log that shows the message
> > coming in, but the adder/subtractor is never invoked for it even though
> it
> > should. So no result gets published to the intermediate topic I have.
> >
> > Thanks,
> >
> > Tomer
> >
> > On Thu, Oct 28, 2021 at 11:57 AM Bill Bejeck <bi...@confluent.io.invalid>
> > wrote:
> >
> > > Tomer,
> > >
> > > As Matthias pointed out for a single, final result you need to use the
> > > `suppress()` operator.
> > >
> > > But back to your original question,
> > >
> > > they are processed by the adder/subtractor and are not
> > > > materialized in the intermediate topics which causes them not to be
> > > > outputted in the final topic
> > > >
> > >
> > > Is the issue you never see a result or were you curious about the
> > > intermediate calculations?
> > > HTH,
> > > Bill
> > >
> > > On Thu, Oct 28, 2021 at 1:05 AM Matthias J. Sax <mj...@apache.org>
> > wrote:
> > >
> > > > For this case, you can call `aggregate(...).suppress()`.
> > > >
> > > > -Matthias
> > > >
> > > > On 10/27/21 12:42 PM, Tomer Cohen wrote:
> > > > > Hi Bill,
> > > > >
> > > > > Thanks for the prompt reply.
> > > > >
> > > > > Setting to 0 forces a no collection window, so if I get 10 messages
> > to
> > > > > aggregate for example, it will send 10 updates. But I only want to
> > > > publish
> > > > > the final state only.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Tomer
> > > > >
> > > > > On Wed, Oct 27, 2021 at 2:10 PM Bill Bejeck
> > <bill@confluent.io.invalid
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Tomer,
> > > > >>
> > > > >> From the description you've provided, it sounds to me like you
> > have a
> > > > >> stateful operation.
> > > > >>
> > > > >> The thing to keep in mind with stateful operations in Kafka
> Streams
> > is
> > > > that
> > > > >> every result is not written to the changelog and forwarded
> > downstream.
> > > > >> Kafka Streams uses a cache for stateful operations and it's only
> on
> > > > cache
> > > > >> flush either when it's full or when Kafka Streams commits (every
> 30
> > > > seconds
> > > > >> by default) that Kafka Streams writes the results of the stateful
> > > > >> operations to the changelog and forwards the records downstream to
> > > other
> > > > >> processors.
> > > > >>
> > > > >> If you want every Kafka Streams to forward every record you'll
> need
> > to
> > > > set
> > > > >> the `StreamConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG` to 0.
> > > > >>
> > > > >> If I haven't understood your experience accurately can you
> provide a
> > > few
> > > > >> more details?
> > > > >>
> > > > >> Thanks,
> > > > >> Bill
> > > > >>
> > > > >>
> > > > >> On Wed, Oct 27, 2021 at 9:48 AM Tomer Cohen <il...@gmail.com>
> > > wrote:
> > > > >>
> > > > >>> Hello Kafka team,
> > > > >>>
> > > > >>> I am seeing an odd behavior when using kafka streams.
> > > > >>>
> > > > >>> During periods of heavier volumes, there are messages coming in.
> > > > However,
> > > > >>> they do look like they are processed by the adder/subtractor and
> > are
> > > > not
> > > > >>> materialized in the intermediate topics which causes them not to
> be
> > > > >>> outputted in the final topic.
> > > > >>>
> > > > >>> Is there any way to debug this or log out when a message is
> dropped
> > > in
> > > > >> the
> > > > >>> stream and not processed for whatever reason?
> > > > >>>
> > > > >>> Thanks,
> > > > >>>
> > > > >>> Tomer
> > > > >>>
> > > > >>
> > > > >
> > > >
> > >
> >
>
Re: Kafka streams - message not materialized in intermediate topics
Posted by Bill Bejeck <bi...@confluent.io.INVALID>.
Hi Tomer,
Can you share your topology and any log files?
Thanks,
Bill
On Thu, Oct 28, 2021 at 12:07 PM Tomer Cohen <il...@gmail.com> wrote:
> Hi Bill/Matthias,
>
> Thanks for the replies.
>
> The issue is I never see a result, I have a log that shows the message
> coming in, but the adder/subtractor is never invoked for it even though it
> should. So no result gets published to the intermediate topic I have.
>
> Thanks,
>
> Tomer
>
> On Thu, Oct 28, 2021 at 11:57 AM Bill Bejeck <bi...@confluent.io.invalid>
> wrote:
>
> > Tomer,
> >
> > As Matthias pointed out for a single, final result you need to use the
> > `suppress()` operator.
> >
> > But back to your original question,
> >
> > they are processed by the adder/subtractor and are not
> > > materialized in the intermediate topics which causes them not to be
> > > outputted in the final topic
> > >
> >
> > Is the issue you never see a result or were you curious about the
> > intermediate calculations?
> > HTH,
> > Bill
> >
> > On Thu, Oct 28, 2021 at 1:05 AM Matthias J. Sax <mj...@apache.org>
> wrote:
> >
> > > For this case, you can call `aggregate(...).suppress()`.
> > >
> > > -Matthias
> > >
> > > On 10/27/21 12:42 PM, Tomer Cohen wrote:
> > > > Hi Bill,
> > > >
> > > > Thanks for the prompt reply.
> > > >
> > > > Setting to 0 forces a no collection window, so if I get 10 messages
> to
> > > > aggregate for example, it will send 10 updates. But I only want to
> > > publish
> > > > the final state only.
> > > >
> > > > Thanks,
> > > >
> > > > Tomer
> > > >
> > > > On Wed, Oct 27, 2021 at 2:10 PM Bill Bejeck
> <bill@confluent.io.invalid
> > >
> > > > wrote:
> > > >
> > > >> Hi Tomer,
> > > >>
> > > >> From the description you've provided, it sounds to me like you
> have a
> > > >> stateful operation.
> > > >>
> > > >> The thing to keep in mind with stateful operations in Kafka Streams
> is
> > > that
> > > >> every result is not written to the changelog and forwarded
> downstream.
> > > >> Kafka Streams uses a cache for stateful operations and it's only on
> > > cache
> > > >> flush either when it's full or when Kafka Streams commits (every 30
> > > seconds
> > > >> by default) that Kafka Streams writes the results of the stateful
> > > >> operations to the changelog and forwards the records downstream to
> > other
> > > >> processors.
> > > >>
> > > >> If you want every Kafka Streams to forward every record you'll need
> to
> > > set
> > > >> the `StreamConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG` to 0.
> > > >>
> > > >> If I haven't understood your experience accurately can you provide a
> > few
> > > >> more details?
> > > >>
> > > >> Thanks,
> > > >> Bill
> > > >>
> > > >>
> > > >> On Wed, Oct 27, 2021 at 9:48 AM Tomer Cohen <il...@gmail.com>
> > wrote:
> > > >>
> > > >>> Hello Kafka team,
> > > >>>
> > > >>> I am seeing an odd behavior when using kafka streams.
> > > >>>
> > > >>> During periods of heavier volumes, there are messages coming in.
> > > However,
> > > >>> they do look like they are processed by the adder/subtractor and
> are
> > > not
> > > >>> materialized in the intermediate topics which causes them not to be
> > > >>> outputted in the final topic.
> > > >>>
> > > >>> Is there any way to debug this or log out when a message is dropped
> > in
> > > >> the
> > > >>> stream and not processed for whatever reason?
> > > >>>
> > > >>> Thanks,
> > > >>>
> > > >>> Tomer
> > > >>>
> > > >>
> > > >
> > >
> >
>
Re: Kafka streams - message not materialized in intermediate topics
Posted by Tomer Cohen <il...@gmail.com>.
Hi Bill/Matthias,
Thanks for the replies.
The issue is I never see a result, I have a log that shows the message
coming in, but the adder/subtractor is never invoked for it even though it
should. So no result gets published to the intermediate topic I have.
Thanks,
Tomer
On Thu, Oct 28, 2021 at 11:57 AM Bill Bejeck <bi...@confluent.io.invalid>
wrote:
> Tomer,
>
> As Matthias pointed out for a single, final result you need to use the
> `suppress()` operator.
>
> But back to your original question,
>
> they are processed by the adder/subtractor and are not
> > materialized in the intermediate topics which causes them not to be
> > outputted in the final topic
> >
>
> Is the issue you never see a result or were you curious about the
> intermediate calculations?
> HTH,
> Bill
>
> On Thu, Oct 28, 2021 at 1:05 AM Matthias J. Sax <mj...@apache.org> wrote:
>
> > For this case, you can call `aggregate(...).suppress()`.
> >
> > -Matthias
> >
> > On 10/27/21 12:42 PM, Tomer Cohen wrote:
> > > Hi Bill,
> > >
> > > Thanks for the prompt reply.
> > >
> > > Setting to 0 forces a no collection window, so if I get 10 messages to
> > > aggregate for example, it will send 10 updates. But I only want to
> > publish
> > > the final state only.
> > >
> > > Thanks,
> > >
> > > Tomer
> > >
> > > On Wed, Oct 27, 2021 at 2:10 PM Bill Bejeck <bill@confluent.io.invalid
> >
> > > wrote:
> > >
> > >> Hi Tomer,
> > >>
> > >> From the description you've provided, it sounds to me like you have a
> > >> stateful operation.
> > >>
> > >> The thing to keep in mind with stateful operations in Kafka Streams is
> > that
> > >> every result is not written to the changelog and forwarded downstream.
> > >> Kafka Streams uses a cache for stateful operations and it's only on
> > cache
> > >> flush either when it's full or when Kafka Streams commits (every 30
> > seconds
> > >> by default) that Kafka Streams writes the results of the stateful
> > >> operations to the changelog and forwards the records downstream to
> other
> > >> processors.
> > >>
> > >> If you want every Kafka Streams to forward every record you'll need to
> > set
> > >> the `StreamConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG` to 0.
> > >>
> > >> If I haven't understood your experience accurately can you provide a
> few
> > >> more details?
> > >>
> > >> Thanks,
> > >> Bill
> > >>
> > >>
> > >> On Wed, Oct 27, 2021 at 9:48 AM Tomer Cohen <il...@gmail.com>
> wrote:
> > >>
> > >>> Hello Kafka team,
> > >>>
> > >>> I am seeing an odd behavior when using kafka streams.
> > >>>
> > >>> During periods of heavier volumes, there are messages coming in.
> > However,
> > >>> they do look like they are processed by the adder/subtractor and are
> > not
> > >>> materialized in the intermediate topics which causes them not to be
> > >>> outputted in the final topic.
> > >>>
> > >>> Is there any way to debug this or log out when a message is dropped
> in
> > >> the
> > >>> stream and not processed for whatever reason?
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Tomer
> > >>>
> > >>
> > >
> >
>
Re: Kafka streams - message not materialized in intermediate topics
Posted by Bill Bejeck <bi...@confluent.io.INVALID>.
Tomer,
As Matthias pointed out for a single, final result you need to use the
`suppress()` operator.
But back to your original question,
they are processed by the adder/subtractor and are not
> materialized in the intermediate topics which causes them not to be
> outputted in the final topic
>
Is the issue you never see a result or were you curious about the
intermediate calculations?
HTH,
Bill
On Thu, Oct 28, 2021 at 1:05 AM Matthias J. Sax <mj...@apache.org> wrote:
> For this case, you can call `aggregate(...).suppress()`.
>
> -Matthias
>
> On 10/27/21 12:42 PM, Tomer Cohen wrote:
> > Hi Bill,
> >
> > Thanks for the prompt reply.
> >
> > Setting to 0 forces a no collection window, so if I get 10 messages to
> > aggregate for example, it will send 10 updates. But I only want to
> publish
> > the final state only.
> >
> > Thanks,
> >
> > Tomer
> >
> > On Wed, Oct 27, 2021 at 2:10 PM Bill Bejeck <bi...@confluent.io.invalid>
> > wrote:
> >
> >> Hi Tomer,
> >>
> >> From the description you've provided, it sounds to me like you have a
> >> stateful operation.
> >>
> >> The thing to keep in mind with stateful operations in Kafka Streams is
> that
> >> every result is not written to the changelog and forwarded downstream.
> >> Kafka Streams uses a cache for stateful operations and it's only on
> cache
> >> flush either when it's full or when Kafka Streams commits (every 30
> seconds
> >> by default) that Kafka Streams writes the results of the stateful
> >> operations to the changelog and forwards the records downstream to other
> >> processors.
> >>
> >> If you want every Kafka Streams to forward every record you'll need to
> set
> >> the `StreamConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG` to 0.
> >>
> >> If I haven't understood your experience accurately can you provide a few
> >> more details?
> >>
> >> Thanks,
> >> Bill
> >>
> >>
> >> On Wed, Oct 27, 2021 at 9:48 AM Tomer Cohen <il...@gmail.com> wrote:
> >>
> >>> Hello Kafka team,
> >>>
> >>> I am seeing an odd behavior when using kafka streams.
> >>>
> >>> During periods of heavier volumes, there are messages coming in.
> However,
> >>> they do look like they are processed by the adder/subtractor and are
> not
> >>> materialized in the intermediate topics which causes them not to be
> >>> outputted in the final topic.
> >>>
> >>> Is there any way to debug this or log out when a message is dropped in
> >> the
> >>> stream and not processed for whatever reason?
> >>>
> >>> Thanks,
> >>>
> >>> Tomer
> >>>
> >>
> >
>
Re: Kafka streams - message not materialized in intermediate topics
Posted by "Matthias J. Sax" <mj...@apache.org>.
For this case, you can call `aggregate(...).suppress()`.
-Matthias
On 10/27/21 12:42 PM, Tomer Cohen wrote:
> Hi Bill,
>
> Thanks for the prompt reply.
>
> Setting to 0 forces a no collection window, so if I get 10 messages to
> aggregate for example, it will send 10 updates. But I only want to publish
> the final state only.
>
> Thanks,
>
> Tomer
>
> On Wed, Oct 27, 2021 at 2:10 PM Bill Bejeck <bi...@confluent.io.invalid>
> wrote:
>
>> Hi Tomer,
>>
>> From the description you've provided, it sounds to me like you have a
>> stateful operation.
>>
>> The thing to keep in mind with stateful operations in Kafka Streams is that
>> every result is not written to the changelog and forwarded downstream.
>> Kafka Streams uses a cache for stateful operations and it's only on cache
>> flush either when it's full or when Kafka Streams commits (every 30 seconds
>> by default) that Kafka Streams writes the results of the stateful
>> operations to the changelog and forwards the records downstream to other
>> processors.
>>
>> If you want every Kafka Streams to forward every record you'll need to set
>> the `StreamConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG` to 0.
>>
>> If I haven't understood your experience accurately can you provide a few
>> more details?
>>
>> Thanks,
>> Bill
>>
>>
>> On Wed, Oct 27, 2021 at 9:48 AM Tomer Cohen <il...@gmail.com> wrote:
>>
>>> Hello Kafka team,
>>>
>>> I am seeing an odd behavior when using kafka streams.
>>>
>>> During periods of heavier volumes, there are messages coming in. However,
>>> they do look like they are processed by the adder/subtractor and are not
>>> materialized in the intermediate topics which causes them not to be
>>> outputted in the final topic.
>>>
>>> Is there any way to debug this or log out when a message is dropped in
>> the
>>> stream and not processed for whatever reason?
>>>
>>> Thanks,
>>>
>>> Tomer
>>>
>>
>
Re: Kafka streams - message not materialized in intermediate topics
Posted by Tomer Cohen <il...@gmail.com>.
Hi Bill,
Thanks for the prompt reply.
Setting to 0 forces a no collection window, so if I get 10 messages to
aggregate for example, it will send 10 updates. But I only want to publish
the final state only.
Thanks,
Tomer
On Wed, Oct 27, 2021 at 2:10 PM Bill Bejeck <bi...@confluent.io.invalid>
wrote:
> Hi Tomer,
>
> From the description you've provided, it sounds to me like you have a
> stateful operation.
>
> The thing to keep in mind with stateful operations in Kafka Streams is that
> every result is not written to the changelog and forwarded downstream.
> Kafka Streams uses a cache for stateful operations and it's only on cache
> flush either when it's full or when Kafka Streams commits (every 30 seconds
> by default) that Kafka Streams writes the results of the stateful
> operations to the changelog and forwards the records downstream to other
> processors.
>
> If you want every Kafka Streams to forward every record you'll need to set
> the `StreamConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG` to 0.
>
> If I haven't understood your experience accurately can you provide a few
> more details?
>
> Thanks,
> Bill
>
>
> On Wed, Oct 27, 2021 at 9:48 AM Tomer Cohen <il...@gmail.com> wrote:
>
> > Hello Kafka team,
> >
> > I am seeing an odd behavior when using kafka streams.
> >
> > During periods of heavier volumes, there are messages coming in. However,
> > they do look like they are processed by the adder/subtractor and are not
> > materialized in the intermediate topics which causes them not to be
> > outputted in the final topic.
> >
> > Is there any way to debug this or log out when a message is dropped in
> the
> > stream and not processed for whatever reason?
> >
> > Thanks,
> >
> > Tomer
> >
>
Re: Kafka streams - message not materialized in intermediate topics
Posted by Bill Bejeck <bi...@confluent.io.INVALID>.
Hi Tomer,
From the description you've provided, it sounds to me like you have a
stateful operation.
The thing to keep in mind with stateful operations in Kafka Streams is that
every result is not written to the changelog and forwarded downstream.
Kafka Streams uses a cache for stateful operations and it's only on cache
flush either when it's full or when Kafka Streams commits (every 30 seconds
by default) that Kafka Streams writes the results of the stateful
operations to the changelog and forwards the records downstream to other
processors.
If you want every Kafka Streams to forward every record you'll need to set
the `StreamConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG` to 0.
If I haven't understood your experience accurately can you provide a few
more details?
Thanks,
Bill
On Wed, Oct 27, 2021 at 9:48 AM Tomer Cohen <il...@gmail.com> wrote:
> Hello Kafka team,
>
> I am seeing an odd behavior when using kafka streams.
>
> During periods of heavier volumes, there are messages coming in. However,
> they do look like they are processed by the adder/subtractor and are not
> materialized in the intermediate topics which causes them not to be
> outputted in the final topic.
>
> Is there any way to debug this or log out when a message is dropped in the
> stream and not processed for whatever reason?
>
> Thanks,
>
> Tomer
>