Customers getting duplicate emails

[ASHOK MACHERLA <iA...@outlook.com>]

Dear Team

In our project, for SMS/Email purpose we are using Kafka cluster and Real-time Notification which is our custom application.

We are sending to messages from Kafka to Real-time Notification, and then SMTP Gateway servers.

Our problem is ,sometimes customers are getting same email for multiple times.
During this time consumer is goes to Rebalancing mode.

How to overcome this,
Right now, we have 10 partitions for Kafka topic and 10 consumers.

Can you please suggest us to fix this one.

If you required any information/logs, I’ll share to you .

Please help us, Thanks


Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

Re: Customers getting duplicate emails

[ASHOK MACHERLA <ia...@outlook.com>]

Dear John

To overcome this issue what parameters I have to add in consumer side??

Please tell me if settings required I'll put it in that ..


Thanks..

Sent from Outlook

Re: Customers getting duplicate emails

[John Roesler <jo...@confluent.io>]

Hi Ashok,

In general, what Ryanne said is correct. For example, if you send an
email, and the send times out (but the email actually did get sent),
your app cannot distinguish this from a failure in which the send
times out before the email makes it out. Therefore, your only option
would be to retry, which may result in duplicate messages. This
obviously has nothing to do with Kafka, it's just a fact of life with
distributed systems.

However, there's some possibility that you can use Kafka's guarantees
to reduce your exposure to duplicate emails... But it's hard to say
without more information about the structure of your application. Feel
free to provide more info, and we'll see if we can give you some
advice.

Thanks,
-John

On Fri, May 10, 2019 at 10:11 AM Ryanne Dolan <ry...@gmail.com> wrote:
>
> Kafka only supports exactly-once and idempotency within the context of
> streams apps where records are consumed and produced within the same
> cluster. As soon as you touch the outside world in a non-idempotent way,
> e.g. by sending an email, these guarantees fall away. It is essentially
> impossible to guarantee that an intended action occurs no more than once
> while also guaranteeing that no intended action is skipped. This is a
> problem with distributed systems in general.
>
> You might use something like a cache to prevent most spurious emails -- if
> the cache already has record of the email being sent, don't resend it --
> but this will not prevent all duplicates.
>
> Ryanne
>
> On Fri, May 10, 2019 at 7:26 AM ASHOK MACHERLA <iA...@outlook.com> wrote:
>
> > *Dear Team*
> >
> >
> >
> > In our project, for SMS/Email purpose we are using Kafka cluster and
> > Real-time Notification which is our custom application.
> >
> >
> >
> > We are sending to messages from *Kafka to Real-time Notification, and
> > then SMTP Gateway* servers.
> >
> >
> >
> > Our problem is ,sometimes customers are getting same email for multiple
> > times.
> >
> > During this time consumer is goes to Rebalancing mode.
> >
> >
> >
> > How to overcome this,
> >
> > Right now, we have 10 partitions for Kafka topic and 10 consumers.
> >
> >
> >
> > Can you please suggest us to fix this one.
> >
> >
> >
> > If you required any information/logs, I’ll share to you .
> >
> >
> >
> > Please help us, Thanks
> >
> >
> >
> >
> >
> > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> > Windows 10
> >
> >
> >

Re: Customers getting duplicate emails

[Ryanne Dolan <ry...@gmail.com>]

Kafka only supports exactly-once and idempotency within the context of
streams apps where records are consumed and produced within the same
cluster. As soon as you touch the outside world in a non-idempotent way,
e.g. by sending an email, these guarantees fall away. It is essentially
impossible to guarantee that an intended action occurs no more than once
while also guaranteeing that no intended action is skipped. This is a
problem with distributed systems in general.

You might use something like a cache to prevent most spurious emails -- if
the cache already has record of the email being sent, don't resend it --
but this will not prevent all duplicates.

Ryanne

On Fri, May 10, 2019 at 7:26 AM ASHOK MACHERLA <iA...@outlook.com> wrote:

> *Dear Team*
>
>
>
> In our project, for SMS/Email purpose we are using Kafka cluster and
> Real-time Notification which is our custom application.
>
>
>
> We are sending to messages from *Kafka to Real-time Notification, and
> then SMTP Gateway* servers.
>
>
>
> Our problem is ,sometimes customers are getting same email for multiple
> times.
>
> During this time consumer is goes to Rebalancing mode.
>
>
>
> How to overcome this,
>
> Right now, we have 10 partitions for Kafka topic and 10 consumers.
>
>
>
> Can you please suggest us to fix this one.
>
>
>
> If you required any information/logs, I’ll share to you .
>
>
>
> Please help us, Thanks
>
>
>
>
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
>

Re: Customers getting duplicate emails

[Wade Chandler <wa...@apache.org>]

> On May 10, 2019, at 8:26 AM, ASHOK MACHERLA <iA...@outlook.com> wrote:
> 
> Dear Team
> 
> In our project, for SMS/Email purpose we are using Kafka cluster and Real-time Notification which is our custom application.
> 
> We are sending to messages from Kafka to Real-time Notification, and then SMTP Gateway servers.
> 
> Our problem is ,sometimes customers are getting same email for multiple times.
> During this time consumer is goes to Rebalancing mode.
> 
> How to overcome this,
> Right now, we have 10 partitions for Kafka topic and 10 consumers.
> 
> Can you please suggest us to fix this one.
> 
> If you required any information/logs, I’ll share to you .


Are you using automatic or async commits versus sync (commitSync); meaning you’d have some processed messages during this time. Now, it is possible that every now and then even considering that you could have this issue; let’s say you have some network issues and can’t commit after you have sent an email, but the other scenarios are more likely I believe. At this link see the section “Offsets and Consumer Position”.

Wade

=======================

Wade Chandler
e: consult@wadechandler.com
t: @wadechandler
https://www.linkedin.com/in/wade-chandler