Ackers and Spout restart

[Srividhya Shanmugam <Sr...@fico.com>]

I am trying to understand the Spout restart scenario in relation to the acker.

If a topology relies on storm acker's tracking, what happens when a Spout task dies due to worker crash and gets restarted by the Supervisor?

When the acker is about to notify the Spout on ack() or fail() it invokes emitDirect on the outputCollector. If a Spout task is unavailable, what happens to the notification?

Thank you.

This email and any files transmitted with it are confidential, proprietary and intended solely for the individual or entity to whom they are addressed. If you have received this email in error please delete it immediately.

RE: Ackers and Spout restart

[Pradeep Badiger <Pr...@fico.com>]

Hi,

Here is the problem that we are facing. If the spout restarts before the tuple tree completes, we don't see the ack/fail called back for the spout tuples which got emitted before the spout restarted. I created a sample topology and tested this with 4 workers, 2 ackers and with parallelism of 1. The tuples which got emitted before the spout restarted need to be acknowledged in some way.

Thanks,
Pradeep V.B.


-----Original Message-----
From: Ganesh Chandrasekaran [mailto:gchandrasekaran@wayfair.com] 
Sent: Tuesday, June 16, 2015 7:58 AM
To: user@storm.apache.org
Subject: RE: Ackers and Spout restart

The source queue will replay all the pending messages when it sees that the spout died.


https://storm.apache.org/documentation/Guaranteeing-message-processing.html


_________________________________
From: Srividhya Shanmugam [SrividhyaShanmugam@fico.com]
Sent: Monday, June 15, 2015 4:35 PM
To: user@storm.apache.org
Subject: Ackers and Spout restart

I am trying to understand the Spout restart scenario in relation to the acker.

If a topology relies on storm acker's tracking, what happens when a Spout task dies due to worker crash and gets restarted by the Supervisor?

When the acker is about to notify the Spout on ack() or fail() it invokes emitDirect on the outputCollector. If a Spout task is unavailable, what happens to the notification?

Thank you.

This email and any files transmitted with it are confidential, proprietary and intended solely for the individual or entity to whom they are addressed. If you have received this email in error please delete it immediately.

This email and any files transmitted with it are confidential, proprietary and intended solely for the individual or entity to whom they are addressed. If you have received this email in error please delete it immediately.

RE: Ackers and Spout restart

[Ganesh Chandrasekaran <gc...@wayfair.com>]

The source queue will replay all the pending messages when it sees that the spout died.


https://storm.apache.org/documentation/Guaranteeing-message-processing.html


_________________________________
From: Srividhya Shanmugam [SrividhyaShanmugam@fico.com]
Sent: Monday, June 15, 2015 4:35 PM
To: user@storm.apache.org
Subject: Ackers and Spout restart

I am trying to understand the Spout restart scenario in relation to the acker.

If a topology relies on storm acker’s tracking, what happens when a Spout task dies due to worker crash and gets restarted by the Supervisor?

When the acker is about to notify the Spout on ack() or fail() it invokes emitDirect on the outputCollector. If a Spout task is unavailable, what happens to the notification?

Thank you.

This email and any files transmitted with it are confidential, proprietary and intended solely for the individual or entity to whom they are addressed. If you have received this email in error please delete it immediately.