You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by pradeep s <sr...@gmail.com> on 2016/09/11 18:09:11 UTC
How to debug spout failures
Hi,
I have a storm topology running in storm 1.0.0 in AWS with one spout which
is reading messages from AWS SQS , then sends to a database insert bolt.
The same tuple is emitted from database bolt to a S3 write bolt. Final bolt
in the topo is a SQS message deletion action.
All these are working fine and there are no bolt failures. Only errors in
the log are some database foreign key related failures since we are doing
individual table inserts.
There is no failure indication from worker logs. But spout shows lot of
failure counts. I have attached the topology screenshot.
Any way to find out the reason for these spout failures?
[image: Inline image 1]
Regards
Pradeep S
Re: How to debug spout failures
Posted by Ravi Sharma <pi...@gmail.com>.
Your tuple may be timing out, try increasing tuple timeout and also check
ur setting of max pending tuple, you may not want unlimited tuple going
through ur pipeline.
Ravi.
On 11 Sep 2016 7:09 p.m., "pradeep s" <sr...@gmail.com> wrote:
Hi,
I have a storm topology running in storm 1.0.0 in AWS with one spout which
is reading messages from AWS SQS , then sends to a database insert bolt.
The same tuple is emitted from database bolt to a S3 write bolt. Final bolt
in the topo is a SQS message deletion action.
All these are working fine and there are no bolt failures. Only errors in
the log are some database foreign key related failures since we are doing
individual table inserts.
There is no failure indication from worker logs. But spout shows lot of
failure counts. I have attached the topology screenshot.
Any way to find out the reason for these spout failures?
[image: Inline image 1]
Regards
Pradeep S
Re: How to debug spout failures
Posted by Ravi Sharma <pi...@gmail.com>.
Your tuple may be timing out, try increasing tuple timeout and also check
ur setting of max pending tuple, you may not want unlimited tuple going
through ur pipeline.
Ravi.
On 11 Sep 2016 7:09 p.m., "pradeep s" <sr...@gmail.com> wrote:
Hi,
I have a storm topology running in storm 1.0.0 in AWS with one spout which
is reading messages from AWS SQS , then sends to a database insert bolt.
The same tuple is emitted from database bolt to a S3 write bolt. Final bolt
in the topo is a SQS message deletion action.
All these are working fine and there are no bolt failures. Only errors in
the log are some database foreign key related failures since we are doing
individual table inserts.
There is no failure indication from worker logs. But spout shows lot of
failure counts. I have attached the topology screenshot.
Any way to find out the reason for these spout failures?
[image: Inline image 1]
Regards
Pradeep S