You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@storm.apache.org by 이승진 <sw...@navercorp.com> on 2015/06/15 11:25:53 UTC

difference between opaque and transactional kafka spout

I briefly tested how these two spouts handle failure in processing tuples.
 
I randomly generated error in updateState method, and batch seems to be replayed.
 
In transactional spout, seems like exactly same batch is replayed but in opaque spout, batch itself is increasing in it's size although it seems to contain all the items in previous batch.
 
Am I getting right?

RE: difference between opaque and transactional kafka spout

Posted by 이승진 <sw...@navercorp.com>.

And also, where do these spouts store offset information in zookeeper?
 
In TridentKafkaConfig, there's no such option which exists in SpoutConfig for traditional topology.
 
forceFromStart, startOffsetTime work same way in trident as it does in traditional topology?

   이승진  Adrian Seungjin Lee 
Distributed System Development Lab   / Researcher 

 NAVER Green Factory, 6, Buljeong-ro, Bundang-gu, Seongnam-si, Gyeonggi-do, 463-867, Korea
Tel 031-784-1274   Mobile 010-7141-7134 
Email  sweetest.sj@navercorp.com   Messenger   

 
 
-----Original Message-----
From: "이승진"&lt;sweetest.sj@navercorp.com&gt; 
To: &lt;user@storm.apache.org&gt;; 
Cc: 
Sent: 2015-06-15 (월) 18:25:53
Subject: difference between opaque and transactional kafka spout
 
I briefly tested how these two spouts handle failure in processing tuples.
 
I randomly generated error in updateState method, and batch seems to be replayed.
 
In transactional spout, seems like exactly same batch is replayed but in opaque spout, batch itself is increasing in it's size although it seems to contain all the items in previous batch.
 
Am I getting right?