You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by bigdata hadoop <ha...@gmail.com> on 2015/12/08 19:50:38 UTC

Offsets not matching with the count

Hi All

I have a topic with encrypted data.
I am using spring-xd streams kafka source and my processor which decrypts
the message and writes to file.
So the issue I am facing is the offsets or throughput isn't matching with
the topic offsets.
I also made a tap on the stream for counting the messages read from kafka
source,where the count shows something different from the source offset.
Here are the streams I deployed:

stream create --name modulename --definition "kafka
--zkconnect=xx.xx.xx:2181,xx.xx.xx:2181,xx.xx.xx:2181 --topic=topicname
--groupId=modulegroup --initialOffsets=0@12,1@18,2@72| decryptor|file"

stream deploy modulename --properties "module.*.count=3"

stream create --name modulenametap --definition "tap:stream:modulename  >
counter --name=modulenamecount" --deploy

counter display modulenamecount


Any suggestions would be helpfull.

Thanks

Sowjanya