You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Michael Strobl <mi...@gameforge.com> on 2015/05/28 17:07:17 UTC
Offset (counter) as String - why?
Hey Samza team!
Please, can you explain me, why the offset counter in the IncomingMessageEnvelope is stored as a String attribute and not Long? As far as I know in Kafka the offset counter is long. Can I simply cast the String offset to long (have to work in my job with it) or does Samza modify (sometimes) this Kafka offset and therefore it is a String?
Thanks for helping,
Michael
Re: Offset (counter) as String - why?
Posted by Yi Pan <ni...@gmail.com>.
Hi, Michael,
The reason that offset in the IncomingMessageEnvelope is string type
instead of long is the following: if you integrate non-Kafka messaging
system in Samza, there is no guarantee that the offset is of type long.
E.g. ActiveMQ uses a composite format from connectionId, sessionId,
producerId, and sequenceId to construct a message key. The abstraction here
is to allow extensibility to various different format of offset. There is
already a JIRA open to discuss about making offset an ordered set:
SAMZA-569.
If you are sure that your application only works w/ Kafka systems,
offset.toLong() would work just fine.
Thanks!
-Yi
On Thu, May 28, 2015 at 8:07 AM, Michael Strobl <
michael.strobl@gameforge.com> wrote:
> Hey Samza team!
>
> Please, can you explain me, why the offset counter in the
> IncomingMessageEnvelope is stored as a String attribute and not Long? As
> far as I know in Kafka the offset counter is long. Can I simply cast the
> String offset to long (have to work in my job with it) or does Samza modify
> (sometimes) this Kafka offset and therefore it is a String?
>
> Thanks for helping,
> Michael
>