Noobie question about consumers and ordering of messages

[Dillian Murphey <cr...@gmail.com>]

So I just realized that when creating multiple partitions for a topic, it
is the responsibility of the producer and the consumer to  decide where to
put data.

So if I am a consumer, how do I know what partition to read data from?
What if the producer only put data in 1 partition out of 50.

And so there isn't any order, here, right? Only order per partition, but
there is no concept of consuming messages in the order they were produced,
correct?

Thanks. Just struggling on the practical aspects now that I've moved past
some of the administration tasks.

Re: Noobie question about consumers and ordering of messages

[Sharninder <sh...@gmail.com>]

The producer can use a key to determine which partition to put data in. For example we use a partitioning scheme based on the header of the message so that all messages with a specific header go to a specific partition. The consumer can also include the same logic to process those messages. 

> On 13-Jun-2015, at 4:01 am, Dillian Murphey <cr...@gmail.com> wrote:
> 
> So I just realized that when creating multiple partitions for a topic, it
> is the responsibility of the producer and the consumer to  decide where to
> put data.
> 
> So if I am a consumer, how do I know what partition to read data from?
> What if the producer only put data in 1 partition out of 50.
> 
> And so there isn't any order, here, right? Only order per partition, but
> there is no concept of consuming messages in the order they were produced,
> correct?
> 
> Thanks. Just struggling on the practical aspects now that I've moved past
> some of the administration tasks.