Camus reads from multiple offsets in parallel?

[Yang <te...@gmail.com>]

we have a single partition, and the topic contains 300k events.

we fired off a camus job, it finished within 1 minute. this is rather fast.

I was guess that the multiple mappers must be reading from multiple offsets
in parallel, right?

otherwise if they are reading in serial (like in a consumer group, which a
previous thead has clarified to be not the case, i.e. camus does not use
group api), it would be very slow.


Thanks
Yang

Re: Camus reads from multiple offsets in parallel?

[Yang <te...@gmail.com>]

Thanks for that info Jun.

On Tue, Mar 3, 2015 at 3:56 PM, Jun Rao <ju...@confluent.io> wrote:

> Camus only fetches from different partitions in parallel.
>
> Thanks,
>
> Jun
>
> On Fri, Feb 27, 2015 at 4:24 PM, Yang <te...@gmail.com> wrote:
>
> > we have a single partition, and the topic contains 300k events.
> >
> > we fired off a camus job, it finished within 1 minute. this is rather
> fast.
> >
> > I was guess that the multiple mappers must be reading from multiple
> offsets
> > in parallel, right?
> >
> > otherwise if they are reading in serial (like in a consumer group, which
> a
> > previous thead has clarified to be not the case, i.e. camus does not use
> > group api), it would be very slow.
> >
> >
> > Thanks
> > Yang
> >
>

Re: Camus reads from multiple offsets in parallel?

[Jun Rao <ju...@confluent.io>]

Camus only fetches from different partitions in parallel.

Thanks,

Jun

On Fri, Feb 27, 2015 at 4:24 PM, Yang <te...@gmail.com> wrote:

> we have a single partition, and the topic contains 300k events.
>
> we fired off a camus job, it finished within 1 minute. this is rather fast.
>
> I was guess that the multiple mappers must be reading from multiple offsets
> in parallel, right?
>
> otherwise if they are reading in serial (like in a consumer group, which a
> previous thead has clarified to be not the case, i.e. camus does not use
> group api), it would be very slow.
>
>
> Thanks
> Yang
>