Kafka basic doubts

[Pariksheet Barapatre <pa...@gmail.com>]

Hi All,

Greetings..!!! This is my first email to Kafka Community.

I have just started exploring Kafka on CDH5.5 cluster which ships with
Kafka 0.8.2.1.

I am able to run sample programs for producer as well as consumer (both
high level and low level).

Now I am trying to load messages from Kafka to HDFS in batch i.e. every
hour.

Managing an offsets at partition level, I guess will do a trick but I am
confused about offset itself. Is it a line number or byte offset.

I tried using Kangaroo project but no luck. It assumes offset as number of
bytes whereas I am getting line number as offset.

Also, Kafka Connect service is introduced in Kafka 0.9, does anybody tried
loading data from Kafka to HDFS using it.

Many Thanks in Advance.

Regards
Pari