Description about readSmallestOffset property in Kafka channel

[Jeong-shik Jang <js...@gmail.com>]

Hi Flume team and users,

User Guide document says:

readSmallestOffset false When set to true, the channel will read all data
in the topic, starting from the oldest event when false, it will read only
events written after the channel started When “parseAsFlumeEvent” is true,
this will be false. Flume source will start prior to the sinks and this
guarantees that events sent by source before sinks start will not be lost.

But I think "When parseAsFlumeEvent is true, this will be *true*" would be
correct.
Could anyone please confirm this?

JFYI, the followings are related source codes in KafkaChannel.java

   if (parseAsFlumeEvent || readSmallest) {

      // readSmallest is eval-ed only if parseAsFlumeEvent is false.

      // The default is largest, so we don't need to set it explicitly.

      kafkaConf.put("auto.offset.reset", "smallest");

    }

And a minor typo, a comma, with package name in
org.apache.flume.source.kafka*,*KafkaSource

thanks,

JS

Re: Description about readSmallestOffset property in Kafka channel

[Gonzalo Herreros <gh...@gmail.com>]

I think you are right

Gonzalo

On 8 April 2016 at 03:39, Jeong-shik Jang <js...@gmail.com> wrote:

> Hi Flume team and users,
>
> User Guide document says:
>
> readSmallestOffset false When set to true, the channel will read all data
> in the topic, starting from the oldest event when false, it will read only
> events written after the channel started When “parseAsFlumeEvent” is true,
> this will be false. Flume source will start prior to the sinks and this
> guarantees that events sent by source before sinks start will not be lost.
>
> But I think "When parseAsFlumeEvent is true, this will be *true*" would
> be correct.
> Could anyone please confirm this?
>
> JFYI, the followings are related source codes in KafkaChannel.java
>
>    if (parseAsFlumeEvent || readSmallest) {
>
>       // readSmallest is eval-ed only if parseAsFlumeEvent is false.
>
>       // The default is largest, so we don't need to set it explicitly.
>
>       kafkaConf.put("auto.offset.reset", "smallest");
>
>     }
>
> And a minor typo, a comma, with package name in
> org.apache.flume.source.kafka*,*KafkaSource
>
> thanks,
>
> JS
>