offset.storage.filename configuration in kafka-connect-hdfs

[FEI Aggie <Ag...@alcatel-lucent.com>]

Hi,
I'm running kafka-connect-hdfs 3.1.1.
offset.storage.filename is a required configuration for standalone connector. But when I set this parameter in worker configuration file for kafka-connect-hdfs with standlone mode:
offset.storage.file.filename=/mnt/data/connect.offsets

It never works. The configured file is not generated.
Anyone know this issue?

Thanks!
Aggie

Re: offset.storage.filename configuration in kafka-connect-hdfs

[Ewen Cheslack-Postava <ew...@confluent.io>]

The offset file is marked as required so we can fail early & fast, but if
you only run sink connectors then offsets will be stored in Kafka's normal
offsets topic and the file will never need to be created. (And the HDFS
connector is even more unusual in that it doesn't even rely on Kafka
offsets because it stores offset information in HDFS to provide exactly
once delivery guarantees.)

-Ewen

On Tue, Mar 7, 2017 at 11:15 PM, FEI Aggie <Ag...@alcatel-lucent.com>
wrote:

> Hi,
> I'm running kafka-connect-hdfs 3.1.1.
> offset.storage.filename is a required configuration for standalone
> connector. But when I set this parameter in worker configuration file for
> kafka-connect-hdfs with standlone mode:
> offset.storage.file.filename=/mnt/data/connect.offsets
>
> It never works. The configured file is not generated.
> Anyone know this issue?
>
> Thanks!
> Aggie
>