You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Per Jahn <PJ...@wideorbit.com> on 2015/12/05 20:05:29 UTC

how robust are file accesses?

Hi

Are files that kafka writes to the file system robust against abrupt termination?
Including fs meta data.
And assuming it's used with a journaling fs.
Or do I need ups and always shut down kafka gracefully to prevent data corruption?
I'm not afraid of losing data (have redundant servers), but I'm afraid of file corruption that prevents kafka from starting without problems.

I'm asking for worst case of course, not was usually happens.

Thanks.

/ Per

Re: how robust are file accesses?

Posted by Jay Kreps <ja...@confluent.io>.

Kafka should be robust against abrupt termination.

-Jay

On Sat, Dec 5, 2015 at 11:05 AM, Per Jahn <PJ...@wideorbit.com> wrote:
> Hi
>
> Are files that kafka writes to the file system robust against abrupt termination?
> Including fs meta data.
> And assuming it's used with a journaling fs.
> Or do I need ups and always shut down kafka gracefully to prevent data corruption?
> I'm not afraid of losing data (have redundant servers), but I'm afraid of file corruption that prevents kafka from starting without problems.
>
> I'm asking for worst case of course, not was usually happens.
>
> Thanks.
>
> / Per
>