Preventing Kafka crashes on disk fill-up

[Tirtha Chatterjee <ti...@gmail.com>]

Hello everyone.

As of today, when a data disk fills up and Kafka is unable to write to it
anymore, the Kafka process crashes. In this situation, users are unable to
use Kafka APIs to recover their cluster, and have to rely on manually
cleaning up disk space.

There are multiple approaches towards addressing this issue. One of the
behaviors that users commonly want is to degrade gracefully, being unable
to produce any new data to the disk, while still retaining the ability to
call admin APIs and delete the data on disk. I am wondering what the views
of the community are on this as a default behavior. Please let me know.

I am sure this is a discussion that has happened multiple times in the past
in the community. I would love to not re-invent the wheel or jump to a
solution without context, and look to learn from the learnings from those
discussions. What are the reasons why this might be challenging
to implement? What do you think are the primary obstacles / unknowns?

-- 
Regards
Tirtha Chatterjee