Interest in automatic topology/rack awareness?

[Michael Gasch <mi...@live.com>]

Hi,

Today, the configuration of rack awareness for brokers (broker.rack=my-rack-id) is either manual or via a script during deployment of a broker.

Other systems, like Hadoop or Kubernetes, have support for auto-detection of topologies, often called well-known labels like racks, zones or regions.

Many Kafka deployments I support on the VMware vSphere hypervisor have rack awareness as a requirement to increase availability for the brokers in case of host (hypervisor) failure. For this to work, several actions (and communication) across different teams has to happen, which is error-prone.

Would there be interest in having a functionality to auto-detect the underlying topology, e.g. via a pluggable mechanism (interface) or a dedicated implementation for vSphere? A Kafka broker could make a call to "GetZones()" (or "GetRacks") and the specific implementation of the cloud provider would respond with the current rack/zone where this broker is running.

Thx for any feedback,
Michael