Dockerfile for testing HA clusters of yarn?

[Randy Chapman <wi...@gmail.com>]

Folks,

Are there any dockerfiles or the like for setting up a local HA cluster with yarn using Docker (or something other than multiple VMs), using a custom build of Hadoop (ideally, straight out of the build tree) ?

Context:

I’m working on changes for the ZKRMStateStore to alleviate a number of potential issues similar to YARN-5006, YARN-6825 and other cases where a change in application state causes a failure of the RM due to a too-large message to ZK.  (In our particular instance, it was a Spark AM failing with a very large exception text.).  I can repro similar problems in test cases, and validate their fixes – but I’d like an easy way to set up an actual running cluster on my laptop as well.

Thanks,
--randy

Sent from Mail for Windows 10