Configuring hdfs with the MesosApplicationMasterRunner

[Guillaume Balaine <ig...@gmail.com>]

Hi,

I get that this might not be a priority given FLIP6, but I set up a mesos
configuration for clustering with Flink today, and one of the painful
points was the hadoop configuration.

It's impossible to pass the hadoop conf folder artefact (containing files
like core-site.xml etc.) to the task managers. The appmaster clearly sees
the hadoop configuration folder but then even if I setup container volumes
for the task managers, I can't mount these files.

I read the whole flink-mesos code but there was no way to pass the hdfs
file.
I see that flink-dcos-service has a FLINK_HDFS_CONFIG_URL env var but it
does nothing...
Which means I have to either put them directly in the container images
(what I did), or download them at startup (inconvenient).

Has anyone come across this kind of ops problem before ?