Spark on yarn logging

[John Vines <vi...@apache.org>]

I followed the instructions for configuring a custom logger per
https://spark.apache.org/docs/2.0.2/running-on-yarn.html (because we have
long running spark jobs, sometimes occasionally get stuck and without a
rolling file appender will fill up disk). This seems to work well for us,
but it breaks the web-ui because it only has links for stderr/stdout.

I can take that url and manually change it, but I'm wondering if there's a
way to configure the spark-ui to look for files of a specific format so
that way no manual url manipulation is necessary to view the logs.

Thanks