Spark logs in standalone clusters

[Thomas Gerber <th...@radius.com>]

Hello,

I was wondering where all the logs files were located on a standalone
cluster:

   1. the executor logs are in the work directory on each slave machine
   (stdout/stderr)
      - I've notice that GC information is in stdout, and stage information
      in stderr
      - *Could we get more information on what is written in stdout vs
      stderr?*
   2. the master log
      - The path to the log file is shown went you launch the master,
      like /mnt/var/log/apps/spark-hadoop-org.apache.spark.deploy.master.Master-MACHINENAME.out;
      - *Could we get more information on where this path is configured?*
   3. driver logs
      - It seems they are only in the console by default (although you can
      override that in the log4j.properties file.
   4. communication manager logs?
   - *Are there any logs for the communication manager (aka the
      MapOutputTracker?)?*
   5. Any other log file?

Thanks,
Thomas