[Discussion] Changing Amaterasu deployment strategy

[Nadav Har Tzvi <na...@gmail.com>]

Hey everyone,

While working on the new CLI (rather, fixing the damage caused by the
rebase ontop of RC3), I came across some issues and questions that, at
least in my opinion, should be addressed when we deploy Amaterasu.


   1. Configuration -
      1. *Status at 0.2.0-rc3:*
      amaterasu.properties file is bundled in the deployment package (the
      .tar file) and resides in the same directory as the entry point
      (ama-start-mesos.sh, ama-start-yarn.sh)
      2. *Suggestion for 0.2.1*:
         1. The new CLI provided the "ama setup" command that generates the
         configuration file based on a set of questions (TODO - change
to try and
         detect the cluster type and vendor - e.g. Hortonworks HDP, AWS EMR,
         Standalone Mesos, DC/OS, etc)
         2. Change amaterasu.properties to amaterasu.conf and have it
         located at /etc/amaterasu by default, to conform to how
things are usually
         done with many other Apache (and non-Apache) projects.
      2. Amaterasu assets -
      1. *Status at 0.2.0-rc3:*
      When ama-start-mesos.sh or ama-start-yarn.sh is invoked for the first
      time, the relevant dependencies are downloaded (mesos - Spark,
Miniconda |
      yarn - Miniconda) into the {AMATERASU_HOME}/dist directory, by
convention,
      AMATERASU_HOME=/ama
      2. *Suggestion for 0.2.1*:
      1. Again, the new CLI "ama setup" command also takes care of
         downloading the relevant dependencies.
         2. We need a default path for {AMATERASU_HOME}, {AMATERASU_HOME}
         will hold the relevant Amaterasu JARs and any files that are
required for
         Amaterasu to work properly (spark, Miniconda, etc). What
should this path
         be?
         My suggestion based on other projects:
         /usr/share - Apache Marathon
         /usr/lib - Apache Zookeeper

         I tend to go with /usr/share, simply because /usr/lib is intended
         for shared objects. So for example we will have
/usr/share/amaterasu, any
         thoughts?
         3. Distribution methods -
      1. *Status at 0.2.0-rc3:*
      Users who wish to use Amaterasu need to get a the distributable TAR
      and manually install and configure Amaterasu.
      2.
*Suggestion for 0.2.1: *
         1. The new CLI takes care of some of the things during "ama
         setup", but not everything. (see my fork for details:
         https://github.com/nadav-har-tzvi/incubator-amaterasu/tree/feature/ama-cli
         )
         2. Create a RPM package, so users will be able to add a repository
         and simply "yum install amaterasu".


If you have any thoughts you'd like to share in the matter, please do.

Cheers,
Nadav