[GitHub] incubator-metron issue #423: METRON-270: Add Zeppelin to the platform

[nickwallen <gi...@git.apache.org>]

Github user nickwallen commented on the issue:

    https://github.com/apache/incubator-metron/pull/423
  
    So far, I have not been able to get the "Zeppelin Dashboard Install" mechanism to work. 
     
    I am not aware of step-by-step instructions to use the MPack, but maybe I am missing them.  Thought it would be beneficial to document here since it took me a while to piece all this together.  Here is exactly what I did.  What did I do wrong?
    
    
    ### Build
    
    * Build Metron
    ```
    cd incubator-metron
    mvn clean package -DskipTests
    ```
    
    * Start Docker on your build machine
    
    * Build Metron RPMs
    ```
    mvn clean package -Dbuild-rpms -DskipTests
    ```
    
    * Validate Metron RPMs
    ```
    $ find ./ -name "*.rpm"
    .//metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/metron-common-0.3.0-201701261504.noarch.rpm
    .//metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/metron-data-management-0.3.0-201701261504.noarch.rpm
    .//metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/metron-elasticsearch-0.3.0-201701261504.noarch.rpm
    .//metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/metron-enrichment-0.3.0-201701261504.noarch.rpm
    .//metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/metron-indexing-0.3.0-201701261504.noarch.rpm
    .//metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/metron-parsers-0.3.0-201701261504.noarch.rpm
    .//metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/metron-pcap-0.3.0-201701261504.noarch.rpm
    .//metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/metron-profiler-0.3.0-201701261504.noarch.rpm
    .//metron-deployment/packaging/docker/rpm-docker/RPMS/noarch/metron-solr-0.3.0-201701261504.noarch.rpm
    .//metron-deployment/packaging/docker/rpm-docker/SRPMS/metron-0.3.0-201701261504.src.rpm
    ```
    
    * Build Ambari MPack
    ```
    cd metron-deployment
    mvn clean package
    ```
    
    * Validate Ambari MPack
    ```
    $ find . -name "*mpack*.tar.gz"
    ./packaging/ambari/metron-mpack/target/metron_mpack-1.0.0.0-SNAPSHOT.tar.gz
    ```
    
    ### Create Test VM
    
    * Launch Vagrant and install Ambari only
    ```
    cd metron-deployment/vagrant/quick-dev
    vagrant --ansible-tags=ambari up
    ```
    
    * Copy Artifacts to VM
    ```
    scp incubator-metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/*.rpm vagrant@node1:/tmp
    scp incubator-metron/metron-deploment/packaging/ambari/metron-mpack/target/metron_mpack-1.0.0.0-SNAPSHOT.tar.gz vagrant@node1:/tmp
    ```
    
    * Prepare RPMS
    ```
    vagrant ssh
    sudo su -
    mkdir /localrepo
    cp /tmp/metron*.rpm /localrepo
    ```
    
    ### Install Ambari MPack
    
    * Install MPack
    ```
    ambari-server install-mpack --mpack=/tmp/metron_mpack-1.0.0.0-SNAPSHOT.tar.gz --verbose
    service ambari-server restart
    ```
    
    ### Deploy Metron with MPack
    
    * You may need to clear the browser cache to see the additional options installed by the MPack.
    
    * Login to Ambari at http://node1:8080
    
    * Click Add Services
    
    * Select "Metron" and then continue the install
    
    * SUCCESS!  Validated that I was prompted to install Zeppelin and Spark as dependencies
    
    ### Install Zeppelin Notebooks
    
    * Create a Zeppelin notebook and drop it at `/usr/metron/0.3.0/zeppelin/`.
    
    * In Ambari, Click Metron > Service Actions > Zeppelin Dashboard Install.  
    
    * This is what I see from the Ambari status window.
    ```
    2017-01-26 16:45:29,926 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.5.0.0-1245
    2017-01-26 16:45:29,926 - Checking if need to create versioned conf dir /etc/hadoop/2.5.0.0-1245/0
    2017-01-26 16:45:29,927 - call[('ambari-python-wrap', '/usr/bin/conf-select', 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.0.0-1245', '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
    2017-01-26 16:45:29,947 - call returned (1, '/etc/hadoop/2.5.0.0-1245/0 exist already', '')
    2017-01-26 16:45:29,948 - checked_call[('ambari-python-wrap', '/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.0.0-1245', '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False}
    2017-01-26 16:45:29,968 - checked_call returned (0, '')
    2017-01-26 16:45:29,972 - Ensuring that hadoop has the correct symlink structure
    2017-01-26 16:45:29,972 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
    
    Command completed successfully!
    ```
    
    * FAILURE! I then go to Zeppelin at `http://node1:9995/` and search for the notebook.  The Notebook does not seem to be installed.
    
    



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---