[GitHub] [bigtop] sekikn commented on pull request #676: BIGTOP-3406. Fix Oozie smoke test to work.

[GitBox <gi...@apache.org>]

sekikn commented on pull request #676:
URL: https://github.com/apache/bigtop/pull/676#issuecomment-696799372


   With this PR, I confirmed Oozie smoke test succeeded on Ubuntu 18.04 as follows.
   
   Build and deploy Oozie and its dependencies:
   ```
   $ ./gradlew oozie-clean oozie-pkg repo -Dbuildwithdeps=true
   
   (snip)
   
   BUILD SUCCESSFUL in 33m 29s
   10 actionable tasks: 10 executed
   $ cat /etc/puppet/hieradata/site.yaml 
   ---
   bigtop::hadoop_head_node: ubuntu1804.localdomain
   
   hadoop::hadoop_storage_dirs:
     - /data
   
   hadoop_cluster_node::cluster_components:
     - hdfs
     - yarn
     - mapreduce
     - hive
     - oozie
   
   bigtop::bigtop_repo_uri: file:///home/vagrant/bigtop/output/apt
   bigtop::jdk_preinstalled: true
   bigtop::bigtop_repo_gpg_check: false
   $ sudo puppet apply --hiera_config=/etc/puppet/hiera.yaml --modulepath=bigtop-deploy/puppet/modules:/usr/share/puppet/modules bigtop-deploy/puppet/manifests
   
   (snip)
   
   Notice: Applied catalog in 320.00 seconds
   ```
   
   Upload Oozie sharelib manually and run Oozie smoke test as root:
   ```
   # . /usr/lib/bigtop-utils/bigtop-detect-javahome
   # export HADOOP_HOME=/usr/lib/hadoop
   # export HADOOP_CONF_DIR=/etc/hadoop/conf
   # export HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce
   # export HIVE_HOME=/usr/lib/hive
   # export HIVE_CONF_DIR=/etc/hive/conf
   # export OOZIE_URL=http://$(hostname -f):11000/oozie/
   # export OOZIE_TAR_HOME=/usr/share/doc/oozie
   # sudo -u hdfs hdfs dfs -chown -R oozie:oozie /user/oozie
   # oozie-setup sharelib create -fs hdfs://$(hostname -f):8020/
   
   (snip)
   
   the destination path for sharelib is: /user/oozie/share/lib/lib_20200922080033
   
   # oozie admin -sharelibupdate
   
   (snip)
   
   [ShareLib update status]
   	sharelibDirOld = /user/oozie/share/lib
   	host = http://ubuntu1804.localdomain:11000/oozie
   	sharelibDirNew = hdfs://ubuntu1804.localdomain:8020/user/oozie/share/lib/lib_20200922080033
   	status = Successful
   
   
   # ./gradlew bigtop-tests:smoke-tests:oozie:test -Psmoke.tests --info
   
   (snip)
   
   > Task :bigtop-tests:smoke-tests:oozie:test
   Finished generating test XML results (0.003 secs) into: /home/vagrant/bigtop/bigtop-tests/smoke-tests/oozie/build/test-results/test
   Generating HTML test report...
   Finished generating test html results (0.015 secs) into: /home/vagrant/bigtop/bigtop-tests/smoke-tests/oozie/build/reports/tests/test
   Now testing...
   :bigtop-tests:smoke-tests:oozie:test (Thread[Daemon worker Thread 3,5,main]) completed. Took 6 mins 58.169 secs.
   
   BUILD SUCCESSFUL in 7m 7s
   6 actionable tasks: 6 executed
   Stopped 1 worker daemon(s).
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org