You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "jay vyas (JIRA)" <ji...@apache.org> on 2014/10/01 03:50:35 UTC

[jira] [Updated] (BIGTOP-1449) Add RC Testing support to vagrant recipe

     [ https://issues.apache.org/jira/browse/BIGTOP-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

jay vyas updated BIGTOP-1449:
-----------------------------
    Attachment: BIGTOP-1449.patch

Heres a patch that automates this. there might be a couple minor tweaks but its pretty much completed .

I'll put some finishing touches on it tomorrow and then we can merge it in for release testing from now on.


> Add RC Testing support to vagrant recipe
> ----------------------------------------
>
>                 Key: BIGTOP-1449
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-1449
>             Project: Bigtop
>          Issue Type: Improvement
>          Components: vm
>    Affects Versions: 0.9.0
>            Reporter: jay vyas
>            Assignee: jay vyas
>         Attachments: BIGTOP-1449.patch
>
>
> We can automatically set up vagrant to install a release candidate on n nodes and  run smoke tests.  Here is how .
> 1) Adding the bigtop folder as a shared folder in Vagrantfile, so that VMs have access to bigtop  source
> {noformat}
>    # the bigtop puppet recipes
>    bigtop_puppet = "../../puppet"
> -
> +  bigtop_home =   "../../../"
> ...
> +  bigtop.vm.synced_folder bigtop_home, "/bigtop-home"
>  {noformat}
> 2) cd /bigtop-home [ {{sudo puppet apply --modulepath=./ -e "include bigtop_toolchain::gradle && alias gradle=/usr/local/gradle/bin/gradle"}}
> 3) Having the bigtop repo in Vagrantfile point at the RC url (modify the line in provision.sh {{bigtop_yumrepo_uri,http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.8.0/label=centos6/6/artifact/output/}}
> 4) Export HADOOP_CONF_DIR, BIGTOP_HOME,HADOOP_MAPRED_HOME,HIVE_HOME,HIVE_CONF_DIR, and so on...
> 5) Ensure that a home dir exists for the user (i.e. as fs superuser (i.e. hdfs or root on a posix filesystem) run {{hadoop fs -mkdir /user/vagrant}} )
> 6) Running the tests {{cd bigtop-tests/smoke-tests/ && gradle compileGroovy test -Dsmoke-tests=mapreduce --info}} 
> Might as well do it so that for next release,  we can automate testing of bigtop on n-node clusters.
> When doing this task, 
> - lets also update {{smoke-tests}} and (1) fix the README (it sais smoke-tests, instead of smoke.tests) (2) parameterize ITest .  Those are both needed for proper automation of RC testing.
> - Also lets bound the amount of hive tests we run. Currently it runs all of them.  I think we only need it to run "basic" or a few others, and we can ensure this by setting system properties like this, so the smoke tests is fast and simple:
>  {noformat} 
> test {
>  systemProperties["org.apache.bigtop.itest.hivesmoke.TestHiveSmokeBulk.test_include"]="basic"
> }
> {noformat}
> So all in all there are basically two tasks:  Add a "test.sh" script to the vagrant provisioner which installs toolchain and runs bigtop smoke tests, and a few complimentary,  minor updates to the smoke-tests .
> *Then, next time an RC comes out, to test, we just update number of nodes and run "vagrant up".*  :)
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)