You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@bigtop.apache.org by Srinivas Kolla <sr...@altiscale.com> on 2015/08/16 12:39:36 UTC
Bigtop test framework documentation
Hi Team,
Where can i get bigtop test framework documentation?
Recently, i have started using bigtop test framework to automation the
hadoop test cases.
Though i could understand it at a high level, i want to learn it in a
detail way.
I could not find valuable resources from google search on bigtop.
In this regard, Can someone please help me to point to correct
documentation of Bigtop.
Regards,
Srinivas.
Re: Bigtop test framework documentation
Posted by Jay Vyas <ja...@gmail.com>.
Hi this is great to hear. There are layers to the testing framework. Here is a summary for you. I'll post this in the docs or wiki if you folks think it's useful.
Bigtop Test Frameworks
First there is itest, a Java wrapper to the shell which allows you to test and verify output from commands easily by writing Java programs using its api and asserting outputs.
Then there are two test frameworks which you can run. Smoke-tests is a pluggable gradle based testing component. You add groovy scripts to it and then run it, there are examples under the smoke tests directory.
Finally The test-artifacts can be compiled to jar files and distributed with a Hadoop distribution you build on top of bigtop.
We don't have comprehensive docs on these, in general they are being updated by developers. However I like your idea to have an introduction to the whole framework.Why not open the code up and ask questions here in the meantime?
We want to merge test-artifacts with smoke-tests soon, but iirc haven't done so yet.
BigPetStore Integration Tests
Finally, after we realized that just isolated smoke tests were of limited use, we built the bigpetstore-mapreduce and bigpetstore-spark integration apps. Those both scale out to any arbitrary size, and use many ecosystem components (hive,pig,mahout) ,spark (mllib,sparksql).
Note that to run all of these understanding gradle and groovy is helpful in case you haven't use those tools and languages that is a great start.
> On Aug 16, 2015, at 6:39 AM, Srinivas Kolla <sr...@altiscale.com> wrote:
>
> Hi Team,
>
> Where can i get bigtop test framework documentation?
> Recently, i have started using bigtop test framework to automation the hadoop test cases.
> Though i could understand it at a high level, i want to learn it in a detail way.
> I could not find valuable resources from google search on bigtop.
> In this regard, Can someone please help me to point to correct documentation of Bigtop.
>
> Regards,
> Srinivas.
Re: Bigtop test framework documentation
Posted by jay vyas <ja...@gmail.com>.
try by removing the -o (at least, remove it the first time you run
it)......
On Tue, Aug 18, 2015 at 9:14 AM, Srinivas Kolla <sr...@altiscale.com>
wrote:
> Thanks Youngwoo and Jay for your inputs. These helped me a lot to
> understand the bigtop tests.
>
> I have cloned the bigtop code from master branch and followed the steps
> mentioned at:
> https://cwiki.apache.org/confluence/display/BIGTOP/Running+integration+and+system+tests
> .
>
>
> Now i wanted to run some/all the tests for hadoop hdfs module. I tried to
> use below to run the tests but it failed to run tests:
>
> cd bigtop-tests/test-execution/smokes/hadoop; mvn verify -o
> -Dorg.apache.maven-failsafe-plugin.testInclude=**/TestHDFS*
>
> I am getting below error:
>
> [main] ERROR org.apache.maven.cli.MavenCli - Failed to execute goal
> org.apache.maven.plugins:maven-dependency-plugin:2.8:copy
> (copy-testartifact-jar) on project hadoop-smoke-execution: Execution
> copy-testartifact-jar of goal
> org.apache.maven.plugins:maven-dependency-plugin:2.8:copy failed: Plugin
> org.apache.maven.plugins:maven-dependency-plugin:2.8 or one of its
> dependencies could not be resolved: The following artifacts could not be
> resolved: org.apache.maven.reporting:maven-reporting-impl:jar:2.0.5,
> org.apache.maven.doxia:doxia-sink-api:jar:1.0,
> org.apache.maven.doxia:doxia-site-renderer:jar:1.0,
> org.codehaus.plexus:plexus-archiver:jar:2.3,
> org.codehaus.plexus:plexus-utils:jar:3.0.9,
> org.codehaus.plexus:plexus-io:jar:2.0.6,
> org.apache.maven.shared:maven-dependency-analyzer:jar:1.4,
> org.apache.maven.shared:maven-dependency-tree:jar:2.1,
> org.apache.maven.shared:maven-invoker:jar:2.0.11: Cannot access internal (
> https://archiva.service.verticloud.com/archiva/repository/internal/) in
> offline mode and the artifact
> org.apache.maven.reporting:maven-reporting-impl:jar:2.0.5 has not been
> downloaded from it before. -> [Help 1]
>
>
>
>
> Can you please help me to find out what exactly i am missing here?
>
> Regards,
> Srinivas.
>
> On Sun, Aug 16, 2015 at 5:52 PM, 김영우 (Youngwoo Kim) <wa...@gmail.com>
> wrote:
>
>> Hi Srinivas,
>>
>> Please refer to this wiki pages:
>>
>> https://cwiki.apache.org/confluence/display/BIGTOP/Running+integration+and+system+tests
>>
>> https://cwiki.apache.org/confluence/display/BIGTOP/Writing+integration+and+system+tests
>>
>> If you have any further question, feel free to let us know.
>>
>> Thanks,
>> Youngwoo
>> 2015. 8. 16. 오후 7:39에 "Srinivas Kolla" <sr...@altiscale.com>님이 작성:
>>
>> Hi Team,
>>>
>>> Where can i get bigtop test framework documentation?
>>> Recently, i have started using bigtop test framework to automation the
>>> hadoop test cases.
>>> Though i could understand it at a high level, i want to learn it in a
>>> detail way.
>>> I could not find valuable resources from google search on bigtop.
>>> In this regard, Can someone please help me to point to correct
>>> documentation of Bigtop.
>>>
>>> Regards,
>>> Srinivas.
>>>
>>
>
--
jay vyas
Re: Bigtop test framework documentation
Posted by Srinivas Kolla <sr...@altiscale.com>.
Thanks Youngwoo and Jay for your inputs. These helped me a lot to
understand the bigtop tests.
I have cloned the bigtop code from master branch and followed the steps
mentioned at:
https://cwiki.apache.org/confluence/display/BIGTOP/Running+integration+and+system+tests
.
Now i wanted to run some/all the tests for hadoop hdfs module. I tried to
use below to run the tests but it failed to run tests:
cd bigtop-tests/test-execution/smokes/hadoop; mvn verify -o
-Dorg.apache.maven-failsafe-plugin.testInclude=**/TestHDFS*
I am getting below error:
[main] ERROR org.apache.maven.cli.MavenCli - Failed to execute goal
org.apache.maven.plugins:maven-dependency-plugin:2.8:copy
(copy-testartifact-jar) on project hadoop-smoke-execution: Execution
copy-testartifact-jar of goal
org.apache.maven.plugins:maven-dependency-plugin:2.8:copy failed: Plugin
org.apache.maven.plugins:maven-dependency-plugin:2.8 or one of its
dependencies could not be resolved: The following artifacts could not be
resolved: org.apache.maven.reporting:maven-reporting-impl:jar:2.0.5,
org.apache.maven.doxia:doxia-sink-api:jar:1.0,
org.apache.maven.doxia:doxia-site-renderer:jar:1.0,
org.codehaus.plexus:plexus-archiver:jar:2.3,
org.codehaus.plexus:plexus-utils:jar:3.0.9,
org.codehaus.plexus:plexus-io:jar:2.0.6,
org.apache.maven.shared:maven-dependency-analyzer:jar:1.4,
org.apache.maven.shared:maven-dependency-tree:jar:2.1,
org.apache.maven.shared:maven-invoker:jar:2.0.11: Cannot access internal (
https://archiva.service.verticloud.com/archiva/repository/internal/) in
offline mode and the artifact
org.apache.maven.reporting:maven-reporting-impl:jar:2.0.5 has not been
downloaded from it before. -> [Help 1]
Can you please help me to find out what exactly i am missing here?
Regards,
Srinivas.
On Sun, Aug 16, 2015 at 5:52 PM, 김영우 (Youngwoo Kim) <wa...@gmail.com>
wrote:
> Hi Srinivas,
>
> Please refer to this wiki pages:
>
> https://cwiki.apache.org/confluence/display/BIGTOP/Running+integration+and+system+tests
>
> https://cwiki.apache.org/confluence/display/BIGTOP/Writing+integration+and+system+tests
>
> If you have any further question, feel free to let us know.
>
> Thanks,
> Youngwoo
> 2015. 8. 16. 오후 7:39에 "Srinivas Kolla" <sr...@altiscale.com>님이 작성:
>
> Hi Team,
>>
>> Where can i get bigtop test framework documentation?
>> Recently, i have started using bigtop test framework to automation the
>> hadoop test cases.
>> Though i could understand it at a high level, i want to learn it in a
>> detail way.
>> I could not find valuable resources from google search on bigtop.
>> In this regard, Can someone please help me to point to correct
>> documentation of Bigtop.
>>
>> Regards,
>> Srinivas.
>>
>
Re: Bigtop test framework documentation
Posted by "김영우 (Youngwoo Kim)" <wa...@gmail.com>.
Hi Srinivas,
Please refer to this wiki pages:
https://cwiki.apache.org/confluence/display/BIGTOP/Running+integration+and+system+tests
https://cwiki.apache.org/confluence/display/BIGTOP/Writing+integration+and+system+tests
If you have any further question, feel free to let us know.
Thanks,
Youngwoo
2015. 8. 16. 오후 7:39에 "Srinivas Kolla" <sr...@altiscale.com>님이 작성:
> Hi Team,
>
> Where can i get bigtop test framework documentation?
> Recently, i have started using bigtop test framework to automation the
> hadoop test cases.
> Though i could understand it at a high level, i want to learn it in a
> detail way.
> I could not find valuable resources from google search on bigtop.
> In this regard, Can someone please help me to point to correct
> documentation of Bigtop.
>
> Regards,
> Srinivas.
>