[jira] [Comment Edited] (BIGTOP-1257) Make TestHadoopExamples modular enough to support longevity tests.

["jay vyas (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-1257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13943401#comment-13943401 ] 

jay vyas edited comment on BIGTOP-1257 at 3/21/14 7:04 PM:
-----------------------------------------------------------

Yup ! Well, I see that Dasha's patch does some things better than HadoopExamples (its got great fine grained control).  But its build specific to DFSIO.  So im basically suggesting that we have one super class, which is a wrapper to hadoop-examples-jar, and then all hadoop-examples tests (DFSIO, TestHadoopExamples, etc...) can use it as a utility to drive their speicif cimplementations of word count, DFSIO, teragen, etc.

So maybe :

- The Suite: is "TestHadoopExamples"
- Run the Suite with "longevity" configuration: You get dasha's tests.
- Run the Suite with "smoke" configuration: You get VM scaled tests.



was (Author: jayunit100):
Yup ! Well, I see that Dasha's patch does some things better than HadoopExamples (its got great fine grained control).  But its build specific to DFSIO.  So im basically suggesting that we have one super class, which is a wrapper to hadoop-examples-jar, and then all hadoop-examples tests (DFSIO, TestHadoopExamples, etc...) can use it as a utility to drive their speicif cimplementations of word count, DFSIO, teragen, etc.

> Make TestHadoopExamples modular enough to support longevity tests.
> ------------------------------------------------------------------
>
>                 Key: BIGTOP-1257
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-1257
>             Project: Bigtop
>          Issue Type: Improvement
>          Components: Tests
>    Affects Versions: 0.7.0
>            Reporter: jay vyas
>
> TestHadoopExamples is really just a super of what we eventually will add in the Longevity suite.  It will be good to create a super class, then, which wraps the hadoop examples jars, whic BOTH the smoke tests AND longevity tests can both use and customize to suite their needs.
> Advantages:
> - less code duplication
> - easier to add in new longevity tests
> - longevity tests focus on customizing , rather than defining the tests classes.
> We can go about this two ways: 
> 1) Use the approach of TestHadoopExamples: define all examples in a k/v map, with parameters or 
> 2) Write individual unit tests for each benchmark (i.e. TestDFSIO), as is done in Dasha's BIGTOP-1208 patch.
> Part of this JIRA i think is to audit the longevity suite for other duplications, to see where we can merge efforts.



--
This message was sent by Atlassian JIRA
(v6.2#6252)