[jira] [Created] (BIGTOP-1066) multifilewc and possibly other tests rely on mapred: should they be separated or replaced?

["jay vyas (JIRA)" <ji...@apache.org>]

jay vyas created BIGTOP-1066:
--------------------------------

             Summary: multifilewc and possibly other tests rely on mapred: should they be separated or replaced?
                 Key: BIGTOP-1066
                 URL: https://issues.apache.org/jira/browse/BIGTOP-1066
             Project: Bigtop
          Issue Type: Bug
            Reporter: jay vyas
            Priority: Minor


The use of old tests (i.e. those that use mapred) can cause issues.  Is is it considered standard pratcice to still support the mapred.* classes? 

I say this because in my bigtop smokes, we fail the multifilewc test, because its RecordReader seems to try to call getPos() after the underlying stream is closed in the FileSystem implementation.  If we update to using the new mapreduce.* implementation of multifilewc, which uses CombineFileSplit instead of the older MultiFileInputFormat, the job works fine. 

So - this is partly a question : should bigtop tests which rely on mapred.* 1) classes be separated out into a different test suite, or 
2) deleted entirely for 2.0 (forward), or 
3) simply ignored on a case-by-case/cluster-by-cluster basis? 

I favor (1) : It would be nice if we could decide, at runtime, declaratively - wether or not we wanted to use tests that exersize the old (2009) mapred classes.  




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira