[jira] [Commented] (MAPREDUCE-207) Computing Input Splits on the MR Cluster

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995583#comment-13995583 ] 

Hadoop QA commented on MAPREDUCE-207:
-------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12644428/MAPREDUCE-207.v02.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:red}-1 core tests{color}.  The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:

                  org.apache.hadoop.mapred.lib.aggregate.TestAggregates
                  org.apache.hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat
                  org.apache.hadoop.mapred.TestFieldSelection
                  org.apache.hadoop.mapred.TestOldCombinerGrouping
                  org.apache.hadoop.mapreduce.TestLocalRunner
                  org.apache.hadoop.mapred.TestUserDefinedCounters
                  org.apache.hadoop.mapreduce.TestMROutputFormat
                  org.apache.hadoop.mapreduce.lib.fieldsel.TestMRFieldSelection
                  org.apache.hadoop.mapred.TestLocalMRNotification
                  org.apache.hadoop.mapred.TestLineRecordReaderJobs
                  org.apache.hadoop.mapreduce.lib.map.TestMultithreadedMapper
                  org.apache.hadoop.mapreduce.TestNewCombinerGrouping
                  org.apache.hadoop.mapred.lib.TestChainMapReduce
                  org.apache.hadoop.mapreduce.TestMapReduce
                  org.apache.hadoop.mapreduce.lib.join.TestJoinDatamerge
                  org.apache.hadoop.mapred.lib.TestKeyFieldBasedComparator
                  org.apache.hadoop.mapred.lib.TestMultithreadedMapRunner
                  org.apache.hadoop.mapreduce.TestMapperReducerCleanup
                  org.apache.hadoop.mapred.lib.TestMultipleOutputs
                  org.apache.hadoop.mapred.TestJavaSerialization
                  org.apache.hadoop.mapreduce.lib.output.TestMRMultipleOutputs
                  org.apache.hadoop.mapred.TestCollect
                  org.apache.hadoop.mapred.join.TestDatamerge
                  org.apache.hadoop.mapreduce.TestMapCollection
                  org.apache.hadoop.mapreduce.lib.aggregate.TestMapReduceAggregates
                  org.apache.hadoop.mapred.TestMapRed
                  org.apache.hadoop.mapred.TestFileOutputFormat
                  org.apache.hadoop.mapreduce.TestValueIterReset
                  org.apache.hadoop.mapred.TestMapOutputType
                  org.apache.hadoop.mapred.TestJobCounters
                  org.apache.hadoop.conf.TestNoDefaultsJobConf
                  org.apache.hadoop.mapred.TestReporter
                  org.apache.hadoop.mapreduce.lib.partition.TestMRKeyFieldBasedComparator
                  org.apache.hadoop.mapreduce.lib.chain.TestChainErrors
                  org.apache.hadoop.mapreduce.lib.chain.TestSingleElementChain
                  org.apache.hadoop.mapreduce.lib.input.TestMultipleInputs
                  org.apache.hadoop.mapred.TestComparators
                  org.apache.hadoop.mapreduce.lib.input.TestLineRecordReaderJobs
                  org.apache.hadoop.mapreduce.lib.chain.TestMapReduceChain
                  org.apache.hadoop.mapred.jobcontrol.TestLocalJobControl
                  org.apache.hadoop.mapreduce.lib.jobcontrol.TestMapReduceJobControl

                                      The following test timeouts occurred in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:

org.apache.hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
org.apache.hadoop.mapreduce.v2.app.job.impl.TestMapReduceChildJVM
org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebApp
org.apache.hadoop.mapreduce.v2.app.TestMRClientService
org.apache.hadoop.mapreduce.v2.app.TestKill
org.apache.hadoop.mapreduce.v2.app.TestMRApp
org.apache.hadoop.mapreduce.v2.app.TestJobEndNotifier
org.apache.hadoop.mapreduce.v2.app.TestFail
org.apache.hadoop.mapreduce.v2.app.TestStagingCleanup
org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator
org.apache.hadoop.mapreduce.v2.app.launcher.TestContainerLauncher
org.apache.hadoop.mapreduce.v2.app.TestRecovery
org.apache.hadoop.mapreduce.v2.app.TestAMInfos
org.apache.hadoop.mapreduce.v2.app.TestMRAppComponentDependencies
org.apache.hadoop.mapreduce.v2.app.TestFetchFailure
org.apache.hadoop.mapreduce.v2.TestSpeculativeExecutionWithMRApp
org.apache.hadoop.mapred.pipes.TestPipeApplication

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4595//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4595//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4595//console

This message is automatically generated.

> Computing Input Splits on the MR Cluster
> ----------------------------------------
>
>                 Key: MAPREDUCE-207
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-207
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: applicationmaster, mrv2
>            Reporter: Philip Zeyliger
>            Assignee: Arun C Murthy
>         Attachments: MAPREDUCE-207.patch, MAPREDUCE-207.v02.patch
>
>
> Instead of computing the input splits as part of job submission, Hadoop could have a separate "job task type" that computes the input splits, therefore allowing that computation to happen on the cluster.



--
This message was sent by Atlassian JIRA
(v6.2#6252)