You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Aaron T. Myers (JIRA)" <ji...@apache.org> on 2012/04/21 00:41:33 UTC
[jira] [Created] (MAPREDUCE-4170) Move "sleep" and "fail" jobs from
tests module to examples module
Aaron T. Myers created MAPREDUCE-4170:
-----------------------------------------
Summary: Move "sleep" and "fail" jobs from tests module to examples module
Key: MAPREDUCE-4170
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4170
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: examples, test
Affects Versions: 2.0.0
Reporter: Aaron T. Myers
Priority: Minor
The sleep job used to be in the "examples" jar in MR1. I'm not quite sure when, but the sleep job has been moved to the "tests" module/jar in MR2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4170) Move "sleep" and "fail" jobs
from tests module to examples module
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258644#comment-13258644 ]
Aaron T. Myers commented on MAPREDUCE-4170:
-------------------------------------------
The "sleep" and "fail" jobs seem like they belong more with examples than with tests, IMO.
Here are the programs in the "tests" jar:
{code}
DFSCIOTest: Distributed i/o benchmark of libhdfs.
DistributedFSCheck: Distributed checkup of the file system consistency.
JHLogAnalyzer: Job History Log analyzer.
MRReliabilityTest: A program that tests the reliability of the MR framework by injecting faults/failures
SliveTest: HDFS Stress Test and Live Data Verification.
TestDFSIO: Distributed i/o benchmark.
fail: a job that always fails
filebench: Benchmark SequenceFile(Input|Output)Format (block,record compressed and uncompressed), Text(Input|Output)Format (compressed and uncompressed)
loadgen: Generic map/reduce load generator
mapredtest: A map/reduce test check.
mrbench: A map/reduce benchmark that can create many small jobs
nnbench: A benchmark that stresses the namenode.
sleep: A job that sleeps at each map and reduce task.
testbigmapoutput: A map/reduce program that works on a very big non-splittable file and does identity map/reduce
testfilesystem: A test for FileSystem read/write.
testmapredsort: A map/reduce program that validates the map-reduce framework's sort.
testsequencefile: A test for flat files of binary key value pairs.
testsequencefileinputformat: A test for sequence file input format.
testtextinputformat: A test for text input format.
threadedmapbench: A map/reduce benchmark that compares the performance of maps with multiple spills over maps with 1 spill
{code}
and here are the programs in the "examples" jar:
{code}
aggregatewordcount: An Aggregate based map/reduce program that counts the words in the input files.
aggregatewordhist: An Aggregate based map/reduce program that computes the histogram of the words in the input files.
bbp: A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact digits of Pi.
dbcount: An example job that count the pageview counts from a database.
distbbp: A map/reduce program that uses a BBP-type formula to compute exact bits of Pi.
grep: A map/reduce program that counts the matches of a regex in the input.
join: A job that effects a join over sorted, equally partitioned datasets
multifilewc: A job that counts words from several files.
pentomino: A map/reduce tile laying program to find solutions to pentomino problems.
pi: A map/reduce program that estimates Pi using a quasi-Monte Carlo method.
randomtextwriter: A map/reduce program that writes 10GB of random textual data per node.
randomwriter: A map/reduce program that writes 10GB of random data per node.
secondarysort: An example defining a secondary sort to the reduce.
sort: A map/reduce program that sorts the data written by the random writer.
sudoku: A sudoku solver.
teragen: Generate data for the terasort
terasort: Run the terasort
teravalidate: Checking results of terasort
wordcount: A map/reduce program that counts the words in the input files.
{code}
> Move "sleep" and "fail" jobs from tests module to examples module
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-4170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4170
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 2.0.0
> Reporter: Aaron T. Myers
> Priority: Minor
>
> The sleep job used to be in the "examples" jar in MR1. I'm not quite sure when, but the sleep job has been moved to the "tests" module/jar in MR2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4170) Move "sleep" and "fail" jobs
from tests module to examples module
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258661#comment-13258661 ]
Chris Douglas commented on MAPREDUCE-4170:
------------------------------------------
MAPREDUCE-973
The test dependency on the example jar was unfortunate. I'm not sure if any dependency still exists.
> Move "sleep" and "fail" jobs from tests module to examples module
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-4170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4170
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 2.0.0
> Reporter: Aaron T. Myers
> Priority: Minor
>
> The sleep job used to be in the "examples" jar in MR1. I'm not quite sure when, but the sleep job has been moved to the "tests" module/jar in MR2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4170) Move "sleep" and "fail" jobs
from tests module to examples module
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258645#comment-13258645 ]
Aaron T. Myers commented on MAPREDUCE-4170:
-------------------------------------------
Come to think of it, maybe JHLogAnalyzer should be moved as well, but that I don't feel as strongly about that as I do about sleep and fail.
> Move "sleep" and "fail" jobs from tests module to examples module
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-4170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4170
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 2.0.0
> Reporter: Aaron T. Myers
> Priority: Minor
>
> The sleep job used to be in the "examples" jar in MR1. I'm not quite sure when, but the sleep job has been moved to the "tests" module/jar in MR2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira