You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Mukund Madhugiri (JIRA)" <ji...@apache.org> on 2007/10/09 20:20:50 UTC
[jira] Updated: (HADOOP-2000) Re-write NNBench to use MapReduce
[ https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mukund Madhugiri updated HADOOP-2000:
-------------------------------------
Status: Patch Available (was: Open)
Attaching patch for code review
Here is a sample run output:
---------------------------------------------
bin/hadoop jar hadoop-0.15.0-dev-test.jar nnbench -maps 1 -reduces 1 -startTime `date +"%s"` -blockSize 1 -bytesPerChecksum 1 -replicationFactorPerFile 1 -numberOfFiles 100
NameNode Benchmark 0.4
07/10/09 18:14:09 INFO dfs.NNBench: Test Inputs:
07/10/09 18:14:09 INFO dfs.NNBench: Test Operation: all
07/10/09 18:14:09 INFO dfs.NNBench: Number of maps: 1
07/10/09 18:14:09 INFO dfs.NNBench: Number of reduces: 1
07/10/09 18:14:09 INFO dfs.NNBench: Block Size: 1
07/10/09 18:14:09 INFO dfs.NNBench: Bytes per checksum: 1
07/10/09 18:14:09 INFO dfs.NNBench: Number of files: 100
07/10/09 18:14:09 INFO dfs.NNBench: Replication factor: 1
07/10/09 18:14:10 INFO dfs.NNBench: Creating control files
07/10/09 18:14:10 INFO mapred.FileInputFormat: Total input paths to process : 1
07/10/09 18:14:10 INFO mapred.JobClient: Running job: job_200710032050_0199
07/10/09 18:14:11 INFO mapred.JobClient: map 0% reduce 0%
07/10/09 18:14:17 INFO mapred.JobClient: map 100% reduce 0%
07/10/09 18:14:18 INFO mapred.JobClient: map 100% reduce 100%
07/10/09 18:14:19 INFO mapred.JobClient: Job complete: job_200710032050_0199
07/10/09 18:14:19 INFO mapred.JobClient: Counters: 10
07/10/09 18:14:19 INFO mapred.JobClient: Job Counters
07/10/09 18:14:19 INFO mapred.JobClient: Launched map tasks=1
07/10/09 18:14:19 INFO mapred.JobClient: Launched reduce tasks=1
07/10/09 18:14:19 INFO mapred.JobClient: Data-local map tasks=1
07/10/09 18:14:19 INFO mapred.JobClient: Map-Reduce Framework
07/10/09 18:14:19 INFO mapred.JobClient: Map input records=1
07/10/09 18:14:19 INFO mapred.JobClient: Map output records=5
07/10/09 18:14:19 INFO mapred.JobClient: Map input bytes=39
07/10/09 18:14:19 INFO mapred.JobClient: Map output bytes=90
07/10/09 18:14:19 INFO mapred.JobClient: Reduce input groups=5
07/10/09 18:14:19 INFO mapred.JobClient: Reduce input records=5
07/10/09 18:14:19 INFO mapred.JobClient: Reduce output records=5
07/10/09 18:14:19 INFO dfs.NNBench: ------ NNBench ------ :
07/10/09 18:14:19 INFO dfs.NNBench: Version: NameNode Benchmark 0.4
07/10/09 18:14:19 INFO dfs.NNBench: Date & time: Tue Oct 09 18:14:19 UTC 2007
07/10/09 18:14:19 INFO dfs.NNBench:
07/10/09 18:14:19 INFO dfs.NNBench: Test Operation: all
07/10/09 18:14:19 INFO dfs.NNBench: Maps: 1
07/10/09 18:14:19 INFO dfs.NNBench: Reduces: 1
07/10/09 18:14:19 INFO dfs.NNBench: Block Size (bytes): 1
07/10/09 18:14:19 INFO dfs.NNBench: Bytes to write: 1
07/10/09 18:14:19 INFO dfs.NNBench: Bytes per checksum: 1
07/10/09 18:14:19 INFO dfs.NNBench: Number of files: 100
07/10/09 18:14:19 INFO dfs.NNBench: Replication factor: 1
07/10/09 18:14:19 INFO dfs.NNBench:
07/10/09 18:14:19 INFO dfs.NNBench: TPS: Create and Close: 72
07/10/09 18:14:19 INFO dfs.NNBench: TPS: Open (Read): 2777
07/10/09 18:14:19 INFO dfs.NNBench: TPS: Rename: 800
07/10/09 18:14:19 INFO dfs.NNBench: TPS: Delete: 7142
07/10/09 18:14:19 INFO dfs.NNBench:
07/10/09 18:14:19 INFO dfs.NNBench: Avg Lat (ms): Create: 1.35
07/10/09 18:14:19 INFO dfs.NNBench: Avg Lat (ms): Close: 26.28
07/10/09 18:14:19 INFO dfs.NNBench: Avg Lat (ms): Open (Read): 0.36
07/10/09 18:14:19 INFO dfs.NNBench: Avg Lat (ms): Rename: 1.25
07/10/09 18:14:19 INFO dfs.NNBench: Avg Lat (ms): Delete: 0.14
---------------------------------------------
> Re-write NNBench to use MapReduce
> ---------------------------------
>
> Key: HADOOP-2000
> URL: https://issues.apache.org/jira/browse/HADOOP-2000
> Project: Hadoop
> Issue Type: Test
> Components: test
> Affects Versions: 0.15.0
> Reporter: Mukund Madhugiri
> Assignee: Mukund Madhugiri
>
> The proposal is to re-write the NNBench benchmark/test to measure Namenode operations using MapReduce. Two buckets of measurements will be done:
> 1. Transactions per second
> 2. Average latency
> for these operations
> - Create and Close file
> - Open file
> - Rename file
> - Delete file
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.