You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Mona Chitnis (JIRA)" <ji...@apache.org> on 2010/07/31 02:54:17 UTC

[jira] Updated: (HDFS-1324) Quantifying HDFS Client Latency to understand performance and scalability

     [ https://issues.apache.org/jira/browse/HDFS-1324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mona Chitnis updated HDFS-1324:
-------------------------------

    Attachment: Paper1.pdf
                Client_Latency_Observation_v4.0.ppt
                watcher.jar

Attachments:
1. Design Document - Whitepaper describing the motivation, test methodology, setup and results
2. Information on advantages of this study and the important findings
3. Watcher JAR comprising of the benchmarking module
4. Helper scripts needed to execute the JAR
5. README to execute unit test


> Quantifying HDFS Client Latency to understand performance and scalability
> -------------------------------------------------------------------------
>
>                 Key: HDFS-1324
>                 URL: https://issues.apache.org/jira/browse/HDFS-1324
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: benchmarks
>    Affects Versions: 0.20-append
>         Environment: An HDFS cluster of 9 nodes (same rack; 1 rack = 40 nodes to one rack switch) was deployed on Yahoo!'s R&D grid cluster. It comprised of 1 Namenode, 1 JobTracker and 7 Datanodes. client assigned to a separate compute node not part of this HDFS. The characteristics of the testbed are as follows:
> •	Hadoop 0.20.1xx - latest Yahoo! Hadoop Security version
> •	OS - GNU/Linux x86_64 kernel version 2.6.18
> •	Java 1.6
> •	Processor - 2 SMP Quad-core Intel Xeon @ 2.5Ghz per node
> •	Memory - 16GB RAM per node
>            Reporter: Mona Chitnis
>         Attachments: Client_Latency_Observation_v4.0.ppt, Paper1.pdf, watcher.jar
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> JAR to measure HDFS client latencies. Runs a process comprising of HDFS operations with helper scripts to specify number of iterations, number of concurrent instances and base directory. Compiles statistics in 2 text files inside base directory which can be read/manipulated using data charts applications (e.g. MS Excel).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.