You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Charley Newtonne <cn...@gmail.com> on 2014/07/22 17:23:08 UTC
Bench-marking Hadoop Performance
This is a new cluster I'm putting up and I need to get an idea on what to
expect from a performance standpoint.
Older docs point to gridmix and TestDFSIO . However, most of this doc is
obsolete and no longer applies on 2.4.
Where can I find benchmarking docs for 2.4? What are my options?
Also, I have searched safari books online including rough cuts, but not
seeing books for the 2.4 release. If you know of a book for this release,
please share.
Thank you.
Re: Bench-marking Hadoop Performance
Posted by jay vyas <ja...@gmail.com>.
There are alot of tests out there and it can be tough to determine what is
a "standard".
- TeraGen/TearSort and testdfsio are starting points.
- Various other non apache projects (such as ycsb or hibench) will have
good benchmarks for certain type sof cases.
-If looking for a more comprehensive long term strategy, I'd suggest the
you ask on the bigtop mailing list, where we are
building a broader community around uniform smoke testing and benchmarking
of hadoop, hadoop compatible file systems, and YARN applications.
On Tue, Jul 22, 2014 at 11:23 AM, Charley Newtonne <cn...@gmail.com>
wrote:
> This is a new cluster I'm putting up and I need to get an idea on what to
> expect from a performance standpoint.
>
> Older docs point to gridmix and TestDFSIO . However, most of this doc is
> obsolete and no longer applies on 2.4.
>
> Where can I find benchmarking docs for 2.4? What are my options?
> Also, I have searched safari books online including rough cuts, but not
> seeing books for the 2.4 release. If you know of a book for this release,
> please share.
>
> Thank you.
>
>
>
--
jay vyas
Re: Bench-marking Hadoop Performance
Posted by jay vyas <ja...@gmail.com>.
There are alot of tests out there and it can be tough to determine what is
a "standard".
- TeraGen/TearSort and testdfsio are starting points.
- Various other non apache projects (such as ycsb or hibench) will have
good benchmarks for certain type sof cases.
-If looking for a more comprehensive long term strategy, I'd suggest the
you ask on the bigtop mailing list, where we are
building a broader community around uniform smoke testing and benchmarking
of hadoop, hadoop compatible file systems, and YARN applications.
On Tue, Jul 22, 2014 at 11:23 AM, Charley Newtonne <cn...@gmail.com>
wrote:
> This is a new cluster I'm putting up and I need to get an idea on what to
> expect from a performance standpoint.
>
> Older docs point to gridmix and TestDFSIO . However, most of this doc is
> obsolete and no longer applies on 2.4.
>
> Where can I find benchmarking docs for 2.4? What are my options?
> Also, I have searched safari books online including rough cuts, but not
> seeing books for the 2.4 release. If you know of a book for this release,
> please share.
>
> Thank you.
>
>
>
--
jay vyas
Re: Bench-marking Hadoop Performance
Posted by jay vyas <ja...@gmail.com>.
There are alot of tests out there and it can be tough to determine what is
a "standard".
- TeraGen/TearSort and testdfsio are starting points.
- Various other non apache projects (such as ycsb or hibench) will have
good benchmarks for certain type sof cases.
-If looking for a more comprehensive long term strategy, I'd suggest the
you ask on the bigtop mailing list, where we are
building a broader community around uniform smoke testing and benchmarking
of hadoop, hadoop compatible file systems, and YARN applications.
On Tue, Jul 22, 2014 at 11:23 AM, Charley Newtonne <cn...@gmail.com>
wrote:
> This is a new cluster I'm putting up and I need to get an idea on what to
> expect from a performance standpoint.
>
> Older docs point to gridmix and TestDFSIO . However, most of this doc is
> obsolete and no longer applies on 2.4.
>
> Where can I find benchmarking docs for 2.4? What are my options?
> Also, I have searched safari books online including rough cuts, but not
> seeing books for the 2.4 release. If you know of a book for this release,
> please share.
>
> Thank you.
>
>
>
--
jay vyas
Re: Bench-marking Hadoop Performance
Posted by jay vyas <ja...@gmail.com>.
There are alot of tests out there and it can be tough to determine what is
a "standard".
- TeraGen/TearSort and testdfsio are starting points.
- Various other non apache projects (such as ycsb or hibench) will have
good benchmarks for certain type sof cases.
-If looking for a more comprehensive long term strategy, I'd suggest the
you ask on the bigtop mailing list, where we are
building a broader community around uniform smoke testing and benchmarking
of hadoop, hadoop compatible file systems, and YARN applications.
On Tue, Jul 22, 2014 at 11:23 AM, Charley Newtonne <cn...@gmail.com>
wrote:
> This is a new cluster I'm putting up and I need to get an idea on what to
> expect from a performance standpoint.
>
> Older docs point to gridmix and TestDFSIO . However, most of this doc is
> obsolete and no longer applies on 2.4.
>
> Where can I find benchmarking docs for 2.4? What are my options?
> Also, I have searched safari books online including rough cuts, but not
> seeing books for the 2.4 release. If you know of a book for this release,
> please share.
>
> Thank you.
>
>
>
--
jay vyas