You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Prasun Ratn <pr...@gmail.com> on 2017/01/10 03:50:31 UTC
Spark performance tests
Hi
Are there performance tests or microbenchmarks for Spark - especially
directed towards the CPU specific parts? I looked at spark-perf but
that doesn't seem to have been updated recently.
Thanks
Prasun
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Spark performance tests
Posted by Prasun Ratn <pr...@gmail.com>.
Thanks Adam, Kazuaki!
On Tue, Jan 10, 2017 at 3:28 PM, Adam Roberts <AR...@uk.ibm.com> wrote:
> Hi, I suggest HiBench and SparkSqlPerf, HiBench features many benchmarks
> within it that exercise several components of Spark (great for stressing
> core, sql, MLlib capabilities), SparkSqlPerf features 99 TPC-DS queries
> (stressing the DataFrame API and therefore the Catalyst optimiser), both
> work well with Spark 2
>
> HiBench: https://github.com/intel-hadoop/HiBench
> SparkSqlPerf: https://github.com/databricks/spark-sql-perf
>
>
>
>
> From: "Kazuaki Ishizaki" <IS...@jp.ibm.com>
> To: Prasun Ratn <pr...@gmail.com>
> Cc: Apache Spark Dev <de...@spark.apache.org>
> Date: 10/01/2017 09:22
> Subject: Re: Spark performance tests
> ________________________________
>
>
>
> Hi,
> You may find several micro-benchmarks under
> https://github.com/apache/spark/tree/master/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark.
>
> Regards,
> Kazuaki Ishizaki
>
>
>
> From: Prasun Ratn <pr...@gmail.com>
> To: Apache Spark Dev <de...@spark.apache.org>
> Date: 2017/01/10 12:52
> Subject: Spark performance tests
> ________________________________
>
>
>
> Hi
>
> Are there performance tests or microbenchmarks for Spark - especially
> directed towards the CPU specific parts? I looked at spark-perf but
> that doesn't seem to have been updated recently.
>
> Thanks
> Prasun
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Spark performance tests
Posted by Adam Roberts <AR...@uk.ibm.com>.
Hi, I suggest HiBench and SparkSqlPerf, HiBench features many benchmarks
within it that exercise several components of Spark (great for stressing
core, sql, MLlib capabilities), SparkSqlPerf features 99 TPC-DS queries
(stressing the DataFrame API and therefore the Catalyst optimiser), both
work well with Spark 2
HiBench: https://github.com/intel-hadoop/HiBench
SparkSqlPerf: https://github.com/databricks/spark-sql-perf
From: "Kazuaki Ishizaki" <IS...@jp.ibm.com>
To: Prasun Ratn <pr...@gmail.com>
Cc: Apache Spark Dev <de...@spark.apache.org>
Date: 10/01/2017 09:22
Subject: Re: Spark performance tests
Hi,
You may find several micro-benchmarks under
https://github.com/apache/spark/tree/master/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark
.
Regards,
Kazuaki Ishizaki
From: Prasun Ratn <pr...@gmail.com>
To: Apache Spark Dev <de...@spark.apache.org>
Date: 2017/01/10 12:52
Subject: Spark performance tests
Hi
Are there performance tests or microbenchmarks for Spark - especially
directed towards the CPU specific parts? I looked at spark-perf but
that doesn't seem to have been updated recently.
Thanks
Prasun
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: Spark performance tests
Posted by Kazuaki Ishizaki <IS...@jp.ibm.com>.
Hi,
You may find several micro-benchmarks under
https://github.com/apache/spark/tree/master/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark
.
Regards,
Kazuaki Ishizaki
From: Prasun Ratn <pr...@gmail.com>
To: Apache Spark Dev <de...@spark.apache.org>
Date: 2017/01/10 12:52
Subject: Spark performance tests
Hi
Are there performance tests or microbenchmarks for Spark - especially
directed towards the CPU specific parts? I looked at spark-perf but
that doesn't seem to have been updated recently.
Thanks
Prasun
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org