You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by rx...@apache.org on 2014/11/06 00:01:18 UTC
svn commit: r1637002 - in /spark:
news/_posts/2014-11-05-spark-wins-daytona-gray-sort-100tb-benchmark.md
site/news/spark-wins-daytona-gray-sort-100tb-benchmark.html
Author: rxin
Date: Wed Nov 5 23:01:17 2014
New Revision: 1637002
URL: http://svn.apache.org/r1637002
Log:
added the sort benchmark news item
Added:
spark/news/_posts/2014-11-05-spark-wins-daytona-gray-sort-100tb-benchmark.md
spark/site/news/spark-wins-daytona-gray-sort-100tb-benchmark.html
Added: spark/news/_posts/2014-11-05-spark-wins-daytona-gray-sort-100tb-benchmark.md
URL: http://svn.apache.org/viewvc/spark/news/_posts/2014-11-05-spark-wins-daytona-gray-sort-100tb-benchmark.md?rev=1637002&view=auto
==============================================================================
--- spark/news/_posts/2014-11-05-spark-wins-daytona-gray-sort-100tb-benchmark.md (added)
+++ spark/news/_posts/2014-11-05-spark-wins-daytona-gray-sort-100tb-benchmark.md Wed Nov 5 23:01:17 2014
@@ -0,0 +1,21 @@
+---
+layout: post
+title: Spark wins Daytona Gray Sort 100TB Benchmark
+categories:
+- News
+tags: []
+status: publish
+type: post
+published: true
+meta:
+ _edit_last: '4'
+ _wpas_done_all: '1'
+---
+
+We are proud to announce that Spark won the <a href="http://sortbenchmark.org/">2014 Gray Sort Benchmark</a> (Daytona 100TB category). A team from <a href="http://databricks.com/">Databricks</a> including Spark committers, Reynold Xin, Xiangrui Meng, and Matei Zaharia, <a href="http://databricks.com/blog/2014/11/05/spark-officially-sets-a-new-record-in-large-scale-sorting.html">entered the benchmark using Spark</a>. Spark won a tie with the Themis team from UCSD, and jointly set a new world record in sorting.
+
+They used Spark and sorted 100TB of data using 206 EC2 i2.8xlarge machines in 23 minutes. The previous world record was 72 minutes, set by a Hadoop MapReduce cluster of 2100 nodes. This means that Spark sorted the same data 3X faster using 10X fewer machines. All the sorting took place on disk (HDFS), without using Sparkâs in-memory cache.
+
+Outperforming large Hadoop MapReduce clusters on sorting not only validates the vision and work done by the Spark community, but also demonstrates that Spark is fulfilling its promise to serve as a faster and more scalable engine for data processing of all sizes.
+
+For more information, see the <a href="http://databricks.com/blog/2014/11/05/spark-officially-sets-a-new-record-in-large-scale-sorting.html">Databricks blog article</a> written by the Reynold Xin.
Added: spark/site/news/spark-wins-daytona-gray-sort-100tb-benchmark.html
URL: http://svn.apache.org/viewvc/spark/site/news/spark-wins-daytona-gray-sort-100tb-benchmark.html?rev=1637002&view=auto
==============================================================================
--- spark/site/news/spark-wins-daytona-gray-sort-100tb-benchmark.html (added)
+++ spark/site/news/spark-wins-daytona-gray-sort-100tb-benchmark.html Wed Nov 5 23:01:17 2014
@@ -0,0 +1,196 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <title>
+ Spark wins Daytona Gray Sort 100TB Benchmark | Apache Spark
+
+ </title>
+
+
+
+ <!-- Bootstrap core CSS -->
+ <link href="/css/cerulean.min.css" rel="stylesheet">
+ <link href="/css/custom.css" rel="stylesheet">
+
+ <script type="text/javascript">
+ <!-- Google Analytics initialization -->
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-32518208-2']);
+ _gaq.push(['_trackPageview']);
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+
+ <!-- Adds slight delay to links to allow async reporting -->
+ function trackOutboundLink(link, category, action) {
+ try {
+ _gaq.push(['_trackEvent', category , action]);
+ } catch(err){}
+
+ setTimeout(function() {
+ document.location.href = link.href;
+ }, 100);
+ }
+ </script>
+
+ <!-- HTML5 shim and Respond.js IE8 support of HTML5 elements and media queries -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
+ <script src="https://oss.maxcdn.com/libs/respond.js/1.3.0/respond.min.js"></script>
+ <![endif]-->
+</head>
+
+<body>
+
+<script src="https://code.jquery.com/jquery.js"></script>
+<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script>
+<script src="/js/lang-tabs.js"></script>
+<script src="/js/downloads.js"></script>
+
+<div class="container" style="max-width: 1200px;">
+
+<div class="masthead">
+
+ <p class="lead">
+ <a href="/">
+ <img src="/images/spark-logo.png"
+ style="height:100px; width:auto; vertical-align: bottom; margin-top: 20px;"></a><span class="tagline">
+ Lightning-fast cluster computing
+ </span>
+ </p>
+
+</div>
+
+<nav class="navbar navbar-default" role="navigation">
+ <!-- Brand and toggle get grouped for better mobile display -->
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle" data-toggle="collapse"
+ data-target="#navbar-collapse-1">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ </div>
+
+ <!-- Collect the nav links, forms, and other content for toggling -->
+ <div class="collapse navbar-collapse" id="navbar-collapse-1">
+ <ul class="nav navbar-nav">
+ <li><a href="/downloads.html">Download</a></li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">
+ Related Projects <b class="caret"></b>
+ </a>
+ <ul class="dropdown-menu">
+
+ <li><a href="/sql/">Spark SQL</a></li>
+ <li><a href="/streaming/">Spark Streaming</a></li>
+ <li><a href="/mllib/">MLlib (machine learning)</a></li>
+ <li><a href="/graphx/">GraphX (graph)</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">
+ Documentation <b class="caret"></b>
+ </a>
+ <ul class="dropdown-menu">
+ <li><a href="/documentation.html">Overview</a></li>
+ <li><a href="/docs/latest/">Latest Release (Spark 1.1.0)</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown">
+ Community <b class="caret"></b>
+ </a>
+ <ul class="dropdown-menu">
+ <li><a href="/community.html">Mailing Lists</a></li>
+ <li><a href="/community.html#events">Events and Meetups</a></li>
+ <li><a href="/community.html#history">Project History</a></li>
+ <li><a href="https://cwiki.apache.org/confluence/display/SPARK/Powered+By+Spark">Powered By</a></li>
+ </ul>
+ </li>
+ <li><a href="/examples.html">Examples</a></li>
+ <li><a href="/faq.html">FAQ</a></li>
+ </ul>
+ </div>
+ <!-- /.navbar-collapse -->
+</nav>
+
+
+<div class="row">
+ <div class="col-md-3 col-md-push-9">
+ <div class="news" style="margin-bottom: 20px;">
+ <h5>Latest News</h5>
+ <ul class="list-unstyled">
+
+ <li><a href="/news/spark-wins-daytona-gray-sort-100tb-benchmark.html">Spark wins Daytona Gray Sort 100TB Benchmark</a>
+ <span class="small">(Nov 05, 2014)</span></li>
+
+ <li><a href="/news/proposals-open-for-spark-summit-east.html">Submissions open for Spark Summit East 2015 in New York</a>
+ <span class="small">(Oct 18, 2014)</span></li>
+
+ <li><a href="/news/spark-1-1-0-released.html">Spark 1.1.0 released</a>
+ <span class="small">(Sep 11, 2014)</span></li>
+
+ <li><a href="/news/spark-1-0-2-released.html">Spark 1.0.2 released</a>
+ <span class="small">(Aug 05, 2014)</span></li>
+
+ </ul>
+ <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
+ </div>
+ <div class="hidden-xs hidden-sm">
+ <a href="/downloads.html" class="btn btn-success btn-lg btn-block" style="margin-bottom: 30px;">
+ Download Spark
+ </a>
+ <p style="font-size: 16px; font-weight: 500; color: #555;">
+ Related Projects:
+ </p>
+ <ul class="list-narrow">
+
+ <li><a href="/sql/">Spark SQL</a></li>
+ <li><a href="/streaming/">Spark Streaming</a></li>
+ <li><a href="/mllib/">MLlib (machine learning)</a></li>
+ <li><a href="/graphx/">GraphX (graph)</a></li>
+ </ul>
+ </div>
+ </div>
+
+ <div class="col-md-9 col-md-pull-3">
+ <h2>Spark wins Daytona Gray Sort 100TB Benchmark</h2>
+
+
+<p>We are proud to announce that Spark won the <a href="http://sortbenchmark.org/">2014 Gray Sort Benchmark</a> (Daytona 100TB category). A team from <a href="http://databricks.com/">Databricks</a> including Spark committers, Reynold Xin, Xiangrui Meng, and Matei Zaharia, <a href="http://databricks.com/blog/2014/11/05/spark-officially-sets-a-new-record-in-large-scale-sorting.html">entered the benchmark using Spark</a>. Spark won a tie with the Themis team from UCSD, and jointly set a new world record in sorting.</p>
+
+<p>They used Spark and sorted 100TB of data using 206 EC2 i2.8xlarge machines in 23 minutes. The previous world record was 72 minutes, set by a Hadoop MapReduce cluster of 2100 nodes. This means that Spark sorted the same data 3X faster using 10X fewer machines. All the sorting took place on disk (HDFS), without using Sparkâs in-memory cache.</p>
+
+<p>Outperforming large Hadoop MapReduce clusters on sorting not only validates the vision and work done by the Spark community, but also demonstrates that Spark is fulfilling its promise to serve as a faster and more scalable engine for data processing of all sizes.</p>
+
+<p>For more information, see the <a href="http://databricks.com/blog/2014/11/05/spark-officially-sets-a-new-record-in-large-scale-sorting.html">Databricks blog article</a> written by the Reynold Xin.</p>
+
+
+<p>
+<br/>
+<a href="/news/">Spark News Archive</a>
+</p>
+
+ </div>
+</div>
+
+
+
+<footer class="small">
+ <hr>
+ Apache Spark, Spark, Apache, and the Spark logo are trademarks of
+ <a href="http://www.apache.org">The Apache Software Foundation</a>.
+</footer>
+
+</div>
+
+</body>
+</html>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org