You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by ag...@apache.org on 2024/05/16 02:07:34 UTC
(datafusion-site) branch init updated (d19baa1 -> 8de43d4)
This is an automated email from the ASF dual-hosted git repository.
agrove pushed a change to branch init
in repository https://gitbox.apache.org/repos/asf/datafusion-site.git
from d19baa1 git ignore
new bcf3842 add comet donation post
new 8de43d4 add images
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
_posts/2024-03-06-comet-donation.md | 90 ++++++++++++++++++++++++++++
assets/datafusion_fast_grouping/full.png | Bin 0 -> 50146 bytes
assets/datafusion_fast_grouping/summary.png | Bin 0 -> 52822 bytes
img/2021-08-18-datafusion500perf.png | Bin 0 -> 21245 bytes
img/datafusion-34.0.0/compare-new.png | Bin 0 -> 176796 bytes
img/datafusion-34.0.0/compare.png | Bin 0 -> 149835 bytes
img/datafusion-comet/comet-architecture.png | Bin 0 -> 30027 bytes
7 files changed, 90 insertions(+)
create mode 100644 _posts/2024-03-06-comet-donation.md
create mode 100644 assets/datafusion_fast_grouping/full.png
create mode 100644 assets/datafusion_fast_grouping/summary.png
create mode 100644 img/2021-08-18-datafusion500perf.png
create mode 100644 img/datafusion-34.0.0/compare-new.png
create mode 100644 img/datafusion-34.0.0/compare.png
create mode 100644 img/datafusion-comet/comet-architecture.png
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org
(datafusion-site) 01/02: add comet donation post
Posted by ag...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch init
in repository https://gitbox.apache.org/repos/asf/datafusion-site.git
commit bcf3842543aee46cb7bcd8cbf511e11e59929018
Author: Andy Grove <ag...@apache.org>
AuthorDate: Wed May 15 20:03:32 2024 -0600
add comet donation post
---
_posts/2024-03-06-comet-donation.md | 90 +++++++++++++++++++++++++++++++++++++
1 file changed, 90 insertions(+)
diff --git a/_posts/2024-03-06-comet-donation.md b/_posts/2024-03-06-comet-donation.md
new file mode 100644
index 0000000..138ae2c
--- /dev/null
+++ b/_posts/2024-03-06-comet-donation.md
@@ -0,0 +1,90 @@
+---
+layout: post
+title: "Announcing Apache Arrow DataFusion Comet"
+date: "2024-03-06 00:00:00"
+author: pmc
+categories: [release]
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+# Introduction
+The Apache Arrow PMC is pleased to announce the donation of the [Comet project],
+a native Spark SQL Accelerator built on [Apache Arrow DataFusion].
+
+Comet is an Apache Spark plugin that uses Apache Arrow DataFusion to
+accelerate Spark workloads. It is designed as a drop-in
+replacement for Spark's JVM based SQL execution engine and offers significant
+performance improvements for some workloads as shown below.
+
+<figure style="text-align: center;">
+ <img
+ src="{{ site.baseurl }}/img/datafusion-comet/comet-architecture.png"
+ width="100%"
+ class="img-responsive"
+ alt="Fig 1: Adaptive Arrow schema architecture overview."
+ >
+ <figcaption>
+ <b>Figure 1</b>: With Comet, users interact with the same Spark ecosystem, tools
+ and APIs such as Spark SQL. Queries still run through Spark's query optimizer and planner.
+ However, the execution is delegated to Comet,
+ which is significantly faster and more resource efficient than a JVM based
+ implementation.
+</figcaption>
+</figure>
+
+
+
+[Rust]: https://www.rust-lang.org/
+
+Comet is one of a growing class of projects that aim to accelerate Spark using
+native columnar engines such as the proprietary [Databricks Photon Engine] and
+open source projects [Gluten], [Spark RAPIDS], and [Blaze] (also built using
+DataFusion).
+
+Comet was originally implemented at Apple and the engineers who worked on the
+project are also significant contributors to Arrow and DataFusion. Bringing
+Comet into the Apache Software Foundation will accelerate its development and
+grow its community of contributors and users.
+
+[Comet project]: https://github.com/apache/arrow-datafusion-comet
+[Apache Arrow DataFusion]: https://arrow.apache.org/datafusion
+[Databricks Photon Engine]: https://www.databricks.com/product/photon
+[Gluten]: https://incubator.apache.org/projects/gluten.html
+[Blaze]: https://github.com/kwai/blaze
+[Spark RAPIDS]: https://github.com/NVIDIA/spark-rapids
+
+# Get Involved
+Comet is still in the early stages of development and we would love to have you
+join us and help shape the project. We are working on an initial release, and
+expect to post another update with more details at that time.
+
+Before then, here are some ways to get involved:
+
+* Learn more by visiting the [Comet project] page and reading the [mailing list
+ discussion] about the initial donation.
+
+* Help us plan out the [roadmap]
+
+* Try out the project and provide feedback, file issues, and contribute code.
+
+[mailing list discussion]: https://lists.apache.org/thread/0q1rb11jtpopc7vt1ffdzro0omblsh0s
+[roadmap]: https://github.com/apache/arrow-datafusion-comet/issues/19
+
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org
(datafusion-site) 02/02: add images
Posted by ag...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch init
in repository https://gitbox.apache.org/repos/asf/datafusion-site.git
commit 8de43d42daecd6b1b14aa01ef60939cfbdf484a4
Author: Andy Grove <ag...@apache.org>
AuthorDate: Wed May 15 20:07:27 2024 -0600
add images
---
assets/datafusion_fast_grouping/full.png | Bin 0 -> 50146 bytes
assets/datafusion_fast_grouping/summary.png | Bin 0 -> 52822 bytes
img/2021-08-18-datafusion500perf.png | Bin 0 -> 21245 bytes
img/datafusion-34.0.0/compare-new.png | Bin 0 -> 176796 bytes
img/datafusion-34.0.0/compare.png | Bin 0 -> 149835 bytes
img/datafusion-comet/comet-architecture.png | Bin 0 -> 30027 bytes
6 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/assets/datafusion_fast_grouping/full.png b/assets/datafusion_fast_grouping/full.png
new file mode 100644
index 0000000..9f07b04
Binary files /dev/null and b/assets/datafusion_fast_grouping/full.png differ
diff --git a/assets/datafusion_fast_grouping/summary.png b/assets/datafusion_fast_grouping/summary.png
new file mode 100644
index 0000000..f92df82
Binary files /dev/null and b/assets/datafusion_fast_grouping/summary.png differ
diff --git a/img/2021-08-18-datafusion500perf.png b/img/2021-08-18-datafusion500perf.png
new file mode 100644
index 0000000..4406473
Binary files /dev/null and b/img/2021-08-18-datafusion500perf.png differ
diff --git a/img/datafusion-34.0.0/compare-new.png b/img/datafusion-34.0.0/compare-new.png
new file mode 100644
index 0000000..9e2ae22
Binary files /dev/null and b/img/datafusion-34.0.0/compare-new.png differ
diff --git a/img/datafusion-34.0.0/compare.png b/img/datafusion-34.0.0/compare.png
new file mode 100644
index 0000000..8360737
Binary files /dev/null and b/img/datafusion-34.0.0/compare.png differ
diff --git a/img/datafusion-comet/comet-architecture.png b/img/datafusion-comet/comet-architecture.png
new file mode 100644
index 0000000..e7c9075
Binary files /dev/null and b/img/datafusion-comet/comet-architecture.png differ
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org