You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2024/04/17 14:18:16 UTC
(spark-website) branch asf-site updated: add pandas on spark page
This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/spark-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 697f9f6d8f add pandas on spark page
697f9f6d8f is described below
commit 697f9f6d8f29a0187e638b49f218ad8c73d0ff63
Author: Matthew Powers <ma...@gmail.com>
AuthorDate: Wed Apr 17 09:17:58 2024 -0500
add pandas on spark page
Sorry for the huge PR!
This PR adds a pandas on Spark page to explain the benefits of pandas on Spark.
The [pandas API on Spark docs](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html) are great, but they get right into the details and don't provide a high-level overview of why someone should use pandas on Spark.
I am hoping that this page serves as a nice entry-point to the existing pandas API on Spark docs.
Author: Matthew Powers <ma...@gmail.com>
Closes #513 from MrPowers/add-pandas-on-spark.
---
_layouts/global.html | 1 +
_layouts/home.html | 1 +
pandas-on-spark/index.md | 194 ++++++++++
site/committers.html | 1 +
site/community.html | 1 +
site/contributing.html | 1 +
site/developer-tools.html | 1 +
site/documentation.html | 1 +
site/downloads.html | 1 +
site/error-message-guidelines.html | 1 +
site/examples.html | 1 +
site/faq.html | 1 +
site/graphx/index.html | 1 +
site/history.html | 1 +
site/improvement-proposals.html | 1 +
site/index.html | 1 +
site/mailing-lists.html | 1 +
site/mllib/index.html | 1 +
site/news/3-1-3-released.html | 1 +
site/news/amp-camp-2013-registration-ope.html | 1 +
site/news/announcing-the-first-spark-summit.html | 1 +
site/news/fourth-spark-screencast-published.html | 1 +
site/news/index.html | 1 +
site/news/new-repository-service.html | 1 +
site/news/next-official-release-spark-3.1.1.html | 1 +
site/news/nsdi-paper.html | 1 +
site/news/one-month-to-spark-summit-2015.html | 1 +
site/news/plan-for-dropping-python-2-support.html | 1 +
.../news/proposals-open-for-spark-summit-east.html | 1 +
.../registration-open-for-spark-summit-east.html | 1 +
site/news/run-spark-and-shark-on-amazon-emr.html | 1 +
site/news/sigmod-system-award.html | 1 +
site/news/spark-0-6-1-and-0-5-2-released.html | 1 +
site/news/spark-0-6-2-released.html | 1 +
site/news/spark-0-7-0-released.html | 1 +
site/news/spark-0-7-2-released.html | 1 +
site/news/spark-0-7-3-released.html | 1 +
site/news/spark-0-8-0-released.html | 1 +
site/news/spark-0-8-1-released.html | 1 +
site/news/spark-0-9-0-released.html | 1 +
site/news/spark-0-9-1-released.html | 1 +
site/news/spark-0-9-2-released.html | 1 +
site/news/spark-1-0-0-released.html | 1 +
site/news/spark-1-0-1-released.html | 1 +
site/news/spark-1-0-2-released.html | 1 +
site/news/spark-1-1-0-released.html | 1 +
site/news/spark-1-1-1-released.html | 1 +
site/news/spark-1-2-0-released.html | 1 +
site/news/spark-1-2-1-released.html | 1 +
site/news/spark-1-2-2-released.html | 1 +
site/news/spark-1-3-0-released.html | 1 +
site/news/spark-1-4-0-released.html | 1 +
site/news/spark-1-4-1-released.html | 1 +
site/news/spark-1-5-0-released.html | 1 +
site/news/spark-1-5-1-released.html | 1 +
site/news/spark-1-5-2-released.html | 1 +
site/news/spark-1-6-0-released.html | 1 +
site/news/spark-1-6-1-released.html | 1 +
site/news/spark-1-6-2-released.html | 1 +
site/news/spark-1-6-3-released.html | 1 +
site/news/spark-2-0-0-released.html | 1 +
site/news/spark-2-0-1-released.html | 1 +
site/news/spark-2-0-2-released.html | 1 +
site/news/spark-2-1-0-released.html | 1 +
site/news/spark-2-1-1-released.html | 1 +
site/news/spark-2-1-2-released.html | 1 +
site/news/spark-2-1-3-released.html | 1 +
site/news/spark-2-2-0-released.html | 1 +
site/news/spark-2-2-1-released.html | 1 +
site/news/spark-2-2-2-released.html | 1 +
site/news/spark-2-3-0-released.html | 1 +
site/news/spark-2-3-1-released.html | 1 +
site/news/spark-2-3-2-released.html | 1 +
site/news/spark-2-3-3-released.html | 1 +
site/news/spark-2-3-4-released.html | 1 +
site/news/spark-2-4-0-released.html | 1 +
site/news/spark-2-4-1-released.html | 1 +
site/news/spark-2-4-2-released.html | 1 +
site/news/spark-2-4-3-released.html | 1 +
site/news/spark-2-4-4-released.html | 1 +
site/news/spark-2-4-5-released.html | 1 +
site/news/spark-2-4-6.html | 1 +
site/news/spark-2-4-7-released.html | 1 +
site/news/spark-2-4-8-released.html | 1 +
site/news/spark-2.0.0-preview.html | 1 +
site/news/spark-3-0-0-released.html | 1 +
site/news/spark-3-0-1-released.html | 1 +
site/news/spark-3-0-2-released.html | 1 +
site/news/spark-3-0-3-released.html | 1 +
site/news/spark-3-1-1-released.html | 1 +
site/news/spark-3-1-2-released.html | 1 +
site/news/spark-3-2-0-released.html | 1 +
site/news/spark-3-2-1-released.html | 1 +
site/news/spark-3-2-2-released.html | 1 +
site/news/spark-3-2-3-released.html | 1 +
site/news/spark-3-2-4-released.html | 1 +
site/news/spark-3-3-0-released.html | 1 +
site/news/spark-3-3-1-released.html | 1 +
site/news/spark-3-3-2-released.html | 1 +
site/news/spark-3-3-3-released.html | 1 +
site/news/spark-3-3-4-released.html | 1 +
site/news/spark-3-4-0-released.html | 1 +
site/news/spark-3-4-1-released.html | 1 +
site/news/spark-3-4-2-released.html | 1 +
site/news/spark-3-5-0-released.html | 1 +
site/news/spark-3-5-1-released.html | 1 +
site/news/spark-3.0.0-preview.html | 1 +
site/news/spark-3.0.0-preview2.html | 1 +
.../news/spark-accepted-into-apache-incubator.html | 1 +
.../spark-ai-summit-apr-2019-agenda-posted.html | 1 +
.../spark-ai-summit-june-2020-agenda-posted.html | 1 +
site/news/spark-and-shark-in-the-news.html | 1 +
site/news/spark-becomes-tlp.html | 1 +
site/news/spark-featured-in-wired.html | 1 +
.../news/spark-mailing-lists-moving-to-apache.html | 1 +
site/news/spark-meetups.html | 1 +
site/news/spark-release-2-2-3.html | 1 +
site/news/spark-screencasts-published.html | 1 +
site/news/spark-summit-2013-is-a-wrap.html | 1 +
site/news/spark-summit-2014-videos-posted.html | 1 +
site/news/spark-summit-2015-videos-posted.html | 1 +
site/news/spark-summit-agenda-posted.html | 1 +
.../news/spark-summit-east-2015-videos-posted.html | 1 +
site/news/spark-summit-east-2016-cfp-closing.html | 1 +
.../news/spark-summit-east-2017-agenda-posted.html | 1 +
.../news/spark-summit-east-agenda-posted-2015.html | 1 +
.../news/spark-summit-east-agenda-posted-2016.html | 1 +
site/news/spark-summit-eu-2017-agenda-posted.html | 1 +
site/news/spark-summit-europe-agenda-posted.html | 1 +
site/news/spark-summit-europe.html | 1 +
.../news/spark-summit-june-2016-agenda-posted.html | 1 +
.../news/spark-summit-june-2017-agenda-posted.html | 1 +
.../news/spark-summit-june-2018-agenda-posted.html | 1 +
site/news/spark-summit-oct-2018-agenda-posted.html | 1 +
site/news/spark-tips-from-quantifind.html | 1 +
.../spark-user-survey-and-powered-by-page.html | 1 +
site/news/spark-version-0-6-0-released.html | 1 +
.../news/spark-wins-cloudsort-100tb-benchmark.html | 1 +
...ark-wins-daytona-gray-sort-100tb-benchmark.html | 1 +
.../strata-exercises-now-available-online.html | 1 +
site/news/submit-talks-to-spark-summit-2014.html | 1 +
site/news/submit-talks-to-spark-summit-2016.html | 1 +
.../submit-talks-to-spark-summit-east-2016.html | 1 +
.../news/submit-talks-to-spark-summit-eu-2016.html | 1 +
site/news/two-weeks-to-spark-summit-2014.html | 1 +
.../video-from-first-spark-development-meetup.html | 1 +
site/pandas-on-spark/index.html | 401 +++++++++++++++++++++
site/powered-by.html | 1 +
site/release-process.html | 1 +
site/releases/spark-release-0-3.html | 1 +
site/releases/spark-release-0-5-0.html | 1 +
site/releases/spark-release-0-5-1.html | 1 +
site/releases/spark-release-0-5-2.html | 1 +
site/releases/spark-release-0-6-0.html | 1 +
site/releases/spark-release-0-6-1.html | 1 +
site/releases/spark-release-0-6-2.html | 1 +
site/releases/spark-release-0-7-0.html | 1 +
site/releases/spark-release-0-7-2.html | 1 +
site/releases/spark-release-0-7-3.html | 1 +
site/releases/spark-release-0-8-0.html | 1 +
site/releases/spark-release-0-8-1.html | 1 +
site/releases/spark-release-0-9-0.html | 1 +
site/releases/spark-release-0-9-1.html | 1 +
site/releases/spark-release-0-9-2.html | 1 +
site/releases/spark-release-1-0-0.html | 1 +
site/releases/spark-release-1-0-1.html | 1 +
site/releases/spark-release-1-0-2.html | 1 +
site/releases/spark-release-1-1-0.html | 1 +
site/releases/spark-release-1-1-1.html | 1 +
site/releases/spark-release-1-2-0.html | 1 +
site/releases/spark-release-1-2-1.html | 1 +
site/releases/spark-release-1-2-2.html | 1 +
site/releases/spark-release-1-3-0.html | 1 +
site/releases/spark-release-1-3-1.html | 1 +
site/releases/spark-release-1-4-0.html | 1 +
site/releases/spark-release-1-4-1.html | 1 +
site/releases/spark-release-1-5-0.html | 1 +
site/releases/spark-release-1-5-1.html | 1 +
site/releases/spark-release-1-5-2.html | 1 +
site/releases/spark-release-1-6-0.html | 1 +
site/releases/spark-release-1-6-1.html | 1 +
site/releases/spark-release-1-6-2.html | 1 +
site/releases/spark-release-1-6-3.html | 1 +
site/releases/spark-release-2-0-0.html | 1 +
site/releases/spark-release-2-0-1.html | 1 +
site/releases/spark-release-2-0-2.html | 1 +
site/releases/spark-release-2-1-0.html | 1 +
site/releases/spark-release-2-1-1.html | 1 +
site/releases/spark-release-2-1-2.html | 1 +
site/releases/spark-release-2-1-3.html | 1 +
site/releases/spark-release-2-2-0.html | 1 +
site/releases/spark-release-2-2-1.html | 1 +
site/releases/spark-release-2-2-2.html | 1 +
site/releases/spark-release-2-2-3.html | 1 +
site/releases/spark-release-2-3-0.html | 1 +
site/releases/spark-release-2-3-1.html | 1 +
site/releases/spark-release-2-3-2.html | 1 +
site/releases/spark-release-2-3-3.html | 1 +
site/releases/spark-release-2-3-4.html | 1 +
site/releases/spark-release-2-4-0.html | 1 +
site/releases/spark-release-2-4-1.html | 1 +
site/releases/spark-release-2-4-2.html | 1 +
site/releases/spark-release-2-4-3.html | 1 +
site/releases/spark-release-2-4-4.html | 1 +
site/releases/spark-release-2-4-5.html | 1 +
site/releases/spark-release-2-4-6.html | 1 +
site/releases/spark-release-2-4-7.html | 1 +
site/releases/spark-release-2-4-8.html | 1 +
site/releases/spark-release-3-0-0.html | 1 +
site/releases/spark-release-3-0-1.html | 1 +
site/releases/spark-release-3-0-2.html | 1 +
site/releases/spark-release-3-0-3.html | 1 +
site/releases/spark-release-3-1-1.html | 1 +
site/releases/spark-release-3-1-2.html | 1 +
site/releases/spark-release-3-1-3.html | 1 +
site/releases/spark-release-3-2-0.html | 1 +
site/releases/spark-release-3-2-1.html | 1 +
site/releases/spark-release-3-2-2.html | 1 +
site/releases/spark-release-3-2-3.html | 1 +
site/releases/spark-release-3-2-4.html | 1 +
site/releases/spark-release-3-3-0.html | 1 +
site/releases/spark-release-3-3-1.html | 1 +
site/releases/spark-release-3-3-2.html | 1 +
site/releases/spark-release-3-3-3.html | 1 +
site/releases/spark-release-3-3-4.html | 1 +
site/releases/spark-release-3-4-0.html | 1 +
site/releases/spark-release-3-4-1.html | 1 +
site/releases/spark-release-3-4-2.html | 1 +
site/releases/spark-release-3-5-0.html | 1 +
site/releases/spark-release-3-5-1.html | 1 +
site/research.html | 1 +
site/screencasts/1-first-steps-with-spark.html | 1 +
.../2-spark-documentation-overview.html | 1 +
.../screencasts/3-transformations-and-caching.html | 1 +
site/screencasts/4-a-standalone-job-in-spark.html | 1 +
site/screencasts/index.html | 1 +
site/security.html | 1 +
site/sitemap.xml | 16 +-
site/spark-connect/index.html | 1 +
site/sql/index.html | 1 +
site/streaming/index.html | 1 +
site/third-party-projects.html | 1 +
site/trademarks.html | 1 +
site/versioning-policy.html | 1 +
244 files changed, 846 insertions(+), 6 deletions(-)
diff --git a/_layouts/global.html b/_layouts/global.html
index 19755a7692..7f42ea4b6a 100644
--- a/_layouts/global.html
+++ b/_layouts/global.html
@@ -71,6 +71,7 @@
<li><a class="dropdown-item" href="{{site.baseurl}}/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="{{site.baseurl}}/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="{{site.baseurl}}/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="{{site.baseurl}}/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="{{site.baseurl}}/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="{{site.baseurl}}/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/_layouts/home.html b/_layouts/home.html
index ad8f255b7c..5afc9ea07d 100644
--- a/_layouts/home.html
+++ b/_layouts/home.html
@@ -75,6 +75,7 @@
<li><a class="dropdown-item" href="{{site.baseurl}}/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="{{site.baseurl}}/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="{{site.baseurl}}/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="{{site.baseurl}}/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="{{site.baseurl}}/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="{{site.baseurl}}/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/pandas-on-spark/index.md b/pandas-on-spark/index.md
new file mode 100644
index 0000000000..ee048f66f3
--- /dev/null
+++ b/pandas-on-spark/index.md
@@ -0,0 +1,194 @@
+---
+layout: global
+type: "page singular"
+title: pandas API on Spark
+description: The pandas API on Spark offers the familiarity of pandas with the power of Spark.
+subproject: pandas API on Spark
+---
+
+# pandas API on Spark
+
+This page describes the advantages of the pandas API on Spark (“pandas on Spark”) and when you should use it instead of pandas (or in conjunction with pandas).
+
+pandas on Spark can be much faster than pandas and offers syntax that is familiar to pandas users. It offers the power of Spark with the familiarity of pandas.
+
+Here are the main advantages of pandas on Spark:
+
+* Faster query execution on single-machine workloads (because pandas on Spark uses all available cores, processes queries in parallel, and optimizes queries)
+* pandas on Spark is scalable to multiple machines in a cluster and can process big data
+* pandas on Spark allows queries to be run on larger than memory datasets
+* pandas on Spark provides familiar syntax for pandas users
+
+pandas has many limitations:
+
+* pandas must load all data into memory before running a query which can be slow
+* pandas cannot process datasets that are larger than the available memory on a single machine (because all data must be loaded into memory)
+* pandas computations are run on a single core and don't leverage all available cores of a single machine
+* pandas computations cannot be scaled out to multiple machines
+* pandas doesn't have a query optimizer, so users have to manually code optimizations or suffer from slow code
+
+Let's look at some simple examples to get a better understanding of how pandas on Spark overcomes the limitations of pandas. We’ll also investigate the limitations of pandas on Spark.
+
+At the end of this page, you will see how you can use both pandas and pandas on Spark in conjunction. It’s not an either/or decision - in many situations using both is a great option.
+
+## pandas on Spark example
+
+This section demonstrates how pandas on Spark can run a query on a single file on localhost faster than pandas. pandas on Spark isn’t necessarily faster for all queries, but this example shows when it provides a nice speed-up.
+
+Suppose you have a Parquet file with 9 columns and 1 billion rows of data. Here are the first three rows of the file:
+
+```
++-------+-------+--------------+-------+-------+--------+------+------+---------+
+| id1 | id2 | id3 | id4 | id5 | id6 | v1 | v2 | v3 |
+|-------+-------+--------------+-------+-------+--------+------+------+---------|
+| id016 | id046 | id0000109363 | 88 | 13 | 146094 | 4 | 6 | 18.8377 |
+| id039 | id087 | id0000466766 | 14 | 30 | 111330 | 4 | 14 | 46.7973 |
+| id047 | id098 | id0000307804 | 85 | 23 | 187639 | 3 | 5 | 47.5773 |
++-------+-------+--------------+-------+-------+--------+------+------+---------+
+```
+
+Here is how you can use pandas on Spark to read the file and run a group by aggregation.
+
+```python
+import pyspark.pandas as ps
+
+df = ps.read_parquet("G1_1e9_1e2_0_0.parquet")[
+ ["id1", "id2", "v3"]
+]
+df.query("id1 > 'id098'").groupby("id2").sum().head(3)
+```
+
+This query runs in 62 seconds when executed on a 2020 M1 Macbook with 64 GB of RAM with Spark 3.5.0.
+
+Let's compare this with unoptimized pandas code.
+
+```python
+import pandas as pd
+
+df = pd.read_parquet("G1_1e9_1e2_0_0.parquet")[
+ ["id1", "id2", "v3"]
+]
+df.query("id1 > 'id098'").groupby("id2").sum().head(3)
+```
+
+This query errors out because the machine with 64GB of RAM does not have enough space to store 1 billion rows of data in memory.
+
+Let’s manually add some pandas optimizations to make the query run:
+
+```python
+df = pd.read_parquet(
+ "G1_1e9_1e2_0_0.parquet",
+ columns=["id1", "id2", "v3"],
+ filters=[("id1", ">", "id098")],
+ engine="pyarrow",
+)
+df.query("id1 > 'id098'").groupby("id2").sum().head(3)
+```
+
+This query runs in 275 seconds with pandas 2.2.0.
+
+Coding these optimizations manually with pandas can potentially give the wrong results. Here’s an example of a group by query that’s correct, but with a row-group filtering predicate that’s wrong:
+
+```python
+df = pd.read_parquet(
+ "G1_1e9_1e2_0_0.parquet",
+ columns=["id1", "id2", "v3"],
+ filters=[("id1", "==", "id001")],
+ engine="pyarrow",
+)
+df.query("id1 > 'id098'").groupby("id2").sum().head(3)
+```
+
+This returns the wrong result, even though the group by aggregation logic is right!
+
+With pandas, you need to manually apply column pruning and row-group filtering when reading a Parquet file. With pandas on Spark, the Spark optimizer automatically applies these query enhancements, so you do not need to type them manually.
+
+Let's investigate the advantages of pandas on Spark in more detail.
+
+## Advantages of pandas on Spark
+
+Let's recap the advantages of pandas on Spark:
+
+**Faster query execution**
+
+pandas on Spark can execute queries faster than pandas because it uses all the available cores to parallelize computations and optimizes queries before running them to allow for efficient execution.
+
+pandas computations only run on a single core.
+
+**Scalable to larger-than-memory datasets**
+
+pandas loads data in memory before running queries, so it can only query datasets that fit into memory.
+
+Spark can query datasets that are larger than memory by streaming the data and incrementally running computations.
+
+pandas is notorious for erroring out when dataset sizes grow and Spark doesn't have this limitation.
+
+**Runnable on a cluster of many machines**
+
+Spark can be run on a single machine or distributed to many machines in a cluster.
+
+When Spark is run on a single machine, the computations are run on all available cores. This is often faster than pandas which only runs computations on a single core.
+
+Scaling computations on multiple machines is great for when you want to run computations on larger data sets or simply access more RAM/cores so the queries run faster.
+
+**Familiar syntax for pandas users**
+
+pandas on Spark was designed to provide familiar syntax for pandas users.
+
+The familiar syntax is the whole point - pandas on Spark provides the power of Spark with the same syntax that pandas users are accustomed to.
+
+**Provides access to Spark's battle-tested query optimizer**
+
+pandas on Spark computations are optimized by Spark’s Catalyst optimizer before they're executed.
+
+These optimizations simplify queries and add optimizations.
+
+Earlier in this post, we saw how Spark automatically adds the column pruning/row group filtering optimizations when reading Parquet files for example. pandas doesn't have a query optimizer, so you need to add these optimizations yourself. Manually adding optimizations is tedious and error-prone. If you don't manually apply the right optimizations, your query will return the wrong result.
+
+## Limitations of pandas on Spark
+
+pandas on Spark doesn't support all the APIs that pandas supports for two reasons:
+
+* some features haven't been added to pandas on Spark yet
+
+* some pandas features don't make sense with Spark's distributed, parallel execution model
+
+Spark breaks up DataFrames into multiple chunks so they can be processed in parallel, so certain pandas operations don't transition well to Spark's execution model.
+
+## Using pandas on Spark in conjunction with regular pandas
+
+It’s often useful to use pandas on Spark and pandas to get the best of both worlds.
+
+Suppose you have a large dataset that you clean and aggregate into a smaller dataset that’s passed into a scikit-learn machine learning model.
+
+You can clean and aggregate the dataset with pandas on Spark to take advantage of fast query times and parallel execution. Once the dataset is processed, you can convert it to a pandas DataFrame with `to_pandas()` and then run the machine learning model with scikit-learn. This approach works well if the dataset can be reduced enough to fit in a pandas DataFrame.
+
+## The pandas on Spark query execution model is different
+
+pandas on Spark executes queries completely differently than pandas.
+
+pandas on Spark uses lazy evaluation. It converts the query to an unresolved logical plan, optimizes it with Spark, and only runs computations when results are requested.
+
+pandas uses eager evaluation. It loads all the data into memory and executes operations immediately when they are invoked. pandas does not apply query optimization and all the data must be loaded into memory before the query is executed.
+
+When comparing pandas on Spark and pandas, you must be careful to factor in how much time it takes to load the data into memory and how much time it takes to run the query. A lot of datasets take a long time to load into pandas.
+
+You can also load data into memory with pandas on Spark, but this is often considered an anti-pattern. A dataset that’s loaded into memory won’t be updated if the data in storage changes (via an append, merge, or deletion). Persisting a Spark DataFrame is wise in certain situations to speed up query times, but must be used carefully because it can cause incorrect query results.
+
+## How pandas on Spark and PySpark differ
+
+pandas on Spark and PySpark both take queries, convert them to unresolved logical plans, and then execute them with Spark.
+
+PySpark and pandas on Spark both have similar query execution models. Converting a query to an unresolved logical plan is relatively quick. Optimizing the query and executing it takes much more time. So PySpark and pandas on Spark should have similar performance.
+
+The main difference between pandas on Spark and PySpark is just the syntax.
+
+## Conclusion
+
+pandas on Spark is a great alternative for pandas users who would like to run their queries faster and want to leverage Spark’s optimizer rather than writing their own optimizations.
+
+pandas on Spark uses syntax that’s familiar to pandas users, so it’s easy to learn.
+
+pandas on Spark is also a great technology to be used in conjunction with pandas. You can use the big-data and performant processing capabilities of pandas on Spark to process datasets before they’re converted into pandas DataFrames that are compatible with other technologies.
+
+Check out [the docs](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html) to learn more about how to use pandas on Spark.
diff --git a/site/committers.html b/site/committers.html
index 263645809e..410166ea44 100644
--- a/site/committers.html
+++ b/site/committers.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/community.html b/site/community.html
index 6eb4341169..a4cbd3fabf 100644
--- a/site/community.html
+++ b/site/community.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/contributing.html b/site/contributing.html
index 44de8d6a39..fc68e568f8 100644
--- a/site/contributing.html
+++ b/site/contributing.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/developer-tools.html b/site/developer-tools.html
index d304f823df..f697929920 100644
--- a/site/developer-tools.html
+++ b/site/developer-tools.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/documentation.html b/site/documentation.html
index 5fcfe03410..8ae01a0719 100644
--- a/site/documentation.html
+++ b/site/documentation.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/downloads.html b/site/downloads.html
index 2d37c09998..912fe57ec2 100644
--- a/site/downloads.html
+++ b/site/downloads.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/error-message-guidelines.html b/site/error-message-guidelines.html
index 0fc2ea5ead..2cc8e9fbc1 100644
--- a/site/error-message-guidelines.html
+++ b/site/error-message-guidelines.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/examples.html b/site/examples.html
index 4fe0989abb..0a1d538c36 100644
--- a/site/examples.html
+++ b/site/examples.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/faq.html b/site/faq.html
index 4a3a281d44..5620223bff 100644
--- a/site/faq.html
+++ b/site/faq.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/graphx/index.html b/site/graphx/index.html
index 7c6f71a931..2ae777b92b 100644
--- a/site/graphx/index.html
+++ b/site/graphx/index.html
@@ -67,6 +67,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/history.html b/site/history.html
index 1e08e02073..f28c55813d 100644
--- a/site/history.html
+++ b/site/history.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/improvement-proposals.html b/site/improvement-proposals.html
index 1da30ff3e1..a99815909e 100644
--- a/site/improvement-proposals.html
+++ b/site/improvement-proposals.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/index.html b/site/index.html
index 3e6072f1d7..e45066d1c4 100644
--- a/site/index.html
+++ b/site/index.html
@@ -71,6 +71,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/mailing-lists.html b/site/mailing-lists.html
index 9b93a60294..38683d8f5e 100644
--- a/site/mailing-lists.html
+++ b/site/mailing-lists.html
@@ -68,6 +68,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/mllib/index.html b/site/mllib/index.html
index 0bc67a3037..aaf57ab609 100644
--- a/site/mllib/index.html
+++ b/site/mllib/index.html
@@ -67,6 +67,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/3-1-3-released.html b/site/news/3-1-3-released.html
index 9e89aa6a20..3ceb8efe71 100644
--- a/site/news/3-1-3-released.html
+++ b/site/news/3-1-3-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/amp-camp-2013-registration-ope.html b/site/news/amp-camp-2013-registration-ope.html
index 23b9216a8a..c9ce441728 100644
--- a/site/news/amp-camp-2013-registration-ope.html
+++ b/site/news/amp-camp-2013-registration-ope.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/announcing-the-first-spark-summit.html b/site/news/announcing-the-first-spark-summit.html
index cd08023831..3fb8f5ca4f 100644
--- a/site/news/announcing-the-first-spark-summit.html
+++ b/site/news/announcing-the-first-spark-summit.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/fourth-spark-screencast-published.html b/site/news/fourth-spark-screencast-published.html
index 6bbd9e7443..c1fb0bc619 100644
--- a/site/news/fourth-spark-screencast-published.html
+++ b/site/news/fourth-spark-screencast-published.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/index.html b/site/news/index.html
index 4f892ea059..a7b2e03117 100644
--- a/site/news/index.html
+++ b/site/news/index.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/new-repository-service.html b/site/news/new-repository-service.html
index 8c6a861811..5c617fae7a 100644
--- a/site/news/new-repository-service.html
+++ b/site/news/new-repository-service.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/next-official-release-spark-3.1.1.html b/site/news/next-official-release-spark-3.1.1.html
index ef6b7634a8..aad159e84d 100644
--- a/site/news/next-official-release-spark-3.1.1.html
+++ b/site/news/next-official-release-spark-3.1.1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/nsdi-paper.html b/site/news/nsdi-paper.html
index 5f121244bc..e6c728b680 100644
--- a/site/news/nsdi-paper.html
+++ b/site/news/nsdi-paper.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/one-month-to-spark-summit-2015.html b/site/news/one-month-to-spark-summit-2015.html
index 1d3f6840a1..e7a2902fe0 100644
--- a/site/news/one-month-to-spark-summit-2015.html
+++ b/site/news/one-month-to-spark-summit-2015.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/plan-for-dropping-python-2-support.html b/site/news/plan-for-dropping-python-2-support.html
index 1debae20de..06314ffa22 100644
--- a/site/news/plan-for-dropping-python-2-support.html
+++ b/site/news/plan-for-dropping-python-2-support.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/proposals-open-for-spark-summit-east.html b/site/news/proposals-open-for-spark-summit-east.html
index 7b1d592f36..5c7be904a9 100644
--- a/site/news/proposals-open-for-spark-summit-east.html
+++ b/site/news/proposals-open-for-spark-summit-east.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/registration-open-for-spark-summit-east.html b/site/news/registration-open-for-spark-summit-east.html
index 078433c86f..e0122f3997 100644
--- a/site/news/registration-open-for-spark-summit-east.html
+++ b/site/news/registration-open-for-spark-summit-east.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/run-spark-and-shark-on-amazon-emr.html b/site/news/run-spark-and-shark-on-amazon-emr.html
index 7b48d65797..0a4fda6360 100644
--- a/site/news/run-spark-and-shark-on-amazon-emr.html
+++ b/site/news/run-spark-and-shark-on-amazon-emr.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/sigmod-system-award.html b/site/news/sigmod-system-award.html
index 854b19a4e2..ce898b6da1 100644
--- a/site/news/sigmod-system-award.html
+++ b/site/news/sigmod-system-award.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-0-6-1-and-0-5-2-released.html b/site/news/spark-0-6-1-and-0-5-2-released.html
index fb65793aa9..1cce367ff4 100644
--- a/site/news/spark-0-6-1-and-0-5-2-released.html
+++ b/site/news/spark-0-6-1-and-0-5-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-0-6-2-released.html b/site/news/spark-0-6-2-released.html
index e16adf5627..2d956e56a4 100644
--- a/site/news/spark-0-6-2-released.html
+++ b/site/news/spark-0-6-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-0-7-0-released.html b/site/news/spark-0-7-0-released.html
index a809e05f2e..aca874e9a5 100644
--- a/site/news/spark-0-7-0-released.html
+++ b/site/news/spark-0-7-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-0-7-2-released.html b/site/news/spark-0-7-2-released.html
index 710934a207..255c07903a 100644
--- a/site/news/spark-0-7-2-released.html
+++ b/site/news/spark-0-7-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-0-7-3-released.html b/site/news/spark-0-7-3-released.html
index e4d308f4d9..da371e763e 100644
--- a/site/news/spark-0-7-3-released.html
+++ b/site/news/spark-0-7-3-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-0-8-0-released.html b/site/news/spark-0-8-0-released.html
index 12ec6a1c29..0d6a50e6db 100644
--- a/site/news/spark-0-8-0-released.html
+++ b/site/news/spark-0-8-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-0-8-1-released.html b/site/news/spark-0-8-1-released.html
index 08b92f5a0d..0afc22a515 100644
--- a/site/news/spark-0-8-1-released.html
+++ b/site/news/spark-0-8-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-0-9-0-released.html b/site/news/spark-0-9-0-released.html
index 4e3a601391..f02b1c9eae 100644
--- a/site/news/spark-0-9-0-released.html
+++ b/site/news/spark-0-9-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-0-9-1-released.html b/site/news/spark-0-9-1-released.html
index 469786f67f..88d34bd089 100644
--- a/site/news/spark-0-9-1-released.html
+++ b/site/news/spark-0-9-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-0-9-2-released.html b/site/news/spark-0-9-2-released.html
index 0ad38ae243..1d28df1215 100644
--- a/site/news/spark-0-9-2-released.html
+++ b/site/news/spark-0-9-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-0-0-released.html b/site/news/spark-1-0-0-released.html
index d6bd0bae99..f044fb9921 100644
--- a/site/news/spark-1-0-0-released.html
+++ b/site/news/spark-1-0-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-0-1-released.html b/site/news/spark-1-0-1-released.html
index 060aaed3d8..a062e700c3 100644
--- a/site/news/spark-1-0-1-released.html
+++ b/site/news/spark-1-0-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-0-2-released.html b/site/news/spark-1-0-2-released.html
index e8d3e3a443..bfb6b5d30a 100644
--- a/site/news/spark-1-0-2-released.html
+++ b/site/news/spark-1-0-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-1-0-released.html b/site/news/spark-1-1-0-released.html
index 80bbabc0fc..9a86a92bdf 100644
--- a/site/news/spark-1-1-0-released.html
+++ b/site/news/spark-1-1-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-1-1-released.html b/site/news/spark-1-1-1-released.html
index 92c9ebe1f4..d828cad3c5 100644
--- a/site/news/spark-1-1-1-released.html
+++ b/site/news/spark-1-1-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-2-0-released.html b/site/news/spark-1-2-0-released.html
index b37aeac928..39f697be86 100644
--- a/site/news/spark-1-2-0-released.html
+++ b/site/news/spark-1-2-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-2-1-released.html b/site/news/spark-1-2-1-released.html
index 79e5280547..607dc5971f 100644
--- a/site/news/spark-1-2-1-released.html
+++ b/site/news/spark-1-2-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-2-2-released.html b/site/news/spark-1-2-2-released.html
index 1643f4b1c4..38a3ea567b 100644
--- a/site/news/spark-1-2-2-released.html
+++ b/site/news/spark-1-2-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-3-0-released.html b/site/news/spark-1-3-0-released.html
index a5d2fc22f5..82c978eb50 100644
--- a/site/news/spark-1-3-0-released.html
+++ b/site/news/spark-1-3-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-4-0-released.html b/site/news/spark-1-4-0-released.html
index b95bf9cd2b..dc3f40ce09 100644
--- a/site/news/spark-1-4-0-released.html
+++ b/site/news/spark-1-4-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-4-1-released.html b/site/news/spark-1-4-1-released.html
index 142de79092..741533a609 100644
--- a/site/news/spark-1-4-1-released.html
+++ b/site/news/spark-1-4-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-5-0-released.html b/site/news/spark-1-5-0-released.html
index 112bf2592a..e560686247 100644
--- a/site/news/spark-1-5-0-released.html
+++ b/site/news/spark-1-5-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-5-1-released.html b/site/news/spark-1-5-1-released.html
index c2585cad1c..4a643ca2da 100644
--- a/site/news/spark-1-5-1-released.html
+++ b/site/news/spark-1-5-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-5-2-released.html b/site/news/spark-1-5-2-released.html
index 979a1c1a1e..f4c3031a81 100644
--- a/site/news/spark-1-5-2-released.html
+++ b/site/news/spark-1-5-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-6-0-released.html b/site/news/spark-1-6-0-released.html
index 73f9de442b..dce2b804b8 100644
--- a/site/news/spark-1-6-0-released.html
+++ b/site/news/spark-1-6-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-6-1-released.html b/site/news/spark-1-6-1-released.html
index 0df853a941..4b5a796637 100644
--- a/site/news/spark-1-6-1-released.html
+++ b/site/news/spark-1-6-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-6-2-released.html b/site/news/spark-1-6-2-released.html
index 55a95e10d6..b30ca06b02 100644
--- a/site/news/spark-1-6-2-released.html
+++ b/site/news/spark-1-6-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-1-6-3-released.html b/site/news/spark-1-6-3-released.html
index 3268b7b21f..4e757e2d32 100644
--- a/site/news/spark-1-6-3-released.html
+++ b/site/news/spark-1-6-3-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-0-0-released.html b/site/news/spark-2-0-0-released.html
index 4f55ff4477..665ee93b1e 100644
--- a/site/news/spark-2-0-0-released.html
+++ b/site/news/spark-2-0-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-0-1-released.html b/site/news/spark-2-0-1-released.html
index d66d399001..af5f9eb4ed 100644
--- a/site/news/spark-2-0-1-released.html
+++ b/site/news/spark-2-0-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-0-2-released.html b/site/news/spark-2-0-2-released.html
index 0be4ec0fce..d89364dd1f 100644
--- a/site/news/spark-2-0-2-released.html
+++ b/site/news/spark-2-0-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-1-0-released.html b/site/news/spark-2-1-0-released.html
index 5a18d1b5f6..bf48a2ab98 100644
--- a/site/news/spark-2-1-0-released.html
+++ b/site/news/spark-2-1-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-1-1-released.html b/site/news/spark-2-1-1-released.html
index a53624b1ff..5899d61651 100644
--- a/site/news/spark-2-1-1-released.html
+++ b/site/news/spark-2-1-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-1-2-released.html b/site/news/spark-2-1-2-released.html
index f5e8b04cae..eb460a3afc 100644
--- a/site/news/spark-2-1-2-released.html
+++ b/site/news/spark-2-1-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-1-3-released.html b/site/news/spark-2-1-3-released.html
index d917019064..3de8b9d186 100644
--- a/site/news/spark-2-1-3-released.html
+++ b/site/news/spark-2-1-3-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-2-0-released.html b/site/news/spark-2-2-0-released.html
index fed5ae2af7..256adc4f7c 100644
--- a/site/news/spark-2-2-0-released.html
+++ b/site/news/spark-2-2-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-2-1-released.html b/site/news/spark-2-2-1-released.html
index 259019f6f6..78802aa958 100644
--- a/site/news/spark-2-2-1-released.html
+++ b/site/news/spark-2-2-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-2-2-released.html b/site/news/spark-2-2-2-released.html
index a485cddb4f..2dd567f745 100644
--- a/site/news/spark-2-2-2-released.html
+++ b/site/news/spark-2-2-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-3-0-released.html b/site/news/spark-2-3-0-released.html
index 7fcf8a453f..f3ea01e7ba 100644
--- a/site/news/spark-2-3-0-released.html
+++ b/site/news/spark-2-3-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-3-1-released.html b/site/news/spark-2-3-1-released.html
index 854da3cbb7..5f78e31477 100644
--- a/site/news/spark-2-3-1-released.html
+++ b/site/news/spark-2-3-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-3-2-released.html b/site/news/spark-2-3-2-released.html
index 2c35a8624e..974c4d5aef 100644
--- a/site/news/spark-2-3-2-released.html
+++ b/site/news/spark-2-3-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-3-3-released.html b/site/news/spark-2-3-3-released.html
index b0bb7d005e..de1de02ef3 100644
--- a/site/news/spark-2-3-3-released.html
+++ b/site/news/spark-2-3-3-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-3-4-released.html b/site/news/spark-2-3-4-released.html
index e951ab4dba..c10abb5ce7 100644
--- a/site/news/spark-2-3-4-released.html
+++ b/site/news/spark-2-3-4-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-4-0-released.html b/site/news/spark-2-4-0-released.html
index cb882ecd28..5dc31a2b9d 100644
--- a/site/news/spark-2-4-0-released.html
+++ b/site/news/spark-2-4-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-4-1-released.html b/site/news/spark-2-4-1-released.html
index c24e068ba1..f3370333f5 100644
--- a/site/news/spark-2-4-1-released.html
+++ b/site/news/spark-2-4-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-4-2-released.html b/site/news/spark-2-4-2-released.html
index c8cd1960e1..791f1cf61f 100644
--- a/site/news/spark-2-4-2-released.html
+++ b/site/news/spark-2-4-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-4-3-released.html b/site/news/spark-2-4-3-released.html
index 678ee52073..e3d98b71b6 100644
--- a/site/news/spark-2-4-3-released.html
+++ b/site/news/spark-2-4-3-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-4-4-released.html b/site/news/spark-2-4-4-released.html
index 21bab8cc29..9265b7decd 100644
--- a/site/news/spark-2-4-4-released.html
+++ b/site/news/spark-2-4-4-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-4-5-released.html b/site/news/spark-2-4-5-released.html
index 7f64cfdc42..5168e28b95 100644
--- a/site/news/spark-2-4-5-released.html
+++ b/site/news/spark-2-4-5-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-4-6.html b/site/news/spark-2-4-6.html
index 4a20be0e1b..23a964f88b 100644
--- a/site/news/spark-2-4-6.html
+++ b/site/news/spark-2-4-6.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-4-7-released.html b/site/news/spark-2-4-7-released.html
index ec7115f3ca..a35db6b8a6 100644
--- a/site/news/spark-2-4-7-released.html
+++ b/site/news/spark-2-4-7-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2-4-8-released.html b/site/news/spark-2-4-8-released.html
index 49f08779db..7e04ecdb68 100644
--- a/site/news/spark-2-4-8-released.html
+++ b/site/news/spark-2-4-8-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-2.0.0-preview.html b/site/news/spark-2.0.0-preview.html
index f7f49f2654..bf6783e7b0 100644
--- a/site/news/spark-2.0.0-preview.html
+++ b/site/news/spark-2.0.0-preview.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-0-0-released.html b/site/news/spark-3-0-0-released.html
index 92347cc6ed..4c35ced3ef 100644
--- a/site/news/spark-3-0-0-released.html
+++ b/site/news/spark-3-0-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-0-1-released.html b/site/news/spark-3-0-1-released.html
index 74916788bc..738f0a9853 100644
--- a/site/news/spark-3-0-1-released.html
+++ b/site/news/spark-3-0-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-0-2-released.html b/site/news/spark-3-0-2-released.html
index 1f6dbd739f..a889746cad 100644
--- a/site/news/spark-3-0-2-released.html
+++ b/site/news/spark-3-0-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-0-3-released.html b/site/news/spark-3-0-3-released.html
index f4281f9130..c3b53cb195 100644
--- a/site/news/spark-3-0-3-released.html
+++ b/site/news/spark-3-0-3-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-1-1-released.html b/site/news/spark-3-1-1-released.html
index fca45e6907..f058ce502e 100644
--- a/site/news/spark-3-1-1-released.html
+++ b/site/news/spark-3-1-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-1-2-released.html b/site/news/spark-3-1-2-released.html
index a2218ad506..68b6d26294 100644
--- a/site/news/spark-3-1-2-released.html
+++ b/site/news/spark-3-1-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-2-0-released.html b/site/news/spark-3-2-0-released.html
index 443fac3e30..c467c6cc96 100644
--- a/site/news/spark-3-2-0-released.html
+++ b/site/news/spark-3-2-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-2-1-released.html b/site/news/spark-3-2-1-released.html
index 3199e2f83b..d7018af5cb 100644
--- a/site/news/spark-3-2-1-released.html
+++ b/site/news/spark-3-2-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-2-2-released.html b/site/news/spark-3-2-2-released.html
index c4dab0a205..beecc4fb9d 100644
--- a/site/news/spark-3-2-2-released.html
+++ b/site/news/spark-3-2-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-2-3-released.html b/site/news/spark-3-2-3-released.html
index 4756a8cec8..28f141674c 100644
--- a/site/news/spark-3-2-3-released.html
+++ b/site/news/spark-3-2-3-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-2-4-released.html b/site/news/spark-3-2-4-released.html
index 9e72fcedb4..6074dbe724 100644
--- a/site/news/spark-3-2-4-released.html
+++ b/site/news/spark-3-2-4-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-3-0-released.html b/site/news/spark-3-3-0-released.html
index d94e1d2382..a8b8a10521 100644
--- a/site/news/spark-3-3-0-released.html
+++ b/site/news/spark-3-3-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-3-1-released.html b/site/news/spark-3-3-1-released.html
index ae233e38f4..b94ce58905 100644
--- a/site/news/spark-3-3-1-released.html
+++ b/site/news/spark-3-3-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-3-2-released.html b/site/news/spark-3-3-2-released.html
index b3e08db2da..ba066b0a53 100644
--- a/site/news/spark-3-3-2-released.html
+++ b/site/news/spark-3-3-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-3-3-released.html b/site/news/spark-3-3-3-released.html
index 6ee1dc9ba6..a9ca4c46ec 100644
--- a/site/news/spark-3-3-3-released.html
+++ b/site/news/spark-3-3-3-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-3-4-released.html b/site/news/spark-3-3-4-released.html
index 2107afaa83..11f2f32be8 100644
--- a/site/news/spark-3-3-4-released.html
+++ b/site/news/spark-3-3-4-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-4-0-released.html b/site/news/spark-3-4-0-released.html
index f7c0b65b6e..1ec7a643e3 100644
--- a/site/news/spark-3-4-0-released.html
+++ b/site/news/spark-3-4-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-4-1-released.html b/site/news/spark-3-4-1-released.html
index 339b28ab55..8fe0cd166a 100644
--- a/site/news/spark-3-4-1-released.html
+++ b/site/news/spark-3-4-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-4-2-released.html b/site/news/spark-3-4-2-released.html
index 49befaf8b7..6438ff6142 100644
--- a/site/news/spark-3-4-2-released.html
+++ b/site/news/spark-3-4-2-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-5-0-released.html b/site/news/spark-3-5-0-released.html
index 4b7c108eec..7b871a7ce5 100644
--- a/site/news/spark-3-5-0-released.html
+++ b/site/news/spark-3-5-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3-5-1-released.html b/site/news/spark-3-5-1-released.html
index b02cda54e9..64613d51fc 100644
--- a/site/news/spark-3-5-1-released.html
+++ b/site/news/spark-3-5-1-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3.0.0-preview.html b/site/news/spark-3.0.0-preview.html
index 548ca1aa84..bb0800f99d 100644
--- a/site/news/spark-3.0.0-preview.html
+++ b/site/news/spark-3.0.0-preview.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-3.0.0-preview2.html b/site/news/spark-3.0.0-preview2.html
index 34831003c8..bd491a6310 100644
--- a/site/news/spark-3.0.0-preview2.html
+++ b/site/news/spark-3.0.0-preview2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-accepted-into-apache-incubator.html b/site/news/spark-accepted-into-apache-incubator.html
index bf307b09f3..322a24b6af 100644
--- a/site/news/spark-accepted-into-apache-incubator.html
+++ b/site/news/spark-accepted-into-apache-incubator.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-ai-summit-apr-2019-agenda-posted.html b/site/news/spark-ai-summit-apr-2019-agenda-posted.html
index 0e549a6544..8f504093fb 100644
--- a/site/news/spark-ai-summit-apr-2019-agenda-posted.html
+++ b/site/news/spark-ai-summit-apr-2019-agenda-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-ai-summit-june-2020-agenda-posted.html b/site/news/spark-ai-summit-june-2020-agenda-posted.html
index 6b4bba76e3..8dffe662fc 100644
--- a/site/news/spark-ai-summit-june-2020-agenda-posted.html
+++ b/site/news/spark-ai-summit-june-2020-agenda-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-and-shark-in-the-news.html b/site/news/spark-and-shark-in-the-news.html
index 692b4f7aad..001a67404a 100644
--- a/site/news/spark-and-shark-in-the-news.html
+++ b/site/news/spark-and-shark-in-the-news.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-becomes-tlp.html b/site/news/spark-becomes-tlp.html
index f9cbf038c4..faf16edafb 100644
--- a/site/news/spark-becomes-tlp.html
+++ b/site/news/spark-becomes-tlp.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-featured-in-wired.html b/site/news/spark-featured-in-wired.html
index 2840c5c2e5..0a2c41fbaa 100644
--- a/site/news/spark-featured-in-wired.html
+++ b/site/news/spark-featured-in-wired.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-mailing-lists-moving-to-apache.html b/site/news/spark-mailing-lists-moving-to-apache.html
index ab24423eb6..5869ebef36 100644
--- a/site/news/spark-mailing-lists-moving-to-apache.html
+++ b/site/news/spark-mailing-lists-moving-to-apache.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-meetups.html b/site/news/spark-meetups.html
index 28ecf5c533..b5024adec1 100644
--- a/site/news/spark-meetups.html
+++ b/site/news/spark-meetups.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-release-2-2-3.html b/site/news/spark-release-2-2-3.html
index 3942414f86..a57a5ee8c7 100644
--- a/site/news/spark-release-2-2-3.html
+++ b/site/news/spark-release-2-2-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-screencasts-published.html b/site/news/spark-screencasts-published.html
index 158919e52d..c39949627b 100644
--- a/site/news/spark-screencasts-published.html
+++ b/site/news/spark-screencasts-published.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-2013-is-a-wrap.html b/site/news/spark-summit-2013-is-a-wrap.html
index 5698a2b888..bc414f0385 100644
--- a/site/news/spark-summit-2013-is-a-wrap.html
+++ b/site/news/spark-summit-2013-is-a-wrap.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-2014-videos-posted.html b/site/news/spark-summit-2014-videos-posted.html
index 84781b5907..7d9ce9b513 100644
--- a/site/news/spark-summit-2014-videos-posted.html
+++ b/site/news/spark-summit-2014-videos-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-2015-videos-posted.html b/site/news/spark-summit-2015-videos-posted.html
index d487ea7694..3c86108137 100644
--- a/site/news/spark-summit-2015-videos-posted.html
+++ b/site/news/spark-summit-2015-videos-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-agenda-posted.html b/site/news/spark-summit-agenda-posted.html
index 2527352ec6..8e2d45b893 100644
--- a/site/news/spark-summit-agenda-posted.html
+++ b/site/news/spark-summit-agenda-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-east-2015-videos-posted.html b/site/news/spark-summit-east-2015-videos-posted.html
index 54fd9258b4..ca340c9133 100644
--- a/site/news/spark-summit-east-2015-videos-posted.html
+++ b/site/news/spark-summit-east-2015-videos-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-east-2016-cfp-closing.html b/site/news/spark-summit-east-2016-cfp-closing.html
index 8d72675427..fe8586bc52 100644
--- a/site/news/spark-summit-east-2016-cfp-closing.html
+++ b/site/news/spark-summit-east-2016-cfp-closing.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-east-2017-agenda-posted.html b/site/news/spark-summit-east-2017-agenda-posted.html
index 7c1ff0ef30..0a15ada5fe 100644
--- a/site/news/spark-summit-east-2017-agenda-posted.html
+++ b/site/news/spark-summit-east-2017-agenda-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-east-agenda-posted-2015.html b/site/news/spark-summit-east-agenda-posted-2015.html
index 5ee713d8e8..3b266411ee 100644
--- a/site/news/spark-summit-east-agenda-posted-2015.html
+++ b/site/news/spark-summit-east-agenda-posted-2015.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-east-agenda-posted-2016.html b/site/news/spark-summit-east-agenda-posted-2016.html
index 631c65815d..e5bb2ae1e8 100644
--- a/site/news/spark-summit-east-agenda-posted-2016.html
+++ b/site/news/spark-summit-east-agenda-posted-2016.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-eu-2017-agenda-posted.html b/site/news/spark-summit-eu-2017-agenda-posted.html
index 7a7dfaaadf..e5681cd51f 100644
--- a/site/news/spark-summit-eu-2017-agenda-posted.html
+++ b/site/news/spark-summit-eu-2017-agenda-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-europe-agenda-posted.html b/site/news/spark-summit-europe-agenda-posted.html
index 96ba51e6fc..1d5a8b106b 100644
--- a/site/news/spark-summit-europe-agenda-posted.html
+++ b/site/news/spark-summit-europe-agenda-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-europe.html b/site/news/spark-summit-europe.html
index fe67bc1260..67f5660930 100644
--- a/site/news/spark-summit-europe.html
+++ b/site/news/spark-summit-europe.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-june-2016-agenda-posted.html b/site/news/spark-summit-june-2016-agenda-posted.html
index a75ce7990b..40f91c8fc9 100644
--- a/site/news/spark-summit-june-2016-agenda-posted.html
+++ b/site/news/spark-summit-june-2016-agenda-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-june-2017-agenda-posted.html b/site/news/spark-summit-june-2017-agenda-posted.html
index d6cea91256..10dd6d6bac 100644
--- a/site/news/spark-summit-june-2017-agenda-posted.html
+++ b/site/news/spark-summit-june-2017-agenda-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-june-2018-agenda-posted.html b/site/news/spark-summit-june-2018-agenda-posted.html
index d0bff0eda7..3d052e9124 100644
--- a/site/news/spark-summit-june-2018-agenda-posted.html
+++ b/site/news/spark-summit-june-2018-agenda-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-summit-oct-2018-agenda-posted.html b/site/news/spark-summit-oct-2018-agenda-posted.html
index 015cb33293..7c9ead54ae 100644
--- a/site/news/spark-summit-oct-2018-agenda-posted.html
+++ b/site/news/spark-summit-oct-2018-agenda-posted.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-tips-from-quantifind.html b/site/news/spark-tips-from-quantifind.html
index a35abdac90..9ddea2f724 100644
--- a/site/news/spark-tips-from-quantifind.html
+++ b/site/news/spark-tips-from-quantifind.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-user-survey-and-powered-by-page.html b/site/news/spark-user-survey-and-powered-by-page.html
index 25382a600e..85b5f6c645 100644
--- a/site/news/spark-user-survey-and-powered-by-page.html
+++ b/site/news/spark-user-survey-and-powered-by-page.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-version-0-6-0-released.html b/site/news/spark-version-0-6-0-released.html
index ab6776c3f7..40d16f4d91 100644
--- a/site/news/spark-version-0-6-0-released.html
+++ b/site/news/spark-version-0-6-0-released.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-wins-cloudsort-100tb-benchmark.html b/site/news/spark-wins-cloudsort-100tb-benchmark.html
index 7ad1d12d45..8fdf36d27b 100644
--- a/site/news/spark-wins-cloudsort-100tb-benchmark.html
+++ b/site/news/spark-wins-cloudsort-100tb-benchmark.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/spark-wins-daytona-gray-sort-100tb-benchmark.html b/site/news/spark-wins-daytona-gray-sort-100tb-benchmark.html
index 9574392a8f..a85cb0ed91 100644
--- a/site/news/spark-wins-daytona-gray-sort-100tb-benchmark.html
+++ b/site/news/spark-wins-daytona-gray-sort-100tb-benchmark.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/strata-exercises-now-available-online.html b/site/news/strata-exercises-now-available-online.html
index bc2dd0918c..fc7ee02c16 100644
--- a/site/news/strata-exercises-now-available-online.html
+++ b/site/news/strata-exercises-now-available-online.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/submit-talks-to-spark-summit-2014.html b/site/news/submit-talks-to-spark-summit-2014.html
index 22b419e1dd..62cfe7e30f 100644
--- a/site/news/submit-talks-to-spark-summit-2014.html
+++ b/site/news/submit-talks-to-spark-summit-2014.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/submit-talks-to-spark-summit-2016.html b/site/news/submit-talks-to-spark-summit-2016.html
index ec58c5b777..850ca2eb0a 100644
--- a/site/news/submit-talks-to-spark-summit-2016.html
+++ b/site/news/submit-talks-to-spark-summit-2016.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/submit-talks-to-spark-summit-east-2016.html b/site/news/submit-talks-to-spark-summit-east-2016.html
index c5bd0e16a7..91d52c0ea4 100644
--- a/site/news/submit-talks-to-spark-summit-east-2016.html
+++ b/site/news/submit-talks-to-spark-summit-east-2016.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/submit-talks-to-spark-summit-eu-2016.html b/site/news/submit-talks-to-spark-summit-eu-2016.html
index 04fff66f65..0c352a31e1 100644
--- a/site/news/submit-talks-to-spark-summit-eu-2016.html
+++ b/site/news/submit-talks-to-spark-summit-eu-2016.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/two-weeks-to-spark-summit-2014.html b/site/news/two-weeks-to-spark-summit-2014.html
index 395ec8a0bb..e4e99931b4 100644
--- a/site/news/two-weeks-to-spark-summit-2014.html
+++ b/site/news/two-weeks-to-spark-summit-2014.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/news/video-from-first-spark-development-meetup.html b/site/news/video-from-first-spark-development-meetup.html
index 138fb38aa5..3ba8ba3d46 100644
--- a/site/news/video-from-first-spark-development-meetup.html
+++ b/site/news/video-from-first-spark-development-meetup.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/pandas-on-spark/index.html b/site/pandas-on-spark/index.html
new file mode 100644
index 0000000000..736a730d66
--- /dev/null
+++ b/site/pandas-on-spark/index.html
@@ -0,0 +1,401 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <title>
+ pandas API on Spark | Apache Spark
+
+ </title>
+
+
+
+
+ <meta name="description" content="The pandas API on Spark offers the familiarity of pandas with the power of Spark.">
+
+
+ <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/css/bootstrap.min.css" rel="stylesheet"
+ integrity="sha384-EVSTQN3/azprG1Anm3QDgpJLIm9Nao0Yz1ztcQTwFspd3yD65VohhpuuCOmLASjC" crossorigin="anonymous">
+ <link rel="preconnect" href="https://fonts.googleapis.com">
+ <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
+ <link href="https://fonts.googleapis.com/css2?family=DM+Sans:ital,wght@0,400;0,500;0,700;1,400;1,500;1,700&Courier+Prime:wght@400;700&display=swap" rel="stylesheet">
+ <link href="/css/custom.css" rel="stylesheet">
+ <!-- Code highlighter CSS -->
+ <link href="/css/pygments-default.css" rel="stylesheet">
+ <link rel="icon" href="/favicon.ico" type="image/x-icon">
+
+ <!-- Matomo -->
+ <script>
+ var _paq = window._paq = window._paq || [];
+ /* tracker methods like "setCustomDimension" should be called before "trackPageView" */
+ _paq.push(["disableCookies"]);
+ _paq.push(['trackPageView']);
+ _paq.push(['enableLinkTracking']);
+ (function() {
+ var u="https://analytics.apache.org/";
+ _paq.push(['setTrackerUrl', u+'matomo.php']);
+ _paq.push(['setSiteId', '40']);
+ var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
+ g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
+ })();
+ </script>
+ <!-- End Matomo Code -->
+</head>
+<body class="global">
+<nav class="navbar navbar-expand-lg navbar-dark p-0 px-4" style="background: #1D6890;">
+ <a class="navbar-brand" href="/">
+ <img src="/images/spark-logo-rev.svg" alt="" width="141" height="72">
+ </a>
+ <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarContent"
+ aria-controls="navbarContent" aria-expanded="false" aria-label="Toggle navigation">
+ <span class="navbar-toggler-icon"></span>
+ </button>
+ <div class="collapse navbar-collapse col-md-12 col-lg-auto pt-4" id="navbarContent">
+
+ <ul class="navbar-nav me-auto">
+ <li class="nav-item">
+ <a class="nav-link active" aria-current="page" href="/downloads.html">Download</a>
+ </li>
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="#" id="libraries" role="button" data-bs-toggle="dropdown"
+ aria-expanded="false">
+ Libraries
+ </a>
+ <ul class="dropdown-menu" aria-labelledby="libraries">
+ <li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
+ <li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
+ <li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
+ <li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
+ <li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
+ <li>
+ <hr class="dropdown-divider">
+ </li>
+ <li><a class="dropdown-item" href="/third-party-projects.html">Third-Party Projects</a></li>
+ </ul>
+ </li>
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="#" id="documentation" role="button" data-bs-toggle="dropdown"
+ aria-expanded="false">
+ Documentation
+ </a>
+ <ul class="dropdown-menu" aria-labelledby="documentation">
+ <li><a class="dropdown-item" href="/docs/latest/">Latest Release</a></li>
+ <li><a class="dropdown-item" href="/documentation.html">Older Versions and Other Resources</a></li>
+ <li><a class="dropdown-item" href="/faq.html">Frequently Asked Questions</a></li>
+ </ul>
+ </li>
+ <li class="nav-item">
+ <a class="nav-link active" aria-current="page" href="/examples.html">Examples</a>
+ </li>
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="#" id="community" role="button" data-bs-toggle="dropdown"
+ aria-expanded="false">
+ Community
+ </a>
+ <ul class="dropdown-menu" aria-labelledby="community">
+ <li><a class="dropdown-item" href="/community.html">Mailing Lists & Resources</a></li>
+ <li><a class="dropdown-item" href="/contributing.html">Contributing to Spark</a></li>
+ <li><a class="dropdown-item" href="/improvement-proposals.html">Improvement Proposals (SPIP)</a>
+ </li>
+ <li><a class="dropdown-item" href="https://issues.apache.org/jira/browse/SPARK">Issue Tracker</a>
+ </li>
+ <li><a class="dropdown-item" href="/powered-by.html">Powered By</a></li>
+ <li><a class="dropdown-item" href="/committers.html">Project Committers</a></li>
+ <li><a class="dropdown-item" href="/history.html">Project History</a></li>
+ </ul>
+ </li>
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="#" id="developers" role="button" data-bs-toggle="dropdown"
+ aria-expanded="false">
+ Developers
+ </a>
+ <ul class="dropdown-menu" aria-labelledby="developers">
+ <li><a class="dropdown-item" href="/developer-tools.html">Useful Developer Tools</a></li>
+ <li><a class="dropdown-item" href="/versioning-policy.html">Versioning Policy</a></li>
+ <li><a class="dropdown-item" href="/release-process.html">Release Process</a></li>
+ <li><a class="dropdown-item" href="/security.html">Security</a></li>
+ </ul>
+ </li>
+ </ul>
+ <ul class="navbar-nav ml-auto">
+ <li class="nav-item dropdown">
+ <a class="nav-link dropdown-toggle" href="#" id="apacheFoundation" role="button"
+ data-bs-toggle="dropdown" aria-expanded="false">
+ Apache Software Foundation
+ </a>
+ <ul class="dropdown-menu" aria-labelledby="apacheFoundation">
+ <li><a class="dropdown-item" href="https://www.apache.org/">Apache Homepage</a></li>
+ <li><a class="dropdown-item" href="https://www.apache.org/licenses/">License</a></li>
+ <li><a class="dropdown-item"
+ href="https://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a class="dropdown-item" href="https://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a class="dropdown-item" href="https://www.apache.org/security/">Security</a></li>
+ <li><a class="dropdown-item" href="https://www.apache.org/events/current-event">Event</a></li>
+ </ul>
+ </li>
+ </ul>
+ </div>
+</nav>
+
+<div class="container">
+ <div class="row mt-4">
+ <div class="col-12 col-md-9">
+ <h1 id="pandas-api-on-spark">pandas API on Spark</h1>
+
+<p>This page describes the advantages of the pandas API on Spark (“pandas on Spark”) and when you should use it instead of pandas (or in conjunction with pandas).</p>
+
+<p>pandas on Spark can be much faster than pandas and offers syntax that is familiar to pandas users. It offers the power of Spark with the familiarity of pandas.</p>
+
+<p>Here are the main advantages of pandas on Spark:</p>
+
+<ul>
+ <li>Faster query execution on single-machine workloads (because pandas on Spark uses all available cores, processes queries in parallel, and optimizes queries)</li>
+ <li>pandas on Spark is scalable to multiple machines in a cluster and can process big data</li>
+ <li>pandas on Spark allows queries to be run on larger than memory datasets</li>
+ <li>pandas on Spark provides familiar syntax for pandas users</li>
+</ul>
+
+<p>pandas has many limitations:</p>
+
+<ul>
+ <li>pandas must load all data into memory before running a query which can be slow</li>
+ <li>pandas cannot process datasets that are larger than the available memory on a single machine (because all data must be loaded into memory)</li>
+ <li>pandas computations are run on a single core and don’t leverage all available cores of a single machine</li>
+ <li>pandas computations cannot be scaled out to multiple machines</li>
+ <li>pandas doesn’t have a query optimizer, so users have to manually code optimizations or suffer from slow code</li>
+</ul>
+
+<p>Let’s look at some simple examples to get a better understanding of how pandas on Spark overcomes the limitations of pandas. We’ll also investigate the limitations of pandas on Spark.</p>
+
+<p>At the end of this page, you will see how you can use both pandas and pandas on Spark in conjunction. It’s not an either/or decision - in many situations using both is a great option.</p>
+
+<h2 id="pandas-on-spark-example">pandas on Spark example</h2>
+
+<p>This section demonstrates how pandas on Spark can run a query on a single file on localhost faster than pandas. pandas on Spark isn’t necessarily faster for all queries, but this example shows when it provides a nice speed-up.</p>
+
+<p>Suppose you have a Parquet file with 9 columns and 1 billion rows of data. Here are the first three rows of the file:</p>
+
+<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>+-------+-------+--------------+-------+-------+--------+------+------+---------+
+| id1 | id2 | id3 | id4 | id5 | id6 | v1 | v2 | v3 |
+|-------+-------+--------------+-------+-------+--------+------+------+---------|
+| id016 | id046 | id0000109363 | 88 | 13 | 146094 | 4 | 6 | 18.8377 |
+| id039 | id087 | id0000466766 | 14 | 30 | 111330 | 4 | 14 | 46.7973 |
+| id047 | id098 | id0000307804 | 85 | 23 | 187639 | 3 | 5 | 47.5773 |
++-------+-------+--------------+-------+-------+--------+------+------+---------+
+</code></pre></div></div>
+
+<p>Here is how you can use pandas on Spark to read the file and run a group by aggregation.</p>
+
+<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">pyspark.pandas</span> <span class="k">as</span> <span class="n">ps</span>
+
+<span class="n">df</span> <span class="o">=</span> <span class="n">ps</span><span class="p">.</span><span class="n">read_parquet</span><span class="p">(</span><span class="s">"G1_1e9_1e2_0_0.parquet"</span><span class="p">)[</span>
+ <span class="p">[</span><span class="s">"id1"</span><span class="p">,</span> <span class="s">"id2"</span><span class="p">,</span> <span class="s">"v3"</span><span class="p">]</span>
+<span class="p">]</span>
+<span class="n">df</span><span class="p">.</span><span class="n">query</span><span class="p">(</span><span class="s">"id1 > 'id098'"</span><span class="p">).</span><span class="n">groupby</span><span class="p">(</span><span class="s">"id2"</span><span class="p">).</span><span class="nb">sum</span><span class="p">().</span><span class="n">head</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
+</code></pre></div></div>
+
+<p>This query runs in 62 seconds when executed on a 2020 M1 Macbook with 64 GB of RAM with Spark 3.5.0.</p>
+
+<p>Let’s compare this with unoptimized pandas code.</p>
+
+<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="n">pd</span>
+
+<span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="p">.</span><span class="n">read_parquet</span><span class="p">(</span><span class="s">"G1_1e9_1e2_0_0.parquet"</span><span class="p">)[</span>
+ <span class="p">[</span><span class="s">"id1"</span><span class="p">,</span> <span class="s">"id2"</span><span class="p">,</span> <span class="s">"v3"</span><span class="p">]</span>
+<span class="p">]</span>
+<span class="n">df</span><span class="p">.</span><span class="n">query</span><span class="p">(</span><span class="s">"id1 > 'id098'"</span><span class="p">).</span><span class="n">groupby</span><span class="p">(</span><span class="s">"id2"</span><span class="p">).</span><span class="nb">sum</span><span class="p">().</span><span class="n">head</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
+</code></pre></div></div>
+
+<p>This query errors out because the machine with 64GB of RAM does not have enough space to store 1 billion rows of data in memory.</p>
+
+<p>Let’s manually add some pandas optimizations to make the query run:</p>
+
+<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="p">.</span><span class="n">read_parquet</span><span class="p">(</span>
+ <span class="s">"G1_1e9_1e2_0_0.parquet"</span><span class="p">,</span>
+ <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="s">"id1"</span><span class="p">,</span> <span class="s">"id2"</span><span class="p">,</span> <span class="s">"v3"</span><span class="p">],</span>
+ <span class="n">filters</span><span class="o">=</span><span class="p">[(</span><span class="s">"id1"</span><span class="p">,</span> <span class="s">">"</span><span class="p">,</span> <span class="s">"id098"</span><span class="p">)],</span>
+ <span class="n">engine</span><span class="o">=</span><span class="s">"pyarrow"</span><span class="p">,</span>
+<span class="p">)</span>
+<span class="n">df</span><span class="p">.</span><span class="n">query</span><span class="p">(</span><span class="s">"id1 > 'id098'"</span><span class="p">).</span><span class="n">groupby</span><span class="p">(</span><span class="s">"id2"</span><span class="p">).</span><span class="nb">sum</span><span class="p">().</span><span class="n">head</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
+</code></pre></div></div>
+
+<p>This query runs in 275 seconds with pandas 2.2.0.</p>
+
+<p>Coding these optimizations manually with pandas can potentially give the wrong results. Here’s an example of a group by query that’s correct, but with a row-group filtering predicate that’s wrong:</p>
+
+<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="p">.</span><span class="n">read_parquet</span><span class="p">(</span>
+ <span class="s">"G1_1e9_1e2_0_0.parquet"</span><span class="p">,</span>
+ <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="s">"id1"</span><span class="p">,</span> <span class="s">"id2"</span><span class="p">,</span> <span class="s">"v3"</span><span class="p">],</span>
+ <span class="n">filters</span><span class="o">=</span><span class="p">[(</span><span class="s">"id1"</span><span class="p">,</span> <span class="s">"=="</span><span class="p">,</span> <span class="s">"id001"</span><span class="p">)],</span>
+ <span class="n">engine</span><span class="o">=</span><span class="s">"pyarrow"</span><span class="p">,</span>
+<span class="p">)</span>
+<span class="n">df</span><span class="p">.</span><span class="n">query</span><span class="p">(</span><span class="s">"id1 > 'id098'"</span><span class="p">).</span><span class="n">groupby</span><span class="p">(</span><span class="s">"id2"</span><span class="p">).</span><span class="nb">sum</span><span class="p">().</span><span class="n">head</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
+</code></pre></div></div>
+
+<p>This returns the wrong result, even though the group by aggregation logic is right!</p>
+
+<p>With pandas, you need to manually apply column pruning and row-group filtering when reading a Parquet file. With pandas on Spark, the Spark optimizer automatically applies these query enhancements, so you do not need to type them manually.</p>
+
+<p>Let’s investigate the advantages of pandas on Spark in more detail.</p>
+
+<h2 id="advantages-of-pandas-on-spark">Advantages of pandas on Spark</h2>
+
+<p>Let’s recap the advantages of pandas on Spark:</p>
+
+<p><strong>Faster query execution</strong></p>
+
+<p>pandas on Spark can execute queries faster than pandas because it uses all the available cores to parallelize computations and optimizes queries before running them to allow for efficient execution.</p>
+
+<p>pandas computations only run on a single core.</p>
+
+<p><strong>Scalable to larger-than-memory datasets</strong></p>
+
+<p>pandas loads data in memory before running queries, so it can only query datasets that fit into memory.</p>
+
+<p>Spark can query datasets that are larger than memory by streaming the data and incrementally running computations.</p>
+
+<p>pandas is notorious for erroring out when dataset sizes grow and Spark doesn’t have this limitation.</p>
+
+<p><strong>Runnable on a cluster of many machines</strong></p>
+
+<p>Spark can be run on a single machine or distributed to many machines in a cluster.</p>
+
+<p>When Spark is run on a single machine, the computations are run on all available cores. This is often faster than pandas which only runs computations on a single core.</p>
+
+<p>Scaling computations on multiple machines is great for when you want to run computations on larger data sets or simply access more RAM/cores so the queries run faster.</p>
+
+<p><strong>Familiar syntax for pandas users</strong></p>
+
+<p>pandas on Spark was designed to provide familiar syntax for pandas users.</p>
+
+<p>The familiar syntax is the whole point - pandas on Spark provides the power of Spark with the same syntax that pandas users are accustomed to.</p>
+
+<p><strong>Provides access to Spark’s battle-tested query optimizer</strong></p>
+
+<p>pandas on Spark computations are optimized by Spark’s Catalyst optimizer before they’re executed.</p>
+
+<p>These optimizations simplify queries and add optimizations.</p>
+
+<p>Earlier in this post, we saw how Spark automatically adds the column pruning/row group filtering optimizations when reading Parquet files for example. pandas doesn’t have a query optimizer, so you need to add these optimizations yourself. Manually adding optimizations is tedious and error-prone. If you don’t manually apply the right optimizations, your query will return the wrong result.</p>
+
+<h2 id="limitations-of-pandas-on-spark">Limitations of pandas on Spark</h2>
+
+<p>pandas on Spark doesn’t support all the APIs that pandas supports for two reasons:</p>
+
+<ul>
+ <li>
+ <p>some features haven’t been added to pandas on Spark yet</p>
+ </li>
+ <li>
+ <p>some pandas features don’t make sense with Spark’s distributed, parallel execution model</p>
+ </li>
+</ul>
+
+<p>Spark breaks up DataFrames into multiple chunks so they can be processed in parallel, so certain pandas operations don’t transition well to Spark’s execution model.</p>
+
+<h2 id="using-pandas-on-spark-in-conjunction-with-regular-pandas">Using pandas on Spark in conjunction with regular pandas</h2>
+
+<p>It’s often useful to use pandas on Spark and pandas to get the best of both worlds.</p>
+
+<p>Suppose you have a large dataset that you clean and aggregate into a smaller dataset that’s passed into a scikit-learn machine learning model.</p>
+
+<p>You can clean and aggregate the dataset with pandas on Spark to take advantage of fast query times and parallel execution. Once the dataset is processed, you can convert it to a pandas DataFrame with <code class="language-plaintext highlighter-rouge">to_pandas()</code> and then run the machine learning model with scikit-learn. This approach works well if the dataset can be reduced enough to fit in a pandas DataFrame.</p>
+
+<h2 id="the-pandas-on-spark-query-execution-model-is-different">The pandas on Spark query execution model is different</h2>
+
+<p>pandas on Spark executes queries completely differently than pandas.</p>
+
+<p>pandas on Spark uses lazy evaluation. It converts the query to an unresolved logical plan, optimizes it with Spark, and only runs computations when results are requested.</p>
+
+<p>pandas uses eager evaluation. It loads all the data into memory and executes operations immediately when they are invoked. pandas does not apply query optimization and all the data must be loaded into memory before the query is executed.</p>
+
+<p>When comparing pandas on Spark and pandas, you must be careful to factor in how much time it takes to load the data into memory and how much time it takes to run the query. A lot of datasets take a long time to load into pandas.</p>
+
+<p>You can also load data into memory with pandas on Spark, but this is often considered an anti-pattern. A dataset that’s loaded into memory won’t be updated if the data in storage changes (via an append, merge, or deletion). Persisting a Spark DataFrame is wise in certain situations to speed up query times, but must be used carefully because it can cause incorrect query results.</p>
+
+<h2 id="how-pandas-on-spark-and-pyspark-differ">How pandas on Spark and PySpark differ</h2>
+
+<p>pandas on Spark and PySpark both take queries, convert them to unresolved logical plans, and then execute them with Spark.</p>
+
+<p>PySpark and pandas on Spark both have similar query execution models. Converting a query to an unresolved logical plan is relatively quick. Optimizing the query and executing it takes much more time. So PySpark and pandas on Spark should have similar performance.</p>
+
+<p>The main difference between pandas on Spark and PySpark is just the syntax.</p>
+
+<h2 id="conclusion">Conclusion</h2>
+
+<p>pandas on Spark is a great alternative for pandas users who would like to run their queries faster and want to leverage Spark’s optimizer rather than writing their own optimizations.</p>
+
+<p>pandas on Spark uses syntax that’s familiar to pandas users, so it’s easy to learn.</p>
+
+<p>pandas on Spark is also a great technology to be used in conjunction with pandas. You can use the big-data and performant processing capabilities of pandas on Spark to process datasets before they’re converted into pandas DataFrames that are compatible with other technologies.</p>
+
+<p>Check out <a href="https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html">the docs</a> to learn more about how to use pandas on Spark.</p>
+
+ </div>
+ <div class="col-12 col-md-3">
+ <div class="news" style="margin-bottom: 20px;">
+ <h5>Latest News</h5>
+ <ul class="list-unstyled">
+
+ <li><a href="/news/spark-3-5-1-released.html">Spark 3.5.1 released</a>
+ <span class="small">(Feb 23, 2024)</span></li>
+
+ <li><a href="/news/spark-3-3-4-released.html">Spark 3.3.4 released</a>
+ <span class="small">(Dec 16, 2023)</span></li>
+
+ <li><a href="/news/spark-3-4-2-released.html">Spark 3.4.2 released</a>
+ <span class="small">(Nov 30, 2023)</span></li>
+
+ <li><a href="/news/spark-3-5-0-released.html">Spark 3.5.0 released</a>
+ <span class="small">(Sep 13, 2023)</span></li>
+
+ </ul>
+ <p class="small" style="text-align: right;"><a href="/news/index.html">Archive</a></p>
+ </div>
+ <div style="text-align:center; margin-bottom: 20px;">
+ <a href="https://www.apache.org/events/current-event.html">
+ <img src="https://www.apache.org/events/current-event-234x60.png" style="max-width: 100%;"/>
+ </a>
+ </div>
+ <div class="hidden-xs hidden-sm">
+ <a href="/downloads.html" class="btn btn-cta btn-lg d-grid" style="margin-bottom: 30px;">
+ Download Spark
+ </a>
+ <p style="font-size: 16px; font-weight: 500; color: #555;">
+ Built-in Libraries:
+ </p>
+ <ul class="list-none">
+ <li><a href="/sql/">SQL and DataFrames</a></li>
+ <li><a href="/streaming/">Spark Streaming</a></li>
+ <li><a href="/mllib/">MLlib (machine learning)</a></li>
+ <li><a href="/graphx/">GraphX (graph)</a></li>
+ </ul>
+ <a href="/third-party-projects.html">Third-Party Projects</a>
+ </div>
+ </div>
+ </div>
+
+
+
+ <footer class="small">
+ <hr>
+ Apache Spark, Spark, Apache, the Apache feather logo, and the Apache Spark project logo are either registered
+ trademarks or trademarks of The Apache Software Foundation in the United States and other countries.
+ See guidance on use of Apache Spark <a href="/trademarks.html">trademarks</a>.
+ All other marks mentioned may be trademarks or registered trademarks of their respective owners.
+ Copyright © 2018 The Apache Software Foundation, Licensed under the
+ <a href="https://www.apache.org/licenses/">Apache License, Version 2.0</a>.
+ </footer>
+</div>
+
+<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/js/bootstrap.bundle.min.js"
+ integrity="sha384-MrcW6ZMFYlzcLA8Nl+NtUVF0sA7MsXsP1UyJoMp4YLEuNSfAP+JcXn/tWtIaxVXM"
+ crossorigin="anonymous"></script>
+<script src="https://code.jquery.com/jquery.js"></script>
+<script src="/js/lang-tabs.js"></script>
+<script src="/js/downloads.js"></script>
+</body>
+</html>
diff --git a/site/powered-by.html b/site/powered-by.html
index 10d756f5da..60504d4bae 100644
--- a/site/powered-by.html
+++ b/site/powered-by.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/release-process.html b/site/release-process.html
index 6e08a40193..35dce8696f 100644
--- a/site/release-process.html
+++ b/site/release-process.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-3.html b/site/releases/spark-release-0-3.html
index aacc655011..46a557de58 100644
--- a/site/releases/spark-release-0-3.html
+++ b/site/releases/spark-release-0-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-5-0.html b/site/releases/spark-release-0-5-0.html
index c2032a71c3..680ca70563 100644
--- a/site/releases/spark-release-0-5-0.html
+++ b/site/releases/spark-release-0-5-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-5-1.html b/site/releases/spark-release-0-5-1.html
index 1741f127ef..a9964e82e2 100644
--- a/site/releases/spark-release-0-5-1.html
+++ b/site/releases/spark-release-0-5-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-5-2.html b/site/releases/spark-release-0-5-2.html
index 2e1af38095..e5afe8d9e7 100644
--- a/site/releases/spark-release-0-5-2.html
+++ b/site/releases/spark-release-0-5-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-6-0.html b/site/releases/spark-release-0-6-0.html
index e0764fe8ba..852f1c41b5 100644
--- a/site/releases/spark-release-0-6-0.html
+++ b/site/releases/spark-release-0-6-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-6-1.html b/site/releases/spark-release-0-6-1.html
index 435294953e..afdbbb97d3 100644
--- a/site/releases/spark-release-0-6-1.html
+++ b/site/releases/spark-release-0-6-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-6-2.html b/site/releases/spark-release-0-6-2.html
index 11abece377..9e36a1479e 100644
--- a/site/releases/spark-release-0-6-2.html
+++ b/site/releases/spark-release-0-6-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-7-0.html b/site/releases/spark-release-0-7-0.html
index 58d9f594d4..85d6f4d736 100644
--- a/site/releases/spark-release-0-7-0.html
+++ b/site/releases/spark-release-0-7-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-7-2.html b/site/releases/spark-release-0-7-2.html
index 74146da0c4..863fd1b9c2 100644
--- a/site/releases/spark-release-0-7-2.html
+++ b/site/releases/spark-release-0-7-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-7-3.html b/site/releases/spark-release-0-7-3.html
index c8030a383f..6608eb5a60 100644
--- a/site/releases/spark-release-0-7-3.html
+++ b/site/releases/spark-release-0-7-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-8-0.html b/site/releases/spark-release-0-8-0.html
index 8873114c69..1e6e77e5af 100644
--- a/site/releases/spark-release-0-8-0.html
+++ b/site/releases/spark-release-0-8-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-8-1.html b/site/releases/spark-release-0-8-1.html
index d2975e3420..b479c59946 100644
--- a/site/releases/spark-release-0-8-1.html
+++ b/site/releases/spark-release-0-8-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-9-0.html b/site/releases/spark-release-0-9-0.html
index 7990a856a1..22950a501c 100644
--- a/site/releases/spark-release-0-9-0.html
+++ b/site/releases/spark-release-0-9-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-9-1.html b/site/releases/spark-release-0-9-1.html
index c838d74545..04cfbba894 100644
--- a/site/releases/spark-release-0-9-1.html
+++ b/site/releases/spark-release-0-9-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-0-9-2.html b/site/releases/spark-release-0-9-2.html
index 6294f62575..d23f57b145 100644
--- a/site/releases/spark-release-0-9-2.html
+++ b/site/releases/spark-release-0-9-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-0-0.html b/site/releases/spark-release-1-0-0.html
index 8ad8d22fce..5989452d43 100644
--- a/site/releases/spark-release-1-0-0.html
+++ b/site/releases/spark-release-1-0-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-0-1.html b/site/releases/spark-release-1-0-1.html
index 660c85960c..604ad82536 100644
--- a/site/releases/spark-release-1-0-1.html
+++ b/site/releases/spark-release-1-0-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-0-2.html b/site/releases/spark-release-1-0-2.html
index aacc5f443a..9d7913cfe0 100644
--- a/site/releases/spark-release-1-0-2.html
+++ b/site/releases/spark-release-1-0-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-1-0.html b/site/releases/spark-release-1-1-0.html
index 79b271b1c3..d15e7772a2 100644
--- a/site/releases/spark-release-1-1-0.html
+++ b/site/releases/spark-release-1-1-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-1-1.html b/site/releases/spark-release-1-1-1.html
index 1b6fadad38..3c72ad8754 100644
--- a/site/releases/spark-release-1-1-1.html
+++ b/site/releases/spark-release-1-1-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-2-0.html b/site/releases/spark-release-1-2-0.html
index f4507394ad..9ab5be32c2 100644
--- a/site/releases/spark-release-1-2-0.html
+++ b/site/releases/spark-release-1-2-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-2-1.html b/site/releases/spark-release-1-2-1.html
index 6e589c560a..21770c3fac 100644
--- a/site/releases/spark-release-1-2-1.html
+++ b/site/releases/spark-release-1-2-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-2-2.html b/site/releases/spark-release-1-2-2.html
index a09a152249..0244dbecd8 100644
--- a/site/releases/spark-release-1-2-2.html
+++ b/site/releases/spark-release-1-2-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-3-0.html b/site/releases/spark-release-1-3-0.html
index b07b33b514..3f5cb5279c 100644
--- a/site/releases/spark-release-1-3-0.html
+++ b/site/releases/spark-release-1-3-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-3-1.html b/site/releases/spark-release-1-3-1.html
index f6acc4e9af..45c17ed312 100644
--- a/site/releases/spark-release-1-3-1.html
+++ b/site/releases/spark-release-1-3-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-4-0.html b/site/releases/spark-release-1-4-0.html
index 667a82cce3..c205df5374 100644
--- a/site/releases/spark-release-1-4-0.html
+++ b/site/releases/spark-release-1-4-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-4-1.html b/site/releases/spark-release-1-4-1.html
index 06b8e9c0af..0754c5ca92 100644
--- a/site/releases/spark-release-1-4-1.html
+++ b/site/releases/spark-release-1-4-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-5-0.html b/site/releases/spark-release-1-5-0.html
index ee3d8ff93b..576ca6e107 100644
--- a/site/releases/spark-release-1-5-0.html
+++ b/site/releases/spark-release-1-5-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-5-1.html b/site/releases/spark-release-1-5-1.html
index cd6fd739aa..026335bd5e 100644
--- a/site/releases/spark-release-1-5-1.html
+++ b/site/releases/spark-release-1-5-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-5-2.html b/site/releases/spark-release-1-5-2.html
index fd86219bff..7a0c2c7792 100644
--- a/site/releases/spark-release-1-5-2.html
+++ b/site/releases/spark-release-1-5-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-6-0.html b/site/releases/spark-release-1-6-0.html
index add6e31388..e5c6d525d6 100644
--- a/site/releases/spark-release-1-6-0.html
+++ b/site/releases/spark-release-1-6-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-6-1.html b/site/releases/spark-release-1-6-1.html
index 0e18ed6cf9..c6a00bf865 100644
--- a/site/releases/spark-release-1-6-1.html
+++ b/site/releases/spark-release-1-6-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-6-2.html b/site/releases/spark-release-1-6-2.html
index 1225dead64..1ed81fb902 100644
--- a/site/releases/spark-release-1-6-2.html
+++ b/site/releases/spark-release-1-6-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-1-6-3.html b/site/releases/spark-release-1-6-3.html
index 987f17a4f5..62004c5d36 100644
--- a/site/releases/spark-release-1-6-3.html
+++ b/site/releases/spark-release-1-6-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-0-0.html b/site/releases/spark-release-2-0-0.html
index 233b0aa8a3..e141ebd9ed 100644
--- a/site/releases/spark-release-2-0-0.html
+++ b/site/releases/spark-release-2-0-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-0-1.html b/site/releases/spark-release-2-0-1.html
index fd9d0ecb46..b5300bdd0c 100644
--- a/site/releases/spark-release-2-0-1.html
+++ b/site/releases/spark-release-2-0-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-0-2.html b/site/releases/spark-release-2-0-2.html
index 6f3b8943b2..6c0da7a155 100644
--- a/site/releases/spark-release-2-0-2.html
+++ b/site/releases/spark-release-2-0-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-1-0.html b/site/releases/spark-release-2-1-0.html
index 36f156efe8..8aea48a934 100644
--- a/site/releases/spark-release-2-1-0.html
+++ b/site/releases/spark-release-2-1-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-1-1.html b/site/releases/spark-release-2-1-1.html
index 8b06027d8e..7366d4df2b 100644
--- a/site/releases/spark-release-2-1-1.html
+++ b/site/releases/spark-release-2-1-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-1-2.html b/site/releases/spark-release-2-1-2.html
index 1bf4576779..866b923da8 100644
--- a/site/releases/spark-release-2-1-2.html
+++ b/site/releases/spark-release-2-1-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-1-3.html b/site/releases/spark-release-2-1-3.html
index 69e5b682db..90bdcc7eca 100644
--- a/site/releases/spark-release-2-1-3.html
+++ b/site/releases/spark-release-2-1-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-2-0.html b/site/releases/spark-release-2-2-0.html
index 76dc29c745..cd4a529b41 100644
--- a/site/releases/spark-release-2-2-0.html
+++ b/site/releases/spark-release-2-2-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-2-1.html b/site/releases/spark-release-2-2-1.html
index a22a2fe066..9c3335cc01 100644
--- a/site/releases/spark-release-2-2-1.html
+++ b/site/releases/spark-release-2-2-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-2-2.html b/site/releases/spark-release-2-2-2.html
index 524bb030c8..b6c922c304 100644
--- a/site/releases/spark-release-2-2-2.html
+++ b/site/releases/spark-release-2-2-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-2-3.html b/site/releases/spark-release-2-2-3.html
index 84f36a4834..2b72da0faa 100644
--- a/site/releases/spark-release-2-2-3.html
+++ b/site/releases/spark-release-2-2-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-3-0.html b/site/releases/spark-release-2-3-0.html
index babf51cdba..406732cd9d 100644
--- a/site/releases/spark-release-2-3-0.html
+++ b/site/releases/spark-release-2-3-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-3-1.html b/site/releases/spark-release-2-3-1.html
index 3272b6619a..bf37e93f46 100644
--- a/site/releases/spark-release-2-3-1.html
+++ b/site/releases/spark-release-2-3-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-3-2.html b/site/releases/spark-release-2-3-2.html
index ffce472090..d62ca60b99 100644
--- a/site/releases/spark-release-2-3-2.html
+++ b/site/releases/spark-release-2-3-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-3-3.html b/site/releases/spark-release-2-3-3.html
index 553cca1c4a..77900e5d71 100644
--- a/site/releases/spark-release-2-3-3.html
+++ b/site/releases/spark-release-2-3-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-3-4.html b/site/releases/spark-release-2-3-4.html
index fdc4ec0c05..61ce9950b3 100644
--- a/site/releases/spark-release-2-3-4.html
+++ b/site/releases/spark-release-2-3-4.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-4-0.html b/site/releases/spark-release-2-4-0.html
index 271afecc27..6dc8c4f60b 100644
--- a/site/releases/spark-release-2-4-0.html
+++ b/site/releases/spark-release-2-4-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-4-1.html b/site/releases/spark-release-2-4-1.html
index cc4f8b639b..278745be48 100644
--- a/site/releases/spark-release-2-4-1.html
+++ b/site/releases/spark-release-2-4-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-4-2.html b/site/releases/spark-release-2-4-2.html
index c43941b91b..a5d9782155 100644
--- a/site/releases/spark-release-2-4-2.html
+++ b/site/releases/spark-release-2-4-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-4-3.html b/site/releases/spark-release-2-4-3.html
index 0554d7df94..0b6bdb1008 100644
--- a/site/releases/spark-release-2-4-3.html
+++ b/site/releases/spark-release-2-4-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-4-4.html b/site/releases/spark-release-2-4-4.html
index 4b97b13c84..b63fed694a 100644
--- a/site/releases/spark-release-2-4-4.html
+++ b/site/releases/spark-release-2-4-4.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-4-5.html b/site/releases/spark-release-2-4-5.html
index 5e9a796ea1..53daaac5fe 100644
--- a/site/releases/spark-release-2-4-5.html
+++ b/site/releases/spark-release-2-4-5.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-4-6.html b/site/releases/spark-release-2-4-6.html
index ad3d9cb351..08a9222999 100644
--- a/site/releases/spark-release-2-4-6.html
+++ b/site/releases/spark-release-2-4-6.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-4-7.html b/site/releases/spark-release-2-4-7.html
index 3a17dacbee..d2cbe2cc2e 100644
--- a/site/releases/spark-release-2-4-7.html
+++ b/site/releases/spark-release-2-4-7.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-2-4-8.html b/site/releases/spark-release-2-4-8.html
index 990b478ece..78f26629f1 100644
--- a/site/releases/spark-release-2-4-8.html
+++ b/site/releases/spark-release-2-4-8.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-0-0.html b/site/releases/spark-release-3-0-0.html
index 6d89caf196..828c860b67 100644
--- a/site/releases/spark-release-3-0-0.html
+++ b/site/releases/spark-release-3-0-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-0-1.html b/site/releases/spark-release-3-0-1.html
index 0c072f84af..d62fc19803 100644
--- a/site/releases/spark-release-3-0-1.html
+++ b/site/releases/spark-release-3-0-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-0-2.html b/site/releases/spark-release-3-0-2.html
index 7116a8ae4b..bae8509166 100644
--- a/site/releases/spark-release-3-0-2.html
+++ b/site/releases/spark-release-3-0-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-0-3.html b/site/releases/spark-release-3-0-3.html
index ce74aecb77..13e36f893e 100644
--- a/site/releases/spark-release-3-0-3.html
+++ b/site/releases/spark-release-3-0-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-1-1.html b/site/releases/spark-release-3-1-1.html
index 80ed25d624..ea5356ec61 100644
--- a/site/releases/spark-release-3-1-1.html
+++ b/site/releases/spark-release-3-1-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-1-2.html b/site/releases/spark-release-3-1-2.html
index 504cf6615c..c4bc4e56d7 100644
--- a/site/releases/spark-release-3-1-2.html
+++ b/site/releases/spark-release-3-1-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-1-3.html b/site/releases/spark-release-3-1-3.html
index ae068bd062..39d5370854 100644
--- a/site/releases/spark-release-3-1-3.html
+++ b/site/releases/spark-release-3-1-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-2-0.html b/site/releases/spark-release-3-2-0.html
index bfc32e27e8..fd4d210b16 100644
--- a/site/releases/spark-release-3-2-0.html
+++ b/site/releases/spark-release-3-2-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-2-1.html b/site/releases/spark-release-3-2-1.html
index 11102d1478..496ee443c2 100644
--- a/site/releases/spark-release-3-2-1.html
+++ b/site/releases/spark-release-3-2-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-2-2.html b/site/releases/spark-release-3-2-2.html
index 3064e0af0e..f705de7f94 100644
--- a/site/releases/spark-release-3-2-2.html
+++ b/site/releases/spark-release-3-2-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-2-3.html b/site/releases/spark-release-3-2-3.html
index 38f8cd1281..2774c820b1 100644
--- a/site/releases/spark-release-3-2-3.html
+++ b/site/releases/spark-release-3-2-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-2-4.html b/site/releases/spark-release-3-2-4.html
index c2203230e4..bd0367ef4c 100644
--- a/site/releases/spark-release-3-2-4.html
+++ b/site/releases/spark-release-3-2-4.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-3-0.html b/site/releases/spark-release-3-3-0.html
index af659454ce..4fd433b7c7 100644
--- a/site/releases/spark-release-3-3-0.html
+++ b/site/releases/spark-release-3-3-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-3-1.html b/site/releases/spark-release-3-3-1.html
index 98144b240a..7308244153 100644
--- a/site/releases/spark-release-3-3-1.html
+++ b/site/releases/spark-release-3-3-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-3-2.html b/site/releases/spark-release-3-3-2.html
index ce1f20bd17..d40aa7da6a 100644
--- a/site/releases/spark-release-3-3-2.html
+++ b/site/releases/spark-release-3-3-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-3-3.html b/site/releases/spark-release-3-3-3.html
index fab73f2c84..8d832e5dca 100644
--- a/site/releases/spark-release-3-3-3.html
+++ b/site/releases/spark-release-3-3-3.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-3-4.html b/site/releases/spark-release-3-3-4.html
index 679aa87908..3e810c578c 100644
--- a/site/releases/spark-release-3-3-4.html
+++ b/site/releases/spark-release-3-3-4.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-4-0.html b/site/releases/spark-release-3-4-0.html
index 38a5591352..7b01c24960 100644
--- a/site/releases/spark-release-3-4-0.html
+++ b/site/releases/spark-release-3-4-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-4-1.html b/site/releases/spark-release-3-4-1.html
index 4f1af965c9..db98d5aecc 100644
--- a/site/releases/spark-release-3-4-1.html
+++ b/site/releases/spark-release-3-4-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-4-2.html b/site/releases/spark-release-3-4-2.html
index 37b41bf983..56c8f8335d 100644
--- a/site/releases/spark-release-3-4-2.html
+++ b/site/releases/spark-release-3-4-2.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-5-0.html b/site/releases/spark-release-3-5-0.html
index cd136a991f..9ad0c5cd29 100644
--- a/site/releases/spark-release-3-5-0.html
+++ b/site/releases/spark-release-3-5-0.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/releases/spark-release-3-5-1.html b/site/releases/spark-release-3-5-1.html
index 3da692692e..b916b6bfb4 100644
--- a/site/releases/spark-release-3-5-1.html
+++ b/site/releases/spark-release-3-5-1.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/research.html b/site/research.html
index abc77a1993..01362acf63 100644
--- a/site/research.html
+++ b/site/research.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/screencasts/1-first-steps-with-spark.html b/site/screencasts/1-first-steps-with-spark.html
index b0176bed38..8af8e43b00 100644
--- a/site/screencasts/1-first-steps-with-spark.html
+++ b/site/screencasts/1-first-steps-with-spark.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/screencasts/2-spark-documentation-overview.html b/site/screencasts/2-spark-documentation-overview.html
index 18073da93a..4632a2a20a 100644
--- a/site/screencasts/2-spark-documentation-overview.html
+++ b/site/screencasts/2-spark-documentation-overview.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/screencasts/3-transformations-and-caching.html b/site/screencasts/3-transformations-and-caching.html
index 7627174776..f6def142eb 100644
--- a/site/screencasts/3-transformations-and-caching.html
+++ b/site/screencasts/3-transformations-and-caching.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/screencasts/4-a-standalone-job-in-spark.html b/site/screencasts/4-a-standalone-job-in-spark.html
index 49e9f78d63..201ce45149 100644
--- a/site/screencasts/4-a-standalone-job-in-spark.html
+++ b/site/screencasts/4-a-standalone-job-in-spark.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/screencasts/index.html b/site/screencasts/index.html
index 08694419ed..74bd2ef439 100644
--- a/site/screencasts/index.html
+++ b/site/screencasts/index.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/security.html b/site/security.html
index e34596670c..e1299ddf4b 100644
--- a/site/security.html
+++ b/site/security.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/sitemap.xml b/site/sitemap.xml
index 2432adb37e..93b09b4711 100644
--- a/site/sitemap.xml
+++ b/site/sitemap.xml
@@ -1041,27 +1041,31 @@
<changefreq>weekly</changefreq>
</url>
<url>
- <loc>https://spark.apache.org/graphx/</loc>
+ <loc>https://spark.apache.org/sql/</loc>
<changefreq>weekly</changefreq>
</url>
<url>
- <loc>https://spark.apache.org/mllib/</loc>
+ <loc>https://spark.apache.org/news/</loc>
<changefreq>weekly</changefreq>
</url>
<url>
- <loc>https://spark.apache.org/streaming/</loc>
+ <loc>https://spark.apache.org/screencasts/</loc>
<changefreq>weekly</changefreq>
</url>
<url>
- <loc>https://spark.apache.org/news/</loc>
+ <loc>https://spark.apache.org/graphx/</loc>
<changefreq>weekly</changefreq>
</url>
<url>
- <loc>https://spark.apache.org/screencasts/</loc>
+ <loc>https://spark.apache.org/pandas-on-spark/</loc>
<changefreq>weekly</changefreq>
</url>
<url>
- <loc>https://spark.apache.org/sql/</loc>
+ <loc>https://spark.apache.org/streaming/</loc>
+ <changefreq>weekly</changefreq>
+</url>
+<url>
+ <loc>https://spark.apache.org/mllib/</loc>
<changefreq>weekly</changefreq>
</url>
<url>
diff --git a/site/spark-connect/index.html b/site/spark-connect/index.html
index 1877c0f32d..223ca03169 100644
--- a/site/spark-connect/index.html
+++ b/site/spark-connect/index.html
@@ -67,6 +67,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/sql/index.html b/site/sql/index.html
index aa4312c04f..d087dd4099 100644
--- a/site/sql/index.html
+++ b/site/sql/index.html
@@ -67,6 +67,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/streaming/index.html b/site/streaming/index.html
index 4be1da7faf..3a0f02b864 100644
--- a/site/streaming/index.html
+++ b/site/streaming/index.html
@@ -67,6 +67,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/third-party-projects.html b/site/third-party-projects.html
index 5a8795fe39..cfa7eaf13a 100644
--- a/site/third-party-projects.html
+++ b/site/third-party-projects.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/trademarks.html b/site/trademarks.html
index 611f081acc..ee5fecd9df 100644
--- a/site/trademarks.html
+++ b/site/trademarks.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
diff --git a/site/versioning-policy.html b/site/versioning-policy.html
index 1fedc4be0f..934d08304a 100644
--- a/site/versioning-policy.html
+++ b/site/versioning-policy.html
@@ -65,6 +65,7 @@
<li><a class="dropdown-item" href="/sql/">SQL and DataFrames</a></li>
<li><a class="dropdown-item" href="/spark-connect/">Spark Connect</a></li>
<li><a class="dropdown-item" href="/streaming/">Spark Streaming</a></li>
+ <li><a class="dropdown-item" href="/pandas-on-spark/">pandas on Spark</a></li>
<li><a class="dropdown-item" href="/mllib/">MLlib (machine learning)</a></li>
<li><a class="dropdown-item" href="/graphx/">GraphX (graph)</a></li>
<li>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org