You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2016/08/06 04:02:58 UTC
spark git commit: [SPARK-16772][PYTHON][DOCS] Fix API doc references
to UDFRegistration + Update "important classes"
Repository: spark
Updated Branches:
refs/heads/master 14dba4520 -> 2dd038861
[SPARK-16772][PYTHON][DOCS] Fix API doc references to UDFRegistration + Update "important classes"
## Proposed Changes
* Update the list of "important classes" in `pyspark.sql` to match 2.0.
* Fix references to `UDFRegistration` so that the class shows up in the docs. It currently [doesn't](http://spark.apache.org/docs/latest/api/python/pyspark.sql.html).
* Remove some unnecessary whitespace in the Python RST doc files.
I reused the [existing JIRA](https://issues.apache.org/jira/browse/SPARK-16772) I created last week for similar API doc fixes.
## How was this patch tested?
* I ran `lint-python` successfully.
* I ran `make clean build` on the Python docs and confirmed the results are as expected locally in my browser.
Author: Nicholas Chammas <ni...@gmail.com>
Closes #14496 from nchammas/SPARK-16772-UDFRegistration.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2dd03886
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2dd03886
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/2dd03886
Branch: refs/heads/master
Commit: 2dd03886173f2f3b5c20fe14e9cdbd33480c1f36
Parents: 14dba45
Author: Nicholas Chammas <ni...@gmail.com>
Authored: Sat Aug 6 05:02:59 2016 +0100
Committer: Sean Owen <so...@cloudera.com>
Committed: Sat Aug 6 05:02:59 2016 +0100
----------------------------------------------------------------------
python/docs/index.rst | 1 -
python/docs/pyspark.sql.rst | 2 --
python/pyspark/sql/__init__.py | 11 +++++------
3 files changed, 5 insertions(+), 9 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/2dd03886/python/docs/index.rst
----------------------------------------------------------------------
diff --git a/python/docs/index.rst b/python/docs/index.rst
index 306ffdb..421c8de 100644
--- a/python/docs/index.rst
+++ b/python/docs/index.rst
@@ -50,4 +50,3 @@ Indices and tables
==================
* :ref:`search`
-
http://git-wip-us.apache.org/repos/asf/spark/blob/2dd03886/python/docs/pyspark.sql.rst
----------------------------------------------------------------------
diff --git a/python/docs/pyspark.sql.rst b/python/docs/pyspark.sql.rst
index 3be9533..09848b8 100644
--- a/python/docs/pyspark.sql.rst
+++ b/python/docs/pyspark.sql.rst
@@ -8,14 +8,12 @@ Module Context
:members:
:undoc-members:
-
pyspark.sql.types module
------------------------
.. automodule:: pyspark.sql.types
:members:
:undoc-members:
-
pyspark.sql.functions module
----------------------------
.. automodule:: pyspark.sql.functions
http://git-wip-us.apache.org/repos/asf/spark/blob/2dd03886/python/pyspark/sql/__init__.py
----------------------------------------------------------------------
diff --git a/python/pyspark/sql/__init__.py b/python/pyspark/sql/__init__.py
index cff73ff..22ec416 100644
--- a/python/pyspark/sql/__init__.py
+++ b/python/pyspark/sql/__init__.py
@@ -18,7 +18,7 @@
"""
Important classes of Spark SQL and DataFrames:
- - :class:`pyspark.sql.SQLContext`
+ - :class:`pyspark.sql.SparkSession`
Main entry point for :class:`DataFrame` and SQL functionality.
- :class:`pyspark.sql.DataFrame`
A distributed collection of data grouped into named columns.
@@ -26,8 +26,6 @@ Important classes of Spark SQL and DataFrames:
A column expression in a :class:`DataFrame`.
- :class:`pyspark.sql.Row`
A row of data in a :class:`DataFrame`.
- - :class:`pyspark.sql.HiveContext`
- Main entry point for accessing data stored in Apache Hive.
- :class:`pyspark.sql.GroupedData`
Aggregation methods, returned by :func:`DataFrame.groupBy`.
- :class:`pyspark.sql.DataFrameNaFunctions`
@@ -45,7 +43,7 @@ from __future__ import absolute_import
from pyspark.sql.types import Row
-from pyspark.sql.context import SQLContext, HiveContext
+from pyspark.sql.context import SQLContext, HiveContext, UDFRegistration
from pyspark.sql.session import SparkSession
from pyspark.sql.column import Column
from pyspark.sql.dataframe import DataFrame, DataFrameNaFunctions, DataFrameStatFunctions
@@ -55,7 +53,8 @@ from pyspark.sql.window import Window, WindowSpec
__all__ = [
- 'SparkSession', 'SQLContext', 'HiveContext', 'DataFrame', 'GroupedData', 'Column',
- 'Row', 'DataFrameNaFunctions', 'DataFrameStatFunctions', 'Window', 'WindowSpec',
+ 'SparkSession', 'SQLContext', 'HiveContext', 'UDFRegistration',
+ 'DataFrame', 'GroupedData', 'Column', 'Row',
+ 'DataFrameNaFunctions', 'DataFrameStatFunctions', 'Window', 'WindowSpec',
'DataFrameReader', 'DataFrameWriter'
]
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org