You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by rx...@apache.org on 2015/02/10 05:58:30 UTC
[4/4] spark git commit: [SPARK-5469] restructure pyspark.sql into
multiple files
[SPARK-5469] restructure pyspark.sql into multiple files
All the DataTypes moved into pyspark.sql.types
The changes can be tracked by `--find-copies-harder -M25`
```
davieslocalhost:~/work/spark/python$ git diff --find-copies-harder -M25 --numstat master..
2 5 python/docs/pyspark.ml.rst
0 3 python/docs/pyspark.mllib.rst
10 2 python/docs/pyspark.sql.rst
1 1 python/pyspark/mllib/linalg.py
21 14 python/pyspark/{mllib => sql}/__init__.py
14 2108 python/pyspark/{sql.py => sql/context.py}
10 1772 python/pyspark/{sql.py => sql/dataframe.py}
7 6 python/pyspark/{sql_tests.py => sql/tests.py}
8 1465 python/pyspark/{sql.py => sql/types.py}
4 2 python/run-tests
1 1 sql/core/src/main/scala/org/apache/spark/sql/test/ExamplePointUDT.scala
```
Also `git blame -C -C python/pyspark/sql/context.py` to track the history.
Author: Davies Liu <da...@databricks.com>
Closes #4479 from davies/sql and squashes the following commits:
1b5f0a5 [Davies Liu] Merge branch 'master' of github.com:apache/spark into sql
2b2b983 [Davies Liu] restructure pyspark.sql
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/08488c17
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/08488c17
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/08488c17
Branch: refs/heads/master
Commit: 08488c175f2e8532cb6aab84da2abd9ad57179cc
Parents: d302c48
Author: Davies Liu <da...@databricks.com>
Authored: Mon Feb 9 20:49:22 2015 -0800
Committer: Reynold Xin <rx...@databricks.com>
Committed: Mon Feb 9 20:49:22 2015 -0800
----------------------------------------------------------------------
python/docs/pyspark.ml.rst | 7 +-
python/docs/pyspark.mllib.rst | 3 -
python/docs/pyspark.sql.rst | 12 +-
python/pyspark/mllib/linalg.py | 2 +-
python/pyspark/sql.py | 2736 ------------------
python/pyspark/sql/__init__.py | 42 +
python/pyspark/sql/context.py | 642 ++++
python/pyspark/sql/dataframe.py | 974 +++++++
python/pyspark/sql/tests.py | 300 ++
python/pyspark/sql/types.py | 1279 ++++++++
python/pyspark/sql_tests.py | 299 --
python/run-tests | 6 +-
.../apache/spark/sql/test/ExamplePointUDT.scala | 2 +-
13 files changed, 3255 insertions(+), 3049 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/08488c17/python/docs/pyspark.ml.rst
----------------------------------------------------------------------
diff --git a/python/docs/pyspark.ml.rst b/python/docs/pyspark.ml.rst
index f10d133..4da6d4a 100644
--- a/python/docs/pyspark.ml.rst
+++ b/python/docs/pyspark.ml.rst
@@ -1,11 +1,8 @@
pyspark.ml package
=====================
-Submodules
-----------
-
-pyspark.ml module
------------------
+Module Context
+--------------
.. automodule:: pyspark.ml
:members:
http://git-wip-us.apache.org/repos/asf/spark/blob/08488c17/python/docs/pyspark.mllib.rst
----------------------------------------------------------------------
diff --git a/python/docs/pyspark.mllib.rst b/python/docs/pyspark.mllib.rst
index 4548b87..21f66ca 100644
--- a/python/docs/pyspark.mllib.rst
+++ b/python/docs/pyspark.mllib.rst
@@ -1,9 +1,6 @@
pyspark.mllib package
=====================
-Submodules
-----------
-
pyspark.mllib.classification module
-----------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/08488c17/python/docs/pyspark.sql.rst
----------------------------------------------------------------------
diff --git a/python/docs/pyspark.sql.rst b/python/docs/pyspark.sql.rst
index 65b3650..80c6f02 100644
--- a/python/docs/pyspark.sql.rst
+++ b/python/docs/pyspark.sql.rst
@@ -1,10 +1,18 @@
pyspark.sql module
==================
-Module contents
----------------
+Module Context
+--------------
.. automodule:: pyspark.sql
:members:
:undoc-members:
:show-inheritance:
+
+
+pyspark.sql.types module
+------------------------
+.. automodule:: pyspark.sql.types
+ :members:
+ :undoc-members:
+ :show-inheritance:
http://git-wip-us.apache.org/repos/asf/spark/blob/08488c17/python/pyspark/mllib/linalg.py
----------------------------------------------------------------------
diff --git a/python/pyspark/mllib/linalg.py b/python/pyspark/mllib/linalg.py
index 7f21190..597012b 100644
--- a/python/pyspark/mllib/linalg.py
+++ b/python/pyspark/mllib/linalg.py
@@ -29,7 +29,7 @@ import copy_reg
import numpy as np
-from pyspark.sql import UserDefinedType, StructField, StructType, ArrayType, DoubleType, \
+from pyspark.sql.types import UserDefinedType, StructField, StructType, ArrayType, DoubleType, \
IntegerType, ByteType
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org