You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2022/04/20 01:49:23 UTC
[spark] branch master updated: [SPARK-38828][PYTHON] Remove TimestampNTZ type Python support in Spark 3.3
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 581000de243 [SPARK-38828][PYTHON] Remove TimestampNTZ type Python support in Spark 3.3
581000de243 is described below
commit 581000de24377ca373df7fa94b214baa7e9b0462
Author: itholic <ha...@databricks.com>
AuthorDate: Wed Apr 20 10:49:07 2022 +0900
[SPARK-38828][PYTHON] Remove TimestampNTZ type Python support in Spark 3.3
### What changes were proposed in this pull request?
This PR proposes to remove `TimestampNTZ` type Python support in Spark 3.3 from documentation and `pyspark.sql.types` module.
The purpose of this PR is just hide `TimestampNTZ` type from end-users.
### Why are the changes needed?
Because the `TimestampNTZ` project is not finished yet:
- Lack Hive metastore support
- Lack JDBC support
- Need to spend time scanning the codebase to find out any missing support. The current code usages of TimestampType are larger than TimestampNTZType
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The existing tests should cover.
Closes #36255 from itholic/SPARK-38828.
Authored-by: itholic <ha...@databricks.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
python/docs/source/reference/pyspark.sql.rst | 1 -
python/pyspark/sql/types.py | 1 -
2 files changed, 2 deletions(-)
diff --git a/python/docs/source/reference/pyspark.sql.rst b/python/docs/source/reference/pyspark.sql.rst
index 1d34961a91a..adc1958822e 100644
--- a/python/docs/source/reference/pyspark.sql.rst
+++ b/python/docs/source/reference/pyspark.sql.rst
@@ -302,7 +302,6 @@ Data Types
StringType
StructField
StructType
- TimestampNTZType
TimestampType
DayTimeIntervalType
diff --git a/python/pyspark/sql/types.py b/python/pyspark/sql/types.py
index 23e54eb8889..2a41508d634 100644
--- a/python/pyspark/sql/types.py
+++ b/python/pyspark/sql/types.py
@@ -59,7 +59,6 @@ __all__ = [
"BooleanType",
"DateType",
"TimestampType",
- "TimestampNTZType",
"DecimalType",
"DoubleType",
"FloatType",
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org