[spark] branch master updated: [SPARK-36897][PYTHON] Use NamedTuple with variable type hints instead of namedtuple

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new c5c6f48  [SPARK-36897][PYTHON] Use NamedTuple with variable type hints instead of namedtuple
c5c6f48 is described below

commit c5c6f48de40a4e012a35ecc78f8db41654d2c8bd
Author: Takuya UESHIN <ue...@databricks.com>
AuthorDate: Sat Oct 9 15:00:20 2021 +0900

    [SPARK-36897][PYTHON] Use NamedTuple with variable type hints instead of namedtuple
    
    ### What changes were proposed in this pull request?
    
    Use `NamedTuple` with variable type hints instead of `namedtuple`.
    
    ### Why are the changes needed?
    
    Per discussion under https://github.com/apache/spark/pull/34133#discussion_r718833451, we wanted to replace `collections.namedtuple()` by `typing.NamedTuple`.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Existing tests.
    
    Closes #34228 from ueshin/issues/SPARK-36897/namedtuple.
    
    Authored-by: Takuya UESHIN <ue...@databricks.com>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 python/pyspark/sql/catalog.py | 35 +++++++++++++++++++++++++++++------
 1 file changed, 29 insertions(+), 6 deletions(-)

diff --git a/python/pyspark/sql/catalog.py b/python/pyspark/sql/catalog.py
index 61167fa..29f22e4 100644
--- a/python/pyspark/sql/catalog.py
+++ b/python/pyspark/sql/catalog.py
@@ -17,8 +17,7 @@
 
 import sys
 import warnings
-from collections import namedtuple
-from typing import Any, Callable, List, Optional, TYPE_CHECKING
+from typing import Any, Callable, NamedTuple, List, Optional, TYPE_CHECKING
 
 from pyspark import since
 from pyspark.sql.dataframe import DataFrame
@@ -30,10 +29,34 @@ if TYPE_CHECKING:
     from pyspark.sql.types import DataType
 
 
-Database = namedtuple("Database", "name description locationUri")
-Table = namedtuple("Table", "name database description tableType isTemporary")
-Column = namedtuple("Column", "name description dataType nullable isPartition isBucket")
-Function = namedtuple("Function", "name description className isTemporary")
+class Database(NamedTuple):
+    name: str
+    description: Optional[str]
+    locationUri: str
+
+
+class Table(NamedTuple):
+    name: str
+    database: Optional[str]
+    description: Optional[str]
+    tableType: str
+    isTemporary: bool
+
+
+class Column(NamedTuple):
+    name: str
+    description: Optional[str]
+    dataType: str
+    nullable: bool
+    isPartition: bool
+    isBucket: bool
+
+
+class Function(NamedTuple):
+    name: str
+    description: Optional[str]
+    className: str
+    isTemporary: bool
 
 
 class Catalog(object):

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org