[GitHub] [spark] HyukjinKwon commented on a diff in pull request #37465: [SPARK-40029][PYTHON][DOC] Make pyspark.sql.types examples self-contained

[GitBox <gi...@apache.org>]

HyukjinKwon commented on code in PR #37465:
URL: https://github.com/apache/spark/pull/37465#discussion_r943071346


##########
python/pyspark/sql/types.py:
##########
@@ -1108,6 +1153,8 @@ def _parse_datatype_json_string(json_string: str) -> DataType:
 
     Examples
     --------
+    >>> from pyspark.sql.types import *
+    >>> from pyspark.sql.types import _all_atomic_types, _parse_datatype_json_string

Review Comment:
   ditto



##########
python/pyspark/sql/types.py:
##########
@@ -511,6 +519,14 @@ class MapType(DataType):
 
     Examples
     --------
+    >>> from pyspark.sql.types import IntegerType, FloatType, MapType, StringType
+
+    The below example demonstrates how to create MapType:
+
+    >>> map_type = MapType(StringType(), IntegerType())
+
+    The values of the map can contain null (None) values by default:

Review Comment:
   ```suggestion
       The values of the map can contain null (``None``) values by default:
   ```



##########
python/pyspark/sql/types.py:
##########
@@ -511,6 +519,14 @@ class MapType(DataType):
 
     Examples
     --------
+    >>> from pyspark.sql.types import IntegerType, FloatType, MapType, StringType
+
+    The below example demonstrates how to create MapType:

Review Comment:
   ```suggestion
       The below example demonstrates how to create class:`MapType`:
   ```



##########
python/pyspark/sql/types.py:
##########
@@ -1036,6 +1080,7 @@ def _parse_datatype_string(s: str) -> DataType:
 
     Examples
     --------
+    >>> from pyspark.sql.types import _parse_datatype_string

Review Comment:
   This is an internal example so let's just leave it.



##########
python/pyspark/sql/types.py:
##########
@@ -1649,6 +1696,8 @@ def _make_type_verifier(
 
     Examples
     --------
+    >>> from pyspark.sql.types import *
+    >>> from pyspark.sql.types import _make_type_verifier

Review Comment:
   ditto



##########
python/pyspark/sql/types.py:
##########
@@ -684,6 +702,30 @@ class StructType(DataType):
     ...     StructField("f2", IntegerType(), False)])
     >>> struct1 == struct2
     False
+
+    The below example demonstrates how to create a struct using StructType & StructField on
+    DataFrame:
+
+    >>> data = [("Alice", ["Java", "Scala"]), ("Bob", ["Python", "Scala"])]
+    >>> schema = StructType([
+    ...     StructField("name", StringType()),
+    ...     StructField("languagesSkills", ArrayType(StringType())),
+    ... ])
+    >>> df = spark.createDataFrame(data=data, schema=schema)
+    >>> df.printSchema()
+    root
+     |-- name: string (nullable = true)
+     |-- languagesSkills: array (nullable = true)
+     |    |-- element: string (containsNull = true)
+    <BLANKLINE>

Review Comment:
   You can remove this blankline and add fix the bottom as below:
   
   ```
   ...
   optionflags=doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org