[GitHub] [spark] xinrong-databricks commented on a change in pull request #34185: [SPARK-36886][PYTHON] Inline type hints for python/pyspark/sql/context.py

[GitBox <gi...@apache.org>]

xinrong-databricks commented on a change in pull request #34185:
URL: https://github.com/apache/spark/pull/34185#discussion_r730042321



##########
File path: python/pyspark/sql/observation.py
##########
@@ -102,13 +102,14 @@ def _on(self, df: DataFrame, *exprs: Column) -> DataFrame:
         assert all(isinstance(c, Column) for c in exprs), "all exprs should be Column"
         assert self._jo is None, "an Observation can be used with a DataFrame only once"
 
-        self._jvm = df._sc._jvm  # type: ignore[assignment, attr-defined]
+        self._jvm = df._sc._jvm  # type: ignore[assignment, attr-defined, has-type]

Review comment:
       May I ask why `has-type` is needed here?

##########
File path: python/pyspark/sql/context.py
##########
@@ -268,7 +320,66 @@ def _inferSchema(self, rdd, samplingRatio=None):
         """
         return self.sparkSession._inferSchema(rdd, samplingRatio)
 
-    def createDataFrame(self, data, schema=None, samplingRatio=None, verifySchema=True):
+    @overload
+    def createDataFrame(
+        self,
+        data: Union["RDD[RowLike]", "Iterable[RowLike]"],

Review comment:
       Will `Iterable["RowLike"]` work?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org