[GitHub] [spark] zero323 commented on a change in pull request #34863: [SPARK-37601][PYTHON] sql.DataFrame.transform accept function parameters

[GitBox <gi...@apache.org>]

zero323 commented on a change in pull request #34863:
URL: https://github.com/apache/spark/pull/34863#discussion_r776353027



##########
File path: python/pyspark/sql/dataframe.py
##########
@@ -3067,6 +3067,10 @@ def transform(self, func: Callable[["DataFrame"], "DataFrame"]) -> "DataFrame":
         ----------
         func : function
             a function that takes and returns a :class:`DataFrame`.
+        args : Any
+            optional positional arguments for the func function
+        kwargs: Any
+            optional keyword arguments for the func function

Review comment:
       > I think they are not the Parameters of `transform` ??
   
   What I meant is the following (totally made up change);
   
   - Let's say we want to add `foo` argument to `transform` i..e
   
       ```python
       def transform(self, func: Callable[["DataFrame"], "DataFrame"], foo: Any) -> "DataFrame"
       ```
   
   - That's possible ATM, but if we add `*args` and `**kwargs` to the signature this becomes problematic as both
    
       ```python
      def transform(self, func: Callable[["DataFrame"], "DataFrame"], foo: Any, *args, **kwargs) -> "DataFrame"
       ```
   
      and
   
       ```python
      def transform(self, func: Callable[["DataFrame"], "DataFrame"], *args,  foo: Any,  **kwargs) -> "DataFrame"
       ```
   
      are limited and / or ambiguous.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org