[GitHub] [spark] amaliujia commented on a diff in pull request #39035: [SPARK-41438][CONNECT][PYTHON] Implement `DataFrame.colRegex`

[GitBox <gi...@apache.org>]

amaliujia commented on code in PR #39035:
URL: https://github.com/apache/spark/pull/39035#discussion_r1046689985


##########
python/pyspark/sql/connect/functions.py:
##########
@@ -89,6 +90,10 @@ def col(col: str) -> Column:
 column = col
 
 
+def colRegex(col: str) -> Column:

Review Comment:
   This one should be in dataframe.py. Refer to https://github.com/apache/spark/blob/af8dd411aa9d15bc59d09cf9959d4a57debc9635/python/pyspark/sql/dataframe.py#L1880



##########
python/pyspark/sql/tests/connect/test_connect_column_expressions.py:
##########
@@ -194,6 +194,15 @@ def test_column_alias(self) -> None:
         self.assertIsNotNone(plan)
         self.assertEqual(plan.alias.metadata, '{"pii": true}')
 
+    def test_column_regexp(self):

Review Comment:
   dito



##########
python/pyspark/sql/tests/connect/test_connect_basic.py:
##########
@@ -942,6 +945,14 @@ def test_alias(self) -> None:
             self.connect.range(1, 10).select(col("id").alias("this", "is", "not")).collect()
         self.assertIn("(this, is, not)", str(exc.exception))
 
+    def test_column_regexp(self) -> None:

Review Comment:
   actually we should follow developer guide here:
   
   ```
       def test_agg_with_two_agg_exprs(self) -> None:
           # SPARK-41230: test dataframe.agg()
   ```
   
   Basically have a JIRA with a description for the test case.
   
          



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org