[GitHub] [spark] HyukjinKwon commented on a change in pull request #23797: [SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs

[GitBox <gi...@apache.org>]

HyukjinKwon commented on a change in pull request #23797: [SPARK-26856][PYSPARK] Python support for from_avro and to_avro APIs
URL: https://github.com/apache/spark/pull/23797#discussion_r261917304
 
 

 ##########
 File path: dev/sparktestsupport/modules.py
 ##########
 @@ -178,6 +178,10 @@ def __hash__(self):
     ],
     sbt_test_goals=[
         "avro/test",
+    ],
+    python_test_goals=[
+        # doctests
+        "pyspark.sql.avro.functions"
 
 Review comment:
   @gaborgsomogyi, I think this isn't detected properly when there are only Avro related changes. Currently, the tests look being ran since `root` module (which means all modules) is detected.
   
   Also, I think we should somehow put this into `pyspark-sql` module. Currently, this PR adds new module called `avro` into python testing script:
   
   ```
   Running PySpark tests. Output is in /home/jenkins/workspace/SparkPullRequestBuilder/python/unit-tests.log
   Will test against the following Python executables: ['python2.7', 'python3.4', 'pypy']
   Will test the following Python modules: ['avro', 'pyspark-core', 'pyspark-ml', 'pyspark-mllib', 'pyspark-sql', 'pyspark-streaming']
   Starting test(pypy): pyspark.sql.tests.test_catalog
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org