[GitHub] [spark] viirya opened a new pull request #25133: [SPARK-28365][Python][TEST] Set default locale for StopWordsRemover tests to prevent invalid locale error during test

[GitBox <gi...@apache.org>]

viirya opened a new pull request #25133: [SPARK-28365][Python][TEST] Set default locale for StopWordsRemover tests to prevent invalid locale error during test
URL: https://github.com/apache/spark/pull/25133
 
 
   ## What changes were proposed in this pull request?
   
   Because the local default locale isn't in available locales at `Locale`, when I did some tests locally with python code, `StopWordsRemover` related python test hits some errors, like:
   
   ```
   Traceback (most recent call last):
     File "/spark-1/python/pyspark/ml/tests/test_feature.py", line 87, in test_stopwordsremover
       stopWordRemover = StopWordsRemover(inputCol="input", outputCol="output")
     File "/spark-1/python/pyspark/__init__.py", line 111, in wrapper
       return func(self, **kwargs)
     File "/spark-1/python/pyspark/ml/feature.py", line 2646, in __init__
       self.uid)
     File "/spark-1/python/pyspark/ml/wrapper.py", line 67, in _new_java_obj
       return java_obj(*java_args)
     File /spark-1/python/lib/py4j-0.10.8.1-src.zip/py4j/java_gateway.py", line 1554, in __call__
       answer, self._gateway_client, None, self._fqn)
     File "/spark-1/python/pyspark/sql/utils.py", line 93, in deco
       raise converted
   pyspark.sql.utils.IllegalArgumentException: 'StopWordsRemover_4598673ee802 parameter locale given invalid value en_TW.'
   ```
   
   This patch sets a default locale `en-US` for such tests.
   
   For scala tests, it already sets this default locale.
   
   ## How was this patch tested?
   
   Existing tests and manual test.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org