[GitHub] [spark] srowen commented on issue #25558: [SPARK-28855][CORE][ML][SQL][STREAMING] Remove outdated usages of Experimental, Evolving annotations

[GitBox <gi...@apache.org>]

srowen commented on issue #25558: [SPARK-28855][CORE][ML][SQL][STREAMING] Remove outdated usages of Experimental, Evolving annotations
URL: https://github.com/apache/spark/pull/25558#issuecomment-524042709
 
 
   Uh oh, this might have uncovered some latent issue with the Python Kinesis tests:
   
   ```
   ======================================================================
   ERROR: test_kinesis_stream (pyspark.streaming.tests.test_kinesis.KinesisStreamTests)
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/home/jenkins/workspace/SparkPullRequestBuilder@2/python/pyspark/streaming/tests/test_kinesis.py", line 44, in test_kinesis_stream
       kinesisTestUtils = self.ssc._jvm.org.apache.spark.streaming.kinesis.KinesisTestUtils(2)
     File "/home/jenkins/workspace/SparkPullRequestBuilder@2/python/lib/py4j-0.10.8.1-src.zip/py4j/java_gateway.py", line 1554, in __call__
       answer, self._gateway_client, None, self._fqn)
     File "/home/jenkins/workspace/SparkPullRequestBuilder@2/python/lib/py4j-0.10.8.1-src.zip/py4j/protocol.py", line 328, in get_return_value
       format(target_id, ".", name), value)
   Py4JJavaError: An error occurred while calling None.org.apache.spark.streaming.kinesis.KinesisTestUtils.
   : java.lang.NoSuchMethodError: com.amazonaws.regions.Region.getAvailableEndpoints()Ljava/util/Collection;
   	at org.apache.spark.streaming.kinesis.KinesisTestUtils$.$anonfun$getRegionNameByEndpoint$1(KinesisTestUtils.scala:211)
   	at org.apache.spark.streaming.kinesis.KinesisTestUtils$.$anonfun$getRegionNameByEndpoint$1$adapted(KinesisTestUtils.scala:211)
   	at scala.collection.Iterator.find(Iterator.scala:993)
   	at scala.collection.Iterator.find$(Iterator.scala:990)
   	at scala.collection.AbstractIterator.find(Iterator.scala:1429)
   	at scala.collection.IterableLike.find(IterableLike.scala:81)
   	at scala.collection.IterableLike.find$(IterableLike.scala:80)
   	at scala.collection.AbstractIterable.find(Iterable.scala:56)
   	at org.apache.spark.streaming.kinesis.KinesisTestUtils$.getRegionNameByEndpoint(KinesisTestUtils.scala:211)
   	at org.apache.spark.streaming.kinesis.KinesisTestUtils.<init>(KinesisTestUtils.scala:46)
   ...
   ```
   
   I'll try to investigate separately.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org