[GitHub] spark issue #22099: [SPARK-25111][BUILD] increment kinesis client/producer &...

[steveloughran <gi...@git.apache.org>]

Github user steveloughran commented on the issue:

    https://github.com/apache/spark/pull/22099
  
    As noted in #22146; stripping off bouncy castle and upgrading the SDK worked. But a local test run of just this patch brought up the same error seen in #22081 
    
    ```
    WithoutAggregationKinesisStreamSuite:
    - KinesisUtils API
    - RDD generation
    - basic operation
    - custom message handling *** FAILED ***
      The code passed to eventually never returned normally. Attempted 20 times over 2.092846262916667 minutes. Last failure message: collected.synchronized[Boolean](KinesisStreamTests.this.convertToEqualizer[scala.collection.mutable.HashSet[Int]](collected).===(modData.toSet[Int])(scalactic.this.Equality.default[scala.collection.mutable.HashSet[Int]])) was false
      Data received does not match data sent. (KinesisStreamSuite.scala:230)
    - Kinesis read with custom configurations
    - split and merge shards in a stream
    - failure recovery *** FAILED ***
      The code passed to eventually never returned normally. Attempted 105 times over 2.0055098129 minutes. Last failure message: isCheckpointPresent was true, but 0 was not greater than 10. (KinesisStreamSuite.scala:398)
    ```
    That wasn't a full clean build, so let's see what Jenkins says and some more test runs tomorrow. It could just be this is all showing up some flakiness in the test case. At the very least, some more details on the failure might be good.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org