You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "LuciferYang (via GitHub)" <gi...@apache.org> on 2023/08/21 08:56:17 UTC
[GitHub] [spark] LuciferYang commented on pull request #42581: [WIP] AWS sdk upgrade
LuciferYang commented on PR #42581:
URL: https://github.com/apache/spark/pull/42581#issuecomment-1685925739
I have two concerns about continuing to maintain this module:
1. This module has 60 UTs, but 24 are ignored due to the environment variable `ENABLE_KINESIS_TESTS` defaulting to 0.
```
[info] Tests: succeeded 34, failed 0, canceled 0, ignored 24, pending 0
[info] All tests passed.
```
However, when I run `ENABLE_KINESIS_TESTS=1 build/sbt "streaming-kinesis-asl/test" -Pkinesis-asl`, these previously ignored test cases fail.
```
[info] *** 4 SUITES ABORTED ***
[error] Error: Total 36, Failed 0, Errors 4, Passed 32
[error] Error during tests:
[error] org.apache.spark.streaming.kinesis.WithoutAggregationKinesisBackedBlockRDDSuite
[error] org.apache.spark.streaming.kinesis.WithAggregationKinesisBackedBlockRDDSuite
[error] org.apache.spark.streaming.kinesis.WithAggregationKinesisStreamSuite
[error] org.apache.spark.streaming.kinesis.WithoutAggregationKinesisStreamSuite
[error] (streaming-kinesis-asl / Test / test) sbt.TestsFailedException: Tests unsuccessful
```
2. The CI does not provide sufficient check for this module.
https://github.com/apache/spark/blob/a21e19b6e7ac4c4c77b39d93a2da2cbe1c88c4c8/.github/workflows/build_and_test.yml#L139
So, I think we should complete the TODO SPARK-32246 before proceeding with this update.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org