You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sa...@apache.org on 2021/11/13 06:22:54 UTC
[spark] branch master updated: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest
This is an automated email from the ASF dual-hosted git repository.
sarutak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 8b45a08 [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest
8b45a08 is described below
commit 8b45a08b763e9ee6c75b039893af3de5e5167643
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Sat Nov 13 15:21:59 2021 +0900
[SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest
### What changes were proposed in this pull request?
This PR is a follow-up of #34548. This is missed due to `-Pyarn` profile.
### Why are the changes needed?
This is required to pass `yarn` module on Apple Silicon.
**BEFORE**
```
$ build/sbt "yarn/test"
...
[info] YarnShuffleServiceSuite:
[info] org.apache.spark.network.yarn.YarnShuffleServiceSuite *** ABORTED *** (20 milliseconds)
[info] java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni64-1.8
...
```
**AFTER**
```
$ build/sbt "yarn/test" -Pyarn -Dtest.exclude.tags=org.apache.spark.tags.ExtendedLevelDBTest
...
[info] Run completed in 4 minutes, 57 seconds.
[info] Total number of tests run: 135
[info] Suites: completed 18, aborted 0
[info] Tests: succeeded 135, failed 0, canceled 1, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 319 s (05:19), completed Nov 12, 2021, 4:53:14 PM
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
A manual test on Apple Silicon.
```
$ build/sbt "yarn/test" -Pyarn -Dtest.exclude.tags=org.apache.spark.tags.ExtendedLevelDBTest
```
Closes #34576 from dongjoon-hyun/SPARK-37282-2.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Kousuke Saruta <sa...@oss.nttdata.com>
---
.../scala/org/apache/spark/network/yarn/YarnShuffleServiceSuite.scala | 2 ++
1 file changed, 2 insertions(+)
diff --git a/resource-managers/yarn/src/test/scala/org/apache/spark/network/yarn/YarnShuffleServiceSuite.scala b/resource-managers/yarn/src/test/scala/org/apache/spark/network/yarn/YarnShuffleServiceSuite.scala
index b2025aa..38d2247 100644
--- a/resource-managers/yarn/src/test/scala/org/apache/spark/network/yarn/YarnShuffleServiceSuite.scala
+++ b/resource-managers/yarn/src/test/scala/org/apache/spark/network/yarn/YarnShuffleServiceSuite.scala
@@ -46,8 +46,10 @@ import org.apache.spark.internal.config._
import org.apache.spark.network.shuffle.{NoOpMergedShuffleFileManager, RemoteBlockPushResolver, ShuffleTestAccessor}
import org.apache.spark.network.shuffle.protocol.ExecutorShuffleInfo
import org.apache.spark.network.util.TransportConf
+import org.apache.spark.tags.ExtendedLevelDBTest
import org.apache.spark.util.Utils
+@ExtendedLevelDBTest
class YarnShuffleServiceSuite extends SparkFunSuite with Matchers with BeforeAndAfterEach {
private[yarn] var yarnConfig: YarnConfiguration = null
private[yarn] val SORT_MANAGER = "org.apache.spark.shuffle.sort.SortShuffleManager"
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org