You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2023/09/25 03:10:21 UTC
[GitHub] [spark] HyukjinKwon opened a new pull request, #43084: [SPARK-45299][TESTS] Remove JDK 8 workaround in UtilsSuite
HyukjinKwon opened a new pull request, #43084:
URL: https://github.com/apache/spark/pull/43084
### What changes were proposed in this pull request?
This PR removes the legacy workaround for JDK 7 and below at SPARK-12486. The main code was cleaned up at SPARK-16182 but the test code was not cleaned up.
### Why are the changes needed?
To remove legacy workaround.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Fixed unittests.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a diff in pull request #43084: [SPARK-45299][TESTS] Remove JDK 8 workaround in UtilsSuite
Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on code in PR #43084:
URL: https://github.com/apache/spark/pull/43084#discussion_r1335366773
##########
core/src/test/scala/org/apache/spark/util/UtilsSuite.scala:
##########
@@ -1018,37 +1012,34 @@ class UtilsSuite extends SparkFunSuite with ResetSystemProperties {
signal(pid, "SIGKILL")
}
- if (SystemUtils.isJavaVersionAtLeast(JavaVersion.JAVA_1_8)) {
- // We'll make sure that forcibly terminating a process works by
- // creating a very misbehaving process. It ignores SIGTERM and has been SIGSTOPed. On
- // older versions of java, this will *not* terminate.
- val file = File.createTempFile("temp-file-name", ".tmp")
- file.deleteOnExit()
- val cmd =
- s"""
- |#!/usr/bin/env bash
- |trap "" SIGTERM
- |sleep 10
- """.stripMargin
- Files.write(cmd.getBytes(UTF_8), file)
- file.getAbsoluteFile.setExecutable(true)
-
- val process = new ProcessBuilder(file.getAbsolutePath).start()
- val pid = getPid(process)
- assert(pidExists(pid))
- try {
- signal(pid, "SIGSTOP")
- val startNs = System.nanoTime()
- val terminated = Utils.terminateProcess(process, 5000)
- assert(terminated.isDefined)
- process.waitFor(5, TimeUnit.SECONDS)
- val duration = System.nanoTime() - startNs
- // add a little extra time to allow a force kill to finish
- assert(duration < TimeUnit.SECONDS.toNanos(6))
- assert(!pidExists(pid))
- } finally {
- signal(pid, "SIGKILL")
- }
+ // We'll make sure that forcibly terminating a process works by
+ // creating a very misbehaving process. It ignores SIGTERM and has been SIGSTOPed. On
+ // older versions of java, this will *not* terminate.
+ val file = File.createTempFile("temp-file-name", ".tmp")
+ file.deleteOnExit()
+ val cmd =
+ s"""
+ |#!/usr/bin/env bash
+ |trap "" SIGTERM
+ |sleep 10
+ """.stripMargin
+ Files.write(cmd.getBytes(UTF_8), file)
+ file.getAbsoluteFile.setExecutable(true)
+
+ val process = new ProcessBuilder(file.getAbsolutePath).start()
+ process.toHandle.pid()
+ try {
+ signal(pid, "SIGSTOP")
Review Comment:
The scope of pid here is changed, we need to change it to var and re-assign
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #43084: [SPARK-45299][TESTS] Remove JDK 8 workaround in UtilsSuite
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #43084: [SPARK-45299][TESTS] Remove JDK 8 workaround in UtilsSuite
URL: https://github.com/apache/spark/pull/43084
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #43084: [SPARK-45299][TESTS] Remove JDK 8 workaround in UtilsSuite
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #43084:
URL: https://github.com/apache/spark/pull/43084#issuecomment-1732836567
cc @srowen
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a diff in pull request #43084: [SPARK-45299][TESTS] Remove JDK 8 workaround in UtilsSuite
Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on code in PR #43084:
URL: https://github.com/apache/spark/pull/43084#discussion_r1335338130
##########
core/src/test/scala/org/apache/spark/util/UtilsSuite.scala:
##########
@@ -1018,37 +1018,35 @@ class UtilsSuite extends SparkFunSuite with ResetSystemProperties {
signal(pid, "SIGKILL")
}
- if (SystemUtils.isJavaVersionAtLeast(JavaVersion.JAVA_1_8)) {
- // We'll make sure that forcibly terminating a process works by
- // creating a very misbehaving process. It ignores SIGTERM and has been SIGSTOPed. On
- // older versions of java, this will *not* terminate.
- val file = File.createTempFile("temp-file-name", ".tmp")
- file.deleteOnExit()
- val cmd =
- s"""
- |#!/usr/bin/env bash
- |trap "" SIGTERM
- |sleep 10
- """.stripMargin
- Files.write(cmd.getBytes(UTF_8), file)
- file.getAbsoluteFile.setExecutable(true)
-
- val process = new ProcessBuilder(file.getAbsolutePath).start()
- val pid = getPid(process)
- assert(pidExists(pid))
- try {
- signal(pid, "SIGSTOP")
- val startNs = System.nanoTime()
- val terminated = Utils.terminateProcess(process, 5000)
- assert(terminated.isDefined)
- process.waitFor(5, TimeUnit.SECONDS)
- val duration = System.nanoTime() - startNs
- // add a little extra time to allow a force kill to finish
- assert(duration < TimeUnit.SECONDS.toNanos(6))
- assert(!pidExists(pid))
- } finally {
- signal(pid, "SIGKILL")
- }
+ // We'll make sure that forcibly terminating a process works by
+ // creating a very misbehaving process. It ignores SIGTERM and has been SIGSTOPed. On
+ // older versions of java, this will *not* terminate.
+ val file = File.createTempFile("temp-file-name", ".tmp")
+ file.deleteOnExit()
+ val cmd =
+ s"""
+ |#!/usr/bin/env bash
+ |trap "" SIGTERM
+ |sleep 10
+ """.stripMargin
+ Files.write(cmd.getBytes(UTF_8), file)
+ file.getAbsoluteFile.setExecutable(true)
+
+ val process = new ProcessBuilder(file.getAbsolutePath).start()
+ val pid = getPid(process)
+ assert(pidExists(pid))
Review Comment:
```suggestion
process.toHandle.pid()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a diff in pull request #43084: [SPARK-45299][TESTS] Remove JDK 8 workaround in UtilsSuite
Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on code in PR #43084:
URL: https://github.com/apache/spark/pull/43084#discussion_r1335338130
##########
core/src/test/scala/org/apache/spark/util/UtilsSuite.scala:
##########
@@ -1018,37 +1018,35 @@ class UtilsSuite extends SparkFunSuite with ResetSystemProperties {
signal(pid, "SIGKILL")
}
- if (SystemUtils.isJavaVersionAtLeast(JavaVersion.JAVA_1_8)) {
- // We'll make sure that forcibly terminating a process works by
- // creating a very misbehaving process. It ignores SIGTERM and has been SIGSTOPed. On
- // older versions of java, this will *not* terminate.
- val file = File.createTempFile("temp-file-name", ".tmp")
- file.deleteOnExit()
- val cmd =
- s"""
- |#!/usr/bin/env bash
- |trap "" SIGTERM
- |sleep 10
- """.stripMargin
- Files.write(cmd.getBytes(UTF_8), file)
- file.getAbsoluteFile.setExecutable(true)
-
- val process = new ProcessBuilder(file.getAbsolutePath).start()
- val pid = getPid(process)
- assert(pidExists(pid))
- try {
- signal(pid, "SIGSTOP")
- val startNs = System.nanoTime()
- val terminated = Utils.terminateProcess(process, 5000)
- assert(terminated.isDefined)
- process.waitFor(5, TimeUnit.SECONDS)
- val duration = System.nanoTime() - startNs
- // add a little extra time to allow a force kill to finish
- assert(duration < TimeUnit.SECONDS.toNanos(6))
- assert(!pidExists(pid))
- } finally {
- signal(pid, "SIGKILL")
- }
+ // We'll make sure that forcibly terminating a process works by
+ // creating a very misbehaving process. It ignores SIGTERM and has been SIGSTOPed. On
+ // older versions of java, this will *not* terminate.
+ val file = File.createTempFile("temp-file-name", ".tmp")
+ file.deleteOnExit()
+ val cmd =
+ s"""
+ |#!/usr/bin/env bash
+ |trap "" SIGTERM
+ |sleep 10
+ """.stripMargin
+ Files.write(cmd.getBytes(UTF_8), file)
+ file.getAbsoluteFile.setExecutable(true)
+
+ val process = new ProcessBuilder(file.getAbsolutePath).start()
+ val pid = getPid(process)
+ assert(pidExists(pid))
Review Comment:
```suggestion
process.toHandle.pid()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org