You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by HyukjinKwon <gi...@git.apache.org> on 2017/08/17 17:08:53 UTC

[GitHub] spark pull request #18971: [WIP][SPARK-21764][TESTS] Fix tests failures on W...

GitHub user HyukjinKwon opened a pull request:

    https://github.com/apache/spark/pull/18971

    [WIP][SPARK-21764][TESTS] Fix tests failures on Windows: resources not being closed and incorrect paths

    ## What changes were proposed in this pull request?
    
    `org.apache.spark.deploy.RPackageUtilsSuite`
    
    ```
     - jars without manifest return false *** FAILED *** (109 milliseconds)
       java.io.IOException: Unable to delete file: C:\projects\spark\target\tmp\1500266936418-0\dep1-c.jar
    ```
    
    `org.apache.spark.deploy.SparkSubmitSuite`
    
    ```
     - download one file to local *** FAILED *** (16 milliseconds)
       java.net.URISyntaxException: Illegal character in authority at index 6: s3a://C:\projects\spark\target\tmp\test2630198944759847458.jar
    
     - download list of files to local *** FAILED *** (0 milliseconds)
       java.net.URISyntaxException: Illegal character in authority at index 6: s3a://C:\projects\spark\target\tmp\test2783551769392880031.jar
    ```
    
    `org.apache.spark.scheduler.ReplayListenerSuite`
    
    ```
     - Replay compressed inprogress log file succeeding on partial read (156 milliseconds)
       Exception encountered when attempting to run a suite with class name: 
       org.apache.spark.scheduler.ReplayListenerSuite *** ABORTED *** (1 second, 391 milliseconds)
       java.io.IOException: Failed to delete: C:\projects\spark\target\tmp\spark-8f3cacd6-faad-4121-b901-ba1bba8025a0
    
     - End-to-end replay *** FAILED *** (62 milliseconds)
       java.io.IOException: No FileSystem for scheme: C
    
     - End-to-end replay with compression *** FAILED *** (110 milliseconds)
       java.io.IOException: No FileSystem for scheme: C
    ```
    
    `org.apache.spark.sql.hive.StatisticsSuite`
    
    ```
     - SPARK-21079 - analyze table with location different than that of individual partitions *** FAILED *** (875 milliseconds)
       org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.IllegalArgumentException: Can not create a Path from an empty string);
    
     - SPARK-21079 - analyze partitioned table with only a subset of partitions visible *** FAILED *** (47 milliseconds)
       org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.IllegalArgumentException: Can not create a Path from an empty string);
    ```
    
    Note that this PR does not fix:
    
    `org.apache.spark.deploy.SparkSubmitSuite`
    
    ```
     - launch simple application with spark-submit with redaction *** FAILED *** (172 milliseconds)
       java.util.NoSuchElementException: next on empty iterator
    ```
    
    I can't reproduce this on my Windows machine but looks appearntly consistently failed on AppVeyor. This one is unclear to me yet and hard to debug so I did not include this one for now.
    
    ## How was this patch tested?
    
    Manually via AppVeyor:
    
    **Before**
    
    - `org.apache.spark.deploy.RPackageUtilsSuite`: https://ci.appveyor.com/project/spark-test/spark/build/771-windows-fix/job/8t8ra3lrljuir7q4
    - `org.apache.spark.deploy.SparkSubmitSuite`: https://ci.appveyor.com/project/spark-test/spark/build/771-windows-fix/job/taquy84yudjjen64
    - `org.apache.spark.scheduler.ReplayListenerSuite`: https://ci.appveyor.com/project/spark-test/spark/build/771-windows-fix/job/24omrfn2k0xfa9xq
    - `org.apache.spark.sql.hive.StatisticsSuite`: https://ci.appveyor.com/project/spark-test/spark/build/771-windows-fix/job/2079y1plgj76dc9l
    
    **After**
    
    - `org.apache.spark.deploy.RPackageUtilsSuite`: https://ci.appveyor.com/project/spark-test/spark/build/775-windows-fix/job/3803dbfn89ne1164
    - `org.apache.spark.deploy.SparkSubmitSuite`: https://ci.appveyor.com/project/spark-test/spark/build/775-windows-fix/job/m5l350dp7u9a4xjr
    - `org.apache.spark.scheduler.ReplayListenerSuite`: https://ci.appveyor.com/project/spark-test/spark/build/775-windows-fix/job/565vf74pp6bfdk18
    - `org.apache.spark.sql.hive.StatisticsSuite`: https://ci.appveyor.com/project/spark-test/spark/build/775-windows-fix/job/qm78tsk8c37jb6s4
    
    Jenkins tests are required and AppVeyor tests will be triggered.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/HyukjinKwon/spark windows-fixes

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/18971.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #18971
    
----
commit b979546312da6d3cee003396c283c090f1715597
Author: hyukjinkwon <gu...@gmail.com>
Date:   2017-08-17T10:16:23Z

    Resource closing and path related problems on Windows in tests

commit 90195485a6e8b95b1cb4c5606e508a05b6fdba11
Author: hyukjinkwon <gu...@gmail.com>
Date:   2017-08-17T10:30:38Z

    Add some comments

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #18971: [SPARK-21764][TESTS] Fix tests failures on Window...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/18971#discussion_r135382368
  
    --- Diff: core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala ---
    @@ -112,17 +112,19 @@ class ReplayListenerSuite extends SparkFunSuite with BeforeAndAfter with LocalSp
     
         // Verify the replay returns the events given the input maybe truncated.
         val logData = EventLoggingListener.openEventLog(logFilePath, fileSystem)
    -    val failingStream = new EarlyEOFInputStream(logData, buffered.size - 10)
    -    replayer.replay(failingStream, logFilePath.toString, true)
    +    Utils.tryWithResource(new EarlyEOFInputStream(logData, buffered.size - 10)) { failingStream =>
    --- End diff --
    
    Here `EarlyEOFInputStream` was not being closed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #18971: [SPARK-21764][TESTS] Fix tests failures on Windows: reso...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/18971
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #18971: [SPARK-21764][TESTS] Fix tests failures on Windows: reso...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/18971
  
    **[Test build #81152 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81152/testReport)** for PR 18971 at commit [`236b986`](https://github.com/apache/spark/commit/236b986bfd5fcedfe390ad3b6b566d53f84dd89c).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #18971: [SPARK-21764][TESTS] Fix tests failures on Window...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/18971#discussion_r135382393
  
    --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala ---
    @@ -203,7 +203,7 @@ class StatisticsSuite extends StatisticsCollectionTestBase with TestHiveSingleto
               sql(s"INSERT INTO TABLE $tableName PARTITION (ds='$ds') SELECT * FROM src")
             }
     
    -        sql(s"ALTER TABLE $tableName SET LOCATION '$path'")
    +        sql(s"ALTER TABLE $tableName SET LOCATION '${path.toURI}'")
    --- End diff --
    
    These tests here do not look dedicated to test path. I have fixed those so far.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #18971: [SPARK-21764][TESTS] Fix tests failures on Windows: reso...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/18971
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81239/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #18971: [WIP][SPARK-21764][TESTS] Fix tests failures on Windows:...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/18971
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #18971: [SPARK-21764][TESTS] Fix tests failures on Window...

Posted by jiangxb1987 <gi...@git.apache.org>.

Github user jiangxb1987 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/18971#discussion_r135624751
  
    --- Diff: core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala ---
    @@ -112,17 +112,19 @@ class ReplayListenerSuite extends SparkFunSuite with BeforeAndAfter with LocalSp
     
         // Verify the replay returns the events given the input maybe truncated.
         val logData = EventLoggingListener.openEventLog(logFilePath, fileSystem)
    -    val failingStream = new EarlyEOFInputStream(logData, buffered.size - 10)
    -    replayer.replay(failingStream, logFilePath.toString, true)
    +    Utils.tryWithResource(new EarlyEOFInputStream(logData, buffered.size - 10)) { failingStream =>
    +      replayer.replay(failingStream, logFilePath.toString, true)
     
    -    assert(eventMonster.loggedEvents.size === 1)
    -    assert(failingStream.didFail)
    +      assert(eventMonster.loggedEvents.size === 1)
    +      assert(failingStream.didFail)
    +    }
     
         // Verify the replay throws the EOF exception since the input may not be truncated.
         val logData2 = EventLoggingListener.openEventLog(logFilePath, fileSystem)
    -    val failingStream2 = new EarlyEOFInputStream(logData2, buffered.size - 10)
    -    intercept[EOFException] {
    -      replayer.replay(failingStream2, logFilePath.toString, false)
    +    Utils.tryWithResource(new EarlyEOFInputStream(logData2, buffered.size - 10)) { failingStream2 =>
    --- End diff --
    
    nit: I think we can still use `failingStream` here?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #18971: [SPARK-21764][TESTS] Fix tests failures on Windows: reso...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/18971
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81151/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #18971: [SPARK-21764][TESTS] Fix tests failures on Window...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/18971#discussion_r135382352
  
    --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala ---
    @@ -824,7 +824,7 @@ class SparkSubmitSuite
         val hadoopConf = new Configuration()
         val tmpDir = Files.createTempDirectory("tmp").toFile
         updateConfWithFakeS3Fs(hadoopConf)
    -    val sourcePath = s"s3a://${jarFile.getAbsolutePath}"
    +    val sourcePath = s"s3a://${jarFile.toURI.getPath}"
    --- End diff --
    
    **Windows:**
    
    Before:
    
    ```
    scala> f.toURI.getPath
    res1: String = /C:/a/b/c
    ```
    
    After:
    
    ```
    scala> f.getAbsolutePath
    res2: String = C:\a\b\c
    ```
    
    **Linux:**
    
    Before:
    
    ```
    scala> new File("/a/b/c").getAbsolutePath
    res0: String = /a/b/c
    ```
    
    After:
    
    ```
    scala> new File("/a/b/c").toURI.getPath
    res1: String = /a/b/c
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #18971: [SPARK-21764][TESTS] Fix tests failures on Window...

Posted by asfgit <gi...@git.apache.org>.

Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/18971


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #18971: [SPARK-21764][TESTS] Fix tests failures on Window...

Posted by sarutak <gi...@git.apache.org>.

Github user sarutak commented on a diff in the pull request:

    https://github.com/apache/spark/pull/18971#discussion_r134659662
  
    --- Diff: core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala ---
    @@ -151,7 +153,10 @@ class ReplayListenerSuite extends SparkFunSuite with BeforeAndAfter with LocalSp
        * assumption that the event logging behavior is correct (tested in a separate suite).
        */
       private def testApplicationReplay(codecName: Option[String] = None) {
    -    val logDirPath = Utils.getFilePath(testDir, "test-replay")
    +    val logDir = new File(testDir.getAbsolutePath, "test-replay")
    +    // Here, it creates `Path` from the URI instead of the absolute path for the explicit file
    +    // scheme so that the string representation of this `Path` has leading file scheme correctly.
    +    val logDirPath = new Path(logDir.toURI)
    --- End diff --
    
    @HyukjinKwon I think this change it self looks reasonable.
    `resolveURI` should seems to be fixed so that Windows' path is handled correctly. 
    If  `C:/path/to/some/file` is passed to `resolveURI`, the letter drive "C" should not parsed as URI scheme.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #18971: [SPARK-21764][TESTS] Fix tests failures on Window...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/18971#discussion_r134633585
  
    --- Diff: core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala ---
    @@ -151,7 +153,10 @@ class ReplayListenerSuite extends SparkFunSuite with BeforeAndAfter with LocalSp
        * assumption that the event logging behavior is correct (tested in a separate suite).
        */
       private def testApplicationReplay(codecName: Option[String] = None) {
    -    val logDirPath = Utils.getFilePath(testDir, "test-replay")
    +    val logDir = new File(testDir.getAbsolutePath, "test-replay")
    +    // Here, it creates `Path` from the URI instead of the absolute path for the explicit file
    +    // scheme so that the string representation of this `Path` has leading file scheme correctly.
    +    val logDirPath = new Path(logDir.toURI)
    --- End diff --
    
    I have Windows one set up properly for dev env. So, it is not really a big deal to test. I am fine with asking more cases in the future.
    
    ```scala
    println("=== org.apache.spark.util.Utils.resolveURI")
    println(Utils.resolveURI("C:\\WINDOWS").toString)
    println(Utils.resolveURI("/C:/WINDOWS").toString)
    println(Utils.resolveURI("C:/WINDOWS").toString)
    println
    println(Utils.resolveURI("C:\\WINDOWS").getScheme)
    println(Utils.resolveURI("/C:/WINDOWS").getScheme)
    println(Utils.resolveURI("C:/WINDOWS").getScheme)
    println
    
    println("=== java.io.File")
    println(new File("C:\\WINDOWS").toURI.toString)
    println(new File("/C:/WINDOWS").toURI.toString)
    println(new File("C:/WINDOWS").toURI.toString)
    println
    println(new File("C:\\WINDOWS").toURI.getScheme)
    println(new File("/C:/WINDOWS").toURI.getScheme)
    println(new File("C:/WINDOWS").toURI.getScheme)
    println
    
    println("=== org.apache.hadoop.fs.Path")
    println(new Path("C:\\WINDOWS").toUri.toString)
    println(new Path("/C:/WINDOWS").toUri.toString)
    println(new Path("C:/WINDOWS").toUri.toString)
    println
    println(new Path("C:\\WINDOWS").toString)
    println(new Path("/C:/WINDOWS").toString)
    println(new Path("C:/WINDOWS").toString)
    println
    println(new Path("C:\\WINDOWS").toUri.getScheme)
    println(new Path("/C:/WINDOWS").toUri.getScheme)
    println(new Path("C:/WINDOWS").toUri.getScheme)
    println
    
    println("=== java.io.File.toURI and org.apache.hadoop.fs.Path")
    println(new Path(new File("C:\\WINDOWS").toURI).toUri.toString)
    println(new Path(new File("/C:/WINDOWS").toURI).toUri.toString)
    println(new Path(new File("C:/WINDOWS").toURI).toUri.toString)
    println
    println(new Path(new File("C:\\WINDOWS").toURI).toString)
    println(new Path(new File("/C:/WINDOWS").toURI).toString)
    println(new Path(new File("C:/WINDOWS").toURI).toString)
    println
    println(new Path(new File("C:\\WINDOWS").toURI).toUri.getScheme)
    println(new Path(new File("/C:/WINDOWS").toURI).toUri.getScheme)
    println(new Path(new File("C:/WINDOWS").toURI).toUri.getScheme)
    ```
    
    produced
    
    ```
    === org.apache.spark.util.Utils.resolveURI
    file:/C:/WINDOWS/
    file:/C:/WINDOWS/
    C:/WINDOWS
    
    file
    file
    C
    
    === java.io.File
    file:/C:/WINDOWS/
    file:/C:/WINDOWS/
    file:/C:/WINDOWS/
    
    file
    file
    file
    
    === org.apache.hadoop.fs.Path
    /C:/WINDOWS
    /C:/WINDOWS
    /C:/WINDOWS
    
    C:/WINDOWS
    C:/WINDOWS
    C:/WINDOWS
    
    null
    null
    null
    
    === java.io.File.toURI and org.apache.hadoop.fs.Path
    file:/C:/WINDOWS/
    file:/C:/WINDOWS/
    file:/C:/WINDOWS/
    
    file:/C:/WINDOWS/
    file:/C:/WINDOWS/
    file:/C:/WINDOWS/
    
    file
    file
    file
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #18971: [SPARK-21764][TESTS] Fix tests failures on Window...

Posted by vanzin <gi...@git.apache.org>.

Github user vanzin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/18971#discussion_r134550113
  
    --- Diff: core/src/test/scala/org/apache/spark/scheduler/ReplayListenerSuite.scala ---
    @@ -151,7 +153,10 @@ class ReplayListenerSuite extends SparkFunSuite with BeforeAndAfter with LocalSp
        * assumption that the event logging behavior is correct (tested in a separate suite).
        */
       private def testApplicationReplay(codecName: Option[String] = None) {
    -    val logDirPath = Utils.getFilePath(testDir, "test-replay")
    +    val logDir = new File(testDir.getAbsolutePath, "test-replay")
    +    // Here, it creates `Path` from the URI instead of the absolute path for the explicit file
    +    // scheme so that the string representation of this `Path` has leading file scheme correctly.
    +    val logDirPath = new Path(logDir.toURI)
    --- End diff --
    
    From your description it sounds like `Utils.resolveURI` might not be correct for Windows paths. I don't have Windows available, so if you could try these it might help in understanding:
    
        Utils.resolveURI("C:\\WINDOWS")
        Utils.resolveURI("/C:/WINDOWS")
        Utils.resolveURI("C:/WINDOWS")
    
    The first two should return the same thing ("file:/C:/WINDOWS" or something along those lines) while the third I'm not sure, since it's ambiguous. But that's probably the cause of the change of behavior.
    
    Anyway the code change looks correct.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #18971: [SPARK-21764][TESTS] Fix tests failures on Windows: reso...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/18971
  
    **[Test build #81151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81151/testReport)** for PR 18971 at commit [`236b986`](https://github.com/apache/spark/commit/236b986bfd5fcedfe390ad3b6b566d53f84dd89c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #18971: [SPARK-21764][TESTS] Fix tests failures on Windows: reso...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/18971
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81152/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #18971: [WIP][SPARK-21764][TESTS] Fix tests failures on Windows:...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/18971
  
    **[Test build #80790 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80790/testReport)** for PR 18971 at commit [`9019548`](https://github.com/apache/spark/commit/90195485a6e8b95b1cb4c5606e508a05b6fdba11).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #18971: [SPARK-21764][TESTS] Fix tests failures on Windows: reso...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/18971
  
    **[Test build #81239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81239/testReport)** for PR 18971 at commit [`7d3716c`](https://github.com/apache/spark/commit/7d3716c07713ae928bdff9202d4a6e97cac2104e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #18971: [SPARK-21764][TESTS] Fix tests failures on Windows: reso...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/18971
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org