You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by HyukjinKwon <gi...@git.apache.org> on 2017/01/15 14:28:01 UTC

[GitHub] spark pull request #16586: [WIP][SPARK-19117][SPARK-18922][TESTS] Fix the re...

GitHub user HyukjinKwon opened a pull request:

    https://github.com/apache/spark/pull/16586

    [WIP][SPARK-19117][SPARK-18922][TESTS] Fix the rest of flaky, newly introduced and missed test failures on Windows

    ## What changes were proposed in this pull request?
    
    **Failed tests**
    
    ```
    org.apache.spark.sql.hive.execution.HiveQuerySuite:
     - transform with SerDe3 *** FAILED ***
     - transform with SerDe4 *** FAILED ***
    ```
    
    ```
    org.apache.spark.sql.hive.execution.HiveDDLSuite:
     - create hive serde table with new syntax *** FAILED ***
     - add/drop partition with location - managed table *** FAILED ***
    ```
    
    **Aborted tests**
    
    ```
    Exception encountered when attempting to run a suite with class name: org.apache.spark.sql.hive.execution.HiveSerDeSuite *** ABORTED *** (157 milliseconds)
       org.apache.spark.sql.AnalysisException: LOAD DATA input path does not exist: C:projectssparksqlhive   argetscala-2.11   est-classesdatafilessales.txt;
    ```
    
    **Flaky tests**
    
    ```
    org.apache.spark.scheduler.SparkListenerSuite:
     - local metrics *** FAILED ***
    ```
    
    ## How was this patch tested?
    
    Manually tested via AppVeyor.
    
    **Failed tests**
    
    ```
    org.apache.spark.sql.hive.execution.HiveQuerySuite:
     - transform with SerDe3 !!! CANCELED !!! (0 milliseconds)
     - transform with SerDe4 !!! CANCELED !!! (0 milliseconds)
    ```
    
    ```
    org.apache.spark.sql.hive.execution.HiveDDLSuite:
     - create hive serde table with new syntax (1 second, 672 milliseconds)
    ```
    
    **Aborted tests**
    
    ```
    spark.sql.hive.execution.HiveSerDeSuite:
     - Read with RegexSerDe (2 seconds, 142 milliseconds)
     - Read and write with LazySimpleSerDe (tab separated) (2 seconds)
     - Read with AvroSerDe (1 second, 47 milliseconds)
     - Read Partitioned with AvroSerDe (1 second, 422 milliseconds)
    ```
    
    **Flaky tests**
    
    ```
    org.apache.spark.scheduler.SparkListenerSuite:
     - local metrics (4 seconds, 562 milliseconds)
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/HyukjinKwon/spark set-path-appveyor

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/16586.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #16586
    
----
commit 9ce8846dfc3678649cc041dc762fcc6ed8038527
Author: hyukjinkwon <gu...@gmail.com>
Date:   2017-01-15T14:11:45Z

    Fix flaky, newly introduced and missed test failures on Windows

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #16586: [WIP][SPARK-19117][SPARK-18922][TESTS] Fix the rest of f...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/16586
  
    **[Test build #71399 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71399/testReport)** for PR 16586 at commit [`9ce8846`](https://github.com/apache/spark/commit/9ce8846dfc3678649cc041dc762fcc6ed8038527).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #16586: [WIP][SPARK-19117][SPARK-18922][TESTS] Fix the rest of f...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

https://github.com/apache/spark/pull/16586

@srowen, @shivaram and @felixcheung this is the problem I previously reported to three of you before - [![PR-16586](https://ci.appveyor.com/api/projects/status/github/spark-test/spark?branch=E7636D1D-41D6-4E36-9E15-B26EBB03B9E1&svg=true)](https://ci.appveyor.com/project/spark-test/spark/branch/E7636D1D-41D6-4E36-9E15-B26EBB03B9E1) I have jobs in the queue at AppVeyor but it does not start.

I observed that in AFS account at Spark - https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/history. This can be simply just jobs queued up in other projects or same thing happening. To verify this, I should check if any job is running in any project in Apache via AppVeyor. In case of mine, it is easy to check because I only have single project, spark.

So, any of you (as a committer or PMC) knows if there is any easy way to retreive any list of projects that use AppVeyor in Apache? I am willing to check each project once I get the list and report this to AppVeyor if the same thing happens to AFS account too.

---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #16586: [WIP][SPARK-19117][SPARK-18922][TESTS] Fix the rest of f...

Posted by srowen <gi...@git.apache.org>.

Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/16586
  
    @HyukjinKwon I have no info on this I'm afraid. From searching the internet for "appveyor apache" I think Thrift and Nifi might be using it too. Are you saying your jobs never run in AppVeyor, or take a long time to schedule? You're welcome to tackle the problem if you can.
    
    what's the status here -- do we still need a positive result from Appveyor before merging this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org