You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by cloud-fan <gi...@git.apache.org> on 2016/05/06 06:30:54 UTC

[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

GitHub user cloud-fan opened a pull request:

    https://github.com/apache/spark/pull/12949

    [SPARK-15173][SQL] DataFrameWriter.insertInto should work with datasource table stored in hive

    ## What changes were proposed in this pull request?
    
    When we read a datasource table stored in hive, we will construct a `DataSource` without `paths`. Then `DataFrameWriter.insertInto` can't work with it.  This PR adds the default table path as the `paths` of `DataSource`.
    
    
    ## How was this patch tested?
    
    new test in `SQLQuerySuite`

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/cloud-fan/spark bug

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/12949.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #12949
    
----
commit d83110921d18ee7f521eed7ba91b6ac1a73517ce
Author: Wenchen Fan <we...@databricks.com>
Date:   2016-05-06T05:54:32Z

    DataFrameWriter.insertInto should work with datasource table stored in hive

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:

    https://github.com/apache/spark/pull/12949#discussion_r62561309
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala ---
    @@ -239,8 +239,13 @@ case class DataSource(
         }
       }
     
    -  /** Create a resolved [[BaseRelation]] that can be used to read data from this [[DataSource]] */
    -  def resolveRelation(): BaseRelation = {
    +  /**
    +   * Create a resolved [[BaseRelation]] that can be used to read data from or write data into this
    +   * [[DataSource]]
    +   *
    +   * @param checkPathExist A flag to indicate whether to check the existence of path or not.
    --- End diff --
    
    It will be good to add why we need to this flag. Also, let's see how to clean it up in future. It's kind of weird to have a flag like this at here.
    
    I will update the doc and merge it. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:

    https://github.com/apache/spark/pull/12949#discussion_r62354994
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala ---
    @@ -239,8 +239,13 @@ case class DataSource(
         }
       }
     
    -  /** Create a resolved [[BaseRelation]] that can be used to read data from this [[DataSource]] */
    -  def resolveRelation(): BaseRelation = {
    +  /**
    +   * Create a resolved [[BaseRelation]] that can be used to read data from or write data into this
    +   * [[DataSource]]
    +   *
    +   * @param checkPathExist A flag to indicate whether to check the existence of path or not.
    +   */
    +  def resolveRelation(checkPathExist: Boolean = true): BaseRelation = {
    --- End diff --
    
    I am not sure I understand this change. For a `FileFormat`, when do we not need to check if the path exists?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217369387
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57966/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217391100
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57970/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:

    https://github.com/apache/spark/pull/12949#discussion_r62454544
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala ---
    @@ -239,8 +239,13 @@ case class DataSource(
         }
       }
     
    -  /** Create a resolved [[BaseRelation]] that can be used to read data from this [[DataSource]] */
    -  def resolveRelation(): BaseRelation = {
    +  /**
    +   * Create a resolved [[BaseRelation]] that can be used to read data from or write data into this
    +   * [[DataSource]]
    +   *
    +   * @param checkPathExist A flag to indicate whether to check the existence of path or not.
    +   */
    +  def resolveRelation(checkPathExist: Boolean = true): BaseRelation = {
    --- End diff --
    
    Anything will break if we do not have this flag?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217369299
  
    **[Test build #57966 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57966/consoleFull)** for PR 12949 at commit [`d831109`](https://github.com/apache/spark/commit/d83110921d18ee7f521eed7ba91b6ac1a73517ce).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217473754
  
    **[Test build #58003 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58003/consoleFull)** for PR 12949 at commit [`8f5b688`](https://github.com/apache/spark/commit/8f5b688754bc493548ecab714cb1a56136c3b02e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217468181
  
    **[Test build #58001 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58001/consoleFull)** for PR 12949 at commit [`167a7d6`](https://github.com/apache/spark/commit/167a7d6d8ad13a9d754e22dbd75cd9a16e9d1a56).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217498532
  
    **[Test build #58003 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58003/consoleFull)** for PR 12949 at commit [`8f5b688`](https://github.com/apache/spark/commit/8f5b688754bc493548ecab714cb1a56136c3b02e).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/12949#discussion_r62456196
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala ---
    @@ -239,8 +239,13 @@ case class DataSource(
         }
       }
     
    -  /** Create a resolved [[BaseRelation]] that can be used to read data from this [[DataSource]] */
    -  def resolveRelation(): BaseRelation = {
    +  /**
    +   * Create a resolved [[BaseRelation]] that can be used to read data from or write data into this
    +   * [[DataSource]]
    +   *
    +   * @param checkPathExist A flag to indicate whether to check the existence of path or not.
    +   */
    +  def resolveRelation(checkPathExist: Boolean = true): BaseRelation = {
    --- End diff --
    
    Previously we always check and throw exception if the path doesn't exist. However, when we create datasource table, the path does not exist and we will fail this check. That's why I added this flag: to disable this check for creating datasource table.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/12949#discussion_r62410521
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala ---
    @@ -239,8 +239,13 @@ case class DataSource(
         }
       }
     
    -  /** Create a resolved [[BaseRelation]] that can be used to read data from this [[DataSource]] */
    -  def resolveRelation(): BaseRelation = {
    +  /**
    +   * Create a resolved [[BaseRelation]] that can be used to read data from or write data into this
    +   * [[DataSource]]
    +   *
    +   * @param checkPathExist A flag to indicate whether to check the existence of path or not.
    +   */
    +  def resolveRelation(checkPathExist: Boolean = true): BaseRelation = {
    --- End diff --
    
    When we wanna read it. But for creating datasource table, the path does not exist and we should skip this check.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217391097
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217357762
  
    cc @yhuai


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217493037
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217390860
  
    **[Test build #57970 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57970/consoleFull)** for PR 12949 at commit [`338ae1d`](https://github.com/apache/spark/commit/338ae1dde1a88a39b3945f333caf2133cdfb2217).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217358270
  
    **[Test build #57966 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57966/consoleFull)** for PR 12949 at commit [`d831109`](https://github.com/apache/spark/commit/d83110921d18ee7f521eed7ba91b6ac1a73517ce).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217369385
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217492842
  
    **[Test build #58001 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58001/consoleFull)** for PR 12949 at commit [`167a7d6`](https://github.com/apache/spark/commit/167a7d6d8ad13a9d754e22dbd75cd9a16e9d1a56).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217493039
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58001/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/12949


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217498855
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217970297
  
    Thanks! Merging to master and branch 2.0.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on a diff in the pull request:

    https://github.com/apache/spark/pull/12949#discussion_r62560894
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala ---
    @@ -239,8 +239,13 @@ case class DataSource(
         }
       }
     
    -  /** Create a resolved [[BaseRelation]] that can be used to read data from this [[DataSource]] */
    -  def resolveRelation(): BaseRelation = {
    +  /**
    +   * Create a resolved [[BaseRelation]] that can be used to read data from or write data into this
    +   * [[DataSource]]
    +   *
    +   * @param checkPathExist A flag to indicate whether to check the existence of path or not.
    +   */
    +  def resolveRelation(checkPathExist: Boolean = true): BaseRelation = {
    --- End diff --
    
    ok. Got it. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217498865
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58003/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-15173][SQL] DataFrameWriter.insertInto ...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/12949#issuecomment-217378124
  
    **[Test build #57970 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57970/consoleFull)** for PR 12949 at commit [`338ae1d`](https://github.com/apache/spark/commit/338ae1dde1a88a39b3945f333caf2133cdfb2217).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org