You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by seancxmao <gi...@git.apache.org> on 2018/10/26 07:35:51 UTC
[GitHub] spark pull request #22846: [SPARK-25797][SQL][DOCS] Add migration doc for so...
GitHub user seancxmao opened a pull request:
https://github.com/apache/spark/pull/22846
[SPARK-25797][SQL][DOCS] Add migration doc for solving issues caused by view canonicalization approach change
## What changes were proposed in this pull request?
Since Spark 2.2, view definitions are stored in a different way from prior versions. This may cause Spark unable to read views created by prior versions. See [SPARK-25797](https://issues.apache.org/jira/browse/SPARK-25797) for more details.
Basically, we have 2 options.
1) Make Spark 2.2+ able to get older view definitions back. Since the expanded text is buggy and unusable, we have to use original text (this is possible with [SPARK-25459](https://issues.apache.org/jira/browse/SPARK-25459)). However, because older Spark versions don't save the context for the database, we cannot always get correct view definitions without view default database.
2) Recreate the views by `ALTER VIEW AS` or `CREATE OR REPLACE VIEW AS`.
This PR aims to add migration doc to help users troubleshoot this issue by above option 2.
## How was this patch tested?
N/A.
Docs are generated and checked locally
```
cd docs
SKIP_API=1 jekyll serve --watch
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/seancxmao/spark SPARK-25797
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22846.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22846
----
commit a6f4d540d948a887d9dd88b5fc83d3dcf065491f
Author: seancxmao <se...@...>
Date: 2018-10-26T06:08:42Z
add migration doc
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22846: [SPARK-25797][SQL][DOCS] Add migration doc for solving i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22846
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22846: [SPARK-25797][SQL][DOCS] Add migration doc for solving i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22846
**[Test build #98077 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98077/testReport)** for PR 22846 at commit [`a6f4d54`](https://github.com/apache/spark/commit/a6f4d540d948a887d9dd88b5fc83d3dcf065491f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22846: [SPARK-25797][SQL][DOCS] Add migration doc for solving i...
Posted by seancxmao <gi...@git.apache.org>.
Github user seancxmao commented on the issue:
https://github.com/apache/spark/pull/22846
@cloud-fan Sure, I will send a new PR for 2.3. Thanks you for review this.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22846: [SPARK-25797][SQL][DOCS] Add migration doc for solving i...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22846
**[Test build #98077 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98077/testReport)** for PR 22846 at commit [`a6f4d54`](https://github.com/apache/spark/commit/a6f4d540d948a887d9dd88b5fc83d3dcf065491f).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22846: [SPARK-25797][SQL][DOCS] Add migration doc for solving i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22846
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22846: [SPARK-25797][SQL][DOCS] Add migration doc for so...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22846
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22846: [SPARK-25797][SQL][DOCS] Add migration doc for solving i...
Posted by seancxmao <gi...@git.apache.org>.
Github user seancxmao commented on the issue:
https://github.com/apache/spark/pull/22846
@cloud-fan PR for 2.3 is submitted. Please see #22851.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22846: [SPARK-25797][SQL][DOCS] Add migration doc for solving i...
Posted by seancxmao <gi...@git.apache.org>.
Github user seancxmao commented on the issue:
https://github.com/apache/spark/pull/22846
@jiangxb1987 @cloud-fan @gatorsmile Could you please kindly review this when you have time?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22846: [SPARK-25797][SQL][DOCS] Add migration doc for solving i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22846
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22846: [SPARK-25797][SQL][DOCS] Add migration doc for solving i...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22846
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98077/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22846: [SPARK-25797][SQL][DOCS] Add migration doc for solving i...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22846
LGTM, merging to master/2.4!
it conflicts with 2.3, can you send a new PR? thanks!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org