You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/09/14 01:49:16 UTC
[GitHub] [spark] TongWeii opened a new pull request #33986: Support sql overwrite a path that is also being read from when partit…
TongWeii opened a new pull request #33986:
URL: https://github.com/apache/spark/pull/33986
### What changes were proposed in this pull request?
```
// non-partitioned table overwrite
CREATE TABLE tbl (col1 INT, col2 STRING) USING PARQUET;
INSERT OVERWRITE TABLE tbl SELECT 0,1;
INSERT OVERWRITE TABLE tbl SELECT * FROM tbl;
// partitioned table static overwrite
CREATE TABLE tbl (col1 INT, col2 STRING) USING PARQUET PARTITIONED BY (pt1 INT);
INSERT OVERWRITE TABLE tbl PARTITION(p1=2021) SELECT 0 AS col1,1 AS col2;
INSERT OVERWRITE TABLE tbl PARTITION(p1=2021) SELECT col1, col2 FROM WHERE p1=2021;
```
When we run the above query, an error will be throwed "Cannot overwrite a path that is also being read from"
We need to support this operation when the spark.sql.sources.partitionOverwriteMode is dynamic
### How was this patch tested?
Unit tests -> InsertSuite.scala
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33986: Support sql overwrite a path that is also being read from when partit…
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33986:
URL: https://github.com/apache/spark/pull/33986#issuecomment-918726537
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #33986: [SPARK-36727][SQL]Support sql overwrite a path that is also being read from when partit…
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #33986:
URL: https://github.com/apache/spark/pull/33986
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33986: Support sql overwrite a path that is also being read from when partit…
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33986:
URL: https://github.com/apache/spark/pull/33986#issuecomment-928993352
The same issue of https://issues.apache.org/jira/browse/SPARK-32838 ? pls add jira id in title. like [SPARK-xxx][SQL] {title}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] TongWeii commented on pull request #33986: Support sql overwrite a path that is also being read from when partit…
Posted by GitBox <gi...@apache.org>.
TongWeii commented on pull request #33986:
URL: https://github.com/apache/spark/pull/33986#issuecomment-920756527
ping @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #33986: Support sql overwrite a path that is also being read from when partit…
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #33986:
URL: https://github.com/apache/spark/pull/33986#issuecomment-928993352
The same issue of https://issues.apache.org/jira/browse/SPARK-32838 ? pls add jira id in title. like [SPARK-xxx][SQL] {title}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #33986: [SPARK-36727][SQL]Support sql overwrite a path that is also being read from when partit…
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #33986:
URL: https://github.com/apache/spark/pull/33986#issuecomment-1033189637
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] TongWeii commented on pull request #33986: Support sql overwrite a path that is also being read from when partit…
Posted by GitBox <gi...@apache.org>.
TongWeii commented on pull request #33986:
URL: https://github.com/apache/spark/pull/33986#issuecomment-918729534
cc @viirya
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] TongWeii removed a comment on pull request #33986: Support sql overwrite a path that is also being read from when partit…
Posted by GitBox <gi...@apache.org>.
TongWeii removed a comment on pull request #33986:
URL: https://github.com/apache/spark/pull/33986#issuecomment-920756014
ping @c lou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] TongWeii commented on pull request #33986: Support sql overwrite a path that is also being read from when partit…
Posted by GitBox <gi...@apache.org>.
TongWeii commented on pull request #33986:
URL: https://github.com/apache/spark/pull/33986#issuecomment-920756014
ping @c lou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org