You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/02/06 01:00:28 UTC
[GitHub] rdblue edited a comment on issue #23606: [SPARK-26666][SQL] Support
DSv2 overwrite and dynamic partition overwrite.
rdblue edited a comment on issue #23606: [SPARK-26666][SQL] Support DSv2 overwrite and dynamic partition overwrite.
URL: https://github.com/apache/spark/pull/23606#issuecomment-460864724
The ORC tests are failing due to commit df4c53e44bc9837a470ec66486237403868cb04f by @gengliangwang.
That commit adds a v2 implementation for ORC and uses it by default, which breaks because Overwrite is implemented in this PR using a v2 plan. In v2, overwrite does not create tables, CTAS creates tables. But CTAS cannot be implemented until a table catalog is available.
The solution is to default ORC to the v1 implementation until CTAS is available in v2.
@rxin, FYI since we recently discussed a migration plan. Defaulting sources to use v2 before there are implementations of the necessary SQL plans blocks the implementation of v2. We should migrate sources to v2 as a default after v2 operations are added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org