You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2024/05/16 08:31:11 UTC
[PR] [SPARK-48031][SQL][FOLLOW-UP] Use ANSI-enabled cast in view lookup test [spark]
HyukjinKwon opened a new pull request, #46614:
URL: https://github.com/apache/spark/pull/46614
### What changes were proposed in this pull request?
This PR is a followup of https://github.com/apache/spark/pull/46267 that uses ANSI-enabled cast in the tests. It intentionally uses ANSI-enabled cast in `castColToType` when you look up a view.
### Why are the changes needed?
In order to fix the scheduled CI build without ANSI:
- https://github.com/apache/spark/actions/runs/9072308206/job/24960016975
- https://github.com/apache/spark/actions/runs/9072308206/job/24960019187
```
[info] - look up view relation *** FAILED *** (72 milliseconds)
[info] == FAIL: Plans do not match ===
[info] 'SubqueryAlias spark_catalog.db3.view1 'SubqueryAlias spark_catalog.db3.view1
[info] +- View (`spark_catalog`.`db3`.`view1`, ['col1, 'col2, 'a, 'b]) +- View (`spark_catalog`.`db3`.`view1`, ['col1, 'col2, 'a, 'b])
[info] +- 'Project [cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, col1, 0, 1) as int) AS col1#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, col2, 0, 1) as string) AS col2#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, a, 0, 1) as int) AS a#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, b, 0, 1) as string) AS b#0] +- 'Project [cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, col1, 0, 1) as int) AS col1#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, col2, 0, 1) as string) AS col2#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, a, 0, 1) as int) AS a#0, cast(getviewcolumnbynameandordinal(`spark_catalog`.`db3`.`view1`, b, 0, 1) as string) AS b#0]
[info] +- 'Project [*] +- 'Project [*]
[info] +- 'UnresolvedRelation [tbl1], [], false
```
```
[info] - look up view created before Spark 3.0 *** FAILED *** (452 milliseconds)
[info] == FAIL: Plans do not match ===
[info] 'SubqueryAlias spark_catalog.db3.view2 'SubqueryAlias spark_catalog.db3.view2
[info] +- View (`db3`.`view2`, ['col1, 'col2, 'a, 'b]) +- View (`db3`.`view2`, ['col1, 'col2, 'a, 'b])
[info] +- 'Project [cast(getviewcolumnbynameandordinal(`db3`.`view2`, col1, 0, 1) as int) AS col1#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, col2, 0, 1) as string) AS col2#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, a, 0, 1) as int) AS a#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, b, 0, 1) as string) AS b#0] +- 'Project [cast(getviewcolumnbynameandordinal(`db3`.`view2`, col1, 0, 1) as int) AS col1#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, col2, 0, 1) as string) AS col2#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, a, 0, 1) as int) AS a#0, cast(getviewcolumnbynameandordinal(`db3`.`view2`, b, 0, 1) as string) AS b#0]
[info] +- 'Project [*] +- 'Project [*]
[info] +- 'UnresolvedRelation [tbl1], [], false +- 'UnresolvedRelation [tbl1], [], false (PlanTest.scala:179)
```
### Does this PR introduce _any_ user-facing change?
No, the main change has not been released yet.
### How was this patch tested?
Manually ran the tests after ANSI disabled.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-48031][SQL][FOLLOW-UP] Use ANSI-enabled cast in view lookup test [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #46614: [SPARK-48031][SQL][FOLLOW-UP] Use ANSI-enabled cast in view lookup test
URL: https://github.com/apache/spark/pull/46614
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-48031][SQL][FOLLOW-UP] Use ANSI-enabled cast in view lookup test [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #46614:
URL: https://github.com/apache/spark/pull/46614#issuecomment-2116375642
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-48031][SQL][FOLLOW-UP] Use ANSI-enabled cast in view lookup test [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #46614:
URL: https://github.com/apache/spark/pull/46614#issuecomment-2114527010
because we don't run the tests without ANSI in PR builder. It only runs in the daily build defined here https://github.com/apache/spark/actions/workflows/build_non_ansi.yml
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-48031][SQL][FOLLOW-UP] Use ANSI-enabled cast in view lookup test [spark]
Posted by "srielau (via GitHub)" <gi...@apache.org>.
srielau commented on PR #46614:
URL: https://github.com/apache/spark/pull/46614#issuecomment-2114516109
@HyukjinKwon Any idea why this did not trip the original PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org