You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "vjovanov (via GitHub)" <gi...@apache.org> on 2024/04/09 09:27:51 UTC
[PR] [SPARK-47780][SQL] Make catalyst-generated classes stable and uniquely named [spark]
vjovanov opened a new pull request, #45955:
URL: https://github.com/apache/spark/pull/45955
### What changes were proposed in this pull request?
This change make the classes generated by catalyst:
1. Stable across different runs. The generated code does not contain random numbers that come from lambda-proxy names (`0x<address>`).
2. Uniquely named. The name uniqueness is achieved by adding the hash code of the generated code to the class name. The generated classes are name `org.apache.spark.sql.catalyst.expressions.GeneratedClass$<codeHash>` instead of `org.apache.spark.sql.catalyst.expressions.GeneratedClass`.
### Why are the changes needed?
The changes were needed to allow running Spark SQL workloads on [GraalVM Native Image](https://www.graalvm.org/).
### Does this PR introduce _any_ user-facing change?
Yes, it removes the random number from the lambda-proxy name in the UDF failure error message. Before this change:
```
[FAILED_EXECUTE_UDF] Failed to execute user defined function (QueryExecutionErrorsSuite$$Lambda$970/0x181260145: (string, int) => string).
```
After this change:
```
[FAILED_EXECUTE_UDF] Failed to execute user defined function (nextChar in QueryExecutionErrorsSuite$$Lambda$970: (string, int) => string).
```
### How was this patch tested?
The test cases for this code path were already introduced in the following [PR](https://github.com/apache/spark/pull/41599). This PR modifies these tests to check for the new format.
### Was this patch authored or co-authored using generative AI tooling?
No!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47780][SQL] Make catalyst-generated classes stable and uniquely named [spark]
Posted by "farquet (via GitHub)" <gi...@apache.org>.
farquet commented on PR #45955:
URL: https://github.com/apache/spark/pull/45955#issuecomment-2408827894
Could someone from the team re-open and remove the `stale` tag? It's unfortunate that this is not being merged yet, because it's a small self-contained fix enabling AOT compilation of Spark projects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47780][SQL] Make catalyst-generated classes stable and uniquely named [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #45955:
URL: https://github.com/apache/spark/pull/45955#issuecomment-2408262799
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47780][SQL] Make catalyst-generated classes stable and uniquely named [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #45955: [SPARK-47780][SQL] Make catalyst-generated classes stable and uniquely named
URL: https://github.com/apache/spark/pull/45955
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org