You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "gbloisi-openaire (via GitHub)" <gi...@apache.org> on 2023/08/07 11:57:46 UTC
[GitHub] [spark] gbloisi-openaire opened a new pull request, #42379: [SPARK-44634][SQL][3.4] Encoders.bean does no longer support nested beans with type arguments
gbloisi-openaire opened a new pull request, #42379:
URL: https://github.com/apache/spark/pull/42379
### What changes were proposed in this pull request?
This is a port of [42327](https://github.com/apache/spark/pull/42327)
This PR fixes a regression introduced in Spark 3.4.x where Encoders.bean is no longer able to process nested beans having type arguments. For example:
```
class A<T> {
T value;
// value getter and setter
}
class B {
A<String> stringHolder;
// stringHolder getter and setter
}
Encoders.bean(B.class); // throws "SparkUnsupportedOperationException: [ENCODER_NOT_FOUND]..."
```
### Why are the changes needed?
JavaTypeInference.encoderFor main match does not manage ParameterizedType and TypeVariable cases. I think this is a regression introduced after getting rid of usage of guava TypeToken: [SPARK-42093 SQL Move JavaTypeInference to AgnosticEncoders](https://github.com/apache/spark/commit/18672003513d5a4aa610b6b94dbbc15c33185d3#diff-1191737b908340a2f4c22b71b1c40ebaa0da9d8b40c958089c346a3bda26943b) @hvanhovell @cloud-fan
In this PR I'm leveraging commons lang3 TypeUtils functionalities to solve ParameterizedType type arguments for classes
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing tests have been extended to check correct encoding of a nested bean having type arguments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] hvanhovell closed pull request #42379: [SPARK-44634][SQL][3.4] Encoders.bean does no longer support nested beans with type arguments
Posted by "hvanhovell (via GitHub)" <gi...@apache.org>.
hvanhovell closed pull request #42379: [SPARK-44634][SQL][3.4] Encoders.bean does no longer support nested beans with type arguments
URL: https://github.com/apache/spark/pull/42379
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] hvanhovell commented on pull request #42379: [SPARK-44634][SQL][3.4] Encoders.bean does no longer support nested beans with type arguments
Posted by "hvanhovell (via GitHub)" <gi...@apache.org>.
hvanhovell commented on PR #42379:
URL: https://github.com/apache/spark/pull/42379#issuecomment-1667844955
Merged this PR to 3.4. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org