You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "gbloisi-openaire (via GitHub)" <gi...@apache.org> on 2023/09/06 07:14:30 UTC
[GitHub] [spark] gbloisi-openaire opened a new pull request, #42829: [SPARK-45081][SQL] Encoders.bean does no longer work with read-only properties
gbloisi-openaire opened a new pull request, #42829:
URL: https://github.com/apache/spark/pull/42829
### What changes were proposed in this pull request?
This PR re-enables Encoders.bean to be called against beans having read-only properties, that is properties that have only getters and no setter method. Beans with read only properties are even used in internal tests.
Setter methods of a Java bean encoder are stored within an Option wrapper because they are missing in case of read-only properties. When a java bean has to be initialized, setter methods for the bean properties have to be called: this PR filters out read-only properties from that process.
### Why are the changes needed?
The changes are required to avoid an exception to the thrown by getting the value of a None option object.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
An additional regression test has been added
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] hvanhovell commented on pull request #42829: [SPARK-45081][SQL] Encoders.bean does no longer work with read-only properties
Posted by "hvanhovell (via GitHub)" <gi...@apache.org>.
hvanhovell commented on PR #42829:
URL: https://github.com/apache/spark/pull/42829#issuecomment-1715809440
Alright merging this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gbloisi-openaire commented on pull request #42829: [SPARK-45081][SQL] Encoders.bean does no longer work with read-only properties
Posted by "gbloisi-openaire (via GitHub)" <gi...@apache.org>.
gbloisi-openaire commented on PR #42829:
URL: https://github.com/apache/spark/pull/42829#issuecomment-1709847444
> Do you happen to know which PR disabled this?
I encountered this problem while upgrading a project from Spark 2.4.0 to 3.4.1, but [here](https://stackoverflow.com/questions/76036349/encoders-bean-doesnt-work-anymore-on-a-java-pojo-with-spark-3-4-0) the problem is reported to be present only in Spark 3.4 and not in 3.2 and 3.3. So I presume the support was disabled by some recent change, likely the ones related and subsequent to [SPARK-41993 Move RowEncoder to AgnosticEncoders](https://issues.apache.org/jira/browse/SPARK-41993) (@hvanhovell)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] hvanhovell commented on pull request #42829: [SPARK-45081][SQL] Encoders.bean does no longer work with read-only properties
Posted by "hvanhovell (via GitHub)" <gi...@apache.org>.
hvanhovell commented on PR #42829:
URL: https://github.com/apache/spark/pull/42829#issuecomment-1715815228
Merged to master and 3.5 (it will be part of Spark 3.5.1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] hvanhovell closed pull request #42829: [SPARK-45081][SQL] Encoders.bean does no longer work with read-only properties
Posted by "hvanhovell (via GitHub)" <gi...@apache.org>.
hvanhovell closed pull request #42829: [SPARK-45081][SQL] Encoders.bean does no longer work with read-only properties
URL: https://github.com/apache/spark/pull/42829
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gbloisi-openaire commented on pull request #42829: [SPARK-45081][SQL] Encoders.bean does no longer work with read-only properties
Posted by "gbloisi-openaire (via GitHub)" <gi...@apache.org>.
gbloisi-openaire commented on PR #42829:
URL: https://github.com/apache/spark/pull/42829#issuecomment-1718170192
I prepared port for 3.4 branch as well, see [PR-42913](https://github.com/apache/spark/pull/42913)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #42829: [SPARK-45081][SQL] Encoders.bean does no longer work with read-only properties
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #42829:
URL: https://github.com/apache/spark/pull/42829#issuecomment-1709343087
Do you happen to know which PR disabled this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org