You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "PJ Fanning (Jira)" <ji...@apache.org> on 2022/10/25 12:53:00 UTC
[jira] [Updated] (SPARK-40911) Upgrade jackson-module-scala to 2.14.0
[ https://issues.apache.org/jira/browse/SPARK-40911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
PJ Fanning updated SPARK-40911:
-------------------------------
Description:
This 2.14.0 release is still a few weeks a way. There is an rc2 release but there will probably be an rc3 before a full release.
The reason I marked the Jira component as 'Java API' is that this issue will affect Java users more than Scala users.
I raised this separately to SPARK-40666 because I have a different reason for this issue. SPARK-40666 can probably already be closed as the CVE is fixed in jackson-databind 2.13.4.2.
There are performance issues in jackson-module-scala 2.13.x that may affect some Spark users. Specifically, the per issue is [https://github.com/FasterXML/jackson-module-scala/issues/576]
Scala3 support added in jackson-module-scala 2.13.0 means that if you use Scala 2.13, you should be able to use Scala3 compiled classes with jackson-module-scala. Scala3 compiled classes are harder to recognise using runtime reflection (and Jackson is built around runtime reflection). Scala2 compiled classes have specific annotations. With Scala3 compiled classes, we need to look for .tasty files. This lookup can be slow if you have a lot of jars (or big jars). Issue [576|https://github.com/FasterXML/jackson-module-scala/issues/576] fixes an issue where this .tasty lookup is done every time you try to serialize/deserialize a Java class with an ObjectMapper that has the DefaultScalaModule registered.
For Scala usage, it may be worth turning off this .tasty file support altogether. This is another enhancement in jackson-module-scala (but not in the RC2 release).
I will follow up and update this issue when the v2.14.0 release is ready. This change will require updating all Jackson jars to v2.14.0 (as Jackson does not support using version mismatches - except at patch version level).
was:
This 2.14.0 release is still a few weeks a way. There is an rc2 release but there will probably be an rc3 before a full release.
The reason I marked the Jira component as 'Java API' is that this issue will affect Java users more than Scala users.
I raised this separately to SPARK-40666 because I have a different reason for this issue. SPARK-40666 can probably already be closed as the CVE is fixed in jackson-databind 2.13.4.2.
There are performance issues in jackson-module-scala 2.13.x that may affect some Spark users. Specifically, the per issue is [https://github.com/FasterXML/jackson-module-scala/issues/576]
Scala3 support added in jackson-module-scala 2.13.0 means that if you use Scala 2.13, you should be able to use Scala3 compiled classes with jackson-module-scala. Scala3 compiled classes are harder to recognise using runtime reflection (and Jackson is built around runtime reflection). Scala2 compiled classes have specific annotations. With Scala3 compiled classes, we need to look for .tasty files. This lookup can be slow if you have a lot of jars (or big jars). Issue [576|https://github.com/FasterXML/jackson-module-scala/issues/576] fixes an issue where this .tasty lookup is done every time you try to serialize/deserialize a Java class with an ObjectMapper that has the DefaultScalaModule registered.
For Scala usage, it may be worth turning off this .tasty file support altogether. This is another enhancement in jackson-module-scala (but not in the RC2 release).
I will follow up and update this issue when the v2.14.0 release is ready.
> Upgrade jackson-module-scala to 2.14.0
> --------------------------------------
>
> Key: SPARK-40911
> URL: https://issues.apache.org/jira/browse/SPARK-40911
> Project: Spark
> Issue Type: Improvement
> Components: Java API
> Affects Versions: 3.3.0
> Reporter: PJ Fanning
> Priority: Major
>
> This 2.14.0 release is still a few weeks a way. There is an rc2 release but there will probably be an rc3 before a full release.
> The reason I marked the Jira component as 'Java API' is that this issue will affect Java users more than Scala users.
> I raised this separately to SPARK-40666 because I have a different reason for this issue. SPARK-40666 can probably already be closed as the CVE is fixed in jackson-databind 2.13.4.2.
> There are performance issues in jackson-module-scala 2.13.x that may affect some Spark users. Specifically, the per issue is [https://github.com/FasterXML/jackson-module-scala/issues/576]
> Scala3 support added in jackson-module-scala 2.13.0 means that if you use Scala 2.13, you should be able to use Scala3 compiled classes with jackson-module-scala. Scala3 compiled classes are harder to recognise using runtime reflection (and Jackson is built around runtime reflection). Scala2 compiled classes have specific annotations. With Scala3 compiled classes, we need to look for .tasty files. This lookup can be slow if you have a lot of jars (or big jars). Issue [576|https://github.com/FasterXML/jackson-module-scala/issues/576] fixes an issue where this .tasty lookup is done every time you try to serialize/deserialize a Java class with an ObjectMapper that has the DefaultScalaModule registered.
> For Scala usage, it may be worth turning off this .tasty file support altogether. This is another enhancement in jackson-module-scala (but not in the RC2 release).
> I will follow up and update this issue when the v2.14.0 release is ready. This change will require updating all Jackson jars to v2.14.0 (as Jackson does not support using version mismatches - except at patch version level).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org