You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Lars Francke (Jira)" <ji...@apache.org> on 2019/11/21 23:31:00 UTC
[jira] [Commented] (SPARK-29226) Upgrade jackson-databind to 2.9.10
and fix vulnerabilities.
[ https://issues.apache.org/jira/browse/SPARK-29226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16979697#comment-16979697 ]
Lars Francke commented on SPARK-29226:
--------------------------------------
This causes the following exception for me:
{code:java}
Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/exc/InputCoercionExceptionException in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/exc/InputCoercionException at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.createArrayDeserializer(BasicDeserializerFactory.java:1141) at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:372) at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:349) at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264) at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:476) at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4389) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4198) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3205) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3173) at org.apache.spark.sql.execution.datasources.v2.FileDataSourceV2.$anonfun$getPaths$1(FileDataSourceV2.scala:47) at scala.Option.map(Option.scala:230) at org.apache.spark.sql.execution.datasources.v2.FileDataSourceV2.getPaths(FileDataSourceV2.scala:46) at org.apache.spark.sql.execution.datasources.v2.FileDataSourceV2.getPaths$(FileDataSourceV2.scala:44) at org.apache.spark.sql.execution.datasources.v2.parquet.ParquetDataSourceV2.getPaths(ParquetDataSourceV2.scala:26) at org.apache.spark.sql.execution.datasources.v2.parquet.ParquetDataSourceV2.getTable(ParquetDataSourceV2.scala:33) at org.apache.spark.sql.DataFrameReader.$anonfun$load$1(DataFrameReader.scala:220) at scala.Option.map(Option.scala:230) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:206) at org.apache.spark.sql.DataFrameReader.parquet(DataFrameReader.scala:674) at org.apache.spark.sql.DataFrameReader.parquet(DataFrameReader.scala:658) at com.opencore.SparkReaderDemo.main(SparkReaderDemo.java:12)Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.core.exc.InputCoercionException at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:602) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) ... 23 more
{code}
This is because arrow brings in Jackson 2.7.9 and that seems to take precedence (at least when I run it from within IntelliJ). I'm not creating a new issue because I don't have enough time to investigate properly.
Declaring an explicit dependency on jackson-core in version 2.10.0 fixes this for me.
> Upgrade jackson-databind to 2.9.10 and fix vulnerabilities.
> -----------------------------------------------------------
>
> Key: SPARK-29226
> URL: https://issues.apache.org/jira/browse/SPARK-29226
> Project: Spark
> Issue Type: Dependency upgrade
> Components: Build
> Affects Versions: 3.0.0
> Reporter: jiaan.geng
> Assignee: jiaan.geng
> Priority: Major
> Fix For: 3.0.0
>
>
> The current code uses com.fasterxml.jackson.core:jackson-databind:jar:2.9.9.3 and it will cause a security vulnerabilities. We could get some security info from https://www.tenable.com/cve/CVE-2019-16335
> This reference remind to upgrate the version of `jackson-databind` to 2.9.10 or later.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org