[jira] [Created] (SPARK-25494) Upgrade Spark's use of Janino to 3.0.10

["Kris Mok (JIRA)" <ji...@apache.org>]

Kris Mok created SPARK-25494:
--------------------------------

             Summary: Upgrade Spark's use of Janino to 3.0.10
                 Key: SPARK-25494
                 URL: https://issues.apache.org/jira/browse/SPARK-25494
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.4.0
            Reporter: Kris Mok


This ticket proposes to upgrade Spark's use of Janino from 3.0.9 to 3.0.10.
Note that 3.0.10 is a out-of-band release specifically for fixing an integer overflow issue in Janino's {{ClassFile}} reader. It is otherwise exactly the same as 3.0.9, so it's a low risk and compatible upgrade.

The integer overflow issue affects Spark SQL's codegen stats collection: when a generated Class file is huge, especially when the constant pool size is above {{Short.MAX_VALUE}}, Janino's {{ClassFile}} reader will throw an exception when Spark wants to parse the generated Class file to collect stats. So we'll miss the stats of some huge Class files.

The Janino fix is tracked by this issue: https://github.com/janino-compiler/janino/issues/58



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org