You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by li...@apache.org on 2018/09/21 05:15:55 UTC
spark git commit: [SPARK-25494][SQL] Upgrade Spark's use of Janino to
3.0.10
Repository: spark
Updated Branches:
refs/heads/master 5d25e1544 -> 596af211a
[SPARK-25494][SQL] Upgrade Spark's use of Janino to 3.0.10
## What changes were proposed in this pull request?
This PR upgrades Spark's use of Janino from 3.0.9 to 3.0.10.
Note that 3.0.10 is a out-of-band release specifically for fixing an integer overflow issue in Janino's `ClassFile` reader. It is otherwise exactly the same as 3.0.9, so it's a low risk and compatible upgrade.
The integer overflow issue affects Spark SQL's codegen stats collection: when a generated Class file is huge, especially when the constant pool size is above `Short.MAX_VALUE`, Janino's `ClassFile reader` will throw an exception when Spark wants to parse the generated Class file to collect stats. So we'll miss the stats of some huge Class files.
The related Janino issue is: https://github.com/janino-compiler/janino/issues/58
## How was this patch tested?
Existing codegen tests.
Closes #22506 from rednaxelafx/upgrade-janino.
Authored-by: Kris Mok <kr...@databricks.com>
Signed-off-by: gatorsmile <ga...@gmail.com>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/596af211
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/596af211
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/596af211
Branch: refs/heads/master
Commit: 596af211a5b5a7468a0e9b840561c9ae2353a29c
Parents: 5d25e15
Author: Kris Mok <kr...@databricks.com>
Authored: Thu Sep 20 22:15:52 2018 -0700
Committer: gatorsmile <ga...@gmail.com>
Committed: Thu Sep 20 22:15:52 2018 -0700
----------------------------------------------------------------------
dev/deps/spark-deps-hadoop-2.6 | 4 ++--
dev/deps/spark-deps-hadoop-2.7 | 4 ++--
dev/deps/spark-deps-hadoop-3.1 | 4 ++--
pom.xml | 2 +-
4 files changed, 7 insertions(+), 7 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/596af211/dev/deps/spark-deps-hadoop-2.6
----------------------------------------------------------------------
diff --git a/dev/deps/spark-deps-hadoop-2.6 b/dev/deps/spark-deps-hadoop-2.6
index 62ae04d..969df4d 100644
--- a/dev/deps/spark-deps-hadoop-2.6
+++ b/dev/deps/spark-deps-hadoop-2.6
@@ -34,7 +34,7 @@ commons-beanutils-core-1.8.0.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
-commons-compiler-3.0.9.jar
+commons-compiler-3.0.10.jar
commons-compress-1.8.1.jar
commons-configuration-1.6.jar
commons-crypto-1.0.0.jar
@@ -98,7 +98,7 @@ jackson-module-jaxb-annotations-2.6.7.jar
jackson-module-paranamer-2.7.9.jar
jackson-module-scala_2.11-2.6.7.1.jar
jackson-xc-1.9.13.jar
-janino-3.0.9.jar
+janino-3.0.10.jar
javassist-3.18.1-GA.jar
javax.annotation-api-1.2.jar
javax.inject-1.jar
http://git-wip-us.apache.org/repos/asf/spark/blob/596af211/dev/deps/spark-deps-hadoop-2.7
----------------------------------------------------------------------
diff --git a/dev/deps/spark-deps-hadoop-2.7 b/dev/deps/spark-deps-hadoop-2.7
index dcb5d63..e827dc6 100644
--- a/dev/deps/spark-deps-hadoop-2.7
+++ b/dev/deps/spark-deps-hadoop-2.7
@@ -34,7 +34,7 @@ commons-beanutils-core-1.8.0.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
-commons-compiler-3.0.9.jar
+commons-compiler-3.0.10.jar
commons-compress-1.8.1.jar
commons-configuration-1.6.jar
commons-crypto-1.0.0.jar
@@ -98,7 +98,7 @@ jackson-module-jaxb-annotations-2.6.7.jar
jackson-module-paranamer-2.7.9.jar
jackson-module-scala_2.11-2.6.7.1.jar
jackson-xc-1.9.13.jar
-janino-3.0.9.jar
+janino-3.0.10.jar
javassist-3.18.1-GA.jar
javax.annotation-api-1.2.jar
javax.inject-1.jar
http://git-wip-us.apache.org/repos/asf/spark/blob/596af211/dev/deps/spark-deps-hadoop-3.1
----------------------------------------------------------------------
diff --git a/dev/deps/spark-deps-hadoop-3.1 b/dev/deps/spark-deps-hadoop-3.1
index 641b4a1..2b12c35 100644
--- a/dev/deps/spark-deps-hadoop-3.1
+++ b/dev/deps/spark-deps-hadoop-3.1
@@ -31,7 +31,7 @@ commons-beanutils-1.9.3.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
-commons-compiler-3.0.9.jar
+commons-compiler-3.0.10.jar
commons-compress-1.8.1.jar
commons-configuration2-2.1.1.jar
commons-crypto-1.0.0.jar
@@ -97,7 +97,7 @@ jackson-mapper-asl-1.9.13.jar
jackson-module-jaxb-annotations-2.6.7.jar
jackson-module-paranamer-2.7.9.jar
jackson-module-scala_2.11-2.6.7.1.jar
-janino-3.0.9.jar
+janino-3.0.10.jar
javassist-3.18.1-GA.jar
javax.annotation-api-1.2.jar
javax.inject-1.jar
http://git-wip-us.apache.org/repos/asf/spark/blob/596af211/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 71d5f94..099a081 100644
--- a/pom.xml
+++ b/pom.xml
@@ -170,7 +170,7 @@
<!-- org.apache.commons/commons-lang3/-->
<commons-lang3.version>3.5</commons-lang3.version>
<datanucleus-core.version>3.2.10</datanucleus-core.version>
- <janino.version>3.0.9</janino.version>
+ <janino.version>3.0.10</janino.version>
<jersey.version>2.22.2</jersey.version>
<joda.version>2.9.3</joda.version>
<jodd.version>3.5.2</jodd.version>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org