You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Izek Greenfield (JIRA)" <ji...@apache.org> on 2018/08/12 07:19:00 UTC
[jira] [Created] (SPARK-25094) proccesNext() failed to compile size
is over 64kb
Izek Greenfield created SPARK-25094:
---------------------------------------
Summary: proccesNext() failed to compile size is over 64kb
Key: SPARK-25094
URL: https://issues.apache.org/jira/browse/SPARK-25094
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.4.0
Reporter: Izek Greenfield
I have this tree:
2018-08-12T07:14:31,289 WARN [] org.apache.spark.sql.execution.WholeStageCodegenExec - Whole-stage codegen disabled for plan (id=1):
*(1) Project [, ... 10 more fields]
+- *(1) Filter NOT exposure_calc_method#10141 IN (UNSETTLED_TRANSACTIONS,FREE_DELIVERIES)
+- InMemoryTableScan [, ... 11 more fields], [NOT exposure_calc_method#10141 IN (UNSETTLED_TRANSACTIONS,FREE_DELIVERIES)]
+- InMemoryRelation [, ... 80 more fields], StorageLevel(memory, deserialized, 1 replicas)
+- *(5) SortMergeJoin [unique_id#8506], [unique_id#8722], Inner
:- *(2) Sort [unique_id#8506 ASC NULLS FIRST], false, 0
: +- Exchange(coordinator id: 1456511137) UnknownPartitioning(9), coordinator[target post-shuffle partition size: 67108864]
: +- *(1) Project [, ... 6 more fields]
: +- *(1) Filter (((((isnotnull(v#49) && isnotnull(run_id#52)) && (asof_date#48 <=> 17531)) && (run_id#52 = DATA_REG)) && (v#49 = DATA_REG)) && isnotnull(unique_id#39))
: +- InMemoryTableScan [, ... 6 more fields], [, ... 6 more fields]
: +- InMemoryRelation [, ... 6 more fields], StorageLevel(memory, deserialized, 1 replicas)
: +- *(1) FileScan csv [,... 6 more fields] , ... 6 more fields
+- *(4) Sort [unique_id#8722 ASC NULLS FIRST], false, 0
+- Exchange(coordinator id: 1456511137) UnknownPartitioning(9), coordinator[target post-shuffle partition size: 67108864]
+- *(3) Project [, ... 74 more fields]
+- *(3) Filter (((isnotnull(v#51) && (asof_date#42 <=> 17531)) && (v#51 = DATA_REG)) && isnotnull(unique_id#54))
+- InMemoryTableScan [, ... 74 more fields], [, ... 4 more fields]
+- InMemoryRelation [, ... 74 more fields], StorageLevel(memory, deserialized, 1 replicas)
+- *(1) FileScan csv [,... 74 more fields] , ... 6 more fields
Compiling "GeneratedClass": Code of method "processNext()V" of class "org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1" grows beyond 64 KB
and the generated code failed to compile.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org