You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Arina Ielchiieva (JIRA)" <ji...@apache.org> on 2017/12/12 17:33:00 UTC
[jira] [Updated] (DRILL-6028) Allow splitting generated code in
ChainedHashTable into blocks to avoid "code too large" error
[ https://issues.apache.org/jira/browse/DRILL-6028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arina Ielchiieva updated DRILL-6028:
------------------------------------
Description:
Allow splitting generated code in ChainedHashTable into blocks to avoid "code too large" error.
*REPRODUCE*
File {{1200_columns.csv}}
{noformat}
0,1,2,3...1200
0,1,2,3...1200
{noformat}
Query
{noformat}
select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
union
select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
{noformat}
Error
{noformat}
Error: SYSTEM ERROR: CompileException: File 'org.apache.drill.exec.compile.DrillJavaFileObject[HashTableGen10.java]', Line -7886, Column 24: HashTableGen10.java:57650: error: code too large
public boolean isKeyMatchInternalBuild(int incomingRowIdx, int htRowIdx)
^ (compiler.err.limit.code)
{noformat}
*ROOT CAUSE*
DRILL-4715 added ability to ensure that methods size won't go beyond the 64k limit imposed by JVM. {{BlkCreateMode.TRUE_IF_BOUND}} was added to create new block only if # of expressions added hit upper-bound defined by {{exec.java.compiler.exp_in_method_size}}. Once number of expressions in methods hits upper bound we create from call inner method.
Example:
{noformat}
public void doSetup(RecordBatch incomingBuild, RecordBatch incomingProbe) throws SchemaChangeException {
// some logic
return doSetup0(incomingBuild, incomingProbe);
}
{noformat}
During code generation {{ChainedHashTable}}
was:
Allow splitting generated code in ChainedHashTable into blocks to avoid "code too large" error.
REPRODUCE
File {{1200_columns.csv}}
{noformat}
0,1,2,3...1200
0,1,2,3...1200
{noformat}
Query
{noformat}
select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
union
select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
{noformat}
Error
{noformat}
Error: SYSTEM ERROR: CompileException: File 'org.apache.drill.exec.compile.DrillJavaFileObject[HashTableGen10.java]', Line -7886, Column 24: HashTableGen10.java:57650: error: code too large
public boolean isKeyMatchInternalBuild(int incomingRowIdx, int htRowIdx)
^ (compiler.err.limit.code)
{noformat}
> Allow splitting generated code in ChainedHashTable into blocks to avoid "code too large" error
> ----------------------------------------------------------------------------------------------
>
> Key: DRILL-6028
> URL: https://issues.apache.org/jira/browse/DRILL-6028
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.10.0
> Reporter: Arina Ielchiieva
> Assignee: Arina Ielchiieva
> Fix For: 1.13.0
>
>
> Allow splitting generated code in ChainedHashTable into blocks to avoid "code too large" error.
> *REPRODUCE*
> File {{1200_columns.csv}}
> {noformat}
> 0,1,2,3...1200
> 0,1,2,3...1200
> {noformat}
> Query
> {noformat}
> select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
> union
> select columns[0], column[1]...columns[1200] from dfs.`1200_columns.csv`
> {noformat}
> Error
> {noformat}
> Error: SYSTEM ERROR: CompileException: File 'org.apache.drill.exec.compile.DrillJavaFileObject[HashTableGen10.java]', Line -7886, Column 24: HashTableGen10.java:57650: error: code too large
> public boolean isKeyMatchInternalBuild(int incomingRowIdx, int htRowIdx)
> ^ (compiler.err.limit.code)
> {noformat}
> *ROOT CAUSE*
> DRILL-4715 added ability to ensure that methods size won't go beyond the 64k limit imposed by JVM. {{BlkCreateMode.TRUE_IF_BOUND}} was added to create new block only if # of expressions added hit upper-bound defined by {{exec.java.compiler.exp_in_method_size}}. Once number of expressions in methods hits upper bound we create from call inner method.
> Example:
> {noformat}
> public void doSetup(RecordBatch incomingBuild, RecordBatch incomingProbe) throws SchemaChangeException {
> // some logic
> return doSetup0(incomingBuild, incomingProbe);
> }
> {noformat}
> During code generation {{ChainedHashTable}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)