[jira] [Commented] (CALCITE-3094) Code of method grows beyond 64 KB when joining two tables with many fields

["Runkang He (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-3094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17722133#comment-17722133 ] 

Runkang He commented on CALCITE-3094:
-------------------------------------

I agree with [~danny0405]'s idea. As I know, Flink's code splitter module is widely used and enabled by default in Flink-1.16, and I think code splitter is a general approach to solve code beyond 64k issue.

> Code of method grows beyond 64 KB when joining two tables with many fields
> --------------------------------------------------------------------------
>
>                 Key: CALCITE-3094
>                 URL: https://issues.apache.org/jira/browse/CALCITE-3094
>             Project: Calcite
>          Issue Type: Bug
>          Components: core, jdbc-adapter
>    Affects Versions: 1.19.0
>            Reporter: Pablo
>            Assignee: Thomas Rebele
>            Priority: Critical
>         Attachments: CTE-3094.zip
>
>
> I am running simple queries like these:
> {{SELECT * FROM t0 INNER JOIN t1 ON t0.`f000000` = t1.`f000003`"}}
>  
> They are running ok, but, when t1 and t0 have about 3290 fields, I get this error. 
> Is there anything I can do? I can't do joins with this amount of fields?
>  
> Thank you in advance
>  
> {noformat}
> Caused by: java.lang.RuntimeException: Error while compiling generated Java code:
> public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root) {
> final org.apache.calcite.rel.RelNode v2stashed = (org.apache.calcite.rel.RelNode) root.get("v2stashed");
> final org.apache.calcite.rel.RelNode v1stashed = (org.apache.calcite.rel.RelNode) root.get("v1stashed");
> final org.apache.calcite.interpreter.Interpreter interpreter = new org.apache.calcite.interpreter.Interpreter(
> root,
> v1stashed);
> final org.apache.calcite.interpreter.Interpreter interpreter0 = new org.apache.calcite.interpreter.Interpreter(
> root,
> v2stashed);
> return interpreter.join(interpreter0, new org.apache.calcite.linq4j.function.Function1() {
> public Byte apply(Object[] v1) {
> return (Byte) v1[0];
> }
> public Object apply(Object v1) {
> return apply(
> (Object[]) v1);
> }
> }
> , new org.apache.calcite.linq4j.function.Function1() {
> public Byte apply(Object[] v1) {
> return (Byte) v1[3];
> }
> public Object apply(Object v1) {
> return apply(
> (Object[]) v1);
> }
> }
> , new org.apache.calcite.linq4j.function.Function2() {
> public Object[] apply(Object[] left, Object[] right) {
> return new Object[] {
> left[0],
> left[1],
> left[2],
> [...elided ...]
> left[3298],
> left[3299],
> right[0],
> right[1],
> [...elided...]]
> right[3295],
> right[3296],
> right[3297],
> right[3298],
> right[3299]};
> }
> public Object[] apply(Object left, Object right) {
> return apply(
> (Object[]) left,
> (Object[]) right);
> }
> }
> , null, false, false);
> }
> public Class getElementType() {
> return java.lang.Object[].class;
> }
> at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
> at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:128)
> at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1233)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:332)
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:231)
> at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:767)
> at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:631)
> at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:601)
> at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:229)
> at org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:211)
> ... 80 more
> Caused by: org.codehaus.janino.InternalCompilerException: Compiling "Baz": Code of method "apply([Ljava/lang/Object;[Ljava/lang/Object;)[Ljava/lang/Object;" of class "Baz$3" grows beyond 64 KB
> at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:382)
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)
> at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465)
> at org.codehaus.janino.ClassBodyEvaluator.compileToClass(ClassBodyEvaluator.java:313)
> at org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:235)
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207)
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:50)
> at org.codehaus.janino.ClassBodyEvaluator.createInstance(ClassBodyEvaluator.java:347)
> at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.getBindable(EnumerableInterpretable.java:162)
> at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:125)
> ... 88 more
> Caused by: org.codehaus.janino.InternalCompilerException: Code of method "apply([Ljava/lang/Object;[Ljava/lang/Object;)[Ljava/lang/Object;" of class "Baz$3" grows beyond 64 KB
> at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:1048)
> at org.codehaus.janino.CodeContext.write(CodeContext.java:925)
> at org.codehaus.janino.UnitCompiler.writeOpcode(UnitCompiler.java:12291)
> at org.codehaus.janino.UnitCompiler.pushConstant(UnitCompiler.java:10746)
> at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5616)
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5592)
> at org.codehaus.janino.UnitCompiler.access$9700(UnitCompiler.java:215)
> at org.codehaus.janino.UnitCompiler$16.visitNewInitializedArray(UnitCompiler.java:4434)
> at org.codehaus.janino.UnitCompiler$16.visitNewInitializedArray(UnitCompiler.java:4396)
> at org.codehaus.janino.Java$NewInitializedArray.accept(Java.java:5373)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
> at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5662)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2649)
> at org.codehaus.janino.UnitCompiler.access$2800(UnitCompiler.java:215)
> at org.codehaus.janino.UnitCompiler$6.visitReturnStatement(UnitCompiler.java:1504)
> at org.codehaus.janino.UnitCompiler$6.visitReturnStatement(UnitCompiler.java:1487)
> at org.codehaus.janino.Java$ReturnStatement.accept(Java.java:3563)
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)
> at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1567)
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1357)
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1330)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:981)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:951)
> at org.codehaus.janino.UnitCompiler.access$200(UnitCompiler.java:215)
> at org.codehaus.janino.UnitCompiler$2.visitAnonymousClassDeclaration(UnitCompiler.java:409)
> at org.codehaus.janino.UnitCompiler$2.visitAnonymousClassDeclaration(UnitCompiler.java:406)
> at org.codehaus.janino.Java$AnonymousClassDeclaration.accept(Java.java:1149)
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5509)
> at org.codehaus.janino.UnitCompiler.access$9500(UnitCompiler.java:215)
> at org.codehaus.janino.UnitCompiler$16.visitNewAnonymousClassInstance(UnitCompiler.java:4432)
> at org.codehaus.janino.UnitCompiler$16.visitNewAnonymousClassInstance(UnitCompiler.java:4396)
> at org.codehaus.janino.Java$NewAnonymousClassInstance.accept(Java.java:5238)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
> at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5662)
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5182)
> at org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215)
> at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4423)
> at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4396)
> at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
> at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5662)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2649)
> at org.codehaus.janino.UnitCompiler.access$2800(UnitCompiler.java:215)
> at org.codehaus.janino.UnitCompiler$6.visitReturnStatement(UnitCompiler.java:1504)
> at org.codehaus.janino.UnitCompiler$6.visitReturnStatement(UnitCompiler.java:1487)
> at org.codehaus.janino.Java$ReturnStatement.accept(Java.java:3563)
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)
> at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1567)
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1357)
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1330)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432)
> at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215)
> at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:411)
> at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:406)
> at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1414)
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)
> at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378)
> ... 97 more
> {noformat}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)