You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yu Li (Jira)" <ji...@apache.org> on 2020/05/01 12:40:00 UTC
[jira] [Updated] (FLINK-16589) Flink Table SQL fails/crashes with
big queries with lots of fields
[ https://issues.apache.org/jira/browse/FLINK-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yu Li updated FLINK-16589:
--------------------------
Fix Version/s: (was: 1.10.1)
1.10.2
> Flink Table SQL fails/crashes with big queries with lots of fields
> ------------------------------------------------------------------
>
> Key: FLINK-16589
> URL: https://issues.apache.org/jira/browse/FLINK-16589
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Affects Versions: 1.10.0
> Reporter: Viet Pham
> Assignee: Benchao Li
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.11.0, 1.10.2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hi,
> My use case is a streaming application with a few streaming tables.
> I was trying to build a SELECT query (and registering it as a temporary view) with about 200 fields/expressions out of another streaming table. The application is successfully submitted to Flink cluster. However the worker processes keep crashing, with the exception as quoted below.
> It clearly mentioned in the log that this is a bug, so I fire this ticket. By the way, if I lower the number of fields down to 100 then it works nicely.
> Please advice.
> Thanks a lot for all the efforts bring Flink up. It is really amazing!
> {code:java}
> java.lang.RuntimeException: Could not instantiate generated class 'GroupAggsHandler$9687' at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:57) at org.apache.flink.table.runtime.operators.aggregate.MiniBatchGroupAggFunction.open(MiniBatchGroupAggFunction.java:136) at org.apache.flink.table.runtime.operators.bundle.AbstractMapBundleOperator.open(AbstractMapBundleOperator.java:84) at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeStateAndOpen(StreamTask.java:1007) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:454) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:94) at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:449) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:461) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:707) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:532) at java.lang.Thread.run(Thread.java:748)Caused by: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue. at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:68) at org.apache.flink.table.runtime.generated.GeneratedClass.compile(GeneratedClass.java:78) at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:52) ... 10 moreCaused by: org.apache.flink.shaded.guava18.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue. at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203) at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache.get(LocalCache.java:3937) at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739) at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:66) ... 12 moreCaused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue. at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:81) at org.apache.flink.table.runtime.generated.CompileUtils.lambda$compile$1(CompileUtils.java:66) at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4742) at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527) at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319) at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282) at org.apache.flink.shaded.guava18.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197) ... 15 moreCaused by: org.codehaus.janino.InternalCompilerException: Compiling "GroupAggsHandler$9687": Code of method "retract(Lorg/apache/flink/table/dataformat/BaseRow;)V" of class "GroupAggsHandler$9687" grows beyond 64 KB at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:382) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237) at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216) at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207) at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80) at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75) at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:78) ... 21 moreCaused by: org.codehaus.janino.InternalCompilerException: Code of method "retract(Lorg/apache/flink/table/dataformat/BaseRow;)V" of class "GroupAggsHandler$9687" grows beyond 64 KB at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:1009) at org.codehaus.janino.CodeContext.write(CodeContext.java:901) at org.codehaus.janino.CodeContext.writeShort(CodeContext.java:1026) at org.codehaus.janino.UnitCompiler.writeConstantLongInfo(UnitCompiler.java:12274) at org.codehaus.janino.UnitCompiler.pushConstant(UnitCompiler.java:10679) at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:4936) at org.codehaus.janino.UnitCompiler.access$8400(UnitCompiler.java:215) at org.codehaus.janino.UnitCompiler$16.visitUnaryOperation(UnitCompiler.java:4414) at org.codehaus.janino.UnitCompiler$16.visitUnaryOperation(UnitCompiler.java:4394) at org.codehaus.janino.Java$UnaryOperation.accept(Java.java:4719) at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4394) at org.codehaus.janino.UnitCompiler.fakeCompile(UnitCompiler.java:3719) at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5569) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2580) at org.codehaus.janino.UnitCompiler.access$2700(UnitCompiler.java:215) at org.codehaus.janino.UnitCompiler$6.visitLocalVariableDeclarationStatement(UnitCompiler.java:1503) at org.codehaus.janino.UnitCompiler$6.visitLocalVariableDeclarationStatement(UnitCompiler.java:1487) at org.codehaus.janino.Java$LocalVariableDeclarationStatement.accept(Java.java:3511) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487) at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1567) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388) at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1357) at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1330) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822) at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432) at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215) at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:411) at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:406) at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1414) at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406) at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378) ... 28 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)