You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "Benenson, Michael" <Mi...@intuit.com> on 2022/06/08 19:45:52 UTC
Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug
Hi, folks
Short description:
I use Flink 1.15.0 sql-client and Java User Define Function in CREATE TABLE … statement to get Timestamp.
It works OK, if I do not use Timestamp in Watermark, but if used in Watermark it causes
java.lang.RuntimeException: Could not instantiate generated class 'WatermarkGenerator$0'
…
Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:107)
…
Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column 75: Cannot determine simple type name "org"
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
Details:
java -version
openjdk version "11.0.14.1" 2022-02-08 LTS
OpenJDK Runtime Environment Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS, mixed mode)
Flink 1.15.0, flink-1.15.0/bin/sql-client.sh
SET 'sql-client.execution.result-mode' = 'tableau';
SET 'table.exec.sink.not-null-enforcer' = 'drop';
CREATE TEMPORARY FUNCTION default_catalog.default_database.fix_instant
AS 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA;
CREATE OR REPLACE TABLE input (
event_header ROW(topic_name STRING),
`timestamp` STRING NOT NULL,
event_time AS TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
properties ROW(company_id STRING NOT NULL, scope_area STRING, action STRING) NOT NULL,
WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND,
proc_time AS PROCTIME()
) WITH (
'connector' = 'kafka',
'topic' = 'mb-1644796800-qbo',
'properties.bootstrap.servers' = 'localhost:9092',
'format' = 'json',
'scan.startup.mode' = 'latest-offset',
'json.ignore-parse-errors' = 'true',
'json.fail-on-missing-field' = 'false'
);
SELECT `timestamp`, event_time, event_header.topic_name AS topic, properties.company_id as company FROM input
LIMIT 10
;
Works fine, if I comment WATERMARK FOR event_time …
Causes an error, if WATERMARK FOR event_time is used:
2022-06-08 12:16:32
java.lang.RuntimeException: Could not instantiate generated class 'WatermarkGenerator$0'
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:74)
at org.apache.flink.table.runtime.generated.GeneratedWatermarkGeneratorSupplier.createWatermarkGenerator(GeneratedWatermarkGeneratorSupplier.java:62)
at org.apache.flink.streaming.api.operators.source.ProgressiveTimestampsAndWatermarks.createMainOutput(ProgressiveTimestampsAndWatermarks.java:104)
at org.apache.flink.streaming.api.operators.SourceOperator.initializeMainOutput(SourceOperator.java:426)
at org.apache.flink.streaming.api.operators.SourceOperator.emitNextNotReading(SourceOperator.java:402)
at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:387)
at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68)
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:519)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:804)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:753)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:948)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:94)
at org.apache.flink.table.runtime.generated.GeneratedClass.compile(GeneratedClass.java:101)
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:68)
... 16 more
Caused by: org.apache.flink.shaded.guava30.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.get(LocalCache.java:3962)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4859)
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:92)
... 18 more
Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:107)
at org.apache.flink.table.runtime.generated.CompileUtils.lambda$compile$0(CompileUtils.java:92)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4864)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
... 21 more
Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column 75: Cannot determine simple type name "org"
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6833)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6594)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6573)
at org.codehaus.janino.UnitCompiler.access$13900(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(UnitCompiler.java:6481)
at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(UnitCompiler.java:6476)
at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3928)
at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:6476)
at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:6469)
at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3927)
at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7121)
at org.codehaus.janino.UnitCompiler.access$17000(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(UnitCompiler.java:6529)
at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(UnitCompiler.java:6490)
at org.codehaus.janino.Java$NewClassInstance.accept(Java.java:5190)
at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:6490)
at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:6469)
at org.codehaus.janino.Java$Rvalue.accept(Java.java:4116)
at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
at org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:9237)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9123)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9025)
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5062)
at org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4423)
at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4396)
at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5662)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3783)
at org.codehaus.janino.UnitCompiler.access$5900(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(UnitCompiler.java:3762)
at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(UnitCompiler.java:3734)
at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3734)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2360)
at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1494)
at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1487)
at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2874)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)
at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1567)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1357)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1330)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432)
at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:411)
at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:406)
at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1414)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)
at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)
at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75)
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:104)
... 27 more
Java code:
package com.intuit.data.strmprocess.udf;
import org.apache.flink.table.functions.ScalarFunction;
/**
Usage in SQL:
CREATE TEMPORARY FUNCTION default_catalog.default_database.fix_instant
AS 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA;
CREATE OR REPLACE TABLE input (
`timestamp` STRING NOT NULL,
event_time AS TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
*/
public class FixInstant extends ScalarFunction
{
private static final long serialVersionUID = -3115461377254640072L;
public String eval(String s) {
if (s == null) return null;
else if (s.indexOf('.') > 0) return s;
else if (s.endsWith("Z")) return s.replace("Z", ".000Z");
else return null;
}
}
Compiled for Java 8 or Java 11, no differences
Any ideas?
Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug
Posted by Shengkai Fang <fs...@gmail.com>.
Hi.
I open a ticket about upgrading the version[1]. Maybe it is worth a try.
Best,
Shengkai
[1] https://issues.apache.org/jira/browse/FLINK-27995
Benenson, Michael <Mi...@intuit.com> 于2022年6月10日周五 04:51写道：
> Hi, David
>
>
>
> Hard to tell for sure, but yes, [1] could also indicate some problems with
> Janio.
>
> Both Flink 1.14.3 & 1.15.0 use <janino.version>3.0.11</janino.version>
>
> The difference is that Flink 1.14.3 has been compiled & run on jvm-8, but
> Flink 1.15.0 has been compiled & run on jvm-11
>
>
>
> In the revision history for Janio at [2] one comment for release 3.0.11 is
>
> - At last, the "jdk-9" implementation works.
>
> Probably, we could imply that jdk-11 does not work in Janio 3.0.11, so
> switch to the latest version of Janio 3.1.7
> <https://mvnrepository.com/artifact/org.codehaus.janino/janino/3.1.7> could
> help.
>
>
>
> Do you know, how to make nightly flink build with janio 3.1.7
> <https://mvnrepository.com/artifact/org.codehaus.janino/janino/3.1.7>?
>
>
>
> [1] https://lists.apache.org/thread/9tw165cgpdqz4ron76b1ckmwm9hy4qfd
>
> [2] https://janino-compiler.github.io/janino/changelog.html
>
>
>
>
>
>
>
> *From: *David Anderson <da...@apache.org>
> *Date: *Thursday, June 9, 2022 at 12:05 PM
> *To: *Benenson, Michael <Mi...@intuit.com>
> *Cc: *user@flink.apache.org <us...@flink.apache.org>, Deshpande, Omkar <
> Omkar_Deshpande@intuit.com>, Waghulde, Suraj <Su...@intuit.com>
> *Subject: *Re: Fink 15: InvalidProgramException: Table program cannot be
> compiled. This is a bug
>
> This email is from an external sender.
>
>
>
> Sorry; I guess I jumped to conclusions and got it wrong. Let's dig in
> deeper.
>
>
>
> This sounds similar to what was reported in this thread [1] where someone
> else ran into problems with Janino after upgrading from 1.14 to 1.15. Could
> this be another instance of the same issue (i.e., Flink ends up using the
> wrong version of Janino)?
>
>
>
> [1] https://lists.apache.org/thread/9tw165cgpdqz4ron76b1ckmwm9hy4qfd
>
>
>
> On Thu, Jun 9, 2022 at 8:18 PM Benenson, Michael <
> Mikhail_Benenson@intuit.com> wrote:
>
> Hi, David
>
>
>
> I have tried CREATE TABLE … without proc_time, but it does not help, I see
> the same exception.
>
>
> And this is a new issue for Flink 1.15.0, for Flink 1.14.3 it works fine,
> even with both event_time & proc_time in CREATE TABLE statement.
>
> Now I use
>
> CREATE OR REPLACE TABLE input (
> event_header ROW(topic_name STRING),
> `timestamp` STRING NOT NULL,
> event_time AS TO_TIMESTAMP(fix_instant(`timestamp`),
> 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
> properties ROW(company_id STRING NOT NULL, scope_area STRING, action STRING)
> NOT NULL,
> WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND
> ) WITH (
> 'connector' = 'kafka',
> 'topic' = 'mb-1644796800-qbo',
> 'properties.bootstrap.servers' = 'localhost:9092',
> 'format' = 'json',
> 'scan.startup.mode' = 'latest-offset',
> 'json.ignore-parse-errors' = 'true',
> 'json.fail-on-missing-field' = 'false'
> );
>
> And got the same exception
>
>
> java.lang.RuntimeException: Could not instantiate generated class
> 'WatermarkGenerator$0'
>
> at
> org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:74)
> ~[flink-table-runtime-1.15.0.jar:1.15.0]
> …
> Caused by: org.apache.flink.util.FlinkRuntimeException:
> org.apache.flink.api.common.InvalidProgramException: Table program cannot
> be compiled. This is a bug. Please file an issue.
>
> at
> org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:94)
> ~[flink-table-runtime-1.15.0.jar:1.15.0]
>
> …
>
> Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column
> 75: Cannot determine simple type name "org"
>
> at
> org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
> ~[flink-table-runtime-1.15.0.jar:1.15.0]
>
>
>
> *From: *David Anderson <da...@apache.org>
> *Date: *Thursday, June 9, 2022 at 5:34 AM
> *To: *Benenson, Michael <Mi...@intuit.com>
> *Cc: *user@flink.apache.org <us...@flink.apache.org>, Deshpande, Omkar <
> Omkar_Deshpande@intuit.com>, Waghulde, Suraj <Su...@intuit.com>
> *Subject: *Re: Fink 15: InvalidProgramException: Table program cannot be
> compiled. This is a bug
>
> This email is from an external sender.
>
>
>
> A Table can have at most one time attribute. In your Table the proc_time
> column is a processing time attribute, and when you define a watermark on
> the event_time column then that column becomes an event-time attribute.
>
>
>
> If you want to combine event time and processing time, you can use
> the PROCTIME() function in your queries without having a processing time
> attribute as one of the columns in the table.
>
>
>
> Best,
>
> David
>
>
>
> On Wed, Jun 8, 2022 at 9:46 PM Benenson, Michael <
> Mikhail_Benenson@intuit.com> wrote:
>
> Hi, folks
>
>
>
> *Short description*:
>
>
>
> I use Flink 1.15.0 sql-client and Java User Define Function in CREATE
> TABLE … statement to get Timestamp.
>
> It works OK, if I do not use Timestamp in Watermark, but if used in
> Watermark it causes
>
> java.lang.RuntimeException: Could not instantiate generated class
> 'WatermarkGenerator$0'
>
> …
>
> Caused by: org.apache.flink.api.common.InvalidProgramException: Table
> program cannot be compiled. This is a bug. Please file an issue.
>
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:107)
>
> …
>
> Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column
> 75: Cannot determine simple type name "org"
>
> at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:
> 12211)
>
>
>
>
>
> *Details*:
>
> java -version
>
> openjdk version "11.0.14.1" 2022-02-08 LTS
>
> OpenJDK Runtime Environment Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS)
>
> OpenJDK 64-Bit Server VM Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS,
> mixed mode)
>
>
>
> Flink 1.15.0, flink-1.15.0/bin/sql-client.sh
>
>
>
> SET 'sql-client.execution.result-mode' = 'tableau';
>
> SET 'table.exec.sink.not-null-enforcer' = 'drop';
>
>
>
> CREATE TEMPORARY FUNCTION default_catalog.default_database.fix_instant
>
> AS 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA;
>
>
>
> CREATE OR REPLACE TABLE input (
>
> event_header ROW(topic_name STRING),
>
> `timestamp` STRING NOT NULL,
>
> event_time AS TO_TIMESTAMP(fix_instant(`timestamp`),
> 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
>
> properties ROW(company_id STRING NOT NULL, scope_area STRING,
> action STRING) NOT NULL,
>
> WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND,
>
> proc_time AS PROCTIME()
>
> ) WITH (
>
> 'connector' = 'kafka',
>
> 'topic' = 'mb-1644796800-qbo',
>
> 'properties.bootstrap.servers' = 'localhost:9092',
>
> 'format' = 'json',
>
> 'scan.startup.mode' = 'latest-offset',
>
> 'json.ignore-parse-errors' = 'true',
>
> 'json.fail-on-missing-field' = 'false'
>
> );
>
>
>
> SELECT `timestamp`, event_time, event_header.topic_name AS topic,
> properties.company_id as company FROM input
>
> LIMIT 10
>
> ;
>
>
>
> Works fine, if I comment WATERMARK FOR event_time …
> Causes an error, if WATERMARK FOR event_time is used:
>
> 2022-06-08 12:16:32
>
> java.lang.RuntimeException: Could not instantiate generated class
> 'WatermarkGenerator$0'
>
> at org.apache.flink.table.runtime.generated.GeneratedClass
> .newInstance(GeneratedClass.java:74)
>
> at org.apache.flink.table.runtime.generated.
> GeneratedWatermarkGeneratorSupplier.createWatermarkGenerator(
> GeneratedWatermarkGeneratorSupplier.java:62)
>
> at org.apache.flink.streaming.api.operators.source.
> ProgressiveTimestampsAndWatermarks.createMainOutput(
> ProgressiveTimestampsAndWatermarks.java:104)
>
> at org.apache.flink.streaming.api.operators.SourceOperator
> .initializeMainOutput(SourceOperator.java:426)
>
> at org.apache.flink.streaming.api.operators.SourceOperator
> .emitNextNotReading(SourceOperator.java:402)
>
> at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(
> SourceOperator.java:387)
>
> at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput
> .emitNext(StreamTaskSourceInput.java:68)
>
> at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor
> .processInput(StreamOneInputProcessor.java:65)
>
> at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(
> StreamTask.java:519)
>
> at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor
> .runMailboxLoop(MailboxProcessor.java:203)
>
> at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(
> StreamTask.java:804)
>
> at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(
> StreamTask.java:753)
>
> at org.apache.flink.runtime.taskmanager.Task
> .runWithSystemExitMonitoring(Task.java:948)
>
> at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task
> .java:927)
>
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
>
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> Caused by: org.apache.flink.util.FlinkRuntimeException:
> org.apache.flink.api.common.InvalidProgramException: Table program cannot
> be compiled. This is a bug. Please file an issue.
>
> at org.apache.flink.table.runtime.generated.CompileUtils.compile(
> CompileUtils.java:94)
>
> at org.apache.flink.table.runtime.generated.GeneratedClass.compile(
> GeneratedClass.java:101)
>
> at org.apache.flink.table.runtime.generated.GeneratedClass
> .newInstance(GeneratedClass.java:68)
>
> ... 16 more
>
> Caused by:
> org.apache.flink.shaded.guava30.com.google.common.util.concurrent.
> UncheckedExecutionException: org.apache.flink.api.common.
> InvalidProgramException: Table program cannot be compiled. This is a bug.
> Please file an issue.
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.get(LocalCache.java:2051)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.
> get(LocalCache.java:3962)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LocalManualCache.get(LocalCache.java:4859)
>
> at org.apache.flink.table.runtime.generated.CompileUtils.compile(
> CompileUtils.java:92)
>
> ... 18 more
>
> Caused by: org.apache.flink.api.common.InvalidProgramException: Table
> program cannot be compiled. This is a bug. Please file an issue.
>
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:107)
>
> at org.apache.flink.table.runtime.generated.CompileUtils
> .lambda$compile$0(CompileUtils.java:92)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LocalManualCache$1.load(LocalCache.java:4864)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.loadSync(LocalCache.java:2278)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.get(LocalCache.java:2045)
>
> ... 21 more
>
> Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column
> 75: Cannot determine simple type name "org"
>
> at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:
> 12211)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6833)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6594)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6573)
>
> at org.codehaus.janino.UnitCompiler.access$13900(UnitCompiler.java:215
> )
>
> at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(
> UnitCompiler.java:6481)
>
> at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(
> UnitCompiler.java:6476)
>
> at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3928)
>
> at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:
> 6476)
>
> at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:
> 6469)
>
> at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3927)
>
> at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
>
> at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7121)
>
> at org.codehaus.janino.UnitCompiler.access$17000(UnitCompiler.java:215
> )
>
> at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(
> UnitCompiler.java:6529)
>
> at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(
> UnitCompiler.java:6490)
>
> at org.codehaus.janino.Java$NewClassInstance.accept(Java.java:5190)
>
> at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:
> 6490)
>
> at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:
> 6469)
>
> at org.codehaus.janino.Java$Rvalue.accept(Java.java:4116)
>
> at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
>
> at org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(
> UnitCompiler.java:9237)
>
> at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9123
> )
>
> at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9025
> )
>
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5062
> )
>
> at org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(
> UnitCompiler.java:4423)
>
> at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(
> UnitCompiler.java:4396)
>
> at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
>
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
>
> at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:
> 5662)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3783)
>
> at org.codehaus.janino.UnitCompiler.access$5900(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(
> UnitCompiler.java:3762)
>
> at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(
> UnitCompiler.java:3734)
>
> at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3734)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2360)
>
> at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(
> UnitCompiler.java:1494)
>
> at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(
> UnitCompiler.java:1487)
>
> at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2874)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)
>
> at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler
> .java:1567)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)
>
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(
> UnitCompiler.java:1357)
>
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(
> UnitCompiler.java:1330)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432)
>
> at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$2
> .visitPackageMemberClassDeclaration(UnitCompiler.java:411)
>
> at org.codehaus.janino.UnitCompiler$2
> .visitPackageMemberClassDeclaration(UnitCompiler.java:406)
>
> at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java
> .java:1414)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)
>
> at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378)
>
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)
>
> at org.codehaus.janino.SimpleCompiler.compileToClassLoader(
> SimpleCompiler.java:465)
>
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216)
>
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207)
>
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80)
>
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75)
>
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:104)
>
> ... 27 more
>
>
>
>
>
> Java code:
>
>
>
> package com.intuit.data.strmprocess.udf;
>
> import org.apache.flink.table.functions.ScalarFunction;
>
>
>
>
>
>
>
>
>
>
>
>
> */** Usage in SQL: CREATE TEMPORARY FUNCTION
> default_catalog.default_database.fix_instant AS
> 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA; CREATE OR
> REPLACE TABLE input ( `timestamp` STRING NOT NULL, event_time AS
> TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
> */ *public class FixInstant extends ScalarFunction
> {
> private static final long *serialVersionUID *= -3115461377254640072L;
>
> public String eval(String s) {
> if (s == null) return null;
> else if (s.indexOf('.') > 0) return s;
> else if (s.endsWith("Z")) return s.replace("Z", ".000Z");
> else return null;
> }
> }
>
>
>
> Compiled for Java 8 or Java 11, no differences
>
>
>
> Any ideas?
>
>
>
>
Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug
Posted by "Benenson, Michael" <Mi...@intuit.com>.
Hi, David
Hard to tell for sure, but yes, [1] could also indicate some problems with Janio.
Both Flink 1.14.3 & 1.15.0 use <janino.version>3.0.11</janino.version>
The difference is that Flink 1.14.3 has been compiled & run on jvm-8, but Flink 1.15.0 has been compiled & run on jvm-11
In the revision history for Janio at [2] one comment for release 3.0.11 is
* At last, the "jdk-9" implementation works.
Probably, we could imply that jdk-11 does not work in Janio 3.0.11, so switch to the latest version of Janio 3.1.7<https://mvnrepository.com/artifact/org.codehaus.janino/janino/3.1.7> could help.
Do you know, how to make nightly flink build with janio 3.1.7<https://mvnrepository.com/artifact/org.codehaus.janino/janino/3.1.7>?
[1] https://lists.apache.org/thread/9tw165cgpdqz4ron76b1ckmwm9hy4qfd
[2] https://janino-compiler.github.io/janino/changelog.html
From: David Anderson <da...@apache.org>
Date: Thursday, June 9, 2022 at 12:05 PM
To: Benenson, Michael <Mi...@intuit.com>
Cc: user@flink.apache.org <us...@flink.apache.org>, Deshpande, Omkar <Om...@intuit.com>, Waghulde, Suraj <Su...@intuit.com>
Subject: Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug
This email is from an external sender.
Sorry; I guess I jumped to conclusions and got it wrong. Let's dig in deeper.
This sounds similar to what was reported in this thread [1] where someone else ran into problems with Janino after upgrading from 1.14 to 1.15. Could this be another instance of the same issue (i.e., Flink ends up using the wrong version of Janino)?
[1] https://lists.apache.org/thread/9tw165cgpdqz4ron76b1ckmwm9hy4qfd
On Thu, Jun 9, 2022 at 8:18 PM Benenson, Michael <Mi...@intuit.com>> wrote:
Hi, David
I have tried CREATE TABLE … without proc_time, but it does not help, I see the same exception.
And this is a new issue for Flink 1.15.0, for Flink 1.14.3 it works fine, even with both event_time & proc_time in CREATE TABLE statement.
Now I use
CREATE OR REPLACE TABLE input (
event_header ROW(topic_name STRING),
`timestamp` STRING NOT NULL,
event_time AS TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
properties ROW(company_id STRING NOT NULL, scope_area STRING, action STRING) NOT NULL,
WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND
) WITH (
'connector' = 'kafka',
'topic' = 'mb-1644796800-qbo',
'properties.bootstrap.servers' = 'localhost:9092',
'format' = 'json',
'scan.startup.mode' = 'latest-offset',
'json.ignore-parse-errors' = 'true',
'json.fail-on-missing-field' = 'false'
);
And got the same exception
java.lang.RuntimeException: Could not instantiate generated class 'WatermarkGenerator$0'
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:74) ~[flink-table-runtime-1.15.0.jar:1.15.0]
…
Caused by: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:94) ~[flink-table-runtime-1.15.0.jar:1.15.0]
…
Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column 75: Cannot determine simple type name "org"
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211) ~[flink-table-runtime-1.15.0.jar:1.15.0]
From: David Anderson <da...@apache.org>>
Date: Thursday, June 9, 2022 at 5:34 AM
To: Benenson, Michael <Mi...@intuit.com>>
Cc: user@flink.apache.org<ma...@flink.apache.org> <us...@flink.apache.org>>, Deshpande, Omkar <Om...@intuit.com>>, Waghulde, Suraj <Su...@intuit.com>>
Subject: Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug
This email is from an external sender.
A Table can have at most one time attribute. In your Table the proc_time column is a processing time attribute, and when you define a watermark on the event_time column then that column becomes an event-time attribute.
If you want to combine event time and processing time, you can use the PROCTIME() function in your queries without having a processing time attribute as one of the columns in the table.
Best,
David
On Wed, Jun 8, 2022 at 9:46 PM Benenson, Michael <Mi...@intuit.com>> wrote:
Hi, folks
Short description:
I use Flink 1.15.0 sql-client and Java User Define Function in CREATE TABLE … statement to get Timestamp.
It works OK, if I do not use Timestamp in Watermark, but if used in Watermark it causes
java.lang.RuntimeException: Could not instantiate generated class 'WatermarkGenerator$0'
…
Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:107)
…
Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column 75: Cannot determine simple type name "org"
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
Details:
java -version
openjdk version "11.0.14.1" 2022-02-08 LTS
OpenJDK Runtime Environment Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS, mixed mode)
Flink 1.15.0, flink-1.15.0/bin/sql-client.sh
SET 'sql-client.execution.result-mode' = 'tableau';
SET 'table.exec.sink.not-null-enforcer' = 'drop';
CREATE TEMPORARY FUNCTION default_catalog.default_database.fix_instant
AS 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA;
CREATE OR REPLACE TABLE input (
event_header ROW(topic_name STRING),
`timestamp` STRING NOT NULL,
event_time AS TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
properties ROW(company_id STRING NOT NULL, scope_area STRING, action STRING) NOT NULL,
WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND,
proc_time AS PROCTIME()
) WITH (
'connector' = 'kafka',
'topic' = 'mb-1644796800-qbo',
'properties.bootstrap.servers' = 'localhost:9092',
'format' = 'json',
'scan.startup.mode' = 'latest-offset',
'json.ignore-parse-errors' = 'true',
'json.fail-on-missing-field' = 'false'
);
SELECT `timestamp`, event_time, event_header.topic_name AS topic, properties.company_id as company FROM input
LIMIT 10
;
Works fine, if I comment WATERMARK FOR event_time …
Causes an error, if WATERMARK FOR event_time is used:
2022-06-08 12:16:32
java.lang.RuntimeException: Could not instantiate generated class 'WatermarkGenerator$0'
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:74)
at org.apache.flink.table.runtime.generated.GeneratedWatermarkGeneratorSupplier.createWatermarkGenerator(GeneratedWatermarkGeneratorSupplier.java:62)
at org.apache.flink.streaming.api.operators.source.ProgressiveTimestampsAndWatermarks.createMainOutput(ProgressiveTimestampsAndWatermarks.java:104)
at org.apache.flink.streaming.api.operators.SourceOperator.initializeMainOutput(SourceOperator.java:426)
at org.apache.flink.streaming.api.operators.SourceOperator.emitNextNotReading(SourceOperator.java:402)
at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:387)
at org.apache.flink.streaming.runtime.io<http://org.apache.flink.streaming.runtime.io>.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68)
at org.apache.flink.streaming.runtime.io<http://org.apache.flink.streaming.runtime.io>.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:519)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:804)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:753)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:948)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:94)
at org.apache.flink.table.runtime.generated.GeneratedClass.compile(GeneratedClass.java:101)
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:68)
... 16 more
Caused by: org.apache.flink.shaded.guava30.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.get(LocalCache.java:3962)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4859)
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:92)
... 18 more
Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:107)
at org.apache.flink.table.runtime.generated.CompileUtils.lambda$compile$0(CompileUtils.java:92)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4864)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
... 21 more
Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column 75: Cannot determine simple type name "org"
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6833)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6594)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6573)
at org.codehaus.janino.UnitCompiler.access$13900(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(UnitCompiler.java:6481)
at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(UnitCompiler.java:6476)
at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3928)
at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:6476)
at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:6469)
at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3927)
at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7121)
at org.codehaus.janino.UnitCompiler.access$17000(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(UnitCompiler.java:6529)
at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(UnitCompiler.java:6490)
at org.codehaus.janino.Java$NewClassInstance.accept(Java.java:5190)
at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:6490)
at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:6469)
at org.codehaus.janino.Java$Rvalue.accept(Java.java:4116)
at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
at org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:9237)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9123)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9025)
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5062)
at org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4423)
at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4396)
at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5662)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3783)
at org.codehaus.janino.UnitCompiler.access$5900(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(UnitCompiler.java:3762)
at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(UnitCompiler.java:3734)
at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3734)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2360)
at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1494)
at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1487)
at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2874)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)
at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1567)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1357)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1330)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432)
at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:411)
at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:406)
at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1414)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)
at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)
at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75)
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:104)
... 27 more
Java code:
package com.intuit.data.strmprocess.udf;
import org.apache.flink.table.functions.ScalarFunction;
/**
Usage in SQL:
CREATE TEMPORARY FUNCTION default_catalog.default_database.fix_instant
AS 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA;
CREATE OR REPLACE TABLE input (
`timestamp` STRING NOT NULL,
event_time AS TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
*/
public class FixInstant extends ScalarFunction
{
private static final long serialVersionUID = -3115461377254640072L;
public String eval(String s) {
if (s == null) return null;
else if (s.indexOf('.') > 0) return s;
else if (s.endsWith("Z")) return s.replace("Z", ".000Z");
else return null;
}
}
Compiled for Java 8 or Java 11, no differences
Any ideas?
Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug
Posted by David Anderson <da...@apache.org>.
Sorry; I guess I jumped to conclusions and got it wrong. Let's dig in
deeper.
This sounds similar to what was reported in this thread [1] where someone
else ran into problems with Janino after upgrading from 1.14 to 1.15. Could
this be another instance of the same issue (i.e., Flink ends up using the
wrong version of Janino)?
[1] https://lists.apache.org/thread/9tw165cgpdqz4ron76b1ckmwm9hy4qfd
On Thu, Jun 9, 2022 at 8:18 PM Benenson, Michael <
Mikhail_Benenson@intuit.com> wrote:
> Hi, David
>
>
>
> I have tried CREATE TABLE … without proc_time, but it does not help, I see
> the same exception.
>
>
> And this is a new issue for Flink 1.15.0, for Flink 1.14.3 it works fine,
> even with both event_time & proc_time in CREATE TABLE statement.
>
> Now I use
>
> CREATE OR REPLACE TABLE input (
> event_header ROW(topic_name STRING),
> `timestamp` STRING NOT NULL,
> event_time AS TO_TIMESTAMP(fix_instant(`timestamp`),
> 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
> properties ROW(company_id STRING NOT NULL, scope_area STRING, action STRING)
> NOT NULL,
> WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND
> ) WITH (
> 'connector' = 'kafka',
> 'topic' = 'mb-1644796800-qbo',
> 'properties.bootstrap.servers' = 'localhost:9092',
> 'format' = 'json',
> 'scan.startup.mode' = 'latest-offset',
> 'json.ignore-parse-errors' = 'true',
> 'json.fail-on-missing-field' = 'false'
> );
>
> And got the same exception
>
>
> java.lang.RuntimeException: Could not instantiate generated class
> 'WatermarkGenerator$0'
>
> at
> org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:74)
> ~[flink-table-runtime-1.15.0.jar:1.15.0]
> …
> Caused by: org.apache.flink.util.FlinkRuntimeException:
> org.apache.flink.api.common.InvalidProgramException: Table program cannot
> be compiled. This is a bug. Please file an issue.
>
> at
> org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:94)
> ~[flink-table-runtime-1.15.0.jar:1.15.0]
>
> …
>
> Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column
> 75: Cannot determine simple type name "org"
>
> at
> org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
> ~[flink-table-runtime-1.15.0.jar:1.15.0]
>
>
>
> *From: *David Anderson <da...@apache.org>
> *Date: *Thursday, June 9, 2022 at 5:34 AM
> *To: *Benenson, Michael <Mi...@intuit.com>
> *Cc: *user@flink.apache.org <us...@flink.apache.org>, Deshpande, Omkar <
> Omkar_Deshpande@intuit.com>, Waghulde, Suraj <Su...@intuit.com>
> *Subject: *Re: Fink 15: InvalidProgramException: Table program cannot be
> compiled. This is a bug
>
> This email is from an external sender.
>
>
>
> A Table can have at most one time attribute. In your Table the proc_time
> column is a processing time attribute, and when you define a watermark on
> the event_time column then that column becomes an event-time attribute.
>
>
>
> If you want to combine event time and processing time, you can use
> the PROCTIME() function in your queries without having a processing time
> attribute as one of the columns in the table.
>
>
>
> Best,
>
> David
>
>
>
> On Wed, Jun 8, 2022 at 9:46 PM Benenson, Michael <
> Mikhail_Benenson@intuit.com> wrote:
>
> Hi, folks
>
>
>
> *Short description*:
>
>
>
> I use Flink 1.15.0 sql-client and Java User Define Function in CREATE
> TABLE … statement to get Timestamp.
>
> It works OK, if I do not use Timestamp in Watermark, but if used in
> Watermark it causes
>
> java.lang.RuntimeException: Could not instantiate generated class
> 'WatermarkGenerator$0'
>
> …
>
> Caused by: org.apache.flink.api.common.InvalidProgramException: Table
> program cannot be compiled. This is a bug. Please file an issue.
>
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:107)
>
> …
>
> Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column
> 75: Cannot determine simple type name "org"
>
> at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:
> 12211)
>
>
>
>
>
> *Details*:
>
> java -version
>
> openjdk version "11.0.14.1" 2022-02-08 LTS
>
> OpenJDK Runtime Environment Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS)
>
> OpenJDK 64-Bit Server VM Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS,
> mixed mode)
>
>
>
> Flink 1.15.0, flink-1.15.0/bin/sql-client.sh
>
>
>
> SET 'sql-client.execution.result-mode' = 'tableau';
>
> SET 'table.exec.sink.not-null-enforcer' = 'drop';
>
>
>
> CREATE TEMPORARY FUNCTION default_catalog.default_database.fix_instant
>
> AS 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA;
>
>
>
> CREATE OR REPLACE TABLE input (
>
> event_header ROW(topic_name STRING),
>
> `timestamp` STRING NOT NULL,
>
> event_time AS TO_TIMESTAMP(fix_instant(`timestamp`),
> 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
>
> properties ROW(company_id STRING NOT NULL, scope_area STRING,
> action STRING) NOT NULL,
>
> WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND,
>
> proc_time AS PROCTIME()
>
> ) WITH (
>
> 'connector' = 'kafka',
>
> 'topic' = 'mb-1644796800-qbo',
>
> 'properties.bootstrap.servers' = 'localhost:9092',
>
> 'format' = 'json',
>
> 'scan.startup.mode' = 'latest-offset',
>
> 'json.ignore-parse-errors' = 'true',
>
> 'json.fail-on-missing-field' = 'false'
>
> );
>
>
>
> SELECT `timestamp`, event_time, event_header.topic_name AS topic,
> properties.company_id as company FROM input
>
> LIMIT 10
>
> ;
>
>
>
> Works fine, if I comment WATERMARK FOR event_time …
> Causes an error, if WATERMARK FOR event_time is used:
>
> 2022-06-08 12:16:32
>
> java.lang.RuntimeException: Could not instantiate generated class
> 'WatermarkGenerator$0'
>
> at org.apache.flink.table.runtime.generated.GeneratedClass
> .newInstance(GeneratedClass.java:74)
>
> at org.apache.flink.table.runtime.generated.
> GeneratedWatermarkGeneratorSupplier.createWatermarkGenerator(
> GeneratedWatermarkGeneratorSupplier.java:62)
>
> at org.apache.flink.streaming.api.operators.source.
> ProgressiveTimestampsAndWatermarks.createMainOutput(
> ProgressiveTimestampsAndWatermarks.java:104)
>
> at org.apache.flink.streaming.api.operators.SourceOperator
> .initializeMainOutput(SourceOperator.java:426)
>
> at org.apache.flink.streaming.api.operators.SourceOperator
> .emitNextNotReading(SourceOperator.java:402)
>
> at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(
> SourceOperator.java:387)
>
> at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput
> .emitNext(StreamTaskSourceInput.java:68)
>
> at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor
> .processInput(StreamOneInputProcessor.java:65)
>
> at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(
> StreamTask.java:519)
>
> at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor
> .runMailboxLoop(MailboxProcessor.java:203)
>
> at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(
> StreamTask.java:804)
>
> at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(
> StreamTask.java:753)
>
> at org.apache.flink.runtime.taskmanager.Task
> .runWithSystemExitMonitoring(Task.java:948)
>
> at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task
> .java:927)
>
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
>
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> Caused by: org.apache.flink.util.FlinkRuntimeException:
> org.apache.flink.api.common.InvalidProgramException: Table program cannot
> be compiled. This is a bug. Please file an issue.
>
> at org.apache.flink.table.runtime.generated.CompileUtils.compile(
> CompileUtils.java:94)
>
> at org.apache.flink.table.runtime.generated.GeneratedClass.compile(
> GeneratedClass.java:101)
>
> at org.apache.flink.table.runtime.generated.GeneratedClass
> .newInstance(GeneratedClass.java:68)
>
> ... 16 more
>
> Caused by:
> org.apache.flink.shaded.guava30.com.google.common.util.concurrent.
> UncheckedExecutionException: org.apache.flink.api.common.
> InvalidProgramException: Table program cannot be compiled. This is a bug.
> Please file an issue.
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.get(LocalCache.java:2051)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.
> get(LocalCache.java:3962)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LocalManualCache.get(LocalCache.java:4859)
>
> at org.apache.flink.table.runtime.generated.CompileUtils.compile(
> CompileUtils.java:92)
>
> ... 18 more
>
> Caused by: org.apache.flink.api.common.InvalidProgramException: Table
> program cannot be compiled. This is a bug. Please file an issue.
>
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:107)
>
> at org.apache.flink.table.runtime.generated.CompileUtils
> .lambda$compile$0(CompileUtils.java:92)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LocalManualCache$1.load(LocalCache.java:4864)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.loadSync(LocalCache.java:2278)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.get(LocalCache.java:2045)
>
> ... 21 more
>
> Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column
> 75: Cannot determine simple type name "org"
>
> at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:
> 12211)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6833)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6594)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6573)
>
> at org.codehaus.janino.UnitCompiler.access$13900(UnitCompiler.java:215
> )
>
> at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(
> UnitCompiler.java:6481)
>
> at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(
> UnitCompiler.java:6476)
>
> at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3928)
>
> at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:
> 6476)
>
> at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:
> 6469)
>
> at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3927)
>
> at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
>
> at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7121)
>
> at org.codehaus.janino.UnitCompiler.access$17000(UnitCompiler.java:215
> )
>
> at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(
> UnitCompiler.java:6529)
>
> at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(
> UnitCompiler.java:6490)
>
> at org.codehaus.janino.Java$NewClassInstance.accept(Java.java:5190)
>
> at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:
> 6490)
>
> at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:
> 6469)
>
> at org.codehaus.janino.Java$Rvalue.accept(Java.java:4116)
>
> at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
>
> at org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(
> UnitCompiler.java:9237)
>
> at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9123
> )
>
> at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9025
> )
>
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5062
> )
>
> at org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(
> UnitCompiler.java:4423)
>
> at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(
> UnitCompiler.java:4396)
>
> at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
>
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
>
> at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:
> 5662)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3783)
>
> at org.codehaus.janino.UnitCompiler.access$5900(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(
> UnitCompiler.java:3762)
>
> at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(
> UnitCompiler.java:3734)
>
> at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3734)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2360)
>
> at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(
> UnitCompiler.java:1494)
>
> at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(
> UnitCompiler.java:1487)
>
> at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2874)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)
>
> at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler
> .java:1567)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)
>
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(
> UnitCompiler.java:1357)
>
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(
> UnitCompiler.java:1330)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432)
>
> at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$2
> .visitPackageMemberClassDeclaration(UnitCompiler.java:411)
>
> at org.codehaus.janino.UnitCompiler$2
> .visitPackageMemberClassDeclaration(UnitCompiler.java:406)
>
> at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java
> .java:1414)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)
>
> at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378)
>
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)
>
> at org.codehaus.janino.SimpleCompiler.compileToClassLoader(
> SimpleCompiler.java:465)
>
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216)
>
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207)
>
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80)
>
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75)
>
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:104)
>
> ... 27 more
>
>
>
>
>
> Java code:
>
>
>
> package com.intuit.data.strmprocess.udf;
>
> import org.apache.flink.table.functions.ScalarFunction;
>
>
>
>
>
>
>
>
>
>
>
>
> */** Usage in SQL: CREATE TEMPORARY FUNCTION
> default_catalog.default_database.fix_instant AS
> 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA; CREATE OR
> REPLACE TABLE input ( `timestamp` STRING NOT NULL, event_time AS
> TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
> */ *public class FixInstant extends ScalarFunction
> {
> private static final long *serialVersionUID *= -3115461377254640072L;
>
> public String eval(String s) {
> if (s == null) return null;
> else if (s.indexOf('.') > 0) return s;
> else if (s.endsWith("Z")) return s.replace("Z", ".000Z");
> else return null;
> }
> }
>
>
>
> Compiled for Java 8 or Java 11, no differences
>
>
>
> Any ideas?
>
>
>
>
Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug
Posted by "Benenson, Michael" <Mi...@intuit.com>.
Hi, David
I have tried CREATE TABLE … without proc_time, but it does not help, I see the same exception.
And this is a new issue for Flink 1.15.0, for Flink 1.14.3 it works fine, even with both event_time & proc_time in CREATE TABLE statement.
Now I use
CREATE OR REPLACE TABLE input (
event_header ROW(topic_name STRING),
`timestamp` STRING NOT NULL,
event_time AS TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
properties ROW(company_id STRING NOT NULL, scope_area STRING, action STRING) NOT NULL,
WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND
) WITH (
'connector' = 'kafka',
'topic' = 'mb-1644796800-qbo',
'properties.bootstrap.servers' = 'localhost:9092',
'format' = 'json',
'scan.startup.mode' = 'latest-offset',
'json.ignore-parse-errors' = 'true',
'json.fail-on-missing-field' = 'false'
);
And got the same exception
java.lang.RuntimeException: Could not instantiate generated class 'WatermarkGenerator$0'
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:74) ~[flink-table-runtime-1.15.0.jar:1.15.0]
…
Caused by: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:94) ~[flink-table-runtime-1.15.0.jar:1.15.0]
…
Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column 75: Cannot determine simple type name "org"
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211) ~[flink-table-runtime-1.15.0.jar:1.15.0]
From: David Anderson <da...@apache.org>
Date: Thursday, June 9, 2022 at 5:34 AM
To: Benenson, Michael <Mi...@intuit.com>
Cc: user@flink.apache.org <us...@flink.apache.org>, Deshpande, Omkar <Om...@intuit.com>, Waghulde, Suraj <Su...@intuit.com>
Subject: Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug
This email is from an external sender.
A Table can have at most one time attribute. In your Table the proc_time column is a processing time attribute, and when you define a watermark on the event_time column then that column becomes an event-time attribute.
If you want to combine event time and processing time, you can use the PROCTIME() function in your queries without having a processing time attribute as one of the columns in the table.
Best,
David
On Wed, Jun 8, 2022 at 9:46 PM Benenson, Michael <Mi...@intuit.com>> wrote:
Hi, folks
Short description:
I use Flink 1.15.0 sql-client and Java User Define Function in CREATE TABLE … statement to get Timestamp.
It works OK, if I do not use Timestamp in Watermark, but if used in Watermark it causes
java.lang.RuntimeException: Could not instantiate generated class 'WatermarkGenerator$0'
…
Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:107)
…
Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column 75: Cannot determine simple type name "org"
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
Details:
java -version
openjdk version "11.0.14.1" 2022-02-08 LTS
OpenJDK Runtime Environment Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS, mixed mode)
Flink 1.15.0, flink-1.15.0/bin/sql-client.sh
SET 'sql-client.execution.result-mode' = 'tableau';
SET 'table.exec.sink.not-null-enforcer' = 'drop';
CREATE TEMPORARY FUNCTION default_catalog.default_database.fix_instant
AS 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA;
CREATE OR REPLACE TABLE input (
event_header ROW(topic_name STRING),
`timestamp` STRING NOT NULL,
event_time AS TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
properties ROW(company_id STRING NOT NULL, scope_area STRING, action STRING) NOT NULL,
WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND,
proc_time AS PROCTIME()
) WITH (
'connector' = 'kafka',
'topic' = 'mb-1644796800-qbo',
'properties.bootstrap.servers' = 'localhost:9092',
'format' = 'json',
'scan.startup.mode' = 'latest-offset',
'json.ignore-parse-errors' = 'true',
'json.fail-on-missing-field' = 'false'
);
SELECT `timestamp`, event_time, event_header.topic_name AS topic, properties.company_id as company FROM input
LIMIT 10
;
Works fine, if I comment WATERMARK FOR event_time …
Causes an error, if WATERMARK FOR event_time is used:
2022-06-08 12:16:32
java.lang.RuntimeException: Could not instantiate generated class 'WatermarkGenerator$0'
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:74)
at org.apache.flink.table.runtime.generated.GeneratedWatermarkGeneratorSupplier.createWatermarkGenerator(GeneratedWatermarkGeneratorSupplier.java:62)
at org.apache.flink.streaming.api.operators.source.ProgressiveTimestampsAndWatermarks.createMainOutput(ProgressiveTimestampsAndWatermarks.java:104)
at org.apache.flink.streaming.api.operators.SourceOperator.initializeMainOutput(SourceOperator.java:426)
at org.apache.flink.streaming.api.operators.SourceOperator.emitNextNotReading(SourceOperator.java:402)
at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:387)
at org.apache.flink.streaming.runtime.io<http://org.apache.flink.streaming.runtime.io>.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68)
at org.apache.flink.streaming.runtime.io<http://org.apache.flink.streaming.runtime.io>.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:519)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:804)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:753)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:948)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:94)
at org.apache.flink.table.runtime.generated.GeneratedClass.compile(GeneratedClass.java:101)
at org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:68)
... 16 more
Caused by: org.apache.flink.shaded.guava30.com.google.common.util.concurrent.UncheckedExecutionException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.get(LocalCache.java:3962)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4859)
at org.apache.flink.table.runtime.generated.CompileUtils.compile(CompileUtils.java:92)
... 18 more
Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:107)
at org.apache.flink.table.runtime.generated.CompileUtils.lambda$compile$0(CompileUtils.java:92)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4864)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
... 21 more
Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column 75: Cannot determine simple type name "org"
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:12211)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6833)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6594)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6607)
at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6573)
at org.codehaus.janino.UnitCompiler.access$13900(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(UnitCompiler.java:6481)
at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(UnitCompiler.java:6476)
at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3928)
at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:6476)
at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:6469)
at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3927)
at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7121)
at org.codehaus.janino.UnitCompiler.access$17000(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(UnitCompiler.java:6529)
at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(UnitCompiler.java:6490)
at org.codehaus.janino.Java$NewClassInstance.accept(Java.java:5190)
at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:6490)
at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:6469)
at org.codehaus.janino.Java$Rvalue.accept(Java.java:4116)
at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
at org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:9237)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9123)
at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9025)
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5062)
at org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4423)
at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(UnitCompiler.java:4396)
at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5662)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3783)
at org.codehaus.janino.UnitCompiler.access$5900(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(UnitCompiler.java:3762)
at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(UnitCompiler.java:3734)
at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3734)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2360)
at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1494)
at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1487)
at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2874)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)
at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1567)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1357)
at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1330)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432)
at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215)
at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:411)
at org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:406)
at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1414)
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)
at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)
at org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:465)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216)
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80)
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75)
at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(CompileUtils.java:104)
... 27 more
Java code:
package com.intuit.data.strmprocess.udf;
import org.apache.flink.table.functions.ScalarFunction;
/**
Usage in SQL:
CREATE TEMPORARY FUNCTION default_catalog.default_database.fix_instant
AS 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA;
CREATE OR REPLACE TABLE input (
`timestamp` STRING NOT NULL,
event_time AS TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
*/
public class FixInstant extends ScalarFunction
{
private static final long serialVersionUID = -3115461377254640072L;
public String eval(String s) {
if (s == null) return null;
else if (s.indexOf('.') > 0) return s;
else if (s.endsWith("Z")) return s.replace("Z", ".000Z");
else return null;
}
}
Compiled for Java 8 or Java 11, no differences
Any ideas?
Re: Fink 15: InvalidProgramException: Table program cannot be compiled. This is a bug
Posted by David Anderson <da...@apache.org>.
A Table can have at most one time attribute. In your Table the proc_time
column is a processing time attribute, and when you define a watermark on
the event_time column then that column becomes an event-time attribute.
If you want to combine event time and processing time, you can use
the PROCTIME() function in your queries without having a processing time
attribute as one of the columns in the table.
Best,
David
On Wed, Jun 8, 2022 at 9:46 PM Benenson, Michael <
Mikhail_Benenson@intuit.com> wrote:
> Hi, folks
>
>
>
> *Short description*:
>
>
>
> I use Flink 1.15.0 sql-client and Java User Define Function in CREATE
> TABLE … statement to get Timestamp.
>
> It works OK, if I do not use Timestamp in Watermark, but if used in
> Watermark it causes
>
> java.lang.RuntimeException: Could not instantiate generated class
> 'WatermarkGenerator$0'
>
> …
>
> Caused by: org.apache.flink.api.common.InvalidProgramException: Table
> program cannot be compiled. This is a bug. Please file an issue.
>
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:107)
>
> …
>
> Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column
> 75: Cannot determine simple type name "org"
>
> at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:
> 12211)
>
>
>
>
>
> *Details*:
>
> java -version
>
> openjdk version "11.0.14.1" 2022-02-08 LTS
>
> OpenJDK Runtime Environment Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS)
>
> OpenJDK 64-Bit Server VM Corretto-11.0.14.10.1 (build 11.0.14.1+10-LTS,
> mixed mode)
>
>
>
> Flink 1.15.0, flink-1.15.0/bin/sql-client.sh
>
>
>
> SET 'sql-client.execution.result-mode' = 'tableau';
>
> SET 'table.exec.sink.not-null-enforcer' = 'drop';
>
>
>
> CREATE TEMPORARY FUNCTION default_catalog.default_database.fix_instant
>
> AS 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA;
>
>
>
> CREATE OR REPLACE TABLE input (
>
> event_header ROW(topic_name STRING),
>
> `timestamp` STRING NOT NULL,
>
> event_time AS TO_TIMESTAMP(fix_instant(`timestamp`),
> 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
>
> properties ROW(company_id STRING NOT NULL, scope_area STRING,
> action STRING) NOT NULL,
>
> WATERMARK FOR event_time AS event_time - INTERVAL '5' SECOND,
>
> proc_time AS PROCTIME()
>
> ) WITH (
>
> 'connector' = 'kafka',
>
> 'topic' = 'mb-1644796800-qbo',
>
> 'properties.bootstrap.servers' = 'localhost:9092',
>
> 'format' = 'json',
>
> 'scan.startup.mode' = 'latest-offset',
>
> 'json.ignore-parse-errors' = 'true',
>
> 'json.fail-on-missing-field' = 'false'
>
> );
>
>
>
> SELECT `timestamp`, event_time, event_header.topic_name AS topic,
> properties.company_id as company FROM input
>
> LIMIT 10
>
> ;
>
>
>
> Works fine, if I comment WATERMARK FOR event_time …
> Causes an error, if WATERMARK FOR event_time is used:
>
> 2022-06-08 12:16:32
>
> java.lang.RuntimeException: Could not instantiate generated class
> 'WatermarkGenerator$0'
>
> at org.apache.flink.table.runtime.generated.GeneratedClass
> .newInstance(GeneratedClass.java:74)
>
> at org.apache.flink.table.runtime.generated.
> GeneratedWatermarkGeneratorSupplier.createWatermarkGenerator(
> GeneratedWatermarkGeneratorSupplier.java:62)
>
> at org.apache.flink.streaming.api.operators.source.
> ProgressiveTimestampsAndWatermarks.createMainOutput(
> ProgressiveTimestampsAndWatermarks.java:104)
>
> at org.apache.flink.streaming.api.operators.SourceOperator
> .initializeMainOutput(SourceOperator.java:426)
>
> at org.apache.flink.streaming.api.operators.SourceOperator
> .emitNextNotReading(SourceOperator.java:402)
>
> at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(
> SourceOperator.java:387)
>
> at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput
> .emitNext(StreamTaskSourceInput.java:68)
>
> at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor
> .processInput(StreamOneInputProcessor.java:65)
>
> at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(
> StreamTask.java:519)
>
> at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor
> .runMailboxLoop(MailboxProcessor.java:203)
>
> at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(
> StreamTask.java:804)
>
> at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(
> StreamTask.java:753)
>
> at org.apache.flink.runtime.taskmanager.Task
> .runWithSystemExitMonitoring(Task.java:948)
>
> at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task
> .java:927)
>
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
>
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
>
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> Caused by: org.apache.flink.util.FlinkRuntimeException:
> org.apache.flink.api.common.InvalidProgramException: Table program cannot
> be compiled. This is a bug. Please file an issue.
>
> at org.apache.flink.table.runtime.generated.CompileUtils.compile(
> CompileUtils.java:94)
>
> at org.apache.flink.table.runtime.generated.GeneratedClass.compile(
> GeneratedClass.java:101)
>
> at org.apache.flink.table.runtime.generated.GeneratedClass
> .newInstance(GeneratedClass.java:68)
>
> ... 16 more
>
> Caused by:
> org.apache.flink.shaded.guava30.com.google.common.util.concurrent.
> UncheckedExecutionException: org.apache.flink.api.common.
> InvalidProgramException: Table program cannot be compiled. This is a bug.
> Please file an issue.
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.get(LocalCache.java:2051)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.LocalCache.
> get(LocalCache.java:3962)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LocalManualCache.get(LocalCache.java:4859)
>
> at org.apache.flink.table.runtime.generated.CompileUtils.compile(
> CompileUtils.java:92)
>
> ... 18 more
>
> Caused by: org.apache.flink.api.common.InvalidProgramException: Table
> program cannot be compiled. This is a bug. Please file an issue.
>
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:107)
>
> at org.apache.flink.table.runtime.generated.CompileUtils
> .lambda$compile$0(CompileUtils.java:92)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LocalManualCache$1.load(LocalCache.java:4864)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.loadSync(LocalCache.java:2278)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
>
> at org.apache.flink.shaded.guava30.com.google.common.cache.
> LocalCache$Segment.get(LocalCache.java:2045)
>
> ... 21 more
>
> Caused by: org.codehaus.commons.compiler.CompileException: Line 30, Column
> 75: Cannot determine simple type name "org"
>
> at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:
> 12211)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6833)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6594)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler
> .java:6607)
>
> at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6573)
>
> at org.codehaus.janino.UnitCompiler.access$13900(UnitCompiler.java:215
> )
>
> at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(
> UnitCompiler.java:6481)
>
> at org.codehaus.janino.UnitCompiler$22$1.visitReferenceType(
> UnitCompiler.java:6476)
>
> at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3928)
>
> at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:
> 6476)
>
> at org.codehaus.janino.UnitCompiler$22.visitType(UnitCompiler.java:
> 6469)
>
> at org.codehaus.janino.Java$ReferenceType.accept(Java.java:3927)
>
> at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
>
> at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7121)
>
> at org.codehaus.janino.UnitCompiler.access$17000(UnitCompiler.java:215
> )
>
> at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(
> UnitCompiler.java:6529)
>
> at org.codehaus.janino.UnitCompiler$22$2.visitNewClassInstance(
> UnitCompiler.java:6490)
>
> at org.codehaus.janino.Java$NewClassInstance.accept(Java.java:5190)
>
> at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:
> 6490)
>
> at org.codehaus.janino.UnitCompiler$22.visitRvalue(UnitCompiler.java:
> 6469)
>
> at org.codehaus.janino.Java$Rvalue.accept(Java.java:4116)
>
> at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6469)
>
> at org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(
> UnitCompiler.java:9237)
>
> at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9123
> )
>
> at org.codehaus.janino.UnitCompiler.findIMethod(UnitCompiler.java:9025
> )
>
> at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5062
> )
>
> at org.codehaus.janino.UnitCompiler.access$9100(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(
> UnitCompiler.java:4423)
>
> at org.codehaus.janino.UnitCompiler$16.visitMethodInvocation(
> UnitCompiler.java:4396)
>
> at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
>
> at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4396)
>
> at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:
> 5662)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3783)
>
> at org.codehaus.janino.UnitCompiler.access$5900(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(
> UnitCompiler.java:3762)
>
> at org.codehaus.janino.UnitCompiler$13.visitMethodInvocation(
> UnitCompiler.java:3734)
>
> at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:5073)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3734)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2360)
>
> at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(
> UnitCompiler.java:1494)
>
> at org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(
> UnitCompiler.java:1487)
>
> at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:2874)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1487)
>
> at org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler
> .java:1567)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3388)
>
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(
> UnitCompiler.java:1357)
>
> at org.codehaus.janino.UnitCompiler.compileDeclaredMethods(
> UnitCompiler.java:1330)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:822)
>
> at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:432)
>
> at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:215)
>
> at org.codehaus.janino.UnitCompiler$2
> .visitPackageMemberClassDeclaration(UnitCompiler.java:411)
>
> at org.codehaus.janino.UnitCompiler$2
> .visitPackageMemberClassDeclaration(UnitCompiler.java:406)
>
> at org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java
> .java:1414)
>
> at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:406)
>
> at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:378)
>
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:237)
>
> at org.codehaus.janino.SimpleCompiler.compileToClassLoader(
> SimpleCompiler.java:465)
>
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:216)
>
> at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:207)
>
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:80)
>
> at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:75)
>
> at org.apache.flink.table.runtime.generated.CompileUtils.doCompile(
> CompileUtils.java:104)
>
> ... 27 more
>
>
>
>
>
> Java code:
>
>
>
> package com.intuit.data.strmprocess.udf;
>
> import org.apache.flink.table.functions.ScalarFunction;
>
>
>
>
>
>
>
>
>
>
>
>
> */** Usage in SQL: CREATE TEMPORARY FUNCTION
> default_catalog.default_database.fix_instant AS
> 'com.intuit.data.strmprocess.udf.FixInstant' LANGUAGE JAVA; CREATE OR
> REPLACE TABLE input ( `timestamp` STRING NOT NULL, event_time AS
> TO_TIMESTAMP(fix_instant(`timestamp`), 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z'''),
> */ *public class FixInstant extends ScalarFunction
> {
> private static final long *serialVersionUID *= -3115461377254640072L;
>
> public String eval(String s) {
> if (s == null) return null;
> else if (s.indexOf('.') > 0) return s;
> else if (s.endsWith("Z")) return s.replace("Z", ".000Z");
> else return null;
> }
> }
>
>
>
> Compiled for Java 8 or Java 11, no differences
>
>
>
> Any ideas?
>
>
>