You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by "casel.chen" <ca...@126.com> on 2021/07/15 03:05:51 UTC
flink UDF使用jsonpath解析出错
项目用到了下面依赖,使用flink-shaded-hadoop-2-uber因为使用到了YarnExecutor提交作业
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner-blink_2.12</artifactId>
<version>1.12.1</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-shaded-hadoop-2-uber</artifactId>
<version>2.8.3-10.0</version>
<exclusions>
<exclusion>
<groupId>net.minidev</groupId>
<artifactId>json-smart</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>net.minidev</groupId>
<artifactId>json-smart</artifactId>
<version>2.3</version>
</dependency>
</dependencies>
在执行JsonPath.read(json, jsonpath)语句时抛了如下异常,查了下是因为flink-shaded-hadoop-2-uber包含的json-smart库版本是1.1.1,太低了。可我在pom.xml中exclude了也不起作用。要怎么fix呢?
21/07/15 10:56:49 [Source: TableSourceScan(table=[[default_catalog, default_database, kafka_table]], fields=[json]) -> MiniBatchAssigner(interval=[1000ms], mode=[ProcTime]) -> Calc(select=[(json JSON_VALUE _UTF-16LE'$.suricata.eve.event_type') AS EXPR$0, (json JSON_VALUE _UTF-16LE'$.suricata.eve.app_proto_orig') AS EXPR$1]) -> Sink: Sink(table=[default_catalog.default_database.print_table], fields=[EXPR$0, EXPR$1]) (1/1)#0] WARN taskmanager.Task: Source: TableSourceScan(table=[[default_catalog, default_database, kafka_table]], fields=[json]) -> MiniBatchAssigner(interval=[1000ms], mode=[ProcTime]) -> Calc(select=[(json JSON_VALUE _UTF-16LE'$.suricata.eve.event_type') AS EXPR$0, (json JSON_VALUE _UTF-16LE'$.suricata.eve.app_proto_orig') AS EXPR$1]) -> Sink: Sink(table=[default_catalog.default_database.print_table], fields=[EXPR$0, EXPR$1]) (1/1)#0 (db2f5cc66b6c627fc4d1abcfb4451b89) switched from RUNNING to FAILED.
java.lang.NoSuchFieldError: defaultReader
at org.apache.flink.calcite.shaded.com.jayway.jsonpath.spi.json.JsonSmartJsonProvider.<init>(JsonSmartJsonProvider.java:39)
at org.apache.flink.calcite.shaded.com.jayway.jsonpath.internal.DefaultsImpl.jsonProvider(DefaultsImpl.java:21)
at org.apache.flink.calcite.shaded.com.jayway.jsonpath.Configuration.defaultConfiguration(Configuration.java:174)
at org.apache.flink.calcite.shaded.com.jayway.jsonpath.internal.ParseContextImpl.<init>(ParseContextImpl.java:21)
at org.apache.flink.calcite.shaded.com.jayway.jsonpath.JsonPath.read(JsonPath.java:502)