You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by "casel.chen" <ca...@126.com> on 2021/07/15 03:05:51 UTC

flink UDF使用jsonpath解析出错

项目用到了下面依赖,使用flink-shaded-hadoop-2-uber因为使用到了YarnExecutor提交作业
<dependencies>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-table-planner-blink_2.12</artifactId>
            <version>1.12.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-shaded-hadoop-2-uber</artifactId>
            <version>2.8.3-10.0</version>
            <exclusions>
                <exclusion>
                    <groupId>net.minidev</groupId>
                    <artifactId>json-smart</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>net.minidev</groupId>
            <artifactId>json-smart</artifactId>
            <version>2.3</version>
        </dependency>
    </dependencies>


在执行JsonPath.read(json, jsonpath)语句时抛了如下异常,查了下是因为flink-shaded-hadoop-2-uber包含的json-smart库版本是1.1.1,太低了。可我在pom.xml中exclude了也不起作用。要怎么fix呢?


21/07/15 10:56:49 [Source: TableSourceScan(table=[[default_catalog, default_database, kafka_table]], fields=[json]) -> MiniBatchAssigner(interval=[1000ms], mode=[ProcTime]) -> Calc(select=[(json JSON_VALUE _UTF-16LE'$.suricata.eve.event_type') AS EXPR$0, (json JSON_VALUE _UTF-16LE'$.suricata.eve.app_proto_orig') AS EXPR$1]) -> Sink: Sink(table=[default_catalog.default_database.print_table], fields=[EXPR$0, EXPR$1]) (1/1)#0] WARN taskmanager.Task: Source: TableSourceScan(table=[[default_catalog, default_database, kafka_table]], fields=[json]) -> MiniBatchAssigner(interval=[1000ms], mode=[ProcTime]) -> Calc(select=[(json JSON_VALUE _UTF-16LE'$.suricata.eve.event_type') AS EXPR$0, (json JSON_VALUE _UTF-16LE'$.suricata.eve.app_proto_orig') AS EXPR$1]) -> Sink: Sink(table=[default_catalog.default_database.print_table], fields=[EXPR$0, EXPR$1]) (1/1)#0 (db2f5cc66b6c627fc4d1abcfb4451b89) switched from RUNNING to FAILED.
 java.lang.NoSuchFieldError: defaultReader
at org.apache.flink.calcite.shaded.com.jayway.jsonpath.spi.json.JsonSmartJsonProvider.<init>(JsonSmartJsonProvider.java:39)
at org.apache.flink.calcite.shaded.com.jayway.jsonpath.internal.DefaultsImpl.jsonProvider(DefaultsImpl.java:21)
at org.apache.flink.calcite.shaded.com.jayway.jsonpath.Configuration.defaultConfiguration(Configuration.java:174)
at org.apache.flink.calcite.shaded.com.jayway.jsonpath.internal.ParseContextImpl.<init>(ParseContextImpl.java:21)
at org.apache.flink.calcite.shaded.com.jayway.jsonpath.JsonPath.read(JsonPath.java:502)