You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Adesh Dsilva <ad...@gmail.com> on 2022/03/02 17:33:50 UTC
Help with pom dependencies for Flink with Table
Hi
I am struggling to run my test Flink project with Table API
I am trying to run a simple piece of code:
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setRuntimeMode(RuntimeExecutionMode.BATCH);
StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env);
Path in = Path.fromLocalFile(new File("part-v001-o000-r-00330.avro"));
AvroInputFormat<BidSample> users = new AvroInputFormat<>(in, BidSample.class);
DataStream<BidSample> bidsDS = env.createInput(users);
Table bidsTable = tableEnv.fromDataStream(bidsDS);
bidsTable.printSchema();
And here is my pom dependencies
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.12</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.12</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-api-java-bridge_2.12</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner_2.12</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-common</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-api-java</artifactId>
<version>1.14.3</version>
</dependency>
But I keep getting below error:
Caused by: java.lang.ClassNotFoundException: org.apache.flink.connector.file.table.factories.BulkReaderFormatFactory
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:766)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
Any help to know why this is happening and fix it would be much appreciated.
Many thanks,
Adesh DSilva
Re: Help with pom dependencies for Flink with Table
Posted by Jing Ge <ji...@ververica.com>.
You are very welcome. Thanks for the feedback, I am glad to know that your
issue is solved.
Best regards,
Jing
On Thu, Mar 3, 2022 at 6:13 PM Adesh Dsilva <ad...@gmail.com> wrote:
> Hi
>
> Thank you very much for your reply.
>
> A complete reset with a fresh pom fixed this.
> Like you said I think I had messed up some dependencies
>
> On Wed, 2 Mar 2022, 21:49 Jing Ge, <ji...@ververica.com> wrote:
>
> > Hi Adesh,
> >
> > have you tried building your project with a clean classpath, i.e. mvn
> clean
> > ...? Additionally, you could also try deleting org/apache/flink from your
> > local .m2 repository to have an even cleaner repo. It looks like a
> > BulkReaderFormatFactory from the current master branch [1] has been
> > referenced while Flink 1.14.3 [2] is used.
> >
> > [1]
> >
> >
> https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/factories/BulkReaderFormatFactory.java
> > [2]
> >
> >
> https://github.com/apache/flink/blob/release-1.14.3/flink-table/flink-table-common/src/main/java/org/apache/flink/table/factories/BulkReaderFormatFactory.java
> >
> >
> > Best regards,
> > Jing
> >
> > On Wed, Mar 2, 2022 at 6:34 PM Adesh Dsilva <ad...@gmail.com>
> > wrote:
> >
> > > Hi
> > >
> > > I am struggling to run my test Flink project with Table API
> > >
> > > I am trying to run a simple piece of code:
> > >
> > > final StreamExecutionEnvironment env =
> > > StreamExecutionEnvironment.getExecutionEnvironment();
> > > env.setRuntimeMode(RuntimeExecutionMode.BATCH);
> > >
> > > StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env);
> > >
> > > Path in = Path.fromLocalFile(new File("part-v001-o000-r-00330.avro"));
> > > AvroInputFormat<BidSample> users = new AvroInputFormat<>(in,
> > > BidSample.class);
> > > DataStream<BidSample> bidsDS = env.createInput(users);
> > >
> > > Table bidsTable = tableEnv.fromDataStream(bidsDS);
> > > bidsTable.printSchema();
> > >
> > >
> > > And here is my pom dependencies
> > >
> > > <dependency>
> > > <groupId>org.apache.flink</groupId>
> > > <artifactId>flink-streaming-java_2.12</artifactId>
> > > <version>1.14.3</version>
> > > </dependency>
> > > <dependency>
> > > <groupId>org.apache.flink</groupId>
> > > <artifactId>flink-clients_2.12</artifactId>
> > > <version>1.14.3</version>
> > > </dependency>
> > > <dependency>
> > > <groupId>org.apache.flink</groupId>
> > > <artifactId>flink-table-api-java-bridge_2.12</artifactId>
> > > <version>1.14.3</version>
> > > </dependency>
> > > <dependency>
> > > <groupId>org.apache.flink</groupId>
> > > <artifactId>flink-table-planner_2.12</artifactId>
> > > <version>1.14.3</version>
> > > </dependency>
> > > <dependency>
> > > <groupId>org.apache.flink</groupId>
> > > <artifactId>flink-table-common</artifactId>
> > > <version>1.14.3</version>
> > > </dependency>
> > > <dependency>
> > > <groupId>org.apache.flink</groupId>
> > > <artifactId>flink-table-api-java</artifactId>
> > > <version>1.14.3</version>
> > > </dependency>
> > >
> > >
> > >
> > > But I keep getting below error:
> > > Caused by: java.lang.ClassNotFoundException:
> > > org.apache.flink.connector.file.table.factories.BulkReaderFormatFactory
> > > at
> > >
> >
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:766)
> > > at
> > >
> >
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
> > >
> > >
> > >
> > > Any help to know why this is happening and fix it would be much
> > > appreciated.
> > >
> > > Many thanks,
> > > Adesh DSilva
> >
>
Re: Help with pom dependencies for Flink with Table
Posted by Adesh Dsilva <ad...@gmail.com>.
Hi
Thank you very much for your reply.
A complete reset with a fresh pom fixed this.
Like you said I think I had messed up some dependencies
On Wed, 2 Mar 2022, 21:49 Jing Ge, <ji...@ververica.com> wrote:
> Hi Adesh,
>
> have you tried building your project with a clean classpath, i.e. mvn clean
> ...? Additionally, you could also try deleting org/apache/flink from your
> local .m2 repository to have an even cleaner repo. It looks like a
> BulkReaderFormatFactory from the current master branch [1] has been
> referenced while Flink 1.14.3 [2] is used.
>
> [1]
>
> https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/factories/BulkReaderFormatFactory.java
> [2]
>
> https://github.com/apache/flink/blob/release-1.14.3/flink-table/flink-table-common/src/main/java/org/apache/flink/table/factories/BulkReaderFormatFactory.java
>
>
> Best regards,
> Jing
>
> On Wed, Mar 2, 2022 at 6:34 PM Adesh Dsilva <ad...@gmail.com>
> wrote:
>
> > Hi
> >
> > I am struggling to run my test Flink project with Table API
> >
> > I am trying to run a simple piece of code:
> >
> > final StreamExecutionEnvironment env =
> > StreamExecutionEnvironment.getExecutionEnvironment();
> > env.setRuntimeMode(RuntimeExecutionMode.BATCH);
> >
> > StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env);
> >
> > Path in = Path.fromLocalFile(new File("part-v001-o000-r-00330.avro"));
> > AvroInputFormat<BidSample> users = new AvroInputFormat<>(in,
> > BidSample.class);
> > DataStream<BidSample> bidsDS = env.createInput(users);
> >
> > Table bidsTable = tableEnv.fromDataStream(bidsDS);
> > bidsTable.printSchema();
> >
> >
> > And here is my pom dependencies
> >
> > <dependency>
> > <groupId>org.apache.flink</groupId>
> > <artifactId>flink-streaming-java_2.12</artifactId>
> > <version>1.14.3</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.flink</groupId>
> > <artifactId>flink-clients_2.12</artifactId>
> > <version>1.14.3</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.flink</groupId>
> > <artifactId>flink-table-api-java-bridge_2.12</artifactId>
> > <version>1.14.3</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.flink</groupId>
> > <artifactId>flink-table-planner_2.12</artifactId>
> > <version>1.14.3</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.flink</groupId>
> > <artifactId>flink-table-common</artifactId>
> > <version>1.14.3</version>
> > </dependency>
> > <dependency>
> > <groupId>org.apache.flink</groupId>
> > <artifactId>flink-table-api-java</artifactId>
> > <version>1.14.3</version>
> > </dependency>
> >
> >
> >
> > But I keep getting below error:
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.flink.connector.file.table.factories.BulkReaderFormatFactory
> > at
> >
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:766)
> > at
> >
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
> >
> >
> >
> > Any help to know why this is happening and fix it would be much
> > appreciated.
> >
> > Many thanks,
> > Adesh DSilva
>
Re: Help with pom dependencies for Flink with Table
Posted by Jing Ge <ji...@ververica.com>.
Hi Adesh,
have you tried building your project with a clean classpath, i.e. mvn clean
...? Additionally, you could also try deleting org/apache/flink from your
local .m2 repository to have an even cleaner repo. It looks like a
BulkReaderFormatFactory from the current master branch [1] has been
referenced while Flink 1.14.3 [2] is used.
[1]
https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/factories/BulkReaderFormatFactory.java
[2]
https://github.com/apache/flink/blob/release-1.14.3/flink-table/flink-table-common/src/main/java/org/apache/flink/table/factories/BulkReaderFormatFactory.java
Best regards,
Jing
On Wed, Mar 2, 2022 at 6:34 PM Adesh Dsilva <ad...@gmail.com> wrote:
> Hi
>
> I am struggling to run my test Flink project with Table API
>
> I am trying to run a simple piece of code:
>
> final StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> env.setRuntimeMode(RuntimeExecutionMode.BATCH);
>
> StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env);
>
> Path in = Path.fromLocalFile(new File("part-v001-o000-r-00330.avro"));
> AvroInputFormat<BidSample> users = new AvroInputFormat<>(in,
> BidSample.class);
> DataStream<BidSample> bidsDS = env.createInput(users);
>
> Table bidsTable = tableEnv.fromDataStream(bidsDS);
> bidsTable.printSchema();
>
>
> And here is my pom dependencies
>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-streaming-java_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-clients_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-api-java-bridge_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-planner_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-common</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-api-java</artifactId>
> <version>1.14.3</version>
> </dependency>
>
>
>
> But I keep getting below error:
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.connector.file.table.factories.BulkReaderFormatFactory
> at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:766)
> at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>
>
>
> Any help to know why this is happening and fix it would be much
> appreciated.
>
> Many thanks,
> Adesh DSilva