You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Adesh Dsilva <ad...@gmail.com> on 2022/03/02 17:37:21 UTC
Help with pom dependencies for Flink with Table API
Hello,
I think I accidentally posted this question on the wrong email list (dev) so I am posting it again here.
I am struggling to run my test Flink project with Table API
I am trying to run a simple piece of code:
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setRuntimeMode(RuntimeExecutionMode.BATCH);
StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env);
Path in = Path.fromLocalFile(new File("part-v001-o000-r-00330.avro"));
AvroInputFormat<BidSample> users = new AvroInputFormat<>(in, BidSample.class);
DataStream<BidSample> bidsDS = env.createInput(users);
Table bidsTable = tableEnv.fromDataStream(bidsDS);
bidsTable.printSchema();
And here is my pom dependencies
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.12</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.12</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-api-java-bridge_2.12</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner_2.12</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-common</artifactId>
<version>1.14.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-api-java</artifactId>
<version>1.14.3</version>
</dependency>
But I keep getting below error:
Caused by: java.lang.ClassNotFoundException: org.apache.flink.connector.file.table.factories.BulkReaderFormatFactory
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:766)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
Any help to know why this is happening and fix it would be much appreciated.
Many thanks,
Adesh DSilva
Re: Help with pom dependencies for Flink with Table API
Posted by Adesh Dsilva <ad...@gmail.com>.
Hi,
Yes, you are right. I was mixing some dependencies without knowing.
I did a complete reset of all dependencies and started with a fresh pom and it fixed it.
Many Thanks!
> On 2 Mar 2022, at 17:37, Adesh Dsilva <ad...@gmail.com> wrote:
>
> Hello,
>
> I think I accidentally posted this question on the wrong email list (dev) so I am posting it again here.
>
>
> I am struggling to run my test Flink project with Table API
>
> I am trying to run a simple piece of code:
>
> final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
> env.setRuntimeMode(RuntimeExecutionMode.BATCH);
>
> StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env);
>
> Path in = Path.fromLocalFile(new File("part-v001-o000-r-00330.avro"));
> AvroInputFormat<BidSample> users = new AvroInputFormat<>(in, BidSample.class);
> DataStream<BidSample> bidsDS = env.createInput(users);
>
> Table bidsTable = tableEnv.fromDataStream(bidsDS);
> bidsTable.printSchema();
>
>
> And here is my pom dependencies
>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-streaming-java_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-clients_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-api-java-bridge_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-planner_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-common</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-api-java</artifactId>
> <version>1.14.3</version>
> </dependency>
>
>
>
> But I keep getting below error:
> Caused by: java.lang.ClassNotFoundException: org.apache.flink.connector.file.table.factories.BulkReaderFormatFactory
> at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:766)
> at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>
>
>
> Any help to know why this is happening and fix it would be much appreciated.
>
> Many thanks,
> Adesh DSilva
>
Re: Help with pom dependencies for Flink with Table API
Posted by Francesco Guardiani <fr...@ververica.com>.
Hi,
The moving of
org.apache.flink.connector.file.table.factories.BulkReaderFormatFactory was
done in master a couple of months ago by me, and it should be only on 1.15+.
Could it be you're somehow mixing master snapshots with 1.14.x? Are you
trying to run the job on a cluster using a Flink distribution downloaded
from the website, which version is snapshot and not 1.14?
FG
On Wed, Mar 2, 2022 at 6:44 PM Adesh Dsilva <ad...@gmail.com> wrote:
> Hello,
>
> I think I accidentally posted this question on the wrong email list (dev)
> so I am posting it again here.
>
>
> I am struggling to run my test Flink project with Table API
>
> I am trying to run a simple piece of code:
>
> final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
> env.setRuntimeMode(RuntimeExecutionMode.BATCH);
>
> StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env);
>
> Path in = Path.fromLocalFile(new File("part-v001-o000-r-00330.avro"));
> AvroInputFormat<BidSample> users = new AvroInputFormat<>(in, BidSample.class);
> DataStream<BidSample> bidsDS = env.createInput(users);
>
> Table bidsTable = tableEnv.fromDataStream(bidsDS);
> bidsTable.printSchema();
>
>
>
> And here is my pom dependencies
>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-streaming-java_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-clients_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-api-java-bridge_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-planner_2.12</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-common</artifactId>
> <version>1.14.3</version>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-table-api-java</artifactId>
> <version>1.14.3</version>
> </dependency>
>
>
>
>
> But I keep getting below error:
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.connector.file.table.factories.BulkReaderFormatFactory
> at
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:766)
> at
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
>
>
>
> Any help to know why this is happening and fix it would be much
> appreciated.
>
> Many thanks,
> Adesh DSilva
>
>