reading from jdbc connection

[Arran Duff <a-...@hotmail.com>]

Hi,

I'm quite new to flink and I'm trying to create an application, which reads ID's from a kinesis stream and then uses these to read from a mysql database. I expect that I would just be doing a join of the id's onto the table

I'm struggling to understand from the documentation how to actually connect to jdbc from flink using java.  For example - the code shown here<https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/common.html#connector-tables>
doesn't give any information about what to provide as arguments in the connect method. Reading the javadoc I can see that it needs to be a ConnectorDescriptor<https://ci.apache.org/projects/flink/flink-docs-release-1.11/api/java/org/apache/flink/table/descriptors/ConnectorDescriptor.html> object. And I can see that the known subclasses include a HBase, Kafka and Hbase connector. But I don't see one for JDBC. Should I just be using the CustomConnectorDescriptor<https://ci.apache.org/projects/flink/flink-docs-release-1.11/api/java/org/apache/flink/table/descriptors/CustomConnectorDescriptor.html> and adding JDBC connection options? Will this work out of the box or am I going down a rabbit hole?

I also note that all of the examples that I see for the jdbc connector are written in SQL, or DDL or yaml - for example here https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connect.html#jdbc-connector
I'm not quite sure how I would go about getting that working in java. Any help would be greatly appreciated

Thanks,
Arran

Re: reading from jdbc connection

[Robert Metzger <rm...@apache.org>]

Hey Arran,

It seems that the preferred way, even in the Java API is to use a DDL
statement:
https://github.com/apache/flink/blob/master/flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/table/api/bridge/java/StreamTableEnvironment.java#L602-L639


Hope this helps!

Best,
Robert

On Thu, Mar 25, 2021 at 2:40 PM Arran Duff <a-...@hotmail.com> wrote:

> Hi,
>
> I'm quite new to flink and I'm trying to create an application, which
> reads ID's from a kinesis stream and then uses these to read from a mysql
> database. I expect that I would just be doing a join of the id's onto the
> table
>
> I'm struggling to understand from the documentation how to actually
> connect to jdbc from flink using java.  For example - the code shown here
> <https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/common.html#connector-tables>
> doesn't give any information about what to provide as arguments in the
> connect method. Reading the javadoc I can see that it needs to be a
> ConnectorDescriptor
> <https://ci.apache.org/projects/flink/flink-docs-release-1.11/api/java/org/apache/flink/table/descriptors/ConnectorDescriptor.html>
> object. And I can see that the known subclasses include a HBase, Kafka and
> Hbase connector. But I don't see one for JDBC. Should I just be using the
> CustomConnectorDescriptor
> <https://ci.apache.org/projects/flink/flink-docs-release-1.11/api/java/org/apache/flink/table/descriptors/CustomConnectorDescriptor.html> and
> adding JDBC connection options? Will this work out of the box or am I going
> down a rabbit hole?
>
> I also note that all of the examples that I see for the jdbc connector are
> written in SQL, or DDL or yaml - for example here
> https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connect.html#jdbc-connector
> I'm not quite sure how I would go about getting that working in java. Any
> help would be greatly appreciated
>
> Thanks,
> Arran
>