Classes missing from jar

[Jason Kania <ja...@ymail.com>]

Hello,
I am just getting started with Flink and am attempting to use the kafka connector.
In particular I am attempting to use the jar flink-connector-kafka-0.11_2.11-1.4.0.jar downloaded from:

https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka-0.11_2.11/1.4.0/
with the latest flink-dist_2.11-1.4.0.jar that was downloaded from the main Flink site.
I attempted to use the constructor:
public FlinkKafkaConsumer011(String topic, KeyedDeserializationSchema<T> deserializer, Properties props)
but the IDE indicated that the KeyedDeserializationSchema interface was not defined.
I see that it should be defined as it is contained in the directory:
https://github.com/apache/flink/tree/release-1.4/flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/util/serialization

and the flink-dist_2.11-1.4.0.jar does contain that path but only contains the classes:
I am guessing I have a mismatch of some sort, but the versions match so I am not sure what to check.

Is the flink connector class in the maven repo old or is there a different location from which it should be retrieved?
Any other suggestions?
Thanks,
Jason

Re: Classes missing from jar

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

Hi Jason,

You actually should not be adding the flink-dist jar as a dependency in your application.
It seems like you are not using a build tool for your application, but adding dependencies manually.
In general, I would recommend build management tools like Maven / Gradle for building Java applications.
That enables dependencies to be automatically included when building the application.

Flink also ships a quickstart Maven archetype, which has all the project build configurations setup properly out-of-the-box.
See here [1] for details. It’s a nice walkthrough of setting up your Flink project using the Java API :)

Cheers,
Gordon

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/quickstart/java_api_quickstart.html

On 13 January 2018 at 1:28:54 AM, Jason Kania (jason.kania@ymail.com) wrote:

Thanks. That resolved it. Also had to pull in the kafka 10 and 9 versions of the connector jars. Once the base jar is in the mvn repository, this won't be as problematic.

On Friday, January 12, 2018, 9:46:22 AM EST, Tzu-Li (Gordon) Tai <tz...@apache.org> wrote:


Hi Jason,

The KeyedDeserializationSchema is located in the flink-connector-kafka-base
module, so you'll need to include the jar for that too [1].

Cheers,
Gordon


[1]

https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka-base_2.11/1.4.0/





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Classes missing from jar

[Jason Kania <ja...@ymail.com>]

 Thanks. That resolved it. Also had to pull in the kafka 10 and 9 versions of the connector jars. Once the base jar is in the mvn repository, this won't be as problematic.

    On Friday, January 12, 2018, 9:46:22 AM EST, Tzu-Li (Gordon) Tai <tz...@apache.org> wrote:  
 
 Hi Jason,

The KeyedDeserializationSchema is located in the flink-connector-kafka-base
module, so you'll need to include the jar for that too [1].

Cheers,
Gordon

[1]
https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka-base_2.11/1.4.0/





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
  

Re: Classes missing from jar

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

Hi Jason,

The KeyedDeserializationSchema is located in the flink-connector-kafka-base
module, so you'll need to include the jar for that too [1].

Cheers,
Gordon

[1]
https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka-base_2.11/1.4.0/





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/