Kafka connector and Python

[Stefano Braga <s....@antgroup.it>]

Hello everyone,
I'm relatively new to Flink (currently exploring the documentation and experimenting with it).
I'm using Flink version 1.20 (with the Python package apache-flink 1.20.0 and the right Kafka connector jar), and I've been struggling with an issue for the past week.
I wrote a simple script that subscribes to a Kafka topic.
When I execute:

$ python3 my_test_script.py

Everything works perfectly fine.
However, when I try to run it with Flink:

$ flink run --python my_test_script.py


I encounter the following exception:

py4j.protocol.Py4JJavaError: An error occurred while calling o0.execute.
: java.net.MalformedURLException: no protocol: ['file:/opt/flink/opt/flink-python-1.20.0.jar']


Interestingly, when I run the word_count.py example script with the same Flink command:

$ flink run --python word_count.py

It works as expected.
After searching online for a potential solution, I came across suggestions that this exception may be caused by an incompatibility between libraries, even though the error message itself points to something different.
Does anyone have any ideas on how to resolve this?
Thanks for the help and support.

Regards.

Stefano

Re: [External] Kafka connector and Python

[Dian Fu <di...@gmail.com>]

Similar issue reported in slack:
https://apache-flink.slack.com/archives/C03G7LJTS2G/p1736405517356729

It was solved by correcting the java version.

[image: image.png]

On Tue, Dec 3, 2024 at 8:01 PM 闵中元 <mi...@bytedance.com> wrote:

> Maybe you can check the kafka connector version and make sure the jar is
> in the lib dic.
> Kafka
> <https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/connectors/datastream/kafka/>
> nightlies.apache.org
> <https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/connectors/datastream/kafka/>
> [image: favicon.png]
> <https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/connectors/datastream/kafka/>
> <https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/connectors/datastream/kafka/>
>
>
> 2024年12月3日 19:36，Stefano Braga <s....@antgroup.it> 写道：
>
> py4j.protocol.Py4JJavaError: An error occurred while calling o0.execute.
> : java.net.MalformedURLException: no protocol
>
>
>

Re: [External] Kafka connector and Python

[闵中元 <mi...@bytedance.com>]

Maybe you can check the kafka connector version and make sure the jar is in the lib dic.
https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/connectors/datastream/kafka/

> 2024年12月3日 19:36，Stefano Braga <s....@antgroup.it> 写道：
> 
> py4j.protocol.Py4JJavaError: An error occurred while calling o0.execute.
> : java.net.MalformedURLException: no protocol