You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Till Rohrmann <tr...@apache.org> on 2020/08/03 08:50:09 UTC
Re: JDBCOutputFormat dependency loading error
Hi Flavio,
I am not a JDBC expert but it looks as if you try to
load com.mysql.cj.jdbc.Driver which is not contained
in mariadb-java-client-2.6.0.jar. mariadb-java-client-2.6.0.jar only
contains org/mariadb/jdbc/Driver.class. com.mysql.cj.jdbc.Driver can be
found in mysql-connector-java.jar, though.
Hence I believe that you are missing some dependencies in your user jar to
make your job run. Please check from where com.mysql.cj.jdbc.Driver is
being loaded when running the job from the IDE.
Cheers,
Till
On Fri, Jul 31, 2020 at 4:55 PM Flavio Pompermaier <po...@okkam.it>
wrote:
> Hi to all,
> I'm trying to run my DataSet job on Flink 1.11.0 and I'm connecting toward
> Mariadb in my code.
> I've put the mariadb-java-client-2.6.0.jar in the lib directory and in the
> pom.xml I set that dependency as provided. The code runs successfully from
> the Ide but when I try to run the code on the cluster I get the following
> error:
>
> Caused by: java.lang.ClassNotFoundException: com.mysql.cj.jdbc.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:471) ~[?:?]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
> at
> org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:61)
> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
> at
> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
> at
> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]
> at java.lang.Class.forName0(Native Method) ~[?:?]
> at java.lang.Class.forName(Class.java:315) ~[?:?]
> at
> org.apache.flink.connector.jdbc.internal.connection.SimpleJdbcConnectionProvider.getConnection(SimpleJdbcConnectionProvider.java:52)
> ~myApp.jar:?]
> at
> org.apache.flink.connector.jdbc.internal.AbstractJdbcOutputFormat.establishConnection(AbstractJdbcOutputFormat.java:66)
> ~myApp.jar:?]
> at
> org.apache.flink.connector.jdbc.internal.AbstractJdbcOutputFormat.open(AbstractJdbcOutputFormat.java:59)
> ~myApp.jar:?]
> at
> org.apache.flink.api.java.io.jdbc.JDBCOutputFormat.open(JDBCOutputFormat.java:82)
> ~myApp.jar:?]
> at
> org.apache.flink.runtime.operators.DataSinkTask.invoke(DataSinkTask.java:205)
> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
> at java.lang.Thread.run(Thread.java:834) ~[?:?]
>
> What should I do?
>
> Thanks in advance,
> Flavio
>
Re: JDBCOutputFormat dependency loading error
Posted by Flavio Pompermaier <po...@okkam.it>.
Yes, the problem indeed was mine (2 different connectors for mariadb, both
mysql and mariadb-client), Sorry for the confusion
On Mon, Aug 3, 2020 at 12:26 PM Till Rohrmann <tr...@apache.org> wrote:
> Glad to hear it!
>
> On Mon, Aug 3, 2020 at 11:59 AM Flavio Pompermaier <po...@okkam.it>
> wrote:
>
>> Yes Till, I was figuring out that I was using 2 different connectors and
>> I forgot the mysql jar..I was going to test and tell if the problem was
>> solved!
>>
>> On Mon, Aug 3, 2020 at 10:50 AM Till Rohrmann <tr...@apache.org>
>> wrote:
>>
>>> Hi Flavio,
>>>
>>> I am not a JDBC expert but it looks as if you try to
>>> load com.mysql.cj.jdbc.Driver which is not contained
>>> in mariadb-java-client-2.6.0.jar. mariadb-java-client-2.6.0.jar only
>>> contains org/mariadb/jdbc/Driver.class. com.mysql.cj.jdbc.Driver can be
>>> found in mysql-connector-java.jar, though.
>>>
>>> Hence I believe that you are missing some dependencies in your user jar
>>> to make your job run. Please check from where com.mysql.cj.jdbc.Driver is
>>> being loaded when running the job from the IDE.
>>>
>>> Cheers,
>>> Till
>>>
>>> On Fri, Jul 31, 2020 at 4:55 PM Flavio Pompermaier <po...@okkam.it>
>>> wrote:
>>>
>>>> Hi to all,
>>>> I'm trying to run my DataSet job on Flink 1.11.0 and I'm connecting
>>>> toward Mariadb in my code.
>>>> I've put the mariadb-java-client-2.6.0.jar in the lib directory and in
>>>> the pom.xml I set that dependency as provided. The code runs successfully
>>>> from the Ide but when I try to run the code on the cluster I get the
>>>> following error:
>>>>
>>>> Caused by: java.lang.ClassNotFoundException: com.mysql.cj.jdbc.Driver
>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:471) ~[?:?]
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
>>>> at
>>>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:61)
>>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>>> at
>>>> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
>>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>>> at
>>>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
>>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]
>>>> at java.lang.Class.forName0(Native Method) ~[?:?]
>>>> at java.lang.Class.forName(Class.java:315) ~[?:?]
>>>> at
>>>> org.apache.flink.connector.jdbc.internal.connection.SimpleJdbcConnectionProvider.getConnection(SimpleJdbcConnectionProvider.java:52)
>>>> ~myApp.jar:?]
>>>> at
>>>> org.apache.flink.connector.jdbc.internal.AbstractJdbcOutputFormat.establishConnection(AbstractJdbcOutputFormat.java:66)
>>>> ~myApp.jar:?]
>>>> at
>>>> org.apache.flink.connector.jdbc.internal.AbstractJdbcOutputFormat.open(AbstractJdbcOutputFormat.java:59)
>>>> ~myApp.jar:?]
>>>> at
>>>> org.apache.flink.api.java.io.jdbc.JDBCOutputFormat.open(JDBCOutputFormat.java:82)
>>>> ~myApp.jar:?]
>>>> at
>>>> org.apache.flink.runtime.operators.DataSinkTask.invoke(DataSinkTask.java:205)
>>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>>> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
>>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
>>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>>> at java.lang.Thread.run(Thread.java:834) ~[?:?]
>>>>
>>>> What should I do?
>>>>
>>>> Thanks in advance,
>>>> Flavio
>>>>
>>>
Re: JDBCOutputFormat dependency loading error
Posted by Till Rohrmann <tr...@apache.org>.
Glad to hear it!
On Mon, Aug 3, 2020 at 11:59 AM Flavio Pompermaier <po...@okkam.it>
wrote:
> Yes Till, I was figuring out that I was using 2 different connectors and I
> forgot the mysql jar..I was going to test and tell if the problem was
> solved!
>
> On Mon, Aug 3, 2020 at 10:50 AM Till Rohrmann <tr...@apache.org>
> wrote:
>
>> Hi Flavio,
>>
>> I am not a JDBC expert but it looks as if you try to
>> load com.mysql.cj.jdbc.Driver which is not contained
>> in mariadb-java-client-2.6.0.jar. mariadb-java-client-2.6.0.jar only
>> contains org/mariadb/jdbc/Driver.class. com.mysql.cj.jdbc.Driver can be
>> found in mysql-connector-java.jar, though.
>>
>> Hence I believe that you are missing some dependencies in your user jar
>> to make your job run. Please check from where com.mysql.cj.jdbc.Driver is
>> being loaded when running the job from the IDE.
>>
>> Cheers,
>> Till
>>
>> On Fri, Jul 31, 2020 at 4:55 PM Flavio Pompermaier <po...@okkam.it>
>> wrote:
>>
>>> Hi to all,
>>> I'm trying to run my DataSet job on Flink 1.11.0 and I'm connecting
>>> toward Mariadb in my code.
>>> I've put the mariadb-java-client-2.6.0.jar in the lib directory and in
>>> the pom.xml I set that dependency as provided. The code runs successfully
>>> from the Ide but when I try to run the code on the cluster I get the
>>> following error:
>>>
>>> Caused by: java.lang.ClassNotFoundException: com.mysql.cj.jdbc.Driver
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:471) ~[?:?]
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
>>> at
>>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:61)
>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>> at
>>> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>> at
>>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]
>>> at java.lang.Class.forName0(Native Method) ~[?:?]
>>> at java.lang.Class.forName(Class.java:315) ~[?:?]
>>> at
>>> org.apache.flink.connector.jdbc.internal.connection.SimpleJdbcConnectionProvider.getConnection(SimpleJdbcConnectionProvider.java:52)
>>> ~myApp.jar:?]
>>> at
>>> org.apache.flink.connector.jdbc.internal.AbstractJdbcOutputFormat.establishConnection(AbstractJdbcOutputFormat.java:66)
>>> ~myApp.jar:?]
>>> at
>>> org.apache.flink.connector.jdbc.internal.AbstractJdbcOutputFormat.open(AbstractJdbcOutputFormat.java:59)
>>> ~myApp.jar:?]
>>> at
>>> org.apache.flink.api.java.io.jdbc.JDBCOutputFormat.open(JDBCOutputFormat.java:82)
>>> ~myApp.jar:?]
>>> at
>>> org.apache.flink.runtime.operators.DataSinkTask.invoke(DataSinkTask.java:205)
>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
>>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>>> at java.lang.Thread.run(Thread.java:834) ~[?:?]
>>>
>>> What should I do?
>>>
>>> Thanks in advance,
>>> Flavio
>>>
>>
Re: JDBCOutputFormat dependency loading error
Posted by Flavio Pompermaier <po...@okkam.it>.
Yes Till, I was figuring out that I was using 2 different connectors and I
forgot the mysql jar..I was going to test and tell if the problem was
solved!
On Mon, Aug 3, 2020 at 10:50 AM Till Rohrmann <tr...@apache.org> wrote:
> Hi Flavio,
>
> I am not a JDBC expert but it looks as if you try to
> load com.mysql.cj.jdbc.Driver which is not contained
> in mariadb-java-client-2.6.0.jar. mariadb-java-client-2.6.0.jar only
> contains org/mariadb/jdbc/Driver.class. com.mysql.cj.jdbc.Driver can be
> found in mysql-connector-java.jar, though.
>
> Hence I believe that you are missing some dependencies in your user jar to
> make your job run. Please check from where com.mysql.cj.jdbc.Driver is
> being loaded when running the job from the IDE.
>
> Cheers,
> Till
>
> On Fri, Jul 31, 2020 at 4:55 PM Flavio Pompermaier <po...@okkam.it>
> wrote:
>
>> Hi to all,
>> I'm trying to run my DataSet job on Flink 1.11.0 and I'm connecting
>> toward Mariadb in my code.
>> I've put the mariadb-java-client-2.6.0.jar in the lib directory and in
>> the pom.xml I set that dependency as provided. The code runs successfully
>> from the Ide but when I try to run the code on the cluster I get the
>> following error:
>>
>> Caused by: java.lang.ClassNotFoundException: com.mysql.cj.jdbc.Driver
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:471) ~[?:?]
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
>> at
>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:61)
>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>> at
>> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>> at
>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]
>> at java.lang.Class.forName0(Native Method) ~[?:?]
>> at java.lang.Class.forName(Class.java:315) ~[?:?]
>> at
>> org.apache.flink.connector.jdbc.internal.connection.SimpleJdbcConnectionProvider.getConnection(SimpleJdbcConnectionProvider.java:52)
>> ~myApp.jar:?]
>> at
>> org.apache.flink.connector.jdbc.internal.AbstractJdbcOutputFormat.establishConnection(AbstractJdbcOutputFormat.java:66)
>> ~myApp.jar:?]
>> at
>> org.apache.flink.connector.jdbc.internal.AbstractJdbcOutputFormat.open(AbstractJdbcOutputFormat.java:59)
>> ~myApp.jar:?]
>> at
>> org.apache.flink.api.java.io.jdbc.JDBCOutputFormat.open(JDBCOutputFormat.java:82)
>> ~myApp.jar:?]
>> at
>> org.apache.flink.runtime.operators.DataSinkTask.invoke(DataSinkTask.java:205)
>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
>> ~[flink-dist_2.12-1.11.0.jar:1.11.0]
>> at java.lang.Thread.run(Thread.java:834) ~[?:?]
>>
>> What should I do?
>>
>> Thanks in advance,
>> Flavio
>>
>