You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Geren White <ge...@1stdibs.com> on 2016/08/04 20:44:18 UTC
PreparedStatement must be reprepared error
Hello all,
I'm having trouble importing data from a mysql database (version 5.26) with
sqoop (version 1.4.6).
I have a number of imports that are happening in AWS EMR and all of them
are working that go against tables. I have one import that goes against a
view that is not working. It works on our dev database but for some reason
not on our prod server.
Here's the command I'm running:
sqoop import --connect jdbc:mysql://$databaseHost/inventory
--connection-manager org.apache.sqoop.manager.MySQLManager --username
$user_name --password-file=s3://$s3Bucket/dibs/settings/sqoop_db_credentials.properties
--as-avrodatafile --target-dir /user/hadoop/inventory/category_view
--delete-target-dir --table category_view --split-by id --verbose
I've tried with and without specifying the jdbc driver and without
specifying the connection-manager. I've also tried overriding the default
SQL it is using but nothing seems to help. I've also seen a number of other
users with this issue that all point to the table_definition_cache but
playing with those values has not fixed the issue.
From the below stacktrace you can see that it is using a
MariaDbServerPreparedStatement. This seems weird to me and would expect it
to be using the Mysql specific PreparedStatement class. Is this the normal
behavior?
Here's the stacktrace:
2016-08-04 19:49:40,589 INFO org.apache.sqoop.manager.SqlManager
(main): Executing SQL statement: SELECT t.* FROM `category_view` AS t
LIMIT 1
2016-08-04 19:49:40,601 ERROR org.apache.sqoop.manager.SqlManager
(main): Error executing statement: java.sql.BatchUpdateException:
Prepared statement needs to be re-prepared
java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:374)
at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQuery(MariaDbServerPreparedStatement.java:341)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:758)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: java.sql.SQLException: Prepared statement needs to be re-prepared
at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:136)
at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69)
at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQueryEpilog(MariaDbServerPreparedStatement.java:336)
at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:291)
at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:369)
... 18 more
Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Prepared
statement needs to be re-prepared
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:592)
at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:279)
... 19 more
2016-08-04 19:49:40,606 ERROR org.apache.sqoop.tool.ImportTool (main):
Encountered IOException running import job: java.io.IOException: No
columns to generate for ClassWriter
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Any thoughts/recommendations would be greatly appreciated.
Thanks,
--
*Geren White | Senior Java Developer*
*(e)* geren@1stdibs.com
Re: PreparedStatement must be reprepared error
Posted by Geren White <ge...@1stdibs.com>.
Thanks so much for that excellent explanation. That all makes sense. I will
open up that feature request. Definitely could see that being helpful.
On Fri, Aug 5, 2016 at 8:23 PM, Attila Szabo <as...@cloudera.com> wrote:
> Hi Geren,
>
> To have all the details highlighted we need some Java "FUN" to understand:
> So MySQLManager.java states perfectly that it's driver is
> com.mysql.jdbc.Driver. (MySQL.java line 57 in the current trunk version)
> And in it's super class SqlManager it also validates if the driver class
> is there, by trying to load it. (SQLManager.java line 873 in the current
> trunk version)
> Although the static initializer of the MariaDB driver also loads the
> org.maraidb.jdbc.Driver class (Driver.java line 67 in the latest mariadb
> connector tgz file). It's done through the DriverManager.registerDriver(java.sql.Driver)
> method.
>
> com.mysql.jdbc.Driver states it's responsible for protocols starting with
> "jdbc:mysql", meanwhile org.mariadb.jdbc.Driver states the very same (as
> mariadb is a fork of MySQL). (this part can be found in both driver's
> overridden acceptsUrl(java.lang.String) method.
>
> Thus from this POV we're in a bit of concurrent race between mariadb
> driver, and mysql driver. And if by any reason the static intializer of
> mariadb driver is executed earlier than MySQLManager (or anyone else) would
> load the mysql jdbc driver, then mariadb would win, as it's states it can
> handle properly MySQL database.
>
> On the top:
> As I've read on the bug lists this bug is quite old in mariadb, so it is
> absolutely possible that in any newer version it is fixed (or maybe not!).
>
> There is a possibility (the community has to investigate that scenario of
> course), that Sqoop could attach a logWriter to DriverManager class, and
> thus those information when drivers are loaded (and in which order) would
> be visible on the std err/out or logfiles.
>
> If you think this feature request would help you in the future, I strongly
> encourage you to open an FR JIRA ticket to the Sqoop development community
> where we could discuss the possible solutions around this problem.
>
> Best regards,
> Attila
>
>
>
>
> On Fri, Aug 5, 2016 at 2:43 PM, Geren White <ge...@1stdibs.com> wrote:
>
>> Attila thank you so much for your responses. Ended up getting me in the
>> right direction and I fixed the issue. It turns out that EMR installs
>> mariadb connectors in the java installation of the server. After removing
>> those the import started working.
>>
>> Any ideas why the mariadb connector would take precedence over the mysql
>> one? It seems like this would be outside of the control of Sqoop. I was
>> taking a look at the sqoop code and when specifying --driver and no manager
>> it's going to use the GenericJdbcManager so would end up using whatever was
>> handling the jdbc:mysql connection strings. Since mariadb connector was in
>> the java installation I guess that takes precedence. If I specify both
>> --driver and --connection-manager it blows up because the MysqlManager
>> doesn't have the appropriate constructor. Specifying just the manager looks
>> like it would force it to use the MysqlManager which would end up doing a
>> Class.forName("com.mysql.jdbc.Driver"). Seems like this isn't enough to
>> make it use the Mysql connector?
>>
>> Thanks,
>> Geren
>>
>
>
>
> --
> Best regards,
>
> Attila Szabo
> Sotware Engineer
>
> <http://www.cloudera.com>
>
--
*Geren White | Senior Java Developer*
*(e)* geren@1stdibs.com
Re: PreparedStatement must be reprepared error
Posted by Attila Szabo <as...@cloudera.com>.
Hi Geren,
To have all the details highlighted we need some Java "FUN" to understand:
So MySQLManager.java states perfectly that it's driver is
com.mysql.jdbc.Driver. (MySQL.java line 57 in the current trunk version)
And in it's super class SqlManager it also validates if the driver class is
there, by trying to load it. (SQLManager.java line 873 in the current trunk
version)
Although the static initializer of the MariaDB driver also loads the
org.maraidb.jdbc.Driver class (Driver.java line 67 in the latest mariadb
connector tgz file). It's done through the
DriverManager.registerDriver(java.sql.Driver) method.
com.mysql.jdbc.Driver states it's responsible for protocols starting with
"jdbc:mysql", meanwhile org.mariadb.jdbc.Driver states the very same (as
mariadb is a fork of MySQL). (this part can be found in both driver's
overridden acceptsUrl(java.lang.String) method.
Thus from this POV we're in a bit of concurrent race between mariadb
driver, and mysql driver. And if by any reason the static intializer of
mariadb driver is executed earlier than MySQLManager (or anyone else) would
load the mysql jdbc driver, then mariadb would win, as it's states it can
handle properly MySQL database.
On the top:
As I've read on the bug lists this bug is quite old in mariadb, so it is
absolutely possible that in any newer version it is fixed (or maybe not!).
There is a possibility (the community has to investigate that scenario of
course), that Sqoop could attach a logWriter to DriverManager class, and
thus those information when drivers are loaded (and in which order) would
be visible on the std err/out or logfiles.
If you think this feature request would help you in the future, I strongly
encourage you to open an FR JIRA ticket to the Sqoop development community
where we could discuss the possible solutions around this problem.
Best regards,
Attila
On Fri, Aug 5, 2016 at 2:43 PM, Geren White <ge...@1stdibs.com> wrote:
> Attila thank you so much for your responses. Ended up getting me in the
> right direction and I fixed the issue. It turns out that EMR installs
> mariadb connectors in the java installation of the server. After removing
> those the import started working.
>
> Any ideas why the mariadb connector would take precedence over the mysql
> one? It seems like this would be outside of the control of Sqoop. I was
> taking a look at the sqoop code and when specifying --driver and no manager
> it's going to use the GenericJdbcManager so would end up using whatever was
> handling the jdbc:mysql connection strings. Since mariadb connector was in
> the java installation I guess that takes precedence. If I specify both
> --driver and --connection-manager it blows up because the MysqlManager
> doesn't have the appropriate constructor. Specifying just the manager looks
> like it would force it to use the MysqlManager which would end up doing a
> Class.forName("com.mysql.jdbc.Driver"). Seems like this isn't enough to
> make it use the Mysql connector?
>
> Thanks,
> Geren
>
--
Best regards,
Attila Szabo
Sotware Engineer
<http://www.cloudera.com>
Re: PreparedStatement must be reprepared error
Posted by Geren White <ge...@1stdibs.com>.
Attila thank you so much for your responses. Ended up getting me in the
right direction and I fixed the issue. It turns out that EMR installs
mariadb connectors in the java installation of the server. After removing
those the import started working.
Any ideas why the mariadb connector would take precedence over the mysql
one? It seems like this would be outside of the control of Sqoop. I was
taking a look at the sqoop code and when specifying --driver and no manager
it's going to use the GenericJdbcManager so would end up using whatever was
handling the jdbc:mysql connection strings. Since mariadb connector was in
the java installation I guess that takes precedence. If I specify both
--driver and --connection-manager it blows up because the MysqlManager
doesn't have the appropriate constructor. Specifying just the manager looks
like it would force it to use the MysqlManager which would end up doing a
Class.forName("com.mysql.jdbc.Driver"). Seems like this isn't enough to
make it use the Mysql connector?
Thanks,
Geren
Re: PreparedStatement must be reprepared error
Posted by Attila Szabo <as...@cloudera.com>.
Hi Geren,
In that case at the moment I won't be able to help you, because I'm not
aware about how EMR machines are setup, but it would be awesome from you if
you would share us your findings at the end!
Many thanks,
Maugli (Attila)
On Fri, Aug 5, 2016 at 9:19 AM, Geren White <ge...@1stdibs.com> wrote:
> Thanks for the responses Attila.
>
> I'll do some more digging for that class because agreed it has to be
> hiding somewhere. I'm using Amazon's EMR so it's their installation of
> Sqoop so a clean install isn't really an option.
>
> On Fri, Aug 5, 2016 at 12:13 PM, Attila Szabo <as...@cloudera.com> wrote:
>
>> Hey Geren,
>>
>> My advise would be for you trying to create a clean environment. If you
>> can do a full clean install that would be the best. On the other hand (if
>> it's not an option) please try to find all of the occurences of the mariadb
>> connector and delete them.
>>
>> Meanwhile I've checked the content of the mysql connector jar (latest
>> one) and there must be 0 org.mariadb.* related class. So most probably some
>> of the mariadb connector jars are hiding on your cluster.
>>
>> Cheers
>> M.
>>
>> On Fri, Aug 5, 2016 at 9:03 AM, Geren White <ge...@1stdibs.com> wrote:
>>
>>> Hey Attila,
>>> Sorry for the confusion, I left out the mysql-connector from that list.
>>> That's just what's in there by default and then I install the mysql
>>> connector before running anything. I just ran a test without the mariadb
>>> connector in there and somehow it's still picking up that
>>> MariaDbServerPreparedStatement which makes no sense to me.
>>>
>>>
>>> On Fri, Aug 5, 2016 at 11:58 AM, Attila Szabo <as...@cloudera.com>
>>> wrote:
>>>
>>>> Hi Geren,
>>>>
>>>> My gut feeling is the presence of the mariadb-connector-java.jar in
>>>> your lib directory and the absence of the mysql-connector-java-5.1.21.ja
>>>> r.
>>>> AFAIK as mariadb is a fork of MySQL it's also listening to the
>>>> jdbc:mysql: protocol, and most probably that's the reason why it tries to
>>>> use mariadb related preparedstatement instead of the mysql one.
>>>>
>>>> Could you please try it by removing the mariadb connector and copy
>>>> there the mysql connector with your exact version?
>>>>
>>>> Thanks
>>>> M.
>>>>
>>>> On Fri, Aug 5, 2016 at 8:30 AM, Geren White <ge...@1stdibs.com> wrote:
>>>>
>>>>> Attached is the verbose log.
>>>>>
>>>>> I'm using 5.1.21 of the mysql connector. I've also tried with the
>>>>> latest (5.1.39) and that didn't work.
>>>>>
>>>>> For jdbc drivers in the lib directory there are:
>>>>>
>>>>> mariadb-connector-java.jar
>>>>>
>>>>> postgresql-jdbc.jar
>>>>>
>>>>> hsqldb-1.8.0.10.jar
>>>>>
>>>>> On Thu, Aug 4, 2016 at 8:36 PM, Attila Szabo <as...@cloudera.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Geren,
>>>>>>
>>>>>> For further investigation:
>>>>>> Could you please send us the whole verbose logfile for further
>>>>>> investigation?
>>>>>> On the top could you please check which version of the MySQL JDBC
>>>>>> driver is present in the lib directory of Sqoop?
>>>>>> We would also appreciate if you could tell which other JDBC drivers
>>>>>> located there!
>>>>>>
>>>>>> Many thanks,
>>>>>> M.
>>>>>>
>>>>>> On Thu, Aug 4, 2016 at 1:44 PM, Geren White <ge...@1stdibs.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello all,
>>>>>>>
>>>>>>> I'm having trouble importing data from a mysql database (version 5.26)
>>>>>>> with sqoop (version 1.4.6).
>>>>>>>
>>>>>>> I have a number of imports that are happening in AWS EMR and all of
>>>>>>> them are working that go against tables. I have one import that goes
>>>>>>> against a view that is not working. It works on our dev database but for
>>>>>>> some reason not on our prod server.
>>>>>>>
>>>>>>> Here's the command I'm running:
>>>>>>> sqoop import --connect jdbc:mysql://$databaseHost/inventory
>>>>>>> --connection-manager org.apache.sqoop.manager.MySQLManager
>>>>>>> --username $user_name --password-file=s3://$s3Bucket
>>>>>>> /dibs/settings/sqoop_db_credentials.properties --as-avrodatafile
>>>>>>> --target-dir /user/hadoop/inventory/category_view
>>>>>>> --delete-target-dir --table category_view --split-by id --verbose
>>>>>>>
>>>>>>> I've tried with and without specifying the jdbc driver and without
>>>>>>> specifying the connection-manager. I've also tried overriding the default
>>>>>>> SQL it is using but nothing seems to help. I've also seen a number of other
>>>>>>> users with this issue that all point to the table_definition_cache but
>>>>>>> playing with those values has not fixed the issue.
>>>>>>>
>>>>>>> From the below stacktrace you can see that it is using a
>>>>>>> MariaDbServerPreparedStatement. This seems weird to me and would
>>>>>>> expect it to be using the Mysql specific PreparedStatement class. Is this
>>>>>>> the normal behavior?
>>>>>>>
>>>>>>> Here's the stacktrace:
>>>>>>>
>>>>>>> 2016-08-04 19:49:40,589 INFO org.apache.sqoop.manager.SqlManager (main): Executing SQL statement: SELECT t.* FROM `category_view` AS t LIMIT 1
>>>>>>> 2016-08-04 19:49:40,601 ERROR org.apache.sqoop.manager.SqlManager (main): Error executing statement: java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>>>>>>> java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:374)
>>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQuery(MariaDbServerPreparedStatement.java:341)
>>>>>>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:758)
>>>>>>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)
>>>>>>> at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
>>>>>>> at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
>>>>>>> at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
>>>>>>> at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
>>>>>>> at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
>>>>>>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
>>>>>>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>>>>>>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>>>>>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>>>>>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>>>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>>>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>>>>>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>>>>>> Caused by: java.sql.SQLException: Prepared statement needs to be re-prepared
>>>>>>> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:136)
>>>>>>> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69)
>>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQueryEpilog(MariaDbServerPreparedStatement.java:336)
>>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:291)
>>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:369)
>>>>>>> ... 18 more
>>>>>>> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Prepared statement needs to be re-prepared
>>>>>>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479)
>>>>>>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:592)
>>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:279)
>>>>>>> ... 19 more
>>>>>>> 2016-08-04 19:49:40,606 ERROR org.apache.sqoop.tool.ImportTool (main): Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
>>>>>>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)
>>>>>>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>>>>>>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>>>>>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>>>>>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>>>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>>>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>>>>>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>>>>>>
>>>>>>>
>>>>>>> Any thoughts/recommendations would be greatly appreciated.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> --
>>>>>>> *Geren White | Senior Java Developer*
>>>>>>> *(e)* geren@1stdibs.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best regards,
>>>>>>
>>>>>> Attila Szabo
>>>>>> Sotware Engineer
>>>>>>
>>>>>> <http://www.cloudera.com>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Geren White | Senior Java Developer*
>>>>> *(e)* geren@1stdibs.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>>
>>>> Attila Szabo
>>>> Sotware Engineer
>>>>
>>>> <http://www.cloudera.com>
>>>>
>>>
>>>
>>>
>>> --
>>> *Geren White | Senior Java Developer*
>>> *(e)* geren@1stdibs.com
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>> Attila Szabo
>> Sotware Engineer
>>
>> <http://www.cloudera.com>
>>
>
>
>
> --
> *Geren White | Senior Java Developer*
> *(e)* geren@1stdibs.com
>
--
Best regards,
Attila Szabo
Sotware Engineer
<http://www.cloudera.com>
Re: PreparedStatement must be reprepared error
Posted by Geren White <ge...@1stdibs.com>.
Thanks for the responses Attila.
I'll do some more digging for that class because agreed it has to be hiding
somewhere. I'm using Amazon's EMR so it's their installation of Sqoop so a
clean install isn't really an option.
On Fri, Aug 5, 2016 at 12:13 PM, Attila Szabo <as...@cloudera.com> wrote:
> Hey Geren,
>
> My advise would be for you trying to create a clean environment. If you
> can do a full clean install that would be the best. On the other hand (if
> it's not an option) please try to find all of the occurences of the mariadb
> connector and delete them.
>
> Meanwhile I've checked the content of the mysql connector jar (latest one)
> and there must be 0 org.mariadb.* related class. So most probably some of
> the mariadb connector jars are hiding on your cluster.
>
> Cheers
> M.
>
> On Fri, Aug 5, 2016 at 9:03 AM, Geren White <ge...@1stdibs.com> wrote:
>
>> Hey Attila,
>> Sorry for the confusion, I left out the mysql-connector from that list.
>> That's just what's in there by default and then I install the mysql
>> connector before running anything. I just ran a test without the mariadb
>> connector in there and somehow it's still picking up that
>> MariaDbServerPreparedStatement which makes no sense to me.
>>
>>
>> On Fri, Aug 5, 2016 at 11:58 AM, Attila Szabo <as...@cloudera.com>
>> wrote:
>>
>>> Hi Geren,
>>>
>>> My gut feeling is the presence of the mariadb-connector-java.jar in your
>>> lib directory and the absence of the mysql-connector-java-5.1.21.jar.
>>> AFAIK as mariadb is a fork of MySQL it's also listening to the
>>> jdbc:mysql: protocol, and most probably that's the reason why it tries to
>>> use mariadb related preparedstatement instead of the mysql one.
>>>
>>> Could you please try it by removing the mariadb connector and copy there
>>> the mysql connector with your exact version?
>>>
>>> Thanks
>>> M.
>>>
>>> On Fri, Aug 5, 2016 at 8:30 AM, Geren White <ge...@1stdibs.com> wrote:
>>>
>>>> Attached is the verbose log.
>>>>
>>>> I'm using 5.1.21 of the mysql connector. I've also tried with the
>>>> latest (5.1.39) and that didn't work.
>>>>
>>>> For jdbc drivers in the lib directory there are:
>>>>
>>>> mariadb-connector-java.jar
>>>>
>>>> postgresql-jdbc.jar
>>>>
>>>> hsqldb-1.8.0.10.jar
>>>>
>>>> On Thu, Aug 4, 2016 at 8:36 PM, Attila Szabo <as...@cloudera.com>
>>>> wrote:
>>>>
>>>>> Hi Geren,
>>>>>
>>>>> For further investigation:
>>>>> Could you please send us the whole verbose logfile for further
>>>>> investigation?
>>>>> On the top could you please check which version of the MySQL JDBC
>>>>> driver is present in the lib directory of Sqoop?
>>>>> We would also appreciate if you could tell which other JDBC drivers
>>>>> located there!
>>>>>
>>>>> Many thanks,
>>>>> M.
>>>>>
>>>>> On Thu, Aug 4, 2016 at 1:44 PM, Geren White <ge...@1stdibs.com> wrote:
>>>>>
>>>>>> Hello all,
>>>>>>
>>>>>> I'm having trouble importing data from a mysql database (version 5.26)
>>>>>> with sqoop (version 1.4.6).
>>>>>>
>>>>>> I have a number of imports that are happening in AWS EMR and all of
>>>>>> them are working that go against tables. I have one import that goes
>>>>>> against a view that is not working. It works on our dev database but for
>>>>>> some reason not on our prod server.
>>>>>>
>>>>>> Here's the command I'm running:
>>>>>> sqoop import --connect jdbc:mysql://$databaseHost/inventory
>>>>>> --connection-manager org.apache.sqoop.manager.MySQLManager
>>>>>> --username $user_name --password-file=s3://$s3Bucket
>>>>>> /dibs/settings/sqoop_db_credentials.properties --as-avrodatafile
>>>>>> --target-dir /user/hadoop/inventory/category_view
>>>>>> --delete-target-dir --table category_view --split-by id --verbose
>>>>>>
>>>>>> I've tried with and without specifying the jdbc driver and without
>>>>>> specifying the connection-manager. I've also tried overriding the default
>>>>>> SQL it is using but nothing seems to help. I've also seen a number of other
>>>>>> users with this issue that all point to the table_definition_cache but
>>>>>> playing with those values has not fixed the issue.
>>>>>>
>>>>>> From the below stacktrace you can see that it is using a
>>>>>> MariaDbServerPreparedStatement. This seems weird to me and would
>>>>>> expect it to be using the Mysql specific PreparedStatement class. Is this
>>>>>> the normal behavior?
>>>>>>
>>>>>> Here's the stacktrace:
>>>>>>
>>>>>> 2016-08-04 19:49:40,589 INFO org.apache.sqoop.manager.SqlManager (main): Executing SQL statement: SELECT t.* FROM `category_view` AS t LIMIT 1
>>>>>> 2016-08-04 19:49:40,601 ERROR org.apache.sqoop.manager.SqlManager (main): Error executing statement: java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>>>>>> java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:374)
>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQuery(MariaDbServerPreparedStatement.java:341)
>>>>>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:758)
>>>>>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)
>>>>>> at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
>>>>>> at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
>>>>>> at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
>>>>>> at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
>>>>>> at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
>>>>>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
>>>>>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>>>>>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>>>>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>>>>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>>>>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>>>>> Caused by: java.sql.SQLException: Prepared statement needs to be re-prepared
>>>>>> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:136)
>>>>>> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69)
>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQueryEpilog(MariaDbServerPreparedStatement.java:336)
>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:291)
>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:369)
>>>>>> ... 18 more
>>>>>> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Prepared statement needs to be re-prepared
>>>>>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479)
>>>>>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:592)
>>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:279)
>>>>>> ... 19 more
>>>>>> 2016-08-04 19:49:40,606 ERROR org.apache.sqoop.tool.ImportTool (main): Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
>>>>>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)
>>>>>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>>>>>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>>>>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>>>>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>>>>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>>>>>
>>>>>>
>>>>>> Any thoughts/recommendations would be greatly appreciated.
>>>>>>
>>>>>> Thanks,
>>>>>> --
>>>>>> *Geren White | Senior Java Developer*
>>>>>> *(e)* geren@1stdibs.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>>
>>>>> Attila Szabo
>>>>> Sotware Engineer
>>>>>
>>>>> <http://www.cloudera.com>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Geren White | Senior Java Developer*
>>>> *(e)* geren@1stdibs.com
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Attila Szabo
>>> Sotware Engineer
>>>
>>> <http://www.cloudera.com>
>>>
>>
>>
>>
>> --
>> *Geren White | Senior Java Developer*
>> *(e)* geren@1stdibs.com
>>
>
>
>
> --
> Best regards,
>
> Attila Szabo
> Sotware Engineer
>
> <http://www.cloudera.com>
>
--
*Geren White | Senior Java Developer*
*(e)* geren@1stdibs.com
Re: PreparedStatement must be reprepared error
Posted by Attila Szabo <as...@cloudera.com>.
Hey Geren,
My advise would be for you trying to create a clean environment. If you can
do a full clean install that would be the best. On the other hand (if it's
not an option) please try to find all of the occurences of the mariadb
connector and delete them.
Meanwhile I've checked the content of the mysql connector jar (latest one)
and there must be 0 org.mariadb.* related class. So most probably some of
the mariadb connector jars are hiding on your cluster.
Cheers
M.
On Fri, Aug 5, 2016 at 9:03 AM, Geren White <ge...@1stdibs.com> wrote:
> Hey Attila,
> Sorry for the confusion, I left out the mysql-connector from that list.
> That's just what's in there by default and then I install the mysql
> connector before running anything. I just ran a test without the mariadb
> connector in there and somehow it's still picking up that
> MariaDbServerPreparedStatement which makes no sense to me.
>
>
> On Fri, Aug 5, 2016 at 11:58 AM, Attila Szabo <as...@cloudera.com> wrote:
>
>> Hi Geren,
>>
>> My gut feeling is the presence of the mariadb-connector-java.jar in your
>> lib directory and the absence of the mysql-connector-java-5.1.21.jar.
>> AFAIK as mariadb is a fork of MySQL it's also listening to the
>> jdbc:mysql: protocol, and most probably that's the reason why it tries to
>> use mariadb related preparedstatement instead of the mysql one.
>>
>> Could you please try it by removing the mariadb connector and copy there
>> the mysql connector with your exact version?
>>
>> Thanks
>> M.
>>
>> On Fri, Aug 5, 2016 at 8:30 AM, Geren White <ge...@1stdibs.com> wrote:
>>
>>> Attached is the verbose log.
>>>
>>> I'm using 5.1.21 of the mysql connector. I've also tried with the latest
>>> (5.1.39) and that didn't work.
>>>
>>> For jdbc drivers in the lib directory there are:
>>>
>>> mariadb-connector-java.jar
>>>
>>> postgresql-jdbc.jar
>>>
>>> hsqldb-1.8.0.10.jar
>>>
>>> On Thu, Aug 4, 2016 at 8:36 PM, Attila Szabo <as...@cloudera.com>
>>> wrote:
>>>
>>>> Hi Geren,
>>>>
>>>> For further investigation:
>>>> Could you please send us the whole verbose logfile for further
>>>> investigation?
>>>> On the top could you please check which version of the MySQL JDBC
>>>> driver is present in the lib directory of Sqoop?
>>>> We would also appreciate if you could tell which other JDBC drivers
>>>> located there!
>>>>
>>>> Many thanks,
>>>> M.
>>>>
>>>> On Thu, Aug 4, 2016 at 1:44 PM, Geren White <ge...@1stdibs.com> wrote:
>>>>
>>>>> Hello all,
>>>>>
>>>>> I'm having trouble importing data from a mysql database (version 5.26)
>>>>> with sqoop (version 1.4.6).
>>>>>
>>>>> I have a number of imports that are happening in AWS EMR and all of
>>>>> them are working that go against tables. I have one import that goes
>>>>> against a view that is not working. It works on our dev database but for
>>>>> some reason not on our prod server.
>>>>>
>>>>> Here's the command I'm running:
>>>>> sqoop import --connect jdbc:mysql://$databaseHost/inventory
>>>>> --connection-manager org.apache.sqoop.manager.MySQLManager --username
>>>>> $user_name --password-file=s3://$s3Bucket
>>>>> /dibs/settings/sqoop_db_credentials.properties --as-avrodatafile
>>>>> --target-dir /user/hadoop/inventory/category_view --delete-target-dir
>>>>> --table category_view --split-by id --verbose
>>>>>
>>>>> I've tried with and without specifying the jdbc driver and without
>>>>> specifying the connection-manager. I've also tried overriding the default
>>>>> SQL it is using but nothing seems to help. I've also seen a number of other
>>>>> users with this issue that all point to the table_definition_cache but
>>>>> playing with those values has not fixed the issue.
>>>>>
>>>>> From the below stacktrace you can see that it is using a
>>>>> MariaDbServerPreparedStatement. This seems weird to me and would
>>>>> expect it to be using the Mysql specific PreparedStatement class. Is this
>>>>> the normal behavior?
>>>>>
>>>>> Here's the stacktrace:
>>>>>
>>>>> 2016-08-04 19:49:40,589 INFO org.apache.sqoop.manager.SqlManager (main): Executing SQL statement: SELECT t.* FROM `category_view` AS t LIMIT 1
>>>>> 2016-08-04 19:49:40,601 ERROR org.apache.sqoop.manager.SqlManager (main): Error executing statement: java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>>>>> java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:374)
>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQuery(MariaDbServerPreparedStatement.java:341)
>>>>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:758)
>>>>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)
>>>>> at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
>>>>> at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
>>>>> at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
>>>>> at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
>>>>> at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
>>>>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
>>>>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>>>>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>>>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>>>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>>>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>>>> Caused by: java.sql.SQLException: Prepared statement needs to be re-prepared
>>>>> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:136)
>>>>> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69)
>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQueryEpilog(MariaDbServerPreparedStatement.java:336)
>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:291)
>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:369)
>>>>> ... 18 more
>>>>> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Prepared statement needs to be re-prepared
>>>>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479)
>>>>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:592)
>>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:279)
>>>>> ... 19 more
>>>>> 2016-08-04 19:49:40,606 ERROR org.apache.sqoop.tool.ImportTool (main): Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
>>>>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)
>>>>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>>>>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>>>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>>>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>>>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>>>>
>>>>>
>>>>> Any thoughts/recommendations would be greatly appreciated.
>>>>>
>>>>> Thanks,
>>>>> --
>>>>> *Geren White | Senior Java Developer*
>>>>> *(e)* geren@1stdibs.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>>
>>>> Attila Szabo
>>>> Sotware Engineer
>>>>
>>>> <http://www.cloudera.com>
>>>>
>>>
>>>
>>>
>>> --
>>> *Geren White | Senior Java Developer*
>>> *(e)* geren@1stdibs.com
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>> Attila Szabo
>> Sotware Engineer
>>
>> <http://www.cloudera.com>
>>
>
>
>
> --
> *Geren White | Senior Java Developer*
> *(e)* geren@1stdibs.com
>
--
Best regards,
Attila Szabo
Sotware Engineer
<http://www.cloudera.com>
Re: PreparedStatement must be reprepared error
Posted by Geren White <ge...@1stdibs.com>.
Hey Attila,
Sorry for the confusion, I left out the mysql-connector from that list.
That's just what's in there by default and then I install the mysql
connector before running anything. I just ran a test without the mariadb
connector in there and somehow it's still picking up that
MariaDbServerPreparedStatement which makes no sense to me.
On Fri, Aug 5, 2016 at 11:58 AM, Attila Szabo <as...@cloudera.com> wrote:
> Hi Geren,
>
> My gut feeling is the presence of the mariadb-connector-java.jar in your
> lib directory and the absence of the mysql-connector-java-5.1.21.jar.
> AFAIK as mariadb is a fork of MySQL it's also listening to the jdbc:mysql:
> protocol, and most probably that's the reason why it tries to use mariadb
> related preparedstatement instead of the mysql one.
>
> Could you please try it by removing the mariadb connector and copy there
> the mysql connector with your exact version?
>
> Thanks
> M.
>
> On Fri, Aug 5, 2016 at 8:30 AM, Geren White <ge...@1stdibs.com> wrote:
>
>> Attached is the verbose log.
>>
>> I'm using 5.1.21 of the mysql connector. I've also tried with the latest
>> (5.1.39) and that didn't work.
>>
>> For jdbc drivers in the lib directory there are:
>>
>> mariadb-connector-java.jar
>>
>> postgresql-jdbc.jar
>>
>> hsqldb-1.8.0.10.jar
>>
>> On Thu, Aug 4, 2016 at 8:36 PM, Attila Szabo <as...@cloudera.com> wrote:
>>
>>> Hi Geren,
>>>
>>> For further investigation:
>>> Could you please send us the whole verbose logfile for further
>>> investigation?
>>> On the top could you please check which version of the MySQL JDBC driver
>>> is present in the lib directory of Sqoop?
>>> We would also appreciate if you could tell which other JDBC drivers
>>> located there!
>>>
>>> Many thanks,
>>> M.
>>>
>>> On Thu, Aug 4, 2016 at 1:44 PM, Geren White <ge...@1stdibs.com> wrote:
>>>
>>>> Hello all,
>>>>
>>>> I'm having trouble importing data from a mysql database (version 5.26)
>>>> with sqoop (version 1.4.6).
>>>>
>>>> I have a number of imports that are happening in AWS EMR and all of
>>>> them are working that go against tables. I have one import that goes
>>>> against a view that is not working. It works on our dev database but for
>>>> some reason not on our prod server.
>>>>
>>>> Here's the command I'm running:
>>>> sqoop import --connect jdbc:mysql://$databaseHost/inventory
>>>> --connection-manager org.apache.sqoop.manager.MySQLManager --username
>>>> $user_name --password-file=s3://$s3Bucket/dibs/settings/sqoop_db_credentials.properties
>>>> --as-avrodatafile --target-dir /user/hadoop/inventory/category_view
>>>> --delete-target-dir --table category_view --split-by id --verbose
>>>>
>>>> I've tried with and without specifying the jdbc driver and without
>>>> specifying the connection-manager. I've also tried overriding the default
>>>> SQL it is using but nothing seems to help. I've also seen a number of other
>>>> users with this issue that all point to the table_definition_cache but
>>>> playing with those values has not fixed the issue.
>>>>
>>>> From the below stacktrace you can see that it is using a
>>>> MariaDbServerPreparedStatement. This seems weird to me and would
>>>> expect it to be using the Mysql specific PreparedStatement class. Is this
>>>> the normal behavior?
>>>>
>>>> Here's the stacktrace:
>>>>
>>>> 2016-08-04 19:49:40,589 INFO org.apache.sqoop.manager.SqlManager (main): Executing SQL statement: SELECT t.* FROM `category_view` AS t LIMIT 1
>>>> 2016-08-04 19:49:40,601 ERROR org.apache.sqoop.manager.SqlManager (main): Error executing statement: java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>>>> java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:374)
>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQuery(MariaDbServerPreparedStatement.java:341)
>>>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:758)
>>>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)
>>>> at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
>>>> at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
>>>> at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
>>>> at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
>>>> at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
>>>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
>>>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>>>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>>> Caused by: java.sql.SQLException: Prepared statement needs to be re-prepared
>>>> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:136)
>>>> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69)
>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQueryEpilog(MariaDbServerPreparedStatement.java:336)
>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:291)
>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:369)
>>>> ... 18 more
>>>> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Prepared statement needs to be re-prepared
>>>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479)
>>>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:592)
>>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:279)
>>>> ... 19 more
>>>> 2016-08-04 19:49:40,606 ERROR org.apache.sqoop.tool.ImportTool (main): Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
>>>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)
>>>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>>>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>>>
>>>>
>>>> Any thoughts/recommendations would be greatly appreciated.
>>>>
>>>> Thanks,
>>>> --
>>>> *Geren White | Senior Java Developer*
>>>> *(e)* geren@1stdibs.com
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Attila Szabo
>>> Sotware Engineer
>>>
>>> <http://www.cloudera.com>
>>>
>>
>>
>>
>> --
>> *Geren White | Senior Java Developer*
>> *(e)* geren@1stdibs.com
>>
>
>
>
> --
> Best regards,
>
> Attila Szabo
> Sotware Engineer
>
> <http://www.cloudera.com>
>
--
*Geren White | Senior Java Developer*
*(e)* geren@1stdibs.com
Re: PreparedStatement must be reprepared error
Posted by Attila Szabo <as...@cloudera.com>.
Hi Geren,
My gut feeling is the presence of the mariadb-connector-java.jar in your
lib directory and the absence of the mysql-connector-java-5.1.21.jar.
AFAIK as mariadb is a fork of MySQL it's also listening to the jdbc:mysql:
protocol, and most probably that's the reason why it tries to use mariadb
related preparedstatement instead of the mysql one.
Could you please try it by removing the mariadb connector and copy there
the mysql connector with your exact version?
Thanks
M.
On Fri, Aug 5, 2016 at 8:30 AM, Geren White <ge...@1stdibs.com> wrote:
> Attached is the verbose log.
>
> I'm using 5.1.21 of the mysql connector. I've also tried with the latest
> (5.1.39) and that didn't work.
>
> For jdbc drivers in the lib directory there are:
>
> mariadb-connector-java.jar
>
> postgresql-jdbc.jar
>
> hsqldb-1.8.0.10.jar
>
> On Thu, Aug 4, 2016 at 8:36 PM, Attila Szabo <as...@cloudera.com> wrote:
>
>> Hi Geren,
>>
>> For further investigation:
>> Could you please send us the whole verbose logfile for further
>> investigation?
>> On the top could you please check which version of the MySQL JDBC driver
>> is present in the lib directory of Sqoop?
>> We would also appreciate if you could tell which other JDBC drivers
>> located there!
>>
>> Many thanks,
>> M.
>>
>> On Thu, Aug 4, 2016 at 1:44 PM, Geren White <ge...@1stdibs.com> wrote:
>>
>>> Hello all,
>>>
>>> I'm having trouble importing data from a mysql database (version 5.26)
>>> with sqoop (version 1.4.6).
>>>
>>> I have a number of imports that are happening in AWS EMR and all of them
>>> are working that go against tables. I have one import that goes against a
>>> view that is not working. It works on our dev database but for some reason
>>> not on our prod server.
>>>
>>> Here's the command I'm running:
>>> sqoop import --connect jdbc:mysql://$databaseHost/inventory
>>> --connection-manager org.apache.sqoop.manager.MySQLManager --username
>>> $user_name --password-file=s3://$s3Bucket/dibs/settings/sqoop_db_credentials.properties
>>> --as-avrodatafile --target-dir /user/hadoop/inventory/category_view
>>> --delete-target-dir --table category_view --split-by id --verbose
>>>
>>> I've tried with and without specifying the jdbc driver and without
>>> specifying the connection-manager. I've also tried overriding the default
>>> SQL it is using but nothing seems to help. I've also seen a number of other
>>> users with this issue that all point to the table_definition_cache but
>>> playing with those values has not fixed the issue.
>>>
>>> From the below stacktrace you can see that it is using a
>>> MariaDbServerPreparedStatement. This seems weird to me and would expect
>>> it to be using the Mysql specific PreparedStatement class. Is this the
>>> normal behavior?
>>>
>>> Here's the stacktrace:
>>>
>>> 2016-08-04 19:49:40,589 INFO org.apache.sqoop.manager.SqlManager (main): Executing SQL statement: SELECT t.* FROM `category_view` AS t LIMIT 1
>>> 2016-08-04 19:49:40,601 ERROR org.apache.sqoop.manager.SqlManager (main): Error executing statement: java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>>> java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:374)
>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQuery(MariaDbServerPreparedStatement.java:341)
>>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:758)
>>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)
>>> at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
>>> at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
>>> at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
>>> at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
>>> at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
>>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
>>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>> Caused by: java.sql.SQLException: Prepared statement needs to be re-prepared
>>> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:136)
>>> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69)
>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQueryEpilog(MariaDbServerPreparedStatement.java:336)
>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:291)
>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:369)
>>> ... 18 more
>>> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Prepared statement needs to be re-prepared
>>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479)
>>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:592)
>>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:279)
>>> ... 19 more
>>> 2016-08-04 19:49:40,606 ERROR org.apache.sqoop.tool.ImportTool (main): Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
>>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)
>>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>>
>>>
>>> Any thoughts/recommendations would be greatly appreciated.
>>>
>>> Thanks,
>>> --
>>> *Geren White | Senior Java Developer*
>>> *(e)* geren@1stdibs.com
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>> Attila Szabo
>> Sotware Engineer
>>
>> <http://www.cloudera.com>
>>
>
>
>
> --
> *Geren White | Senior Java Developer*
> *(e)* geren@1stdibs.com
>
--
Best regards,
Attila Szabo
Sotware Engineer
<http://www.cloudera.com>
Re: PreparedStatement must be reprepared error
Posted by Geren White <ge...@1stdibs.com>.
Attached is the verbose log.
I'm using 5.1.21 of the mysql connector. I've also tried with the latest
(5.1.39) and that didn't work.
For jdbc drivers in the lib directory there are:
mariadb-connector-java.jar
postgresql-jdbc.jar
hsqldb-1.8.0.10.jar
On Thu, Aug 4, 2016 at 8:36 PM, Attila Szabo <as...@cloudera.com> wrote:
> Hi Geren,
>
> For further investigation:
> Could you please send us the whole verbose logfile for further
> investigation?
> On the top could you please check which version of the MySQL JDBC driver
> is present in the lib directory of Sqoop?
> We would also appreciate if you could tell which other JDBC drivers
> located there!
>
> Many thanks,
> M.
>
> On Thu, Aug 4, 2016 at 1:44 PM, Geren White <ge...@1stdibs.com> wrote:
>
>> Hello all,
>>
>> I'm having trouble importing data from a mysql database (version 5.26)
>> with sqoop (version 1.4.6).
>>
>> I have a number of imports that are happening in AWS EMR and all of them
>> are working that go against tables. I have one import that goes against a
>> view that is not working. It works on our dev database but for some reason
>> not on our prod server.
>>
>> Here's the command I'm running:
>> sqoop import --connect jdbc:mysql://$databaseHost/inventory
>> --connection-manager org.apache.sqoop.manager.MySQLManager --username
>> $user_name --password-file=s3://$s3Bucket/dibs/settings/sqoop_db_credentials.properties
>> --as-avrodatafile --target-dir /user/hadoop/inventory/category_view
>> --delete-target-dir --table category_view --split-by id --verbose
>>
>> I've tried with and without specifying the jdbc driver and without
>> specifying the connection-manager. I've also tried overriding the default
>> SQL it is using but nothing seems to help. I've also seen a number of other
>> users with this issue that all point to the table_definition_cache but
>> playing with those values has not fixed the issue.
>>
>> From the below stacktrace you can see that it is using a
>> MariaDbServerPreparedStatement. This seems weird to me and would expect
>> it to be using the Mysql specific PreparedStatement class. Is this the
>> normal behavior?
>>
>> Here's the stacktrace:
>>
>> 2016-08-04 19:49:40,589 INFO org.apache.sqoop.manager.SqlManager (main): Executing SQL statement: SELECT t.* FROM `category_view` AS t LIMIT 1
>> 2016-08-04 19:49:40,601 ERROR org.apache.sqoop.manager.SqlManager (main): Error executing statement: java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>> java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:374)
>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQuery(MariaDbServerPreparedStatement.java:341)
>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:758)
>> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)
>> at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
>> at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
>> at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
>> at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
>> at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>> Caused by: java.sql.SQLException: Prepared statement needs to be re-prepared
>> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:136)
>> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69)
>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQueryEpilog(MariaDbServerPreparedStatement.java:336)
>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:291)
>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:369)
>> ... 18 more
>> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Prepared statement needs to be re-prepared
>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479)
>> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:592)
>> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:279)
>> ... 19 more
>> 2016-08-04 19:49:40,606 ERROR org.apache.sqoop.tool.ImportTool (main): Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
>> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)
>> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
>> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
>> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
>> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
>> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
>> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>>
>>
>> Any thoughts/recommendations would be greatly appreciated.
>>
>> Thanks,
>> --
>> *Geren White | Senior Java Developer*
>> *(e)* geren@1stdibs.com
>>
>
>
>
> --
> Best regards,
>
> Attila Szabo
> Sotware Engineer
>
> <http://www.cloudera.com>
>
--
*Geren White | Senior Java Developer*
*(e)* geren@1stdibs.com
Re: PreparedStatement must be reprepared error
Posted by Attila Szabo <as...@cloudera.com>.
Hi Geren,
For further investigation:
Could you please send us the whole verbose logfile for further
investigation?
On the top could you please check which version of the MySQL JDBC driver is
present in the lib directory of Sqoop?
We would also appreciate if you could tell which other JDBC drivers located
there!
Many thanks,
M.
On Thu, Aug 4, 2016 at 1:44 PM, Geren White <ge...@1stdibs.com> wrote:
> Hello all,
>
> I'm having trouble importing data from a mysql database (version 5.26)
> with sqoop (version 1.4.6).
>
> I have a number of imports that are happening in AWS EMR and all of them
> are working that go against tables. I have one import that goes against a
> view that is not working. It works on our dev database but for some reason
> not on our prod server.
>
> Here's the command I'm running:
> sqoop import --connect jdbc:mysql://$databaseHost/inventory
> --connection-manager org.apache.sqoop.manager.MySQLManager --username
> $user_name --password-file=s3://$s3Bucket/dibs/settings/sqoop_db_credentials.properties
> --as-avrodatafile --target-dir /user/hadoop/inventory/category_view
> --delete-target-dir --table category_view --split-by id --verbose
>
> I've tried with and without specifying the jdbc driver and without
> specifying the connection-manager. I've also tried overriding the default
> SQL it is using but nothing seems to help. I've also seen a number of other
> users with this issue that all point to the table_definition_cache but
> playing with those values has not fixed the issue.
>
> From the below stacktrace you can see that it is using a
> MariaDbServerPreparedStatement. This seems weird to me and would expect
> it to be using the Mysql specific PreparedStatement class. Is this the
> normal behavior?
>
> Here's the stacktrace:
>
> 2016-08-04 19:49:40,589 INFO org.apache.sqoop.manager.SqlManager (main): Executing SQL statement: SELECT t.* FROM `category_view` AS t LIMIT 1
> 2016-08-04 19:49:40,601 ERROR org.apache.sqoop.manager.SqlManager (main): Error executing statement: java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
> java.sql.BatchUpdateException: Prepared statement needs to be re-prepared
> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:374)
> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQuery(MariaDbServerPreparedStatement.java:341)
> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:758)
> at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)
> at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
> at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
> at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227)
> at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
> at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> Caused by: java.sql.SQLException: Prepared statement needs to be re-prepared
> at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:136)
> at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69)
> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeQueryEpilog(MariaDbServerPreparedStatement.java:336)
> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:291)
> at org.mariadb.jdbc.MariaDbServerPreparedStatement.execute(MariaDbServerPreparedStatement.java:369)
> ... 18 more
> Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Prepared statement needs to be re-prepared
> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479)
> at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executePreparedQuery(AbstractQueryProtocol.java:592)
> at org.mariadb.jdbc.MariaDbServerPreparedStatement.executeInternal(MariaDbServerPreparedStatement.java:279)
> ... 19 more
> 2016-08-04 19:49:40,606 ERROR org.apache.sqoop.tool.ImportTool (main): Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651)
> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
>
>
> Any thoughts/recommendations would be greatly appreciated.
>
> Thanks,
> --
> *Geren White | Senior Java Developer*
> *(e)* geren@1stdibs.com
>
--
Best regards,
Attila Szabo
Sotware Engineer
<http://www.cloudera.com>