You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by pratik khadloya <ti...@gmail.com> on 2014/09/11 03:17:51 UTC
Hive import stuck
Hello,
My hive table import is getting stuck after the line which prints the derby
database classpath.
Am not sure what is missing here. Can anyone please give me some pointers.
4/09/10 21:11:52 INFO hive.HiveImport: Logging initialized using
configuration in
jar:file:/usr/lib/hive/lib/hive-common-0.13.1.jar!/hive-log4j.properties
14/09/10 21:11:53 INFO hive.HiveImport: Wed Sep 10 21:11:53 EDT 2014
Thread[main,5,main] java.io.FileNotFoundException: derby.log (Permission
denied)
14/09/10 21:11:53 INFO hive.HiveImport:
----------------------------------------------------------------
14/09/10 21:11:53 INFO hive.HiveImport: Wed Sep 10 21:11:53 EDT 2014:
14/09/10 21:11:53 INFO hive.HiveImport: Booting Derby (version The Apache
Software Foundation - Apache Derby - 10.10.1.1 - (1458268)) instance
a816c00e-0148-6244-425a-000001726a08
14/09/10 21:11:53 INFO hive.HiveImport: on database directory
/home/pkhadloya/sqoop-2e1e094/metastore_db in READ ONLY mode with class
loader sun.misc.Launcher$AppClassLoader@77cde100.
14/09/10 21:11:53 INFO hive.HiveImport: Loaded from
file:/usr/lib/hive/lib/derby-10.10.1.1.jar.
14/09/10 21:11:53 INFO hive.HiveImport: java.vendor=Sun Microsystems Inc.
14/09/10 21:11:53 INFO hive.HiveImport: java.runtime.version=1.6.0_16-b01
14/09/10 21:11:53 INFO hive.HiveImport:
user.dir=/home/pkhadloya/sqoop-2e1e094
14/09/10 21:11:53 INFO hive.HiveImport: os.name=Linux
14/09/10 21:11:53 INFO hive.HiveImport: os.arch=amd64
14/09/10 21:11:53 INFO hive.HiveImport:
os.version=2.6.32-279.22.1.el6.x86_64
14/09/10 21:11:53 INFO hive.HiveImport: derby.system.home=null
14/09/10 21:11:54 INFO hive.HiveImport: Database Class Loader started -
derby.database.classpath=''
Thanks,
Pratik
Re: Hive import stuck
Posted by pratik khadloya <ti...@gmail.com>.
Finally figured out the problem. Hive was starting its own derby based
metastore whereas it is configured to use a mysql meta store.
Turns out that sqoop could not know about the hive conf.
I set the HIVE_CONF_DIR to the appropriate directory in sqoop-env.sh and it
worked.
On Thu, Sep 11, 2014 at 12:26 PM, pratik khadloya <ti...@gmail.com>
wrote:
> I got some clue here, seems like the derby database is either read only or
> the user that i am using is read only.
> Got to know this from the hive query logs.
>
> $ tail -f /srv/logs/hive/query_logs/<myuser>/hive.log
> 2014-09-11 15:21:30,664 WARN [main]: common.LogUtils
> (LogUtils.java:logConfigLocation(145)) - hive-site.xml not found on
> CLASSPATH
> 2014-09-11 15:21:30,796 INFO [main]: SessionState
> (SessionState.java:printInfo(543)) -
> Logging initialized using configuration in
> jar:file:/usr/lib/hive/lib/hive-common-0.13.1.jar!/hive-log4j.properties
> 2014-09-11 15:21:31,014 INFO [main]: metastore.HiveMetaStore
> (HiveMetaStore.java:newRawStore(496)) - 0: Opening raw store with
> implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 2014-09-11 15:21:31,050 INFO [main]: metastore.ObjectStore
> (ObjectStore.java:initialize(246)) - ObjectStore, initialize called
> 2014-09-11 15:21:32,216 ERROR [BoneCP-pool-watch-thread]:
> bonecp.PoolWatchThread (PoolWatchThread.java:fillConnections(118)) - Error
> in trying to obtain a connection. Retrying in 7000ms
> java.sql.SQLException: *A read-only user or a user in a read-only
> database is not permitted to disable read-only mode on a connection.*
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown
> Source)
> at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
> at
> org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
> Source)
> at org.apache.derby.impl.jdbc.EmbedConnection.setReadOnly(Unknown Source)
> at
> com.jolbox.bonecp.ConnectionHandle.setReadOnly(ConnectionHandle.java:1324)
> at com.jolbox.bonecp.ConnectionHandle.<init>(ConnectionHandle.java:262)
> at
> com.jolbox.bonecp.PoolWatchThread.fillConnections(PoolWatchThread.java:115)
> at com.jolbox.bonecp.PoolWatchThread.run(PoolWatchThread.java:82)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.sql.SQLException: A read-only user or a user in a
> read-only database is not permitted to disable read-only mode on a
> connection.
> at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
> Source)
> at
> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
> Source)
> ... 13 more
> Caused by: ERROR 25505: A read-only user or a user in a read-only database
> is not permitted to disable read-only mode on a connection.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.sql.conn.GenericAuthorizer.setReadOnlyConnection(Unknown
> Source)
> at
> org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.setReadOnly(Unknown
> Source)
> ... 8 more
>
> On Wed, Sep 10, 2014 at 6:17 PM, pratik khadloya <ti...@gmail.com>
> wrote:
>
>> Hello,
>>
>> My hive table import is getting stuck after the line which prints the
>> derby database classpath.
>> Am not sure what is missing here. Can anyone please give me some pointers.
>>
>> 4/09/10 21:11:52 INFO hive.HiveImport: Logging initialized using
>> configuration in
>> jar:file:/usr/lib/hive/lib/hive-common-0.13.1.jar!/hive-log4j.properties
>> 14/09/10 21:11:53 INFO hive.HiveImport: Wed Sep 10 21:11:53 EDT 2014
>> Thread[main,5,main] java.io.FileNotFoundException: derby.log (Permission
>> denied)
>> 14/09/10 21:11:53 INFO hive.HiveImport:
>> ----------------------------------------------------------------
>> 14/09/10 21:11:53 INFO hive.HiveImport: Wed Sep 10 21:11:53 EDT 2014:
>> 14/09/10 21:11:53 INFO hive.HiveImport: Booting Derby (version The Apache
>> Software Foundation - Apache Derby - 10.10.1.1 - (1458268)) instance
>> a816c00e-0148-6244-425a-000001726a08
>> 14/09/10 21:11:53 INFO hive.HiveImport: on database directory
>> /home/pkhadloya/sqoop-2e1e094/metastore_db in READ ONLY mode with class
>> loader sun.misc.Launcher$AppClassLoader@77cde100.
>> 14/09/10 21:11:53 INFO hive.HiveImport: Loaded from
>> file:/usr/lib/hive/lib/derby-10.10.1.1.jar.
>> 14/09/10 21:11:53 INFO hive.HiveImport: java.vendor=Sun Microsystems Inc.
>> 14/09/10 21:11:53 INFO hive.HiveImport: java.runtime.version=1.6.0_16-b01
>> 14/09/10 21:11:53 INFO hive.HiveImport:
>> user.dir=/home/pkhadloya/sqoop-2e1e094
>> 14/09/10 21:11:53 INFO hive.HiveImport: os.name=Linux
>> 14/09/10 21:11:53 INFO hive.HiveImport: os.arch=amd64
>> 14/09/10 21:11:53 INFO hive.HiveImport:
>> os.version=2.6.32-279.22.1.el6.x86_64
>> 14/09/10 21:11:53 INFO hive.HiveImport: derby.system.home=null
>> 14/09/10 21:11:54 INFO hive.HiveImport: Database Class Loader started -
>> derby.database.classpath=''
>>
>>
>> Thanks,
>> Pratik
>>
>
>
Re: Hive import stuck
Posted by pratik khadloya <ti...@gmail.com>.
I got some clue here, seems like the derby database is either read only or
the user that i am using is read only.
Got to know this from the hive query logs.
$ tail -f /srv/logs/hive/query_logs/<myuser>/hive.log
2014-09-11 15:21:30,664 WARN [main]: common.LogUtils
(LogUtils.java:logConfigLocation(145)) - hive-site.xml not found on
CLASSPATH
2014-09-11 15:21:30,796 INFO [main]: SessionState
(SessionState.java:printInfo(543)) -
Logging initialized using configuration in
jar:file:/usr/lib/hive/lib/hive-common-0.13.1.jar!/hive-log4j.properties
2014-09-11 15:21:31,014 INFO [main]: metastore.HiveMetaStore
(HiveMetaStore.java:newRawStore(496)) - 0: Opening raw store with
implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2014-09-11 15:21:31,050 INFO [main]: metastore.ObjectStore
(ObjectStore.java:initialize(246)) - ObjectStore, initialize called
2014-09-11 15:21:32,216 ERROR [BoneCP-pool-watch-thread]:
bonecp.PoolWatchThread (PoolWatchThread.java:fillConnections(118)) - Error
in trying to obtain a connection. Retrying in 7000ms
java.sql.SQLException: *A read-only user or a user in a read-only database
is not permitted to disable read-only mode on a connection.*
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.setReadOnly(Unknown Source)
at
com.jolbox.bonecp.ConnectionHandle.setReadOnly(ConnectionHandle.java:1324)
at com.jolbox.bonecp.ConnectionHandle.<init>(ConnectionHandle.java:262)
at
com.jolbox.bonecp.PoolWatchThread.fillConnections(PoolWatchThread.java:115)
at com.jolbox.bonecp.PoolWatchThread.run(PoolWatchThread.java:82)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.sql.SQLException: A read-only user or a user in a read-only
database is not permitted to disable read-only mode on a connection.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
Source)
... 13 more
Caused by: ERROR 25505: A read-only user or a user in a read-only database
is not permitted to disable read-only mode on a connection.
at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)
at
org.apache.derby.impl.sql.conn.GenericAuthorizer.setReadOnlyConnection(Unknown
Source)
at
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.setReadOnly(Unknown
Source)
... 8 more
On Wed, Sep 10, 2014 at 6:17 PM, pratik khadloya <ti...@gmail.com>
wrote:
> Hello,
>
> My hive table import is getting stuck after the line which prints the
> derby database classpath.
> Am not sure what is missing here. Can anyone please give me some pointers.
>
> 4/09/10 21:11:52 INFO hive.HiveImport: Logging initialized using
> configuration in
> jar:file:/usr/lib/hive/lib/hive-common-0.13.1.jar!/hive-log4j.properties
> 14/09/10 21:11:53 INFO hive.HiveImport: Wed Sep 10 21:11:53 EDT 2014
> Thread[main,5,main] java.io.FileNotFoundException: derby.log (Permission
> denied)
> 14/09/10 21:11:53 INFO hive.HiveImport:
> ----------------------------------------------------------------
> 14/09/10 21:11:53 INFO hive.HiveImport: Wed Sep 10 21:11:53 EDT 2014:
> 14/09/10 21:11:53 INFO hive.HiveImport: Booting Derby (version The Apache
> Software Foundation - Apache Derby - 10.10.1.1 - (1458268)) instance
> a816c00e-0148-6244-425a-000001726a08
> 14/09/10 21:11:53 INFO hive.HiveImport: on database directory
> /home/pkhadloya/sqoop-2e1e094/metastore_db in READ ONLY mode with class
> loader sun.misc.Launcher$AppClassLoader@77cde100.
> 14/09/10 21:11:53 INFO hive.HiveImport: Loaded from
> file:/usr/lib/hive/lib/derby-10.10.1.1.jar.
> 14/09/10 21:11:53 INFO hive.HiveImport: java.vendor=Sun Microsystems Inc.
> 14/09/10 21:11:53 INFO hive.HiveImport: java.runtime.version=1.6.0_16-b01
> 14/09/10 21:11:53 INFO hive.HiveImport:
> user.dir=/home/pkhadloya/sqoop-2e1e094
> 14/09/10 21:11:53 INFO hive.HiveImport: os.name=Linux
> 14/09/10 21:11:53 INFO hive.HiveImport: os.arch=amd64
> 14/09/10 21:11:53 INFO hive.HiveImport:
> os.version=2.6.32-279.22.1.el6.x86_64
> 14/09/10 21:11:53 INFO hive.HiveImport: derby.system.home=null
> 14/09/10 21:11:54 INFO hive.HiveImport: Database Class Loader started -
> derby.database.classpath=''
>
>
> Thanks,
> Pratik
>