You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by DIPESH KUMAR SINGH <di...@gmail.com> on 2012/04/22 07:21:29 UTC
Error while importing from Teradata to HDFS
I am trying to import the data from Teradata to hdfs referring Cloudera
Connector guide. I added the property
(The Hadoop environment i am working in is a shared Hadoop installation, So
users are not allowed to
add any jars in lib directory or make changes in Sqoop-site.xml of central
installation.
However, users can use .profile i.e. user's configuration files to specify
CLASSPATH )
I added the line com.teradata.jdbc.TeraDriver=<path to jar file
terajdbc4.jar> along with
com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
sqoop-td-connector-1.0.5.jar>
in connectors file inside managers.d directory. In my CLASSPATH, i have
added path to
SQOOP_CONF i.e. a Sqoop folder where i have kept my managers.d and all
jars.
With these I suppose i am able to connect to teradata using the teradata
manager factory using the teradriver as well
but, still i am facing this when i issue the sqoop import command as below
sqoop import
-libjars <path to tdgssconfig.jar>
--driver com.teradata.jdbc.TeraDriver
--connect jdbc:teradata://xxxxxx/DATABASE=dbname
--username hadoop --P
--table SQOOP_TEST
--num-mappers 3 --verbose
12/04/13 02:29:32 DEBUG tool.BaseSqoopTool: Enabled debug logging.
12/04/13 02:29:32 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing class:
com.cloudera.sqoop.manager.TeradataManagerFactory
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
through URL: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
java.net.URLClassLoader@6a8c436b
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
com.cloudera.sqoop.manager.TeradataManagerFactory
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current JVM:
jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar
/Sqoop/sqoop-td-connector-1.0.5.jar: java.net.FactoryURLClassLoader@45db05b2
12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
com.cloudera.sqoop.manager.TeradataManagerFactory in jar
/Sqoop/sqoop-td-connector-1.0.5.jar specified by
/Sqoop/conf/managers.d/connectors
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing class:
com.teradata.jdbc.TeraDriver
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
through URL: jar:file:///Sqoop/lib/terajdbc4.jar!/
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
java.net.FactoryURLClassLoader@45db05b2
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
com.teradata.jdbc.TeraDriver
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current JVM:
jar:file:///Sqoop/lib/terajdbc4.jar!/
12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar
/Sqoop/lib/terajdbc4.jar: java.net.FactoryURLClassLoader@518f5824
12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
com.teradata.jdbc.TeraDriver in jar /Sqoop/lib/terajdbc4.jar specified by
/Sqoop/conf/managers.d/connectors
12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Loaded manager factory:
com.cloudera.sqoop.manager.TeradataManagerFactory
12/04/13 02:29:32 ERROR tool.BaseSqoopTool: Got error creating database
manager: java.lang.ClassCastException: com.teradata.jdbc.TeraDriver cannot
be cast to com.cloudera.sqoop.manager.ManagerFactory
at com.cloudera.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:90)
at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:60)
at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:186)
at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81)
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
Please help.
Thanks & Regards,
--
Dipesh Kr. Singh
Re: Error while importing from Teradata to HDFS
Posted by Technical Anirudh <te...@gmail.com>.
Yes it should but it is more of a hack :). I see no reasons why libjars option wouldn't work. And yes besides using libjars option, the jdbc driver also needs to be in the class path as sqoop will use it for fetching metadata in order to determine the splits.
Thanks,
Anirudh
Sent from my iPhone
On Apr 23, 2012, at 11:57, Cheolsoo Park <ch...@cloudera.com> wrote:
> Hi Dipesh,
>
> I was looking for such an alternative because currently i don't have permissions to add jars in Sqoop/lib.
>
> OK, that's unfortunate.
>
> If none of these suggestions is feasible for your case, there is the last thing that you can do: install Sqoop from the tarball in a directory where you have full permissions and use it instead. As long as HADOOP_HOME is set to the installed hadoop, everything will work.
>
> Thanks,
> Cheolsoo
>
>
> On Mon, Apr 23, 2012 at 11:17 AM, Anirudh <te...@gmail.com> wrote:
> Dipesh,
>
> It would be ideal if the jdbc driver is placed under Sqoop/lib as other users too would be able to leverage the same. But given your situation, use the -libjars option to point to the driver.
>
> Hope this helps.
>
> Thanks,
> Anirudh
>
>
> On Mon, Apr 23, 2012 at 11:01 AM, DIPESH KUMAR SINGH <di...@gmail.com> wrote:
> Hi Cheolsoo,
>
> Yeah, I know. I was looking for an alternative way by which we could tell Sqoop to load jars from a location different than Sqoop/lib. I was looking for such an alternative because currently i don't have
> permissions to add jars in Sqoop/lib, though i have initiated the process.
>
> Thanks & Regards,
>
> Dipesh
>
>
>
> On Mon, Apr 23, 2012 at 11:13 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
> Hi Dipesh,
>
> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.teradata.jdbc.TeraDriver
> java.lang.RuntimeException: Could not load db driver class: com.teradata.jdbc.TeraDrive
>
> Your Teradata jdbc jar is not found by Sqoop. The easiest way to fix it is to put terajdbc4.jar and tdgssconfig.jar in /usr/lib/sqoop/lib.
>
> Thanks,
> Cheolsoo
>
> On Mon, Apr 23, 2012 at 10:32 AM, DIPESH KUMAR SINGH <di...@gmail.com> wrote:
> Hi Cheolsoo,
>
> As you suggested, I tried after removing "TeraDriver" line from connectors file but that didn't worked
> too. Please help..!!
>
> Here is the stack trace:
>
> 12/04/23 01:02:35 DEBUG tool.BaseSqoopTool: Enabled debug logging.
> Enter password:
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Checking for existing class: com.cloudera.sqoop.manager.TeradataManagerFactory
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Attempting to load jar through URL: jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Previous classloader is java.net.URLClassLoader@27b15692
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Testing class in jar: com.cloudera.sqoop.manager.TeradataManagerFactory
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Loaded jar into current JVM: jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Added classloader for jar /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar: java.net.FactoryURLClassLoader@19e3cd51
> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Added factory com.cloudera.sqoop.manager.TeradataManagerFactory in jar /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar specified by /ms/user/a/sqoop-user/Sqoop/conf/managers.d/connectors
> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.TeradataManagerFactory
> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.TeradataManagerFactory
> 12/04/23 01:02:42 DEBUG manager.TeradataManagerFactory: Trying with scheme: jdbc:teradata:
> 12/04/23 01:02:42 INFO manager.SqlManager: Using default fetchSize of 1000
> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Instantiated ConnManager com.cloudera.sqoop.teradata.TeradataManager@220ca470
> 12/04/23 01:02:42 INFO tool.CodeGenTool: Beginning code generation
> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.teradata.jdbc.TeraDriver
> java.lang.RuntimeException: Could not load db driver class: com.teradata.jdbc.TeraDriver
> at com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:634)
> at com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:51)
> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:456)
> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:479)
> at com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:192)
> at com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:176)
> at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1017)
> at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
> at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>
>
> Thanks,
>
> Dipesh
>
>
>
> On Sun, Apr 22, 2012 at 3:01 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
> Hi Dispesh,
>
> I added the line com.teradata.jdbc.TeraDriver=<path to jar file terajdbc4.jar> along with
> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to sqoop-td-connector-1.0.5.jar>
> in connectors file inside managers.d directory.
>
> Please remove "com.teradata.jdbc.TeraDriver=<path to jar file terajdbc4.jar>" from your connector file.
>
> It should only contain "com.cloudera.sqoop.manager.TeradataManagerFactory=<path to sqoop-td-connector-1.0.5.jar>".
>
> Thanks,
> Cheolsoo
>
>
> On Sat, Apr 21, 2012 at 10:21 PM, DIPESH KUMAR SINGH <di...@gmail.com> wrote:
> I am trying to import the data from Teradata to hdfs referring Cloudera Connector guide. I added the property
>
> (The Hadoop environment i am working in is a shared Hadoop installation, So users are not allowed to
> add any jars in lib directory or make changes in Sqoop-site.xml of central installation.
> However, users can use .profile i.e. user's configuration files to specify CLASSPATH )
>
> I added the line com.teradata.jdbc.TeraDriver=<path to jar file terajdbc4.jar> along with
> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to sqoop-td-connector-1.0.5.jar>
> in connectors file inside managers.d directory. In my CLASSPATH, i have added path to
> SQOOP_CONF i.e. a Sqoop folder where i have kept my managers.d and all jars.
>
>
> With these I suppose i am able to connect to teradata using the teradata manager factory using the teradriver as well
> but, still i am facing this when i issue the sqoop import command as below
>
> sqoop import
> -libjars <path to tdgssconfig.jar>
> --driver com.teradata.jdbc.TeraDriver
> --connect jdbc:teradata://xxxxxx/DATABASE=dbname
> --username hadoop --P
> --table SQOOP_TEST
> --num-mappers 3 --verbose
>
> 12/04/13 02:29:32 DEBUG tool.BaseSqoopTool: Enabled debug logging.
> 12/04/13 02:29:32 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing class: com.cloudera.sqoop.manager.TeradataManagerFactory
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar through URL: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is java.net.URLClassLoader@6a8c436b
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar: com.cloudera.sqoop.manager.TeradataManagerFactory
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current JVM: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar /Sqoop/sqoop-td-connector-1.0.5.jar: java.net.FactoryURLClassLoader@45db05b2
> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory com.cloudera.sqoop.manager.TeradataManagerFactory in jar /Sqoop/sqoop-td-connector-1.0.5.jar specified by /Sqoop/conf/managers.d/connectors
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing class: com.teradata.jdbc.TeraDriver
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar through URL: jar:file:///Sqoop/lib/terajdbc4.jar!/
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is java.net.FactoryURLClassLoader@45db05b2
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar: com.teradata.jdbc.TeraDriver
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current JVM: jar:file:///Sqoop/lib/terajdbc4.jar!/
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar /Sqoop/lib/terajdbc4.jar: java.net.FactoryURLClassLoader@518f5824
> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory com.teradata.jdbc.TeraDriver in jar /Sqoop/lib/terajdbc4.jar specified by /Sqoop/conf/managers.d/connectors
> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.TeradataManagerFactory
> 12/04/13 02:29:32 ERROR tool.BaseSqoopTool: Got error creating database manager: java.lang.ClassCastException: com.teradata.jdbc.TeraDriver cannot be cast to com.cloudera.sqoop.manager.ManagerFactory
> at com.cloudera.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:90)
> at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:60)
> at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:186)
> at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81)
> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411)
> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>
>
> Please help.
>
> Thanks & Regards,
> --
> Dipesh Kr. Singh
>
>
>
>
>
>
>
>
> --
> Dipesh Kr. Singh
>
>
>
>
>
>
>
>
> --
> Dipesh Kr. Singh
>
>
>
>
>
>
Re: Error while importing from Teradata to HDFS
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi Dipesh,
I was looking for such an alternative because currently i don't
> have permissions to add jars in Sqoop/lib.
OK, that's unfortunate.
If none of these suggestions is feasible for your case, there is the last
thing that you can do: install Sqoop from the tarball in a directory where
you have full permissions and use it instead. As long as HADOOP_HOME is set
to the installed hadoop, everything will work.
Thanks,
Cheolsoo
On Mon, Apr 23, 2012 at 11:17 AM, Anirudh <te...@gmail.com> wrote:
> Dipesh,
>
> It would be ideal if the jdbc driver is placed under Sqoop/lib as other
> users too would be able to leverage the same. But given your situation, use
> the -libjars option to point to the driver.
>
> Hope this helps.
>
> Thanks,
> Anirudh
>
>
> On Mon, Apr 23, 2012 at 11:01 AM, DIPESH KUMAR SINGH <
> dipesh.tech@gmail.com> wrote:
>
>> Hi Cheolsoo,
>>
>> Yeah, I know. I was looking for an alternative way by which we could tell
>> Sqoop to load jars from a location different than Sqoop/lib. I was looking
>> for such an alternative because currently i don't have
>> permissions to add jars in Sqoop/lib, though i have initiated the process.
>>
>> Thanks & Regards,
>>
>> Dipesh
>>
>>
>>
>> On Mon, Apr 23, 2012 at 11:13 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
>>
>>> Hi Dipesh,
>>>
>>> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
>>>> java.lang.RuntimeException: Could not load db driver class:
>>>> com.teradata.jdbc.TeraDriver
>>>>
>>> java.lang.RuntimeException: Could not load db driver class:
>>>> com.teradata.jdbc.TeraDrive
>>>
>>>
>>> Your Teradata jdbc jar is not found by Sqoop. The easiest way to fix it
>>> is to put *terajdbc4.jar* and *tdgssconfig.jar* in */usr/lib/sqoop/lib*.
>>>
>>> Thanks,
>>> Cheolsoo
>>>
>>> On Mon, Apr 23, 2012 at 10:32 AM, DIPESH KUMAR SINGH <
>>> dipesh.tech@gmail.com> wrote:
>>>
>>>> Hi Cheolsoo,
>>>>
>>>> As you suggested, I tried after removing "TeraDriver" line from
>>>> connectors file but that didn't worked
>>>> too. Please help..!!
>>>>
>>>> Here is the stack trace:
>>>>
>>>>>
>>>>> 12/04/23 01:02:35 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>>>>>
>>>> Enter password:
>>>>>
>>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Checking for existing
>>>>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>>
>>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>>> through URL:
>>>>> jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>>
>>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Previous classloader is
>>>>> java.net.URLClassLoader@27b15692
>>>>>
>>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>>
>>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Loaded jar into current
>>>>> JVM: jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>>
>>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Added classloader for
>>>>> jar /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar:
>>>>> java.net.FactoryURLClassLoader@19e3cd51
>>>>>
>>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Added factory
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>>>>> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar specified by
>>>>> /ms/user/a/sqoop-user/Sqoop/conf/managers.d/connectors
>>>>>
>>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>>
>>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>>>> com.cloudera.sqoop.manager.DefaultManagerFactory
>>>>>
>>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>>
>>>> 12/04/23 01:02:42 DEBUG manager.TeradataManagerFactory: Trying with
>>>>> scheme: jdbc:teradata:
>>>>>
>>>> 12/04/23 01:02:42 INFO manager.SqlManager: Using default fetchSize of
>>>>> 1000
>>>>>
>>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Instantiated ConnManager
>>>>> com.cloudera.sqoop.teradata.TeradataManager@220ca470
>>>>>
>>>> 12/04/23 01:02:42 INFO tool.CodeGenTool: Beginning code generation
>>>>>
>>>> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
>>>>> java.lang.RuntimeException: Could not load db driver class:
>>>>> com.teradata.jdbc.TeraDriver
>>>>>
>>>> java.lang.RuntimeException: Could not load db driver class:
>>>>> com.teradata.jdbc.TeraDriver
>>>>>
>>>> at
>>>>> com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:634)
>>>>>
>>>> at
>>>>> com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:51)
>>>>>
>>>> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:456)
>>>>>
>>>> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:479)
>>>>>
>>>> at
>>>>> com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:192)
>>>>>
>>>> at
>>>>> com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:176)
>>>>>
>>>> at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1017)
>>>>>
>>>> at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
>>>>>
>>>> at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
>>>>>
>>>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
>>>>>
>>>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>>>>>
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>
>>>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>>>>>
>>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>>>>>
>>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>>>>>
>>>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>>>>
>>>>
>>>>>
>>>> Thanks,
>>>>
>>>> Dipesh
>>>>
>>>>
>>>>
>>>> On Sun, Apr 22, 2012 at 3:01 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
>>>>
>>>>> Hi Dispesh,
>>>>>
>>>>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>>>>>> terajdbc4.jar> along with
>>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>>>>>> sqoop-td-connector-1.0.5.jar>
>>>>>> in connectors file inside managers.d directory.
>>>>>
>>>>>
>>>>> Please remove "com.teradata.jdbc.TeraDriver=<path to jar file
>>>>> terajdbc4.jar>" from your connector file.
>>>>>
>>>>> It should only contain "com.cloudera.sqoop.manager.TeradataManagerFactory=<path
>>>>> to sqoop-td-connector-1.0.5.jar>".
>>>>>
>>>>> Thanks,
>>>>> Cheolsoo
>>>>>
>>>>>
>>>>> On Sat, Apr 21, 2012 at 10:21 PM, DIPESH KUMAR SINGH <
>>>>> dipesh.tech@gmail.com> wrote:
>>>>>
>>>>>> I am trying to import the data from Teradata to hdfs referring
>>>>>> Cloudera Connector guide. I added the property
>>>>>>
>>>>>> (The Hadoop environment i am working in is a shared Hadoop
>>>>>> installation, So users are not allowed to
>>>>>> add any jars in lib directory or make changes in Sqoop-site.xml of
>>>>>> central installation.
>>>>>> However, users can use .profile i.e. user's configuration files to
>>>>>> specify CLASSPATH )
>>>>>>
>>>>>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>>>>>> terajdbc4.jar> along with
>>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>>>>>> sqoop-td-connector-1.0.5.jar>
>>>>>> in connectors file inside managers.d directory. In my CLASSPATH, i
>>>>>> have added path to
>>>>>> SQOOP_CONF i.e. a Sqoop folder where i have kept my managers.d and
>>>>>> all jars.
>>>>>>
>>>>>>
>>>>>> With these I suppose i am able to connect to teradata using the
>>>>>> teradata manager factory using the teradriver as well
>>>>>> but, still i am facing this when i issue the sqoop import command as
>>>>>> below
>>>>>>
>>>>>> sqoop import
>>>>>> -libjars <path to tdgssconfig.jar>
>>>>>> --driver com.teradata.jdbc.TeraDriver
>>>>>> --connect jdbc:teradata://xxxxxx/DATABASE=dbname
>>>>>> --username hadoop --P
>>>>>> --table SQOOP_TEST
>>>>>> --num-mappers 3 --verbose
>>>>>>
>>>>>> 12/04/13 02:29:32 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>>>>>> 12/04/13 02:29:32 WARN tool.BaseSqoopTool: Setting your password on
>>>>>> the command-line is insecure. Consider using -P instead.
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>>>>>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>>>> through URL: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader
>>>>>> is java.net.URLClassLoader@6a8c436b
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into
>>>>>> current JVM: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for
>>>>>> jar /Sqoop/sqoop-td-connector-1.0.5.jar:
>>>>>> java.net.FactoryURLClassLoader@45db05b2
>>>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>>>>>> /Sqoop/sqoop-td-connector-1.0.5.jar specified by
>>>>>> /Sqoop/conf/managers.d/connectors
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>>>>>> class: com.teradata.jdbc.TeraDriver
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>>>> through URL: jar:file:///Sqoop/lib/terajdbc4.jar!/
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader
>>>>>> is java.net.FactoryURLClassLoader@45db05b2
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>>>> com.teradata.jdbc.TeraDriver
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into
>>>>>> current JVM: jar:file:///Sqoop/lib/terajdbc4.jar!/
>>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for
>>>>>> jar /Sqoop/lib/terajdbc4.jar: java.net.FactoryURLClassLoader@518f5824
>>>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>>>>>> com.teradata.jdbc.TeraDriver in jar /Sqoop/lib/terajdbc4.jar specified by
>>>>>> /Sqoop/conf/managers.d/connectors
>>>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>>> 12/04/13 02:29:32 ERROR tool.BaseSqoopTool: Got error creating
>>>>>> database manager: java.lang.ClassCastException:
>>>>>> com.teradata.jdbc.TeraDriver cannot be cast to
>>>>>> com.cloudera.sqoop.manager.ManagerFactory
>>>>>> at
>>>>>> com.cloudera.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:90)
>>>>>> at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:60)
>>>>>> at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:186)
>>>>>> at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81)
>>>>>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411)
>>>>>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>>>>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>>>>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>>>>>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>>>>>
>>>>>>
>>>>>> Please help.
>>>>>>
>>>>>> Thanks & Regards,
>>>>>> --
>>>>>> Dipesh Kr. Singh
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Dipesh Kr. Singh
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Dipesh Kr. Singh
>>
>>
>>
>>
>>
>
Re: Error while importing from Teradata to HDFS
Posted by Anirudh <te...@gmail.com>.
Dipesh,
It would be ideal if the jdbc driver is placed under Sqoop/lib as other
users too would be able to leverage the same. But given your situation, use
the -libjars option to point to the driver.
Hope this helps.
Thanks,
Anirudh
On Mon, Apr 23, 2012 at 11:01 AM, DIPESH KUMAR SINGH
<di...@gmail.com>wrote:
> Hi Cheolsoo,
>
> Yeah, I know. I was looking for an alternative way by which we could tell
> Sqoop to load jars from a location different than Sqoop/lib. I was looking
> for such an alternative because currently i don't have
> permissions to add jars in Sqoop/lib, though i have initiated the process.
>
> Thanks & Regards,
>
> Dipesh
>
>
>
> On Mon, Apr 23, 2012 at 11:13 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
>
>> Hi Dipesh,
>>
>> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
>>> java.lang.RuntimeException: Could not load db driver class:
>>> com.teradata.jdbc.TeraDriver
>>>
>> java.lang.RuntimeException: Could not load db driver class:
>>> com.teradata.jdbc.TeraDrive
>>
>>
>> Your Teradata jdbc jar is not found by Sqoop. The easiest way to fix it
>> is to put *terajdbc4.jar* and *tdgssconfig.jar* in */usr/lib/sqoop/lib*.
>>
>> Thanks,
>> Cheolsoo
>>
>> On Mon, Apr 23, 2012 at 10:32 AM, DIPESH KUMAR SINGH <
>> dipesh.tech@gmail.com> wrote:
>>
>>> Hi Cheolsoo,
>>>
>>> As you suggested, I tried after removing "TeraDriver" line from
>>> connectors file but that didn't worked
>>> too. Please help..!!
>>>
>>> Here is the stack trace:
>>>
>>>>
>>>> 12/04/23 01:02:35 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>>>>
>>> Enter password:
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Checking for existing
>>>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>> through URL:
>>>> jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Previous classloader is
>>>> java.net.URLClassLoader@27b15692
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Loaded jar into current
>>>> JVM: jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>
>>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Added classloader for jar
>>>> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar:
>>>> java.net.FactoryURLClassLoader@19e3cd51
>>>>
>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Added factory
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>>>> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar specified by
>>>> /ms/user/a/sqoop-user/Sqoop/conf/managers.d/connectors
>>>>
>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>
>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>>> com.cloudera.sqoop.manager.DefaultManagerFactory
>>>>
>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>
>>> 12/04/23 01:02:42 DEBUG manager.TeradataManagerFactory: Trying with
>>>> scheme: jdbc:teradata:
>>>>
>>> 12/04/23 01:02:42 INFO manager.SqlManager: Using default fetchSize of
>>>> 1000
>>>>
>>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Instantiated ConnManager
>>>> com.cloudera.sqoop.teradata.TeradataManager@220ca470
>>>>
>>> 12/04/23 01:02:42 INFO tool.CodeGenTool: Beginning code generation
>>>>
>>> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
>>>> java.lang.RuntimeException: Could not load db driver class:
>>>> com.teradata.jdbc.TeraDriver
>>>>
>>> java.lang.RuntimeException: Could not load db driver class:
>>>> com.teradata.jdbc.TeraDriver
>>>>
>>> at
>>>> com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:634)
>>>>
>>> at
>>>> com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:51)
>>>>
>>> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:456)
>>>>
>>> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:479)
>>>>
>>> at
>>>> com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:192)
>>>>
>>> at
>>>> com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:176)
>>>>
>>> at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1017)
>>>>
>>> at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
>>>>
>>> at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
>>>>
>>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
>>>>
>>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>>>>
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>
>>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>>>>
>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>>>>
>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>>>>
>>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>>>
>>>
>>>>
>>> Thanks,
>>>
>>> Dipesh
>>>
>>>
>>>
>>> On Sun, Apr 22, 2012 at 3:01 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
>>>
>>>> Hi Dispesh,
>>>>
>>>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>>>>> terajdbc4.jar> along with
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>>>>> sqoop-td-connector-1.0.5.jar>
>>>>> in connectors file inside managers.d directory.
>>>>
>>>>
>>>> Please remove "com.teradata.jdbc.TeraDriver=<path to jar file
>>>> terajdbc4.jar>" from your connector file.
>>>>
>>>> It should only contain "com.cloudera.sqoop.manager.TeradataManagerFactory=<path
>>>> to sqoop-td-connector-1.0.5.jar>".
>>>>
>>>> Thanks,
>>>> Cheolsoo
>>>>
>>>>
>>>> On Sat, Apr 21, 2012 at 10:21 PM, DIPESH KUMAR SINGH <
>>>> dipesh.tech@gmail.com> wrote:
>>>>
>>>>> I am trying to import the data from Teradata to hdfs referring
>>>>> Cloudera Connector guide. I added the property
>>>>>
>>>>> (The Hadoop environment i am working in is a shared Hadoop
>>>>> installation, So users are not allowed to
>>>>> add any jars in lib directory or make changes in Sqoop-site.xml of
>>>>> central installation.
>>>>> However, users can use .profile i.e. user's configuration files to
>>>>> specify CLASSPATH )
>>>>>
>>>>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>>>>> terajdbc4.jar> along with
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>>>>> sqoop-td-connector-1.0.5.jar>
>>>>> in connectors file inside managers.d directory. In my CLASSPATH, i
>>>>> have added path to
>>>>> SQOOP_CONF i.e. a Sqoop folder where i have kept my managers.d and all
>>>>> jars.
>>>>>
>>>>>
>>>>> With these I suppose i am able to connect to teradata using the
>>>>> teradata manager factory using the teradriver as well
>>>>> but, still i am facing this when i issue the sqoop import command as
>>>>> below
>>>>>
>>>>> sqoop import
>>>>> -libjars <path to tdgssconfig.jar>
>>>>> --driver com.teradata.jdbc.TeraDriver
>>>>> --connect jdbc:teradata://xxxxxx/DATABASE=dbname
>>>>> --username hadoop --P
>>>>> --table SQOOP_TEST
>>>>> --num-mappers 3 --verbose
>>>>>
>>>>> 12/04/13 02:29:32 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>>>>> 12/04/13 02:29:32 WARN tool.BaseSqoopTool: Setting your password on
>>>>> the command-line is insecure. Consider using -P instead.
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>>>>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>>> through URL: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
>>>>> java.net.URLClassLoader@6a8c436b
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
>>>>> JVM: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for
>>>>> jar /Sqoop/sqoop-td-connector-1.0.5.jar:
>>>>> java.net.FactoryURLClassLoader@45db05b2
>>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>>>>> /Sqoop/sqoop-td-connector-1.0.5.jar specified by
>>>>> /Sqoop/conf/managers.d/connectors
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>>>>> class: com.teradata.jdbc.TeraDriver
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>>> through URL: jar:file:///Sqoop/lib/terajdbc4.jar!/
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
>>>>> java.net.FactoryURLClassLoader@45db05b2
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>>> com.teradata.jdbc.TeraDriver
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
>>>>> JVM: jar:file:///Sqoop/lib/terajdbc4.jar!/
>>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for
>>>>> jar /Sqoop/lib/terajdbc4.jar: java.net.FactoryURLClassLoader@518f5824
>>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>>>>> com.teradata.jdbc.TeraDriver in jar /Sqoop/lib/terajdbc4.jar specified by
>>>>> /Sqoop/conf/managers.d/connectors
>>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>>> 12/04/13 02:29:32 ERROR tool.BaseSqoopTool: Got error creating
>>>>> database manager: java.lang.ClassCastException:
>>>>> com.teradata.jdbc.TeraDriver cannot be cast to
>>>>> com.cloudera.sqoop.manager.ManagerFactory
>>>>> at
>>>>> com.cloudera.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:90)
>>>>> at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:60)
>>>>> at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:186)
>>>>> at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81)
>>>>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411)
>>>>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>>>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>>>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>>>>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>>>>
>>>>>
>>>>> Please help.
>>>>>
>>>>> Thanks & Regards,
>>>>> --
>>>>> Dipesh Kr. Singh
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Dipesh Kr. Singh
>>>
>>>
>>>
>>>
>>>
>>
>
>
> --
> Dipesh Kr. Singh
>
>
>
>
>
Re: Error while importing from Teradata to HDFS
Posted by DIPESH KUMAR SINGH <di...@gmail.com>.
Hi Cheolsoo,
Yeah, I know. I was looking for an alternative way by which we could tell
Sqoop to load jars from a location different than Sqoop/lib. I was looking
for such an alternative because currently i don't have
permissions to add jars in Sqoop/lib, though i have initiated the process.
Thanks & Regards,
Dipesh
On Mon, Apr 23, 2012 at 11:13 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
> Hi Dipesh,
>
> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
>> java.lang.RuntimeException: Could not load db driver class:
>> com.teradata.jdbc.TeraDriver
>>
> java.lang.RuntimeException: Could not load db driver class:
>> com.teradata.jdbc.TeraDrive
>
>
> Your Teradata jdbc jar is not found by Sqoop. The easiest way to fix it is
> to put *terajdbc4.jar* and *tdgssconfig.jar* in */usr/lib/sqoop/lib*.
>
> Thanks,
> Cheolsoo
>
> On Mon, Apr 23, 2012 at 10:32 AM, DIPESH KUMAR SINGH <
> dipesh.tech@gmail.com> wrote:
>
>> Hi Cheolsoo,
>>
>> As you suggested, I tried after removing "TeraDriver" line from
>> connectors file but that didn't worked
>> too. Please help..!!
>>
>> Here is the stack trace:
>>
>>>
>>> 12/04/23 01:02:35 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>>>
>> Enter password:
>>>
>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Checking for existing
>>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>>>
>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Attempting to load jar
>>> through URL:
>>> jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>
>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Previous classloader is
>>> java.net.URLClassLoader@27b15692
>>>
>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Testing class in jar:
>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>
>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Loaded jar into current
>>> JVM: jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>
>> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Added classloader for jar
>>> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar:
>>> java.net.FactoryURLClassLoader@19e3cd51
>>>
>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Added factory
>>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>>> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar specified by
>>> /ms/user/a/sqoop-user/Sqoop/conf/managers.d/connectors
>>>
>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>
>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>> com.cloudera.sqoop.manager.DefaultManagerFactory
>>>
>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>
>> 12/04/23 01:02:42 DEBUG manager.TeradataManagerFactory: Trying with
>>> scheme: jdbc:teradata:
>>>
>> 12/04/23 01:02:42 INFO manager.SqlManager: Using default fetchSize of 1000
>>>
>> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Instantiated ConnManager
>>> com.cloudera.sqoop.teradata.TeradataManager@220ca470
>>>
>> 12/04/23 01:02:42 INFO tool.CodeGenTool: Beginning code generation
>>>
>> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
>>> java.lang.RuntimeException: Could not load db driver class:
>>> com.teradata.jdbc.TeraDriver
>>>
>> java.lang.RuntimeException: Could not load db driver class:
>>> com.teradata.jdbc.TeraDriver
>>>
>> at
>>> com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:634)
>>>
>> at
>>> com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:51)
>>>
>> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:456)
>>>
>> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:479)
>>>
>> at
>>> com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:192)
>>>
>> at
>>> com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:176)
>>>
>> at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1017)
>>>
>> at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
>>>
>> at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
>>>
>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
>>>
>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>>>
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>
>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>>>
>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>>>
>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>>>
>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>>
>>
>>>
>> Thanks,
>>
>> Dipesh
>>
>>
>>
>> On Sun, Apr 22, 2012 at 3:01 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
>>
>>> Hi Dispesh,
>>>
>>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>>>> terajdbc4.jar> along with
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>>>> sqoop-td-connector-1.0.5.jar>
>>>> in connectors file inside managers.d directory.
>>>
>>>
>>> Please remove "com.teradata.jdbc.TeraDriver=<path to jar file
>>> terajdbc4.jar>" from your connector file.
>>>
>>> It should only contain "com.cloudera.sqoop.manager.TeradataManagerFactory=<path
>>> to sqoop-td-connector-1.0.5.jar>".
>>>
>>> Thanks,
>>> Cheolsoo
>>>
>>>
>>> On Sat, Apr 21, 2012 at 10:21 PM, DIPESH KUMAR SINGH <
>>> dipesh.tech@gmail.com> wrote:
>>>
>>>> I am trying to import the data from Teradata to hdfs referring Cloudera
>>>> Connector guide. I added the property
>>>>
>>>> (The Hadoop environment i am working in is a shared Hadoop
>>>> installation, So users are not allowed to
>>>> add any jars in lib directory or make changes in Sqoop-site.xml of
>>>> central installation.
>>>> However, users can use .profile i.e. user's configuration files to
>>>> specify CLASSPATH )
>>>>
>>>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>>>> terajdbc4.jar> along with
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>>>> sqoop-td-connector-1.0.5.jar>
>>>> in connectors file inside managers.d directory. In my CLASSPATH, i have
>>>> added path to
>>>> SQOOP_CONF i.e. a Sqoop folder where i have kept my managers.d and all
>>>> jars.
>>>>
>>>>
>>>> With these I suppose i am able to connect to teradata using the
>>>> teradata manager factory using the teradriver as well
>>>> but, still i am facing this when i issue the sqoop import command as
>>>> below
>>>>
>>>> sqoop import
>>>> -libjars <path to tdgssconfig.jar>
>>>> --driver com.teradata.jdbc.TeraDriver
>>>> --connect jdbc:teradata://xxxxxx/DATABASE=dbname
>>>> --username hadoop --P
>>>> --table SQOOP_TEST
>>>> --num-mappers 3 --verbose
>>>>
>>>> 12/04/13 02:29:32 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>>>> 12/04/13 02:29:32 WARN tool.BaseSqoopTool: Setting your password on the
>>>> command-line is insecure. Consider using -P instead.
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>>>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>> through URL: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
>>>> java.net.URLClassLoader@6a8c436b
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
>>>> JVM: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for
>>>> jar /Sqoop/sqoop-td-connector-1.0.5.jar:
>>>> java.net.FactoryURLClassLoader@45db05b2
>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>>>> /Sqoop/sqoop-td-connector-1.0.5.jar specified by
>>>> /Sqoop/conf/managers.d/connectors
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>>>> class: com.teradata.jdbc.TeraDriver
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>>>> through URL: jar:file:///Sqoop/lib/terajdbc4.jar!/
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
>>>> java.net.FactoryURLClassLoader@45db05b2
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>>>> com.teradata.jdbc.TeraDriver
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
>>>> JVM: jar:file:///Sqoop/lib/terajdbc4.jar!/
>>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for
>>>> jar /Sqoop/lib/terajdbc4.jar: java.net.FactoryURLClassLoader@518f5824
>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>>>> com.teradata.jdbc.TeraDriver in jar /Sqoop/lib/terajdbc4.jar specified by
>>>> /Sqoop/conf/managers.d/connectors
>>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>>> 12/04/13 02:29:32 ERROR tool.BaseSqoopTool: Got error creating database
>>>> manager: java.lang.ClassCastException: com.teradata.jdbc.TeraDriver cannot
>>>> be cast to com.cloudera.sqoop.manager.ManagerFactory
>>>> at
>>>> com.cloudera.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:90)
>>>> at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:60)
>>>> at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:186)
>>>> at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81)
>>>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411)
>>>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>>>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>>>
>>>>
>>>> Please help.
>>>>
>>>> Thanks & Regards,
>>>> --
>>>> Dipesh Kr. Singh
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Dipesh Kr. Singh
>>
>>
>>
>>
>>
>
--
Dipesh Kr. Singh
Re: Error while importing from Teradata to HDFS
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi Dipesh,
12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.RuntimeException: Could not load db driver class:
> com.teradata.jdbc.TeraDriver
>
java.lang.RuntimeException: Could not load db driver class:
> com.teradata.jdbc.TeraDrive
Your Teradata jdbc jar is not found by Sqoop. The easiest way to fix it is
to put *terajdbc4.jar* and *tdgssconfig.jar* in */usr/lib/sqoop/lib*.
Thanks,
Cheolsoo
On Mon, Apr 23, 2012 at 10:32 AM, DIPESH KUMAR SINGH
<di...@gmail.com>wrote:
> Hi Cheolsoo,
>
> As you suggested, I tried after removing "TeraDriver" line from
> connectors file but that didn't worked
> too. Please help..!!
>
> Here is the stack trace:
>
>>
>> 12/04/23 01:02:35 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>>
> Enter password:
>>
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Checking for existing
>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>>
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Attempting to load jar
>> through URL:
>> jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>>
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Previous classloader is
>> java.net.URLClassLoader@27b15692
>>
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Testing class in jar:
>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Loaded jar into current
>> JVM: jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>>
> 12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Added classloader for jar
>> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar:
>> java.net.FactoryURLClassLoader@19e3cd51
>>
> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Added factory
>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar specified by
>> /ms/user/a/sqoop-user/Sqoop/conf/managers.d/connectors
>>
> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>
> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
>> com.cloudera.sqoop.manager.DefaultManagerFactory
>>
> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>
> 12/04/23 01:02:42 DEBUG manager.TeradataManagerFactory: Trying with
>> scheme: jdbc:teradata:
>>
> 12/04/23 01:02:42 INFO manager.SqlManager: Using default fetchSize of 1000
>>
> 12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Instantiated ConnManager
>> com.cloudera.sqoop.teradata.TeradataManager@220ca470
>>
> 12/04/23 01:02:42 INFO tool.CodeGenTool: Beginning code generation
>>
> 12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
>> java.lang.RuntimeException: Could not load db driver class:
>> com.teradata.jdbc.TeraDriver
>>
> java.lang.RuntimeException: Could not load db driver class:
>> com.teradata.jdbc.TeraDriver
>>
> at
>> com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:634)
>>
> at
>> com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:51)
>>
> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:456)
>>
> at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:479)
>>
> at
>> com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:192)
>>
> at
>> com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:176)
>>
> at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1017)
>>
> at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
>>
> at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
>>
> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
>>
> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>>
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>
> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>>
> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>>
> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>>
> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>
>
>>
> Thanks,
>
> Dipesh
>
>
>
> On Sun, Apr 22, 2012 at 3:01 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
>
>> Hi Dispesh,
>>
>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>>> terajdbc4.jar> along with
>>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>>> sqoop-td-connector-1.0.5.jar>
>>> in connectors file inside managers.d directory.
>>
>>
>> Please remove "com.teradata.jdbc.TeraDriver=<path to jar file
>> terajdbc4.jar>" from your connector file.
>>
>> It should only contain "com.cloudera.sqoop.manager.TeradataManagerFactory=<path
>> to sqoop-td-connector-1.0.5.jar>".
>>
>> Thanks,
>> Cheolsoo
>>
>>
>> On Sat, Apr 21, 2012 at 10:21 PM, DIPESH KUMAR SINGH <
>> dipesh.tech@gmail.com> wrote:
>>
>>> I am trying to import the data from Teradata to hdfs referring Cloudera
>>> Connector guide. I added the property
>>>
>>> (The Hadoop environment i am working in is a shared Hadoop installation,
>>> So users are not allowed to
>>> add any jars in lib directory or make changes in Sqoop-site.xml of
>>> central installation.
>>> However, users can use .profile i.e. user's configuration files to
>>> specify CLASSPATH )
>>>
>>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>>> terajdbc4.jar> along with
>>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>>> sqoop-td-connector-1.0.5.jar>
>>> in connectors file inside managers.d directory. In my CLASSPATH, i have
>>> added path to
>>> SQOOP_CONF i.e. a Sqoop folder where i have kept my managers.d and all
>>> jars.
>>>
>>>
>>> With these I suppose i am able to connect to teradata using the teradata
>>> manager factory using the teradriver as well
>>> but, still i am facing this when i issue the sqoop import command as
>>> below
>>>
>>> sqoop import
>>> -libjars <path to tdgssconfig.jar>
>>> --driver com.teradata.jdbc.TeraDriver
>>> --connect jdbc:teradata://xxxxxx/DATABASE=dbname
>>> --username hadoop --P
>>> --table SQOOP_TEST
>>> --num-mappers 3 --verbose
>>>
>>> 12/04/13 02:29:32 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>>> 12/04/13 02:29:32 WARN tool.BaseSqoopTool: Setting your password on the
>>> command-line is insecure. Consider using -P instead.
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>>> through URL: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
>>> java.net.URLClassLoader@6a8c436b
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
>>> JVM: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar
>>> /Sqoop/sqoop-td-connector-1.0.5.jar: java.net.FactoryURLClassLoader@45db05b2
>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>>> /Sqoop/sqoop-td-connector-1.0.5.jar specified by
>>> /Sqoop/conf/managers.d/connectors
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>>> class: com.teradata.jdbc.TeraDriver
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>>> through URL: jar:file:///Sqoop/lib/terajdbc4.jar!/
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
>>> java.net.FactoryURLClassLoader@45db05b2
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>>> com.teradata.jdbc.TeraDriver
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
>>> JVM: jar:file:///Sqoop/lib/terajdbc4.jar!/
>>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar
>>> /Sqoop/lib/terajdbc4.jar: java.net.FactoryURLClassLoader@518f5824
>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>>> com.teradata.jdbc.TeraDriver in jar /Sqoop/lib/terajdbc4.jar specified by
>>> /Sqoop/conf/managers.d/connectors
>>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Loaded manager factory:
>>> com.cloudera.sqoop.manager.TeradataManagerFactory
>>> 12/04/13 02:29:32 ERROR tool.BaseSqoopTool: Got error creating database
>>> manager: java.lang.ClassCastException: com.teradata.jdbc.TeraDriver cannot
>>> be cast to com.cloudera.sqoop.manager.ManagerFactory
>>> at
>>> com.cloudera.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:90)
>>> at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:60)
>>> at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:186)
>>> at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81)
>>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411)
>>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>>
>>>
>>> Please help.
>>>
>>> Thanks & Regards,
>>> --
>>> Dipesh Kr. Singh
>>>
>>>
>>>
>>>
>>>
>>
>
>
> --
> Dipesh Kr. Singh
>
>
>
>
>
Re: Error while importing from Teradata to HDFS
Posted by DIPESH KUMAR SINGH <di...@gmail.com>.
Hi Cheolsoo,
As you suggested, I tried after removing "TeraDriver" line from connectors
file but that didn't worked
too. Please help..!!
Here is the stack trace:
>
> 12/04/23 01:02:35 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>
Enter password:
>
12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Checking for existing class:
> com.cloudera.sqoop.manager.TeradataManagerFactory
>
12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Attempting to load jar
> through URL:
> jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>
12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Previous classloader is
> java.net.URLClassLoader@27b15692
>
12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Testing class in jar:
> com.cloudera.sqoop.manager.TeradataManagerFactory
>
12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Loaded jar into current JVM:
> jar:file:///ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar!/
>
12/04/23 01:02:42 DEBUG util.ClassLoaderStack: Added classloader for jar
> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar:
> java.net.FactoryURLClassLoader@19e3cd51
>
12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Added factory
> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
> /ms/user/a/sqoop-user/Sqoop/sqoop-td-connector-1.0.5.jar specified by
> /ms/user/a/sqoop-user/Sqoop/conf/managers.d/connectors
>
12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
> com.cloudera.sqoop.manager.TeradataManagerFactory
>
12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Loaded manager factory:
> com.cloudera.sqoop.manager.DefaultManagerFactory
>
12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
> com.cloudera.sqoop.manager.TeradataManagerFactory
>
12/04/23 01:02:42 DEBUG manager.TeradataManagerFactory: Trying with scheme:
> jdbc:teradata:
>
12/04/23 01:02:42 INFO manager.SqlManager: Using default fetchSize of 1000
>
12/04/23 01:02:42 DEBUG sqoop.ConnFactory: Instantiated ConnManager
> com.cloudera.sqoop.teradata.TeradataManager@220ca470
>
12/04/23 01:02:42 INFO tool.CodeGenTool: Beginning code generation
>
12/04/23 01:02:42 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.RuntimeException: Could not load db driver class:
> com.teradata.jdbc.TeraDriver
>
java.lang.RuntimeException: Could not load db driver class:
> com.teradata.jdbc.TeraDriver
>
at
> com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:634)
>
at
> com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:51)
>
at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:456)
>
at com.cloudera.sqoop.manager.SqlManager.execute(SqlManager.java:479)
>
at
> com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:192)
>
at com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:176)
>
at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1017)
>
at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
>
at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
>
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
>
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>
>
Thanks,
Dipesh
On Sun, Apr 22, 2012 at 3:01 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
> Hi Dispesh,
>
> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>> terajdbc4.jar> along with
>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>> sqoop-td-connector-1.0.5.jar>
>> in connectors file inside managers.d directory.
>
>
> Please remove "com.teradata.jdbc.TeraDriver=<path to jar file
> terajdbc4.jar>" from your connector file.
>
> It should only contain "com.cloudera.sqoop.manager.TeradataManagerFactory=<path
> to sqoop-td-connector-1.0.5.jar>".
>
> Thanks,
> Cheolsoo
>
>
> On Sat, Apr 21, 2012 at 10:21 PM, DIPESH KUMAR SINGH <
> dipesh.tech@gmail.com> wrote:
>
>> I am trying to import the data from Teradata to hdfs referring Cloudera
>> Connector guide. I added the property
>>
>> (The Hadoop environment i am working in is a shared Hadoop installation,
>> So users are not allowed to
>> add any jars in lib directory or make changes in Sqoop-site.xml of
>> central installation.
>> However, users can use .profile i.e. user's configuration files to
>> specify CLASSPATH )
>>
>> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
>> terajdbc4.jar> along with
>> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
>> sqoop-td-connector-1.0.5.jar>
>> in connectors file inside managers.d directory. In my CLASSPATH, i have
>> added path to
>> SQOOP_CONF i.e. a Sqoop folder where i have kept my managers.d and all
>> jars.
>>
>>
>> With these I suppose i am able to connect to teradata using the teradata
>> manager factory using the teradriver as well
>> but, still i am facing this when i issue the sqoop import command as below
>>
>> sqoop import
>> -libjars <path to tdgssconfig.jar>
>> --driver com.teradata.jdbc.TeraDriver
>> --connect jdbc:teradata://xxxxxx/DATABASE=dbname
>> --username hadoop --P
>> --table SQOOP_TEST
>> --num-mappers 3 --verbose
>>
>> 12/04/13 02:29:32 DEBUG tool.BaseSqoopTool: Enabled debug logging.
>> 12/04/13 02:29:32 WARN tool.BaseSqoopTool: Setting your password on the
>> command-line is insecure. Consider using -P instead.
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>> class: com.cloudera.sqoop.manager.TeradataManagerFactory
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>> through URL: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
>> java.net.URLClassLoader@6a8c436b
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>> com.cloudera.sqoop.manager.TeradataManagerFactory
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
>> JVM: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar
>> /Sqoop/sqoop-td-connector-1.0.5.jar: java.net.FactoryURLClassLoader@45db05b2
>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
>> /Sqoop/sqoop-td-connector-1.0.5.jar specified by
>> /Sqoop/conf/managers.d/connectors
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
>> class: com.teradata.jdbc.TeraDriver
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
>> through URL: jar:file:///Sqoop/lib/terajdbc4.jar!/
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
>> java.net.FactoryURLClassLoader@45db05b2
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
>> com.teradata.jdbc.TeraDriver
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
>> JVM: jar:file:///Sqoop/lib/terajdbc4.jar!/
>> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar
>> /Sqoop/lib/terajdbc4.jar: java.net.FactoryURLClassLoader@518f5824
>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
>> com.teradata.jdbc.TeraDriver in jar /Sqoop/lib/terajdbc4.jar specified by
>> /Sqoop/conf/managers.d/connectors
>> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Loaded manager factory:
>> com.cloudera.sqoop.manager.TeradataManagerFactory
>> 12/04/13 02:29:32 ERROR tool.BaseSqoopTool: Got error creating database
>> manager: java.lang.ClassCastException: com.teradata.jdbc.TeraDriver cannot
>> be cast to com.cloudera.sqoop.manager.ManagerFactory
>> at
>> com.cloudera.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:90)
>> at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:60)
>> at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:186)
>> at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81)
>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411)
>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>>
>>
>> Please help.
>>
>> Thanks & Regards,
>> --
>> Dipesh Kr. Singh
>>
>>
>>
>>
>>
>
--
Dipesh Kr. Singh
Re: Error while importing from Teradata to HDFS
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi Dispesh,
I added the line com.teradata.jdbc.TeraDriver=<path to jar file
> terajdbc4.jar> along with
> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
> sqoop-td-connector-1.0.5.jar>
> in connectors file inside managers.d directory.
Please remove "com.teradata.jdbc.TeraDriver=<path to jar file
terajdbc4.jar>" from your connector file.
It should only contain "com.cloudera.sqoop.manager.TeradataManagerFactory=<path
to sqoop-td-connector-1.0.5.jar>".
Thanks,
Cheolsoo
On Sat, Apr 21, 2012 at 10:21 PM, DIPESH KUMAR SINGH
<di...@gmail.com>wrote:
> I am trying to import the data from Teradata to hdfs referring Cloudera
> Connector guide. I added the property
>
> (The Hadoop environment i am working in is a shared Hadoop installation,
> So users are not allowed to
> add any jars in lib directory or make changes in Sqoop-site.xml of central
> installation.
> However, users can use .profile i.e. user's configuration files to specify
> CLASSPATH )
>
> I added the line com.teradata.jdbc.TeraDriver=<path to jar file
> terajdbc4.jar> along with
> com.cloudera.sqoop.manager.TeradataManagerFactory=<path to
> sqoop-td-connector-1.0.5.jar>
> in connectors file inside managers.d directory. In my CLASSPATH, i have
> added path to
> SQOOP_CONF i.e. a Sqoop folder where i have kept my managers.d and all
> jars.
>
>
> With these I suppose i am able to connect to teradata using the teradata
> manager factory using the teradriver as well
> but, still i am facing this when i issue the sqoop import command as below
>
> sqoop import
> -libjars <path to tdgssconfig.jar>
> --driver com.teradata.jdbc.TeraDriver
> --connect jdbc:teradata://xxxxxx/DATABASE=dbname
> --username hadoop --P
> --table SQOOP_TEST
> --num-mappers 3 --verbose
>
> 12/04/13 02:29:32 DEBUG tool.BaseSqoopTool: Enabled debug logging.
> 12/04/13 02:29:32 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
> class: com.cloudera.sqoop.manager.TeradataManagerFactory
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
> through URL: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
> java.net.URLClassLoader@6a8c436b
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
> com.cloudera.sqoop.manager.TeradataManagerFactory
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
> JVM: jar:file:///Sqoop/sqoop-td-connector-1.0.5.jar!/
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar
> /Sqoop/sqoop-td-connector-1.0.5.jar: java.net.FactoryURLClassLoader@45db05b2
> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
> com.cloudera.sqoop.manager.TeradataManagerFactory in jar
> /Sqoop/sqoop-td-connector-1.0.5.jar specified by
> /Sqoop/conf/managers.d/connectors
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Checking for existing
> class: com.teradata.jdbc.TeraDriver
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Attempting to load jar
> through URL: jar:file:///Sqoop/lib/terajdbc4.jar!/
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Previous classloader is
> java.net.FactoryURLClassLoader@45db05b2
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Testing class in jar:
> com.teradata.jdbc.TeraDriver
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Loaded jar into current
> JVM: jar:file:///Sqoop/lib/terajdbc4.jar!/
> 12/04/13 02:29:32 DEBUG util.ClassLoaderStack: Added classloader for jar
> /Sqoop/lib/terajdbc4.jar: java.net.FactoryURLClassLoader@518f5824
> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Added factory
> com.teradata.jdbc.TeraDriver in jar /Sqoop/lib/terajdbc4.jar specified by
> /Sqoop/conf/managers.d/connectors
> 12/04/13 02:29:32 DEBUG sqoop.ConnFactory: Loaded manager factory:
> com.cloudera.sqoop.manager.TeradataManagerFactory
> 12/04/13 02:29:32 ERROR tool.BaseSqoopTool: Got error creating database
> manager: java.lang.ClassCastException: com.teradata.jdbc.TeraDriver cannot
> be cast to com.cloudera.sqoop.manager.ManagerFactory
> at com.cloudera.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:90)
> at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:60)
> at com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:186)
> at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:81)
> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:411)
> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:219)
> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:228)
> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:237)
>
>
> Please help.
>
> Thanks & Regards,
> --
> Dipesh Kr. Singh
>
>
>
>
>