You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Giovanni Ruggiero (JIRA)" <ji...@apache.org> on 2012/08/07 15:10:11 UTC
[jira] [Created] (SQOOP-542) Error in importing data in Avro format
Giovanni Ruggiero created SQOOP-542:
---------------------------------------
Summary: Error in importing data in Avro format
Key: SQOOP-542
URL: https://issues.apache.org/jira/browse/SQOOP-542
Project: Sqoop
Issue Type: Bug
Affects Versions: 1.4.1-incubating, 1.4.2
Environment: Mac Os X 10.7
Sqoop: sqoop-1.4.2-incubating-SNAPSHOT.jar(built from trunk with -Dhadoopversion=20)
Hadoop: hadoop-0.20.2-cdh3u4
Reporter: Giovanni Ruggiero
When performing an import as avro datafile, Sqoop fails with the following exception:
{code}
Error: java.lang.ClassNotFoundException: org.apache.avro.io.DatumWriter
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:951)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1002)
at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:253)
at org.apache.hadoop.mapred.Task.initialize(Task.java:509)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security
attempt_201208071052_0007_m_000004_2: 2012-08-07 13:51:05.324 java[8527:1903] Unable to load realm info from SCDynamicStore
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-542) Error in importing data in Avro
format
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448980#comment-13448980 ]
Cheolsoo Park commented on SQOOP-542:
-------------------------------------
I am going to close this jira unless anyone has objections.
Thanks!
> Error in importing data in Avro format
> --------------------------------------
>
> Key: SQOOP-542
> URL: https://issues.apache.org/jira/browse/SQOOP-542
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.1-incubating, 1.4.2
> Environment: Mac Os X 10.7
> Sqoop: sqoop-1.4.2-incubating-SNAPSHOT.jar(built from trunk with -Dhadoopversion=20)
> Hadoop: hadoop-0.20.2-cdh3u4
> Reporter: Giovanni Ruggiero
>
> When performing an import as avro datafile, Sqoop fails with the following exception:
> {code}
> Error: java.lang.ClassNotFoundException: org.apache.avro.io.DatumWriter
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:951)
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1002)
> at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:253)
> at org.apache.hadoop.mapred.Task.initialize(Task.java:509)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security
> attempt_201208071052_0007_m_000004_2: 2012-08-07 13:51:05.324 java[8527:1903] Unable to load realm info from SCDynamicStore
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-542) Error in importing data in Avro
format
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448186#comment-13448186 ]
Cheolsoo Park commented on SQOOP-542:
-------------------------------------
Hi Giovanni,
This is because the avro jar file (avro-1.5.3.jar) is not present in classpath.
Given that you're building Sqoop from the source, I assume that you're running "ant clean *jar* -Dhadoopversion=20" from the top directory where you build sqoop. If so, you need to put dependency jars in the lib directory by yourself, or you will get ClassNotFound error.
Instead, you could run "ant clean *package* -Dhadoopversion=20". This command will create the following directory for you:
{code}
<top directory>/build/sqoop-1.4.3-SNAPSHOT.bin__hadoop-0.20
{code}
And you can find dependency jars are downloaded for you:
{code}
cheolsoo@localhost:~/workspace/sqoop1/build/sqoop-1.4.3-SNAPSHOT.bin__hadoop-0.20 $ls -1 lib
ant-contrib-1.0b3.jar
ant-eclipse-1.0-jvm1.2.jar
avro-1.5.3.jar
avro-ipc-1.5.3.jar
avro-mapred-1.5.3.jar
commons-io-1.4.jar
hsqldb-1.8.0.10.jar
jackson-core-asl-1.7.3.jar
jackson-mapper-asl-1.7.3.jar
jopt-simple-3.2.jar
mysql-connector-java-5.1.18-bin.jar
paranamer-2.3.jar
snappy-java-1.0.3.2.jar
{code}
Using build/sqoop-1.4.3-SNAPSHOT.bin__hadoop-0.20 as the sqoop home directory, you will no longer have this problem.
Lastly, if you don't want to deal with dependency jars at all, you can download the binaries from:
http://www.gtlib.gatech.edu/pub/apache/sqoop/1.4.2/sqoop-1.4.2.bin__hadoop-0.20.tar.gz
When you untar the tarball, you will see the dependency jars are present in the lib directory for you:
{code}
cheolsoo@localhost:~/workspace/sqoop-1.4.2.bin__hadoop-0.20 $ls -1 lib
ant-contrib-1.0b3.jar
ant-eclipse-1.0-jvm1.2.jar
avro-1.5.3.jar
avro-ipc-1.5.3.jar
avro-mapred-1.5.3.jar
commons-io-1.4.jar
hsqldb-1.8.0.10.jar
jackson-core-asl-1.7.3.jar
jackson-mapper-asl-1.7.3.jar
jopt-simple-3.2.jar
paranamer-2.3.jar
snappy-java-1.0.3.2.jar
{code}
> Error in importing data in Avro format
> --------------------------------------
>
> Key: SQOOP-542
> URL: https://issues.apache.org/jira/browse/SQOOP-542
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.1-incubating, 1.4.2
> Environment: Mac Os X 10.7
> Sqoop: sqoop-1.4.2-incubating-SNAPSHOT.jar(built from trunk with -Dhadoopversion=20)
> Hadoop: hadoop-0.20.2-cdh3u4
> Reporter: Giovanni Ruggiero
>
> When performing an import as avro datafile, Sqoop fails with the following exception:
> {code}
> Error: java.lang.ClassNotFoundException: org.apache.avro.io.DatumWriter
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:951)
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1002)
> at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:253)
> at org.apache.hadoop.mapred.Task.initialize(Task.java:509)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security
> attempt_201208071052_0007_m_000004_2: 2012-08-07 13:51:05.324 java[8527:1903] Unable to load realm info from SCDynamicStore
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-542) Error in importing data in Avro
format
Posted by "Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448989#comment-13448989 ]
Jarek Jarcec Cecho commented on SQOOP-542:
------------------------------------------
No objections on my side.
Jarcec
> Error in importing data in Avro format
> --------------------------------------
>
> Key: SQOOP-542
> URL: https://issues.apache.org/jira/browse/SQOOP-542
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.1-incubating, 1.4.2
> Environment: Mac Os X 10.7
> Sqoop: sqoop-1.4.2-incubating-SNAPSHOT.jar(built from trunk with -Dhadoopversion=20)
> Hadoop: hadoop-0.20.2-cdh3u4
> Reporter: Giovanni Ruggiero
>
> When performing an import as avro datafile, Sqoop fails with the following exception:
> {code}
> Error: java.lang.ClassNotFoundException: org.apache.avro.io.DatumWriter
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:951)
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1002)
> at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:253)
> at org.apache.hadoop.mapred.Task.initialize(Task.java:509)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security
> attempt_201208071052_0007_m_000004_2: 2012-08-07 13:51:05.324 java[8527:1903] Unable to load realm info from SCDynamicStore
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (SQOOP-542) Error in importing data in Avro
format
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park resolved SQOOP-542.
---------------------------------
Resolution: Not A Problem
> Error in importing data in Avro format
> --------------------------------------
>
> Key: SQOOP-542
> URL: https://issues.apache.org/jira/browse/SQOOP-542
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.1-incubating, 1.4.2
> Environment: Mac Os X 10.7
> Sqoop: sqoop-1.4.2-incubating-SNAPSHOT.jar(built from trunk with -Dhadoopversion=20)
> Hadoop: hadoop-0.20.2-cdh3u4
> Reporter: Giovanni Ruggiero
>
> When performing an import as avro datafile, Sqoop fails with the following exception:
> {code}
> Error: java.lang.ClassNotFoundException: org.apache.avro.io.DatumWriter
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:951)
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1002)
> at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:253)
> at org.apache.hadoop.mapred.Task.initialize(Task.java:509)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:306)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security
> attempt_201208071052_0007_m_000004_2: 2012-08-07 13:51:05.324 java[8527:1903] Unable to load realm info from SCDynamicStore
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira