You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Abraham Elmahrek (JIRA)" <ji...@apache.org> on 2015/03/26 09:21:53 UTC
[jira] [Updated] (SQOOP-2238) Sqoop2: Parquet integration - method
not found
[ https://issues.apache.org/jira/browse/SQOOP-2238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Abraham Elmahrek updated SQOOP-2238:
------------------------------------
Description:
{code}
2015-03-25 23:49:10,831 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0018:Error occurs during loader run
at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:261)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.NoClassDefFoundError: parquet/hadoop/metadata/CompressionCodecName
at org.kitesdk.data.spi.filesystem.ParquetAppender.open(ParquetAppender.java:62)
at org.kitesdk.data.spi.filesystem.FileSystemWriter.initialize(FileSystemWriter.java:135)
at org.kitesdk.data.spi.filesystem.FileSystemView.newWriter(FileSystemView.java:101)
at org.kitesdk.data.spi.AbstractDataset.newWriter(AbstractDataset.java:58)
at org.apache.sqoop.connector.kite.KiteDatasetExecutor.getOrNewWriter(KiteDatasetExecutor.java:91)
at org.apache.sqoop.connector.kite.KiteDatasetExecutor.writeRecord(KiteDatasetExecutor.java:86)
at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:72)
at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:36)
at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:250)
... 6 more
Caused by: java.lang.ClassNotFoundException: parquet.hadoop.metadata.CompressionCodecName
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 15 more
{code}
was:
{code}
2015-03-19 11:26:33,910 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0018:Error occurs during loader run
at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:261)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.NoSuchMethodError: parquet.avro.AvroParquetWriter.<init>(Lorg/apache/hadoop/fs/Path;Lorg/apache/avro/Schema;Lparquet/hadoop/metadata/CompressionCodecName;IIZLorg/apache/hadoop/conf/Configuration;)V
at org.kitesdk.data.spi.filesystem.ParquetAppender.open(ParquetAppender.java:66)
at org.kitesdk.data.spi.filesystem.FileSystemWriter.initialize(FileSystemWriter.java:135)
at org.kitesdk.data.spi.filesystem.FileSystemView.newWriter(FileSystemView.java:101)
at org.kitesdk.data.spi.AbstractDataset.newWriter(AbstractDataset.java:58)
at org.apache.sqoop.connector.kite.KiteDatasetExecutor.getOrNewWriter(KiteDatasetExecutor.java:89)
at org.apache.sqoop.connector.kite.KiteDatasetExecutor.writeRecord(KiteDatasetExecutor.java:84)
at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:72)
at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:36)
at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:250)
... 6 more
{code}
I had an extra jar on my classpath, but it looks like a bunch of jars are missing in this connector.
> Sqoop2: Parquet integration - method not found
> ----------------------------------------------
>
> Key: SQOOP-2238
> URL: https://issues.apache.org/jira/browse/SQOOP-2238
> Project: Sqoop
> Issue Type: Bug
> Components: sqoop2-kite-connector
> Reporter: Abraham Elmahrek
>
> {code}
> 2015-03-25 23:49:10,831 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0018:Error occurs during loader run
> at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:261)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: java.lang.NoClassDefFoundError: parquet/hadoop/metadata/CompressionCodecName
> at org.kitesdk.data.spi.filesystem.ParquetAppender.open(ParquetAppender.java:62)
> at org.kitesdk.data.spi.filesystem.FileSystemWriter.initialize(FileSystemWriter.java:135)
> at org.kitesdk.data.spi.filesystem.FileSystemView.newWriter(FileSystemView.java:101)
> at org.kitesdk.data.spi.AbstractDataset.newWriter(AbstractDataset.java:58)
> at org.apache.sqoop.connector.kite.KiteDatasetExecutor.getOrNewWriter(KiteDatasetExecutor.java:91)
> at org.apache.sqoop.connector.kite.KiteDatasetExecutor.writeRecord(KiteDatasetExecutor.java:86)
> at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:72)
> at org.apache.sqoop.connector.kite.KiteLoader.load(KiteLoader.java:36)
> at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:250)
> ... 6 more
> Caused by: java.lang.ClassNotFoundException: parquet.hadoop.metadata.CompressionCodecName
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 15 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)