You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Madhan Neethiraj (JIRA)" <ji...@apache.org> on 2017/01/04 09:35:58 UTC
[jira] [Updated] (ATLAS-1051) Sqoop Hook does not package HDFS
model jars which is required
[ https://issues.apache.org/jira/browse/ATLAS-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Madhan Neethiraj updated ATLAS-1051:
------------------------------------
Fix Version/s: 0.7.1-incubating
Committed to 0.7-incubating branch: http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/3f853af0892cef927bf737ef35e0eddc968e8b9d
> Sqoop Hook does not package HDFS model jars which is required
> -------------------------------------------------------------
>
> Key: ATLAS-1051
> URL: https://issues.apache.org/jira/browse/ATLAS-1051
> Project: Atlas
> Issue Type: Bug
> Affects Versions: 0.7-incubating, 0.8-incubating
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Fix For: 0.8-incubating, 0.7.1-incubating
>
> Attachments: ATLAS-1051.patch
>
>
> Sqoop hook should package hdfs model jar and when sqoop commands to import are run, see this error since hdfs model jar is missing resulting in errors while loading data.
> {noformat}
> 2016-07-22 17:06:36,181 DEBUG [main]: ql.Driver (DriverContext.java:shutdown(132)) - Shutting down query
> LOAD DATA INPATH 'hdfs://localhost.localdomain:8020/user/hive/x_service' INTO TABLE `x_service`
> 2016-07-22 17:06:36,181 INFO [main]: CliDriver (SessionState.java:printInfo(939)) - Time taken: 4.791 seconds
> 2016-07-22 17:06:36,181 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(148)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
> 2016-07-22 17:06:36,181 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(176)) - </PERFLOG method=releaseLocks start=1469232396181 end=1469232396181 duration=0 from=org.apache.hadoop.hive.ql.Driver>
> 2016-07-22 17:06:36,182 DEBUG [main]: session.SessionState (SessionState.java:close(1452)) - Removing resource dir /tmp/f60f6488-691f-440d-8fa0-01e26873f8a5_resources
> 2016-07-22 17:06:36,191 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(115)) - ==> AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path)
> 2016-07-22 17:06:36,192 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(123)) - AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): calling pluginClassLoader.loadClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(79)) - ==> AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path)
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(87)) - AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path): calling pluginClassLoader.findClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(97)) - AtlasPluginClassLoader.findClass(org.apache.hadoop.fs.Path): calling componentClassLoader.findClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(133)) - AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): calling componentClassLoader.loadClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(142)) - <== AtlasPluginClassLoader.loadClass(org.apache.hadoop.fs.Path): class org.apache.hadoop.fs.Path
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: hook.HiveHook (HiveHook.java:processHiveEntity(632)) - Registering DFS Path hdfs://localhost.localdomain:8020/user/hive/x_service
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(115)) - ==> AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes)
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(123)) - AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes): calling pluginClassLoader.loadClass()
> 2016-07-22 17:06:36,193 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(79)) - ==> AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes)
> 2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(87)) - AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes): calling pluginClassLoader.findClass()
> 2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:findClass(97)) - AtlasPluginClassLoader.findClass(org.apache.atlas.fs.model.FSDataTypes): calling componentClassLoader.findClass()
> 2016-07-22 17:06:36,194 DEBUG [Atlas Logger 0]: classloader.AtlasPluginClassLoader (AtlasPluginClassLoader.java:loadClass(133)) - AtlasPluginClassLoader.loadClass(org.apache.atlas.fs.model.FSDataTypes): calling componentClassLoader.loadClass()
> 2016-07-22 17:06:36,197 ERROR [Atlas Logger 0]: hook.HiveHook (HiveHook.java:run(188)) - Atlas hook failed due to error
> java.lang.NoClassDefFoundError: org/apache/atlas/fs/model/FSDataTypes
> at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.fillHDFSDataSet(HiveMetaStoreBridge.java:556)
> at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:634)
> at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:599)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:227)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:82)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:186)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: org.apache.atlas.fs.model.FSDataTypes
> at java.lang.ClassLoader.findClass(ClassLoader.java:530)
> at org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:333)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:137)
> ... 11 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)