You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "Mingda Li (JIRA)" <ji...@apache.org> on 2016/11/16 18:01:07 UTC
[jira] [Updated] (ASTERIXDB-1735) Unable to load HDFS data to
AsterixDB
[ https://issues.apache.org/jira/browse/ASTERIXDB-1735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mingda Li updated ASTERIXDB-1735:
---------------------------------
Summary: Unable to load HDFS data to AsterixDB (was: Unable to load HDFS to AsterixDB)
> Unable to load HDFS data to AsterixDB
> -------------------------------------
>
> Key: ASTERIXDB-1735
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1735
> Project: Apache AsterixDB
> Issue Type: Bug
> Reporter: Mingda Li
>
> I met a problem when loading data use following query:
> use dataverse tpcds3;
> load dataset inventory
> using hdfs(("hdfs"="hdfs://SCAI01.CS.UCLA.EDU:9000"),("path"="/clash/datasets/tpcds/10/inventory"),("format"="delimited-text"),("delimiter"="|"));
> The Error in web interface is:
> Internal error. Please check instance logs for further details. [NullPointerException]
> I check the cc.log (cluster controller) and find the following problem:
> SEVERE: Unable to create adapter\
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: Unable to create adapter\
> at org.apache.asterix.metadata.declared.AqlMetadataProvider.getConfiguredAdapterFactory(AqlMetadataProvider.java:990)\
> at org.apache.asterix.metadata.declared.LoadableDataSource.buildDatasourceScanRuntime(LoadableDataSource.java:141)\
> at org.apache.asterix.metadata.declared.AqlMetadataProvider.getScannerRuntime(AqlMetadataProvider.java:383)\
> at org.apache.hyracks.algebricks.core.algebra.operators.physical.DataSourceScanPOperator.contributeRuntimeOperator(DataSourceScanPOperator.java:112)\
> at org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator.contributeRuntimeOperator(AbstractLogicalOperator.java:166)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:98)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compilePlan(PlanCompiler.java:61)\
> at org.apache.hyracks.algebricks.compiler.api.HeuristicCompilerFactoryBuilder$1$1.createJob(HeuristicCompilerFactoryBuilder.java:107)\
> at org.apache.asterix.api.common.APIFramework.compileQuery(APIFramework.java:344)\
> at org.apache.asterix.app.translator.QueryTranslator.handleLoadStatement(QueryTranslator.java:1845)\
> at org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:341)\
> at org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:268)\
> at org.apache.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:132)\
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)\
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)\
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)\
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)\
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\
> at org.eclipse.jetty.server.Server.handle(Server.java:524)\
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)\
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)\
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\
> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\
> at java.lang.Thread.run(Thread.java:745)\
> Caused by: java.lang.NullPointerException\
> at org.apache.asterix.external.util.HDFSUtils.getInputFormatClassName(HDFSUtils.java:155)\
> at org.apache.asterix.external.util.HDFSUtils.configureHDFSJobConf(HDFSUtils.java:186)\
> at org.apache.asterix.external.input.HDFSDataSourceFactory.configure(HDFSDataSourceFactory.java:83)\
> at org.apache.asterix.external.adapter.factory.GenericAdapterFactory.configure(GenericAdapterFactory.java:139)\
> at org.apache.asterix.external.provider.AdapterFactoryProvider.getAdapterFactory(AdapterFactoryProvider.java:49)\
> at org.apache.asterix.metadata.declared.AqlMetadataProvider.getConfiguredAdapterFactory(AqlMetadataProvider.java:969)\
> ... 45 more\
> \
> 8267828 [qtp1281025083-167] DEBUG org.eclipse.jetty.server.Server - RESPONSE for / h=true\
> 200 null\
> Date: Wed, 16 Nov 2016 04:34:31 GMT
> I change the query by adding the ("input-format"="text-input-format"), and the error becomes:
> Message
> Internal error. Please check instance logs for further details. [EOFException]
> The log file for it is:
> SEVERE: Unable to create adapter
> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: Unable to create adapter
> at org.apache.asterix.metadata.declared.AqlMetadataProvider.getConfiguredAdapterFactory(AqlMetadataProvider.java:990)
> at org.apache.asterix.metadata.declared.LoadableDataSource.buildDatasourceScanRuntime(LoadableDataSource.java:141)
> at org.apache.asterix.metadata.declared.AqlMetadataProvider.getScannerRuntime(AqlMetadataProvider.java:383)
> at org.apache.hyracks.algebricks.core.algebra.operators.physical.DataSourceScanPOperator.contributeRuntimeOperator(DataSourceScanPOperator.java:112)
> at org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator.contributeRuntimeOperator(AbstractLogicalOperator.java:166)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:98)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
> at org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compilePlan(PlanCompiler.java:61)
> at org.apache.hyracks.algebricks.compiler.api.HeuristicCompilerFactoryBuilder$1$1.createJob(HeuristicCompilerFactoryBuilder.java:107)
> at org.apache.asterix.api.common.APIFramework.compileQuery(APIFramework.java:344)
> at org.apache.asterix.app.translator.QueryTranslator.handleLoadStatement(QueryTranslator.java:1845)
> at org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:341)
> at org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:268)
> at org.apache.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:132)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:524)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.asterix.common.exceptions.AsterixException: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "SCAI01/131.179.64.20"; destination host is: "SCAI01.CS.UCLA.EDU":9000;
> at org.apache.asterix.external.input.HDFSDataSourceFactory.configure(HDFSDataSourceFactory.java:112)
> at org.apache.asterix.external.adapter.factory.GenericAdapterFactory.configure(GenericAdapterFactory.java:139)
> at org.apache.asterix.external.provider.AdapterFactoryProvider.getAdapterFactory(AdapterFactoryProvider.java:49)
> at org.apache.asterix.metadata.declared.AqlMetadataProvider.getConfiguredAdapterFactory(AqlMetadataProvider.java:969)
> ... 45 more
> Caused by: java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "SCAI01/131.179.64.20"; destination host is: "SCAI01.CS.UCLA.EDU":9000;
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
> at org.apache.hadoop.ipc.Client.call(Client.java:1351)
> at org.apache.hadoop.ipc.Client.call(Client.java:1300)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy18.getFileInfo(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy18.getFileInfo(Unknown Source)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:651)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1679)
> at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1106)
> at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1102)
> at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1102)
> at org.apache.hadoop.fs.FileSystem.globStatusInternal(FileSystem.java:1701)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1647)
> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:222)
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:270)
> at org.apache.asterix.external.input.HDFSDataSourceFactory.configure(HDFSDataSourceFactory.java:90)
> ... 48 more
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:392)
> at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:995)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:891)
> 11147300 [qtp1281025083-525] DEBUG org.eclipse.jetty.server.Server - RESPONSE for / h=true
> 200 null
> Date: Wed, 16 Nov 2016 05:22:30 GMT
> Content-Type: text/html;charset=utf-8
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)