You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Manish Bhoge <ma...@rocketmail.com> on 2012/04/05 22:14:36 UTC
Hive table creation error
When i am creating table from comma delimited csv file then i am getting below error:
Failed to open file '/user/manish/minfo.csv': Could not read block Block(genStamp=1094, blockId=6807603852292275080L, numBytes=44429, token='AAAAAA', startOffset=0, path=u'/user/manish/minfo.csv', nodes=[DatanodeInfo(xceiverCount=1, millisSinceUpdate=344, capacity=486933626880L, name='127.0.0.1:50010', thriftPort=32959, state=1, remaining=456354410496L, httpPort=50075, host='127.0.0.1', storageID='DS-78851148-10.135.211.227-50010-1332868668762', dfsUsed=307200)]) from any replicas: [IOException(msg='Unknown exception in doAs', stack='org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1170)\norg.apache.hadoop.thriftfs.ThriftHandlerBase.assumeUserContextAndExecute(ThriftHandlerBase.java:196)\norg.apache.hadoop.thriftfs.DatanodePlugin$ThriftHandler.readBlock(DatanodePlugin.java:87)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:616)\norg.apache.hadoop.thriftfs.ThriftUtils$SecurityCheckingProxy.invoke(ThriftUtils.java:228)\n$Proxy7.readBlock(Unknown Source)\norg.apache.hadoop.thriftfs.api.Datanode$Processor$readBlock.process(Datanode.java:249)\norg.apache.hadoop.thriftfs.api.Datanode$Processor.process(Datanode.java:227)\norg.apache.hadoop.thriftfs.SanerThreadPoolServer$WorkerProcess.run(SanerThreadPoolServer.java:277)\njava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\njava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\njava.lang.Thread.run(Thread.java:636)\n', clazz='java.lang.reflect.UndeclaredThrowableExceptio)]
Thank You,
Manish
Re: Hive table creation error
Posted by Ashwanth Kumar <as...@googlemail.com>.
Seems most likely that the CSV file block is lost or unreadable. Quick
questions?
- What is the dfs.replication in your Hadoop cluster?
- Can you share the Query used for creating the table?
- Can you load the same data from local path rather than from the HDFS?
On Fri, Apr 6, 2012 at 1:44 AM, Manish Bhoge <ma...@rocketmail.com>wrote:
> When i am creating table from comma delimited csv file then i am getting
> below error:
>
> Failed to open file '/user/manish/minfo.csv': Could not read block
> Block(genStamp=1094, blockId=6807603852292275080L, numBytes=44429,
> token='AAAAAA', startOffset=0, path=u'/user/manish/minfo.csv',
> nodes=[DatanodeInfo(xceiverCount=1, millisSinceUpdate=344,
> capacity=486933626880L, name='127.0.0.1:50010', thriftPort=32959,
> state=1, remaining=456354410496L, httpPort=50075, host='127.0.0.1',
> storageID='DS-78851148-10.135.211.227-50010-1332868668762',
> dfsUsed=307200)]) from any replicas: [IOException(msg='Unknown exception in
> doAs',
> stack='org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1170)\norg.apache.hadoop.thriftfs.ThriftHandlerBase.assumeUserContextAndExecute(ThriftHandlerBase.java:196)\norg.apache.hadoop.thriftfs.DatanodePlugin$ThriftHandler.readBlock(DatanodePlugin.java:87)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:616)\norg.apache.hadoop.thriftfs.ThriftUtils$SecurityCheckingProxy.invoke(ThriftUtils.java:228)\n$Proxy7.readBlock(Unknown
> Source)\norg.apache.hadoop.thriftfs.api.Datanode$Processor$readBlock.process(Datanode.java:249)\norg.apache.hadoop.thriftfs.api.Datanode$Processor.process(Datanode.java:227)\norg.apache.hadoop.thriftfs.SanerThreadPoolServer$WorkerProcess.run(SanerThreadPoolServer.java:277)\njava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)\njava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)\njava.lang.Thread.run(Thread.java:636)\n',
> clazz='java.lang.reflect.UndeclaredThrowableExceptio)]
>
> Thank You,
> Manish
>
--
Ashwanth Kumar / ashwanthkumar.in