You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "Jihoon Son (JIRA)" <ji...@apache.org> on 2015/01/08 16:41:34 UTC
[jira] [Created] (TAJO-1289) History reader fails to get the query
information after a successful query execution
Jihoon Son created TAJO-1289:
--------------------------------
Summary: History reader fails to get the query information after a successful query execution
Key: TAJO-1289
URL: https://issues.apache.org/jira/browse/TAJO-1289
Project: Tajo
Issue Type: Bug
Components: tajo master
Reporter: Jihoon Son
Priority: Critical
Fix For: 0.10
When a query execution is successfully finished, its status is written into a query history. Once the history is written, the tajo master gets the status of the finished query from that written history.
Here, when the tajo master reads the query history, an I/O exception occurs and the tajo cli stops forever. Here is the full stack trace.
{noformat}
2015-01-09 00:27:54,019 INFO org.apache.tajo.master.querymaster.QueryInProgress: Stop query:q_1420730837752_0001
2015-01-09 00:27:54,019 INFO org.apache.tajo.master.rm.TajoWorkerResourceManager: Release Resource: 0.0,512
2015-01-09 00:27:54,019 INFO org.apache.tajo.master.rm.TajoWorkerResourceManager: Released QueryMaster (q_1420730837752_0001) resource.
2015-01-09 00:27:54,019 INFO org.apache.tajo.master.querymaster.QueryInProgress: q_1420730837752_0001 QueryMaster stopped
2015-01-09 00:27:54,032 INFO org.apache.tajo.util.history.HistoryWriter: Create query history file: hdfs://localhost:7020/tmp/tajo-jihoon/staging/history/20150109/query-list/query-list-002754.hist
2015-01-09 00:27:54,899 ERROR org.apache.tajo.util.history.HistoryReader: Reading error:hdfs://localhost:7020/tmp/tajo-jihoon/staging/history/20150107/query-list/query-list-131932.hist, Cannot obtain block length for LocatedBlock{BP-1604697128-192.168.0.12-1412676388616:blk_1073741964_1140; getBlockSize()=1356; corrupt=false; offset=0; locs=[127.0.0.1:50010]}
java.io.IOException: Cannot obtain block length for LocatedBlock{BP-1604697128-192.168.0.12-1412676388616:blk_1073741964_1140; getBlockSize()=1356; corrupt=false; offset=0; locs=[127.0.0.1:50010]}
at org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:350)
at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:294)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:231)
at org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:224)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1295)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:300)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:296)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:296)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
at org.apache.tajo.util.history.HistoryReader.getQueries(HistoryReader.java:88)
at org.apache.tajo.util.history.HistoryReader.getQueryInfo(HistoryReader.java:294)
at org.apache.tajo.master.querymaster.QueryJobManager.getFinishedQuery(QueryJobManager.java:131)
at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.getQueryStatus(TajoMasterClientService.java:471)
at org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:551)
at org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)