You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "Jaehwa Jung (JIRA)" <ji...@apache.org> on 2015/09/17 09:21:45 UTC

[jira] [Created] (TAJO-1865) When building filtered partition path, check partition directory exists on filesystem.

Jaehwa Jung created TAJO-1865:
---------------------------------

             Summary: When building filtered partition path, check partition directory exists on filesystem.
                 Key: TAJO-1865
                 URL: https://issues.apache.org/jira/browse/TAJO-1865
             Project: Tajo
          Issue Type: Sub-task
          Components: Planner/Optimizer
            Reporter: Jaehwa Jung
            Assignee: Jaehwa Jung


When building filtered partition path, Tajo need to check the path exists on file system. Currently, if the partition exists on catalog and it doesn't exist on file system, users would fail to execute a count all query as following:

{code:xml}
2015-09-17 15:54:50,219 ERROR org.apache.tajo.master.GlobalEngine: 
Stack Trace:
org.apache.tajo.exception.TajoInternalError: internal error: File does not exist: /tajo/warehouse/tpch_partition/customer/c_nationkey=9
	at org.apache.hadoop.hdfs.server.namenode.FSDirectory.getContentSummary(FSDirectory.java:2157)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getContentSummary(FSNamesystem.java:4398)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getContentSummary(NameNodeRpcServer.java:998)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getContentSummary(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)

	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter.updateTableStat(PartitionedTableRewriter.java:439)
	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter.access$100(PartitionedTableRewriter.java:49)
	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter$Rewriter.visitScan(PartitionedTableRewriter.java:523)
	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter$Rewriter.visitScan(PartitionedTableRewriter.java:508)
	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:110)
	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visitGroupBy(BasicLogicalPlanVisitor.java:221)
	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:83)
	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visitRoot(BasicLogicalPlanVisitor.java:163)
	at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:62)
	at org.apache.tajo.plan.rewrite.rules.PartitionedTableRewriter.rewrite(PartitionedTableRewriter.java:82)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)