You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "Jinho Kim (JIRA)" <ji...@apache.org> on 2016/01/11 03:02:39 UTC
[jira] [Commented] (TAJO-2047) Implement S3TableSpace::commitTable.
[ https://issues.apache.org/jira/browse/TAJO-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15091334#comment-15091334 ]
Jinho Kim commented on TAJO-2047:
---------------------------------
Guys,
I did report to TAJO-1905. please refer to description
> Implement S3TableSpace::commitTable.
> ------------------------------------
>
> Key: TAJO-2047
> URL: https://issues.apache.org/jira/browse/TAJO-2047
> Project: Tajo
> Issue Type: Sub-task
> Components: S3, Storage
> Reporter: Jaehwa Jung
> Assignee: Jaehwa Jung
>
> Query hangs while executing CTAS for partition table on aws as following:
> * Root directory exists on s3
> * Dataset : tpc-h 1G
> * Query
> {code:xml}
> CREATE TABLE LINEITEM_P (L_ORDERKEY bigint, L_PARTKEY bigint, L_SUPPKEY bigint, L_LINENUMBER bigint, L_QUANTITY double, L_EXTENDEDPRICE double, L_DISCOUNT double, L_TAX double, L_RETURNFLAG text, L_LINESTATUS text, L_COMMITDATE text, L_RECEIPTDATE text, L_SHIPINSTRUCT text, L_SHIPMODE text, L_COMMENT text)
> USING TEXT WITH ('text.delimiter'='|')
> PARTITION BY COLUMN (L_SHIPDATE text) AS
> SELECT L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_LINENUMBER, L_QUANTITY, L_EXTENDEDPRICE, L_DISCOUNT, L_TAX, L_RETURNFLAG, L_LINESTATUS, L_COMMITDATE, L_RECEIPTDATE, L_SHIPINSTRUCT, L_SHIPMODE, L_COMMENT, L_SHIPDATE FROM LINEITEM;
> {code}
> * TSql logs
> {code:xml}
> Progress: 99%, response time: 1157.913 sec
> Progress: 99%, response time: 1158.914 sec
> Progress: 100%, response time: 1159.914 sec
> Progress: 100%, response time: 1160.915 sec
> Progress: 100%, response time: 1161.916 sec
> Progress: 100%, response time: 1162.917 sec
> Progress: 100%, response time: 1163.917 sec
> Progress: 100%, response time: 1164.918 sec
> Progress: 100%, response time: 1165.919 sec
> Progress: 100%, response time: 1166.919 sec
> Progress: 100%, response time: 1167.92 sec
> Progress: 100%, response time: 1168.921 sec
> Progress: 100%, response time: 1169.921 sec
> Progress: 100%, response time: 1170.922 sec
> Progress: 100%, response time: 1171.923 sec
> Progress: 100%, response time: 1172.924 sec
> Progress: 100%, response time: 1173.924 sec
> Progress: 100%, response time: 1174.925 sec
> {code}
> * TajoWorker logs
> {code:xml}
> 2016-01-08 07:50:56,158 INFO org.apache.tajo.querymaster.Stage: Stage completed - eb_1452238256580_0001_000002 (total=6, success=6, killed=0)
> 2016-01-08 07:50:56,159 INFO org.apache.tajo.querymaster.Query: Processing q_1452238256580_0001 of type STAGE_COMPLETED
> 2016-01-08 07:50:56,159 INFO org.apache.tajo.engine.planner.global.ParallelExecutionQueue: Next executable block eb_1452238256580_0001_000003
> 2016-01-08 07:50:56,159 INFO org.apache.tajo.querymaster.Query: Complete Stage[eb_1452238256580_0001_000002], State: SUCCEEDED, 2/2.
> 2016-01-08 07:50:56,159 INFO org.apache.tajo.querymaster.Query: Processing q_1452238256580_0001 of type QUERY_COMPLETED
> 2016-01-08 07:50:56,161 INFO org.apache.tajo.worker.ExecutionBlockContext: Worker's task counter - total:6, succeeded: 6, killed: 0, failed: 0
> 2016-01-08 07:50:56,194 INFO org.apache.tajo.worker.TaskManager: Stopped execution block:eb_1452238256580_0001_000002
> {code}
> I found that query hangs on FileTableSpace::commitTable. We need to implement commitTable in S3TableSpace by considering s3 features.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)