You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "Jihoon Son (JIRA)" <ji...@apache.org> on 2015/08/18 12:41:45 UTC
[jira] [Commented] (TAJO-1781) Join condition is still not found
when it exists in OR clause
[ https://issues.apache.org/jira/browse/TAJO-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701059#comment-14701059 ]
Jihoon Son commented on TAJO-1781:
----------------------------------
I've changed the title to represent the problem directly.
> Join condition is still not found when it exists in OR clause
> -------------------------------------------------------------
>
> Key: TAJO-1781
> URL: https://issues.apache.org/jira/browse/TAJO-1781
> Project: Tajo
> Issue Type: Bug
> Affects Versions: 0.11.0
> Reporter: Dongkyu Hwangbo
> Assignee: Jihoon Son
> Labels: sql
>
> This is TPC-DS q13.
> {code:SQL}
> select avg(ss_quantity)
> ,avg(ss_ext_sales_price)
> ,avg(ss_ext_wholesale_cost)
> ,sum(ss_ext_wholesale_cost)
> from store_sales
> ,store
> ,customer_demographics
> ,household_demographics
> ,customer_address
> ,date_dim
> where s_store_sk = ss_store_sk
> and ss_sold_date_sk = d_date_sk and d_year = 2001
> and((ss_hdemo_sk=hd_demo_sk
> and cd_demo_sk = ss_cdemo_sk
> and cd_marital_status = 'M'
> and cd_education_status = 'Advanced Degree'
> and ss_sales_price between 100.00 and 150.00
> and hd_dep_count = 3
> )or
> (ss_hdemo_sk=hd_demo_sk
> and cd_demo_sk = ss_cdemo_sk
> and cd_marital_status = 'S'
> and cd_education_status = 'College'
> and ss_sales_price between 50.00 and 100.00
> and hd_dep_count = 1
> ) or
> (ss_hdemo_sk=hd_demo_sk
> and cd_demo_sk = ss_cdemo_sk
> and cd_marital_status = 'W'
> and cd_education_status = '2 yr Degree'
> and ss_sales_price between 150.00 and 200.00
> and hd_dep_count = 1
> ))
> and((ss_addr_sk = ca_address_sk
> and ca_country = 'United States'
> and ca_state in ('TX', 'OH', 'TX')
> and ss_net_profit between 100 and 200
> ) or
> (ss_addr_sk = ca_address_sk
> and ca_country = 'United States'
> and ca_state in ('OR', 'NM', 'KY')
> and ss_net_profit between 150 and 300
> ) or
> (ss_addr_sk = ca_address_sk
> and ca_country = 'United States'
> and ca_state in ('VA', 'TX', 'MS')
> and ss_net_profit between 50 and 250
> ));
> {code}
> after (almost) 10 minutes, this error log is written. (in worker log)
> {noformat}
> 2015-08-18 14:29:41,220 INFO org.apache.tajo.worker.TaskImpl: One local chunk is added to listTablets
> 2015-08-18 14:29:41,221 INFO org.apache.tajo.worker.TaskImpl: One local chunk is added to listTablets
> 2015-08-18 14:40:26,950 INFO org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 2309ms
> GC pool 'PS MarkSweep' had collection(s): count=1 time=1988ms
> GC pool 'PS Scavenge' had collection(s): count=1 time=3ms
> 2015-08-18 15:02:56,295 ERROR org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec: 장치에 남은 공간이 없음
> java.io.IOException: 장치에 남은 공간이 없음
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
> at sun.nio.ch.IOUtil.write(IOUtil.java:51)
> at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:205)
> at org.apache.tajo.storage.RawFile$RawFileAppender.flushBuffer(RawFile.java:532)
> at org.apache.tajo.storage.RawFile$RawFileAppender.addTuple(RawFile.java:640)
> at org.apache.tajo.storage.HashShuffleAppender.addTuples(HashShuffleAppender.java:92)
> at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:117)
> at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:403)
> at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2015-08-18 15:02:56,347 ERROR org.apache.tajo.worker.TaskImpl: java.io.IOException: 장치에 남은 공간이 없음
> java.io.IOException: java.io.IOException: 장치에 남은 공간이 없음
> at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:147)
> at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:403)
> at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: 장치에 남은 공간이 없음
> at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
> at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
> at sun.nio.ch.IOUtil.write(IOUtil.java:51)
> at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:205)
> at org.apache.tajo.storage.RawFile$RawFileAppender.flushBuffer(RawFile.java:532)
> at org.apache.tajo.storage.RawFile$RawFileAppender.addTuple(RawFile.java:640)
> at org.apache.tajo.storage.HashShuffleAppender.addTuples(HashShuffleAppender.java:92)
> at org.apache.tajo.engine.planner.physical.HashShuffleFileWriteExec.next(HashShuffleFileWriteExec.java:117)
> ... 7 more
> 2015-08-18 15:02:56,348 INFO org.apache.tajo.worker.TaskAttemptContext: Query status of ta_1439875670149_0001_000007_000000_00 is changed to TA_FAILED
> 2015-08-18 15:02:56,411 INFO org.apache.tajo.worker.TaskImpl: ta_1439875670149_0001_000007_000000_00 completed. Worker's task counter - total:1, succeeded: 0, killed: 0, failed: 1
> 2015-08-18 15:02:56,413 ERROR org.apache.tajo.querymaster.TaskAttempt: ta_1439875670149_0001_000007_000000_00 FROM dkhwangbo >> java.io.IOException: 장치에 남은 공간이 없음
> 2015-08-18 15:02:56,413 INFO org.apache.tajo.querymaster.Task: ====================================================================================
> 2015-08-18 15:02:56,413 INFO org.apache.tajo.querymaster.Task: >>> Task Failed: ta_1439875670149_0001_000007_000000_00, retry:true, attempts:1 <<<
> 2015-08-18 15:02:56,413 INFO org.apache.tajo.querymaster.Task: ====================================================================================
> {noformat}
> I think tajo run with inefficient way to run this query.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)