[jira] [Created] (TAJO-2036) Prevent out of memory in the master server, if the query result is large

["Byunghwa Yun (JIRA)" <ji...@apache.org>]

Byunghwa Yun created TAJO-2036:
----------------------------------

             Summary: Prevent out of memory in the master server, if the query result is large
                 Key: TAJO-2036
                 URL: https://issues.apache.org/jira/browse/TAJO-2036
             Project: Tajo
          Issue Type: Improvement
          Components: Web UI
    Affects Versions: 0.11.1
            Reporter: Byunghwa Yun
            Assignee: Byunghwa Yun
            Priority: Minor
             Fix For: 0.12.0, 0.11.1


Tajo Master server is a no response below situation.

Test SQL:
select col1, col2, .... col100 from big_table
(big_table has rows: 1,000,000,000)

Web UI parameters:
Limit : 10MB
Rows/Page : 25

Master Server Log:
2016-01-06 10:35:59,973 INFO org.apache.tajo.tuple.memory.ResizableMemoryBlock: Increase DirectRowBlock to 64.0 KiB
2016-01-06 10:36:00,739 INFO org.apache.tajo.tuple.memory.ResizableMemoryBlock: Increase DirectRowBlock to 64.0 KiB
2016-01-06 10:36:01,316 INFO org.apache.tajo.master.exec.NonForwardQueryResultFileScanner: "Sent result to client for 7cba1175-e77b-4876-957d-2ff8722e940b, queryId: q_1452043069985_0001  rows: 3986313
2016-01-06 10:36:49,561 INFO org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 6385ms
GC pool 'PS MarkSweep' had collection(s): count=1 time=5460ms
GC pool 'PS Scavenge' had collection(s): count=1 time=993ms
2016-01-06 10:37:03,102 WARN org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 10587ms
GC pool 'PS MarkSweep' had collection(s): count=1 time=10669ms
GC pool 'PS Scavenge' had collection(s): count=1 time=401ms

Master Server Stacktrace: 
"pool-11-thread-1" #379 prio=5 os_prio=0 tid=0x00007f47d424d000 nid=0xe26c waiting on condition [0x00007f4834cfa000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x000000076e9c7220> (a com.google.common.util.concurrent.AbstractFuture$Sync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
        at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:275)
        at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111)
        at org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:77)
        at org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:711)
        at org.apache.tajo.webapp.QueryExecutorServlet$QueryRunner.MakeResultText(QueryExecutorServlet.java:511)
        at org.apache.tajo.webapp.QueryExecutorServlet$QueryRunner.getQueryResult(QueryExecutorServlet.java:466)
        at org.apache.tajo.webapp.QueryExecutorServlet$QueryRunner.run(QueryExecutorServlet.java:347)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)