You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/01/06 07:54:39 UTC
[jira] [Commented] (TAJO-2036) Prevent out of memory in the master
server, if the query result is large
[ https://issues.apache.org/jira/browse/TAJO-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085085#comment-15085085 ]
ASF GitHub Bot commented on TAJO-2036:
--------------------------------------
GitHub user combineads opened a pull request:
https://github.com/apache/tajo/pull/928
TAJO-2036: Prevent out of memory in the master server, if the query result is large
This problem will occur frequently in our production server.
Please confirm.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/combineads/tajo TAJO-2036
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/928.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #928
----
commit 09387b86df03cb1a53fd82da650038a8a0dd364a
Author: Byunghwa Yun <co...@combineads.co.kr>
Date: 2016-01-06T06:24:26Z
TAJO-2036: Prevent out of memory in the master server, if the query
result is large
----
> Prevent out of memory in the master server, if the query result is large
> ------------------------------------------------------------------------
>
> Key: TAJO-2036
> URL: https://issues.apache.org/jira/browse/TAJO-2036
> Project: Tajo
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 0.11.1
> Reporter: Byunghwa Yun
> Assignee: Byunghwa Yun
> Priority: Minor
> Fix For: 0.12.0, 0.11.1
>
>
> Tajo Master server is a no response below situation.
> Test SQL:
> select col1, col2, .... col100 from big_table
> (big_table has rows: 1,000,000,000)
> Web UI parameters:
> Limit : 10MB
> Rows/Page : 25
> Master Server Log:
> 2016-01-06 10:35:59,973 INFO org.apache.tajo.tuple.memory.ResizableMemoryBlock: Increase DirectRowBlock to 64.0 KiB
> 2016-01-06 10:36:00,739 INFO org.apache.tajo.tuple.memory.ResizableMemoryBlock: Increase DirectRowBlock to 64.0 KiB
> 2016-01-06 10:36:01,316 INFO org.apache.tajo.master.exec.NonForwardQueryResultFileScanner: "Sent result to client for 7cba1175-e77b-4876-957d-2ff8722e940b, queryId: q_1452043069985_0001 rows: 3986313
> 2016-01-06 10:36:49,561 INFO org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 6385ms
> GC pool 'PS MarkSweep' had collection(s): count=1 time=5460ms
> GC pool 'PS Scavenge' had collection(s): count=1 time=993ms
> 2016-01-06 10:37:03,102 WARN org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 10587ms
> GC pool 'PS MarkSweep' had collection(s): count=1 time=10669ms
> GC pool 'PS Scavenge' had collection(s): count=1 time=401ms
> Master Server Stacktrace:
> "pool-11-thread-1" #379 prio=5 os_prio=0 tid=0x00007f47d424d000 nid=0xe26c waiting on condition [0x00007f4834cfa000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x000000076e9c7220> (a com.google.common.util.concurrent.AbstractFuture$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:275)
> at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:111)
> at org.apache.tajo.jdbc.FetchResultSet.nextTuple(FetchResultSet.java:77)
> at org.apache.tajo.jdbc.TajoResultSetBase.next(TajoResultSetBase.java:711)
> at org.apache.tajo.webapp.QueryExecutorServlet$QueryRunner.MakeResultText(QueryExecutorServlet.java:511)
> at org.apache.tajo.webapp.QueryExecutorServlet$QueryRunner.getQueryResult(QueryExecutorServlet.java:466)
> at org.apache.tajo.webapp.QueryExecutorServlet$QueryRunner.run(QueryExecutorServlet.java:347)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)