You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/12/07 17:05:10 UTC
[jira] [Commented] (TAJO-1950) Query master uses too much memory
during range shuffle
[ https://issues.apache.org/jira/browse/TAJO-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045137#comment-15045137 ]
ASF GitHub Bot commented on TAJO-1950:
--------------------------------------
Github user jihoonson commented on the pull request:
https://github.com/apache/tajo/pull/884#issuecomment-162568053
Hi, this patch is ready for review.
Here are highlights of this patch.
* To reduce the number of fetches created in Repartitioner, I've merged fetches with their pull server host name. As a result, the total number of fetches is ```H * R```, where ```H``` is the number of hosts and ```R``` is the number of ranges.
* To reduce the size of fetches created in Repartitioner, I've changed to keep FetchProto instead of FetchImpl.
* To reduce the number of index opens and closes, I've added an index reader cache to PullServerService.
> Query master uses too much memory during range shuffle
> ------------------------------------------------------
>
> Key: TAJO-1950
> URL: https://issues.apache.org/jira/browse/TAJO-1950
> Project: Tajo
> Issue Type: Improvement
> Components: distributed query plan
> Reporter: Jihoon Son
> Assignee: Jihoon Son
> Priority: Critical
> Fix For: 0.11.1
>
> Attachments: TAJO-1950proposal.pdf
>
>
> I ran a simple sort query on a 8TB table as follows.
> {noformat}
> tpch10tb> select * from lineitem order by l_orderkey;
> {noformat}
> After the first stage is completed, query master divides the range of the sort key (l_orderkey) into multiple partitions for range shuffle. Here, the partitioning time took about 9 minutes.
> Here is the log.
> {noformat}
> ...
> 2015-10-26 14:23:10,782 INFO org.apache.tajo.engine.planner.global.ParallelExecutionQueue: Next executable block eb_1445835438802_0004_000002
> 2015-10-26 14:23:10,782 INFO org.apache.tajo.querymaster.Query: Scheduling Stage:eb_1445835438802_0004_000002
> 2015-10-26 14:23:10,796 INFO org.apache.tajo.querymaster.Stage: org.apache.tajo.querymaster.DefaultTaskScheduler is chosen for the task scheduling for eb_1445835438802_0004_000002
> 2015-10-26 14:23:10,796 INFO org.apache.tajo.querymaster.Stage: eb_1445835438802_0004_000002, Table's volume is approximately 663647 MB
> 2015-10-26 14:23:10,796 INFO org.apache.tajo.querymaster.Stage: eb_1445835438802_0004_000002, The determined number of non-leaf tasks is 10370
> 2015-10-26 14:23:10,816 INFO org.apache.tajo.querymaster.Repartitioner: eb_1445835438802_0004_000002, Try to divide [(6000000000), (1)) into 10370 sub ranges (total units: 10370)
> 2015-10-26 14:24:58,996 INFO org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 2440ms
> GC pool 'PS MarkSweep' had collection(s): count=1 time=2214ms
> GC pool 'PS Scavenge' had collection(s): count=1 time=622ms
> 2015-10-26 14:27:24,040 WARN org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 13237ms
> GC pool 'PS MarkSweep' had collection(s): count=1 time=12635ms
> GC pool 'PS Scavenge' had collection(s): count=1 time=674ms
> 2015-10-26 14:28:51,914 WARN org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 20873ms
> GC pool 'PS MarkSweep' had collection(s): count=1 time=20486ms
> GC pool 'PS Scavenge' had collection(s): count=1 time=644ms
> 2015-10-26 14:30:52,392 WARN org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 30986ms
> GC pool 'PS MarkSweep' had collection(s): count=1 time=30546ms
> GC pool 'PS Scavenge' had collection(s): count=1 time=696ms
> 2015-10-26 14:32:07,550 WARN org.apache.tajo.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 15449ms
> GC pool 'PS MarkSweep' had collection(s): count=1 time=14593ms
> GC pool 'PS Scavenge' had collection(s): count=1 time=1148ms
> 2015-10-26 14:32:15,807 INFO org.apache.tajo.querymaster.Stage: 10370 objects are scheduled
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)