You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/06/21 17:13:01 UTC
[jira] [Commented] (TAJO-1397) Resource allocation should be fine
grained.
[ https://issues.apache.org/jira/browse/TAJO-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14595074#comment-14595074 ]
ASF GitHub Bot commented on TAJO-1397:
--------------------------------------
GitHub user jinossy opened a pull request:
https://github.com/apache/tajo/pull/608
TAJO-1397
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jinossy/tajo TAJO-1397
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/608.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #608
----
commit 6dd0c0a90b1c2aaa56e3f144ecc57c996715c481
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-01T08:58:08Z
merge from TAJO-1615
commit 91319ee3eed1c950551071eca10a6587f635a48e
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-08T07:57:23Z
TAJO-1397
commit aba1188da4452c3c641e48411d6dd4a24998d93f
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-08T07:57:39Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1397
commit 1bfa430fd65006b79bfcdc382b0b7aeb17bd9686
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-16T11:13:30Z
TAJO-1397
commit 13e71b99ea0136fe5ad7f2489af0883664ca6547
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-16T11:13:46Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1397
commit 133e3f2c04a435a6da9ae823be0fa2dd803db377
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-17T15:50:18Z
TAJO-1397
commit b347e04aa5372bfb34f0e283aead42f9ff904112
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-18T02:16:12Z
TAJO-1397
commit b397ff15eedf982de704fb38ec1abe9a0f69df69
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-18T07:16:28Z
TAJO-1397
commit 4fab8787591302dace885417f4487976d586642d
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-18T08:01:07Z
remove unused codes
commit b82916043dccb26dd4dcb20f5ae70c1c202d5327
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-18T08:38:46Z
add debug message
commit e5318b8030bbf1821aacbe0ddae45d94bc403afa
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-18T09:23:07Z
add changable heartBeatInterval
commit 7ec50d034f9cd1bbc2fc7cce3eda35324681fee8
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-19T03:40:06Z
TAJO-1397
commit a8424871c2fb9a6f7af700f198ebe2f332852a80
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-19T04:15:14Z
fix wrong QM address
commit 50b78e6d4e4f6f9464f149cf3452d51115d9c22e
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-19T04:28:30Z
TAJO-1397
commit a520494bd141da189074e1ea170efaadd3bbd3a0
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-19T07:33:35Z
TAJO-1397
commit 3983008808e3351ba723941095643feae458eb49
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-21T14:42:53Z
remove StartExecutionBlock event
commit 749cfa2bf742bd913d4faf57b5ba7a673ebabb5b
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-21T14:44:03Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1397
commit 07b7b91f8c3cb8614dcf4ecc6e19d6e790a220ec
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-21T14:47:44Z
remove unused class
commit ee0c34e59df250e5b88f91b2d8c2d6b231318bde
Author: Jinho Kim <jh...@apache.org>
Date: 2015-06-21T15:07:40Z
add internal error handling in DefaultTaskScheduler
----
> Resource allocation should be fine grained.
> -------------------------------------------
>
> Key: TAJO-1397
> URL: https://issues.apache.org/jira/browse/TAJO-1397
> Project: Tajo
> Issue Type: Improvement
> Components: query master, resource manager, worker
> Reporter: Hyunsik Choi
> Assignee: Jinho Kim
> Fix For: 0.11.0
>
> Attachments: ResoruceSequence.jpg, old_resource_circuit.png, resource_circuit.png
>
>
> See the comment:
> https://issues.apache.org/jira/browse/TAJO-540?focusedCommentId=14359478&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14359478
> From the discussion in TAJO-540
> {quote}
> In general, query (or job) scheduler aims at the maximum resource utilization. For multi-tenancy, we also need to consider the fairness for multiple users (or queries). BTW, the maximum resource utilization and fairness are usually conflict to each other in many cases. To mitigate this problem, many scheduler seems to use preemption approach.
> In this point, our resource and scheduler system has the following problems:
> * A query exclusively uses allocated resources at the first time until the query is completed or failed.
> * There is no mechanism to deallocate resources during query processing.
> * Preempt is also not allowed.
> To achieve the multi tenancy, we should change our resource circulation. Especially, resource allocation must be fine grained instead of per query.
> So, I'll create a jira issue to change the resource circulation. We have to do this issue firstly in my opinion. If we achieve this, implementing multi-tenant scheduler would be much easier than now. It would be a good starting point of this issue.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)