You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/08/31 13:09:45 UTC
[jira] [Commented] (TAJO-1802) PythonScriptEngine copies controller
and tajo util whenever it is initialized
[ https://issues.apache.org/jira/browse/TAJO-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14723303#comment-14723303 ]
ASF GitHub Bot commented on TAJO-1802:
--------------------------------------
GitHub user jihoonson opened a pull request:
https://github.com/apache/tajo/pull/715
TAJO-1802: PythonScriptEngine copies controller and tajo util whenever it is initialized
I've changed to copy python files exactly once before script engine is started in master. In worker, python files are treated as one of shared resources.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jihoonson/tajo-2 TAJO-1802
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/715.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #715
----
commit 37eabf2f274f8d09b34e52e511e2f1844d6d8233
Author: Jihoon Son <ji...@apache.org>
Date: 2015-08-31T09:51:40Z
TAJO-1802
commit d335fd25f2751a210fbc25ebf4360cacea0ff1f2
Author: Jihoon Son <ji...@apache.org>
Date: 2015-08-31T11:06:37Z
TAJO-1802
----
> PythonScriptEngine copies controller and tajo util whenever it is initialized
> -----------------------------------------------------------------------------
>
> Key: TAJO-1802
> URL: https://issues.apache.org/jira/browse/TAJO-1802
> Project: Tajo
> Issue Type: Bug
> Components: Function/UDF
> Reporter: Jihoon Son
> Assignee: Jihoon Son
> Fix For: 0.11.0
>
>
> In the current implementation, python script files are copied and deleted whenever a task is launched and stopped. This incurs two problems.
> * Frequent copy of script files degrades the overall performance.
> * Among python scripts, tajo_util.py is shared by multiple tasks of a worker. This incurs I/O problem during unit tests because the shared file is also removed when a task is stopped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)