[jira] [Commented] (IGNITE-4426) Hadoop: allow optional classloader reuse between tasks.

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-4426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748053#comment-15748053 ] 

ASF GitHub Bot commented on IGNITE-4426:
----------------------------------------

GitHub user devozerov opened a pull request:

    https://github.com/apache/ignite/pull/1344

    IGNITE-4426

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gridgain/apache-ignite ignite-4426

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/ignite/pull/1344.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1344
    
----
commit df99bd4bba938862f2a942d1f470dcfbb9090bdd
Author: devozerov <vo...@gridgain.com>
Date:   2016-12-14T11:13:16Z

    Implemented.

----


> Hadoop: allow optional classloader reuse between tasks.
> -------------------------------------------------------
>
>                 Key: IGNITE-4426
>                 URL: https://issues.apache.org/jira/browse/IGNITE-4426
>             Project: Ignite
>          Issue Type: Sub-task
>          Components: hadoop
>    Affects Versions: 1.8
>            Reporter: Vladimir Ozerov
>            Assignee: Vladimir Ozerov
>             Fix For: 2.0
>
>
> *Problem*
> Currently we create separate {{HadoopClassLoader}} for each and every MR task. It has serious performance implications:
> 1) Every task have to reload classes form disk;
> 2) Every task starts from "cold" state with not-compiled classes;
> 3) Very high permgen/metaspace and code cache regions usage.
> We can observe, that every task classloader works with the same JARs. It means that the only case when we really need to isolate tasks through classloaders is when they depend on some static data, which should not be shared.
> *Proposal*
> - Share {{HadoopClassLoader}} between tasks by default;
> - Allow to optionally disable this opto.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)