You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by "Vladimir Ozerov (JIRA)" <ji...@apache.org> on 2016/12/14 10:40:58 UTC
[jira] [Created] (IGNITE-4426) Hadoop: allow optional classloader
reuse between tasks.
Vladimir Ozerov created IGNITE-4426:
---------------------------------------
Summary: Hadoop: allow optional classloader reuse between tasks.
Key: IGNITE-4426
URL: https://issues.apache.org/jira/browse/IGNITE-4426
Project: Ignite
Issue Type: Sub-task
Components: hadoop
Affects Versions: 1.8
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
Fix For: 2.0
*Problem*
Currently we create separate {{HadoopClassLoader}} for each and every MR task. It has serious performance implications:
1) Every task have to reload classes form disk;
2) Every task starts from "cold" state with not-compiled classes;
3) Very high permgen/metaspace and code cache regions usage.
We can observe, that every task classloader works with the same JARs. It means that the only case when we really need to isolate tasks through classloaders is when they depend on some static data, which should not be shared.
*Proposal*
- Share {{HadoopClassLoader}} between tasks by default;
- Allow to optionally disable this opto.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)