You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Adrian Muraru (JIRA)" <ji...@apache.org> on 2012/11/22 00:42:58 UTC
[jira] [Created] (HBASE-7205) Coprocessor classloader is replicated
for all regions in the HRegionServer
Adrian Muraru created HBASE-7205:
------------------------------------
Summary: Coprocessor classloader is replicated for all regions in the HRegionServer
Key: HBASE-7205
URL: https://issues.apache.org/jira/browse/HBASE-7205
Project: HBase
Issue Type: Bug
Components: Coprocessors
Affects Versions: 0.94.2, 0.92.2
Reporter: Adrian Muraru
HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
Having the table coprocessor jailed in a separate classloader is good however we should create only one for all table regions in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510241#comment-13510241 ]
Ted Yu commented on HBASE-7205:
-------------------------------
{code}
+ classLoaders.add(cl);
{code}
I only found one reference to classLoaders, shown above. I wonder what purpose classLoaders would serve.
{code}
+ protected Set<ClassLoader> classLoaders = new HashSet<ClassLoader>();
+ static ConcurrentMap<Path, ClassLoader> classLoadersCache = new MapMaker()
{code}
Shouldn't the above be declared as static ? (CoprocessorHost is per-region).
I think we can unload ClassLoader from the cache using DeleteTableHandler and DisableTableHandler.
The assumption is that different tables have different schemas. Coprocessor for one table wouldn't directly fit another table.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511114#comment-13511114 ]
Ted Yu commented on HBASE-7205:
-------------------------------
In RegionCoprocessorHost.postClose(), we can retrieve activeCoprocessorClassLoaders and release the (strong) references.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502887#comment-13502887 ]
Lars Hofhansl commented on HBASE-7205:
--------------------------------------
The other option is to revert HBASE-6308 from 0.94 and only put that fix into 0.96. (At this point I am favoring that)
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503304#comment-13503304 ]
Andrew Purtell commented on HBASE-7205:
---------------------------------------
CoprocessorClassLoader extends URLClassloader, and is used the same way. Reverting doesn't make sense if the effects in PermGen are not a consequence of this change.
We need a test for checking PermGen use in the presence of coprocessors or not - open a few hundred regions and see.
The suggestion to cache the CP classloader according to CP path is a good one, we should pursue that because we don't support replacing jars out underneath the RS after a CP is loaded anyway.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7205) Coprocessor classloader is replicated
for all regions in the HRegionServer
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-7205:
---------------------------------
Fix Version/s: 0.94.4
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Adrian Muraru (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510755#comment-13510755 ]
Adrian Muraru commented on HBASE-7205:
--------------------------------------
[~ted_yu]
bq. I only found one reference to classLoaders, shown above. I wonder what purpose classLoaders would serve.
instance attribute "classLoaders" is keeping (strong) references to (possible multiple) region coproc classloaders.
classLoadersCache on the other hand is the global cache (static) shared by all regions in RS.
The idea is that classloaders instances (values in classLoadersCache keyed by jar path) are WeakReferences so that they are GC eligible once there is no
region using them (what keeps them in cache is the above "active classLoaders")
Agree, the naming is a bit misleading in my patch, should be:
{code:java}
+ protected Set<ClassLoader> activeCoprocessorClassLoaders = new HashSet<ClassLoader>();
+ static ConcurrentMap<Path, ClassLoader> classLoadersCache = new MapMaker()
{code}
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510247#comment-13510247 ]
Andrew Purtell commented on HBASE-7205:
---------------------------------------
bq. I think we can unload ClassLoader from the cache using DeleteTableHandler and DisableTableHandler. The assumption is that different tables have different schemas. Coprocessor for one table wouldn't directly fit another table.
AccessController? TokenProvider? Every table with security enabled will have them loaded.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7205) Coprocessor classloader is replicated
for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-7205:
--------------------------
Status: Patch Available (was: Open)
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.94.2, 0.92.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510249#comment-13510249 ]
Ted Yu commented on HBASE-7205:
-------------------------------
How about putting security related classes in a set and exclude them from unloading ?
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Adrian Muraru (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502763#comment-13502763 ]
Adrian Muraru commented on HBASE-7205:
--------------------------------------
[~tedyu@apache.org] CHecking the thread classloader would be a quick fix in the current implementation, however this would break the following usecase:
1. User creates a table and adds a RegionObserver CP (say org.mycompany.MyRegionObserver) available in say hdfs:///lib/mycp-*0.1*.jar
(in turn, a new classloader will be cached on thread context)
2. The user updates the implementation and overwrite the COPROCESSOR att on table to load from a new jar (say hdfs:///lib/mycp-*0.2*.jar)
In turn, the old/cached classloader will be used to load MyRegionObserver class - in other ways there is no way to drop an old classloader
Though, this might be considered a corner case and documented properly
The real solution here in my mind would be to cache the classloaders keyed by external jar canonical path:
This would require two new methods on org.apache.hadoop.hbase.Server
1. getExternalClassLoader(String key)/
2. setExternalClassLoader(String key, ClassLoader cl)
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502888#comment-13502888 ]
Lars Hofhansl commented on HBASE-7205:
--------------------------------------
[~jbaldassari]
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Adrian Muraru (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502738#comment-13502738 ]
Adrian Muraru commented on HBASE-7205:
--------------------------------------
There is a caching tentative in the current implementation of {code}org.apache.hadoop.hbase.coprocessor.CoprocessorHost#load{code} :
and the custom class loader is cached on thread context:
{code}
Thread.currentThread().setContextClassLoader(cl);
{code}
However the classes are loaded from the current class-loader (different from thread classloader)
{code}
implClass = getClass().getClassLoader().loadClass(className);
{code}
[~tedyu@apache.org] Not sure if RegionCoprocessorHost is the right place, we need similar behavior on MasterCoprocessorHost so should be one level down.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510234#comment-13510234 ]
Ted Yu commented on HBASE-7205:
-------------------------------
{code}
} catch (ClassNotFoundException e) {
+ classLoadersCache.remove(path);
{code}
Is the above code for point #2 ?
When would the catch clause be triggered ?
Thanks
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509340#comment-13509340 ]
Ted Yu commented on HBASE-7205:
-------------------------------
{code}
public CoprocessorClassLoader(List<URL> paths, ClassLoader parent) {
super(paths.toArray(new URL[]{}), parent);
{code}
CoprocessorClassLoader is given a list of URLs.
{code}
synchronized public Class<?> loadClass(String name)
{code}
loadClass() delegates locating class to URLClassLoader.findClass().
Looks like we can extract the path where the class is actually loaded by calling http://docs.oracle.com/javase/6/docs/api/java/net/URLClassLoader.html#findResource%28java.lang.String%29.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502471#comment-13502471 ]
Ted Yu commented on HBASE-7205:
-------------------------------
Looking at the change from HBASE-6308:
{code}
- ClassLoader cl = new URLClassLoader(paths.toArray(new URL[]{}),
+ ClassLoader cl = new CoprocessorClassLoader(paths,
{code}
I wonder if the duplication of CoprocessorClassLoader across regions is caused by the way JVM handles CoprocessorClassLoader differently from handling of URLClassLoader.
RegionCoprocessorHost has access to RegionServerServices. One approach for solving this issue is to expose table to CoprocessorClassLoader mapping through RegionServerServices. loadTableCoprocessors() can consult the mapping and avoid creating duplicate CoprocessorClassLoader's.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7205) Coprocessor classloader is replicated
for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-7205:
--------------------------
Attachment: 7205-v1.txt
Patch v1 uses cache mapping Path to CoprocessorClassLoader.
TestClassLoading and TestAggregateProtocol pass.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7205) Coprocessor classloader is replicated
for all regions in the HRegionServer
Posted by "Adrian Muraru (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Muraru updated HBASE-7205:
---------------------------------
Attachment: HBASE-7205_v2.patch
Adding v2 for this patch. We need to merge the tests though once discussed.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502744#comment-13502744 ]
Ted Yu commented on HBASE-7205:
-------------------------------
@Adrian:
Were you suggesting that we should retrieve thread classloader first so that we can reuse CoprocessorClassLoader instance ?
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7205) Coprocessor classloader is replicated
for all regions in the HRegionServer
Posted by "Adrian Muraru (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Muraru updated HBASE-7205:
---------------------------------
Description:
HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
was:
HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
Having the table coprocessor jailed in a separate classloader is good however we should create only one for all table regions in each HRS.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510254#comment-13510254 ]
Andrew Purtell commented on HBASE-7205:
---------------------------------------
bq. How about putting security related classes in a set and exclude them from unloading ?
What about any other type of coprocessor that might be registered on multiple tables but not on all tables? Security was just a good example.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502856#comment-13502856 ]
Lars Hofhansl commented on HBASE-7205:
--------------------------------------
[~apurtell] Want have a look?
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502881#comment-13502881 ]
Lars Hofhansl commented on HBASE-7205:
--------------------------------------
Looking at HBASE-6308 it seems the memory behavior didn't change principally.
If this is indeed a new problem I will make this a blocker and sink the current 0.94.3RC for this.
Any patch here should also explore the memory behavior in a test (although I am not entirely sure how one would do that)
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu reassigned HBASE-7205:
-----------------------------
Assignee: Ted Yu
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510275#comment-13510275 ]
Ted Yu commented on HBASE-7205:
-------------------------------
{code}
public E load(Path path, String className, int priority,
Configuration conf) throws IOException {
{code}
The above method is only called by RegionCoprocessorHost.loadTableCoprocessors().
We can establish a MultiMap from class name to table name. When there is no more table corresponding to given class, we can unload the class' ClassLoader.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502820#comment-13502820 ]
Ted Yu commented on HBASE-7205:
-------------------------------
As I said earlier, RegionCoprocessorHost has access to RegionServerServices. Should the new methods be added to RegionServerServices ?
In the case you described above, should we allow the path for ver 0.1 of the jar to be purged from the new classloader map ?
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511126#comment-13511126 ]
Ted Yu commented on HBASE-7205:
-------------------------------
Using patch v2, I got the following test failure:
{code}
testHBase3810(org.apache.hadoop.hbase.coprocessor.TestClassLoading) Time elapsed: 7.976 sec <<< FAILURE!
java.lang.AssertionError: Class SimpleRegionObserver was missing on a region
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.hadoop.hbase.coprocessor.TestClassLoading.testHBase3810(TestClassLoading.java:415)
{code}
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7205) Coprocessor classloader is replicated
for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-7205:
--------------------------
Priority: Critical (was: Major)
Fix Version/s: 0.96.0
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Adrian Muraru (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510166#comment-13510166 ]
Adrian Muraru commented on HBASE-7205:
--------------------------------------
[~ted_yu] This is really similar with my wip patch - unfortunately I did not find enough time to clean it up until now.
Couple of comments on your patch
{noformat}101 + if (clsLoaderCache.containsKey(path)) {{noformat}
1. CPH.load() is executed concurrently from multiple RS threads so clsLoaderCache needs to be synchronized/ConcurrentHashMap.
2. We need a way to drop cached classloaders that are not being used by any online Region.
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511098#comment-13511098 ]
Andrew Purtell commented on HBASE-7205:
---------------------------------------
The patch HBASE-7205_v2.patch looks good except for this:
{noformat}
- "com.hadoop",
{noformat}
See the discussion over on HBASE-6843.
Adding logging and ZooKeeper classes to the exceptions list is a good idea, thanks for fixing that oversight.
Should have a test that opens a table with a few regions and confirms there's only one CP classloader instance for all of them?
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Assignee: Ted Yu
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7205-v1.txt, HBASE-7205_v2.patch
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7205) Coprocessor classloader is
replicated for all regions in the HRegionServer
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510091#comment-13510091 ]
Ted Yu commented on HBASE-7205:
-------------------------------
Since loadClass(String name) method is defined in ClassLoader, if we override loadClass() in CoprocessorClassLoader() to return both Class and URL, we must name that method differently from loadClass().
> Coprocessor classloader is replicated for all regions in the HRegionServer
> --------------------------------------------------------------------------
>
> Key: HBASE-7205
> URL: https://issues.apache.org/jira/browse/HBASE-7205
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2
> Reporter: Adrian Muraru
> Priority: Critical
> Fix For: 0.96.0, 0.94.4
>
>
> HBASE-6308 introduced a new custom CoprocessorClassLoader to load the coprocessor classes and a new instance of this CL is created for each single HRegion opened. This leads to OOME-PermGen when the number of regions go above hundres / region server.
> Having the table coprocessor jailed in a separate classloader is good however we should create only one for all regions of a table in each HRS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira