[jira] [Commented] (HADOOP-13070) classloading isolation improvements for cleaner and stricter dependencies

["Sean Busbey (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322828#comment-15322828 ] 

Sean Busbey commented on HADOOP-13070:
--------------------------------------

+1 on removing Configuration's dedicated classloader. That simplification helps limit our pain to the ones java folks expect to have in TCCL.

{quote}
We need to explore an op on that can let you determine the calling class and only block a user calling class to load a parent class (rule #4). We might be able to accomplish this by trying to determine the calling class and its classloader from the stack trace. This is something that the JDK’s ClassLoader does (via a non‐public JDK‐internal method), and we may be able to implement something similar.
{quote}

Ugh. this gives me all kinds of bad feels, though I think I might agree. I know [~stevel@apache.org] has strong feelings on the maintenance burden of this kind of custom classloader work, so let's ping him early.

If we go down this path, how concerned are we going to be with maintaining cross-JVM compatibility (vs falling back to some kind of "no isolation" approach)?

If we're at this point, is just shading every 3rd party dependency we use easier (barring the usual non-relocatable bits)? That would also prevent downstream folks from relying on them without a very clear at-your-own-risk step.

> classloading isolation improvements for cleaner and stricter dependencies
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-13070
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13070
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>            Priority: Critical
>         Attachments: classloading-improvements-ideas-v.3.pdf, classloading-improvements-ideas.pdf, classloading-improvements-ideas.v.2.pdf
>
>
> Related to HADOOP-11656, we would like to make a number of improvements in terms of classloading isolation so that user-code can run safely without worrying about dependency collisions with the Hadoop dependencies.
> By the same token, it should raised the quality of the user code and its specified classpath so that users get clear signals if they specify incorrect classpaths.
> This will contain a proposal that will include several improvements some of which may not be backward compatible. As such, it should be targeted to the next major revision of Hadoop.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org