[jira] [Commented] (HADOOP-11211) mapreduce.job.classloader.system.classes property behaves differently when the exclusion and inclusion order is different

["Sangjin Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-11211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177453#comment-14177453 ] 

Sangjin Lee commented on HADOOP-11211:
--------------------------------------

Thanks for reporting this [~timyitong]!

Actually this needs a little discussion. I agree that the current behavior is not as well documented as it should. As you stated, it is a "first-match-wins" behavior.

What you propose also seems reasonable, but needs more clarification. Your proposal is to disregard the order and consider all matches and do their logical AND's. But there can be cases where that rule is counter-intuitive. In your example, if you have "foo.bar.,-foo.bar." it would mean that the class is not considered a system class. Does that mean that an exclusion match is stronger than an inclusion match?

Here is an even more interesting case: how about "foo.bar.tar.,-foo.bar."? One reasonable interpretation of this may be that "exclude everything that belongs to foo.bar. but still include foo.bar.tar.". However, using the AND rule, it would mean that all classes under "foo.bar" would be excluded and the "foo.bar.tar." rule would be effectively ignored.

Also, how about a exact match (e.g. "foo.bar.MyClass")? Would it need to be considered over package matches?

> mapreduce.job.classloader.system.classes property behaves differently when the exclusion and inclusion order is different
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-11211
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11211
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Yitong Zhou
>            Assignee: Yitong Zhou
>
> If we want to include package foo.bar.* but exclude all sub packages named foo.bar.tar.* in system classes, configuring "mapreduce.job.classloader.system.classes=foo.bar.,-foo.bar.tar." won't work. foo.bar.tar will still be pulled in. But if we change the order:
> "mapreduce.job.classloader.system.classes=-foo.bar.tar.,foo.bar.", then it will work.
> This bug is due to the implementation of ApplicationClassLoaser#isSystemClass in hadoop-common, where we simply return the matching result immediately when the class name hits the first match (either positive or negative).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)