[jira] Commented: (MAPREDUCE-1938) Ability for having user's classes take precedence over the system classes for tasks' classpath

["Scott Carey (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981413#action_12981413 ] 

Scott Carey commented on MAPREDUCE-1938:
----------------------------------------

{quote}
Users get in trouble without making conflicts in a deliberate way, they get into those conflicts because of the lack of isolation of the current model (refer to second paragraph of answer to Mahadev).

Having the use in control as the default is, IMO, the wrong way to go, all the testing you've done on a release becomes meaningless as the users classpath will override any library used by the cluster by default.

My take is that we should fix this via classloader isolation from the get go. 
{quote}

I agree and disagree.  If someone volunteered and fixed the whole situation with classloader and/or shade and/or OSGi then great, this would not be needed.  However, at this point I'd rather be given a crude tool with varioius risks than no tool at all.  No cluster since 0.18 has worked for me without classpath collisions requiring manual replacement of libraries.  

So all that precious testing on the release you mention is useless to me since such a cluster can't run my jobs!


{quote}
Too late for 0.22. Moving Fix Version from 0.22 to 0.23. 
{quote}

Considering this is in the Y! disro (and perhaps CDH3 ?) is there actually any veto for getting this in 0.22?    Its simplistic and does provide the user with a gun to shoot themselves in the foot, but the user interested in this already has a different gun, thats just harder to use and affects all tasks in the cluster instead of selected ones.

> Ability for having user's classes take precedence over the system classes for tasks' classpath
> ----------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1938
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1938
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: job submission, task, tasktracker
>            Reporter: Devaraj Das
>            Assignee: Krishna Ramachandran
>            Priority: Blocker
>             Fix For: 0.23.0
>
>         Attachments: mapred-1938-1, mapred-1938-2.patch, mapred-1938-3.patch, mr-1938-bp20.1.patch, mr-1938-bp20.patch
>
>
> It would be nice to have the ability in MapReduce to allow users to specify for their jobs alternate implementations of classes that are already defined in the MapReduce libraries. For example, an alternate implementation for CombineFileInputFormat. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.