[jira] [Commented] (HADOOP-12406) AbstractMapWritable.readFields throws ClassNotFoundException with custom writables

["Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-12406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14975384#comment-14975384 ] 

Vinod Kumar Vavilapalli commented on HADOOP-12406:
--------------------------------------------------

[~ndouba], is this happening inside a MapReduce job running on top of YARN? MapReduce does have job.jar in the system classpath, so that's not explainable.

The only way this may happen is if you are using the MapReduce JobClassLoader for this - please let us know the values for mapreduce.job.classloader in your job.

> AbstractMapWritable.readFields throws ClassNotFoundException with custom writables
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-12406
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12406
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 2.7.1
>         Environment: Ubuntu Linux 14.04 LTS amd64
>            Reporter: Nadeem Douba
>            Priority: Blocker
>              Labels: bug, hadoop, io, newbie, patch-available
>         Attachments: HADOOP-12406.patch
>
>
> Note: I am not an expert at JAVA, Class loaders, or Hadoop. I am just a hacker. My solution might be entirely wrong.
> AbstractMapWritable.readFields throws a ClassNotFoundException when reading custom writables. Debugging the job using remote debugging in IntelliJ revealed that the class loader being used in Class.forName() is different than that used by the Thread's current context (Thread.currentThread().getContextClassLoader()). The class path for the system class loader does not include the libraries of the job jar. However, the class path for the context class loader does. The proposed patch changes the class loading mechanism in readFields to use the Thread's context class loader instead of the system's default class loader.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)