ExportSnapshot webhdfs problems

[Brian Jeltema <br...@digitalenvoy.net>]

I’m trying to use ExportSnapshot to copy a snapshot from a Hadoop 1 to a Hadoop 2 cluster using the webhdfs protocol.
I’ve done this successfully before, though there are always mapper failures and retries in the job log. However, I’m not
having success with a rather large table due to an excessive number of failures. The exceptions in the job log are always:

14/09/29 20:28:11 INFO mapred.JobClient: Task Id : attempt_201409241055_0024_m_000005_1, Status : FAILED
org.apache.hadoop.ipc.RemoteException
	at org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:114)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:290)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$500(WebHdfsFileSystem.java:98)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$2.close(WebHdfsFileSystem.java:653)
	at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.copyFile(ExportSnapshot.java:204)
	at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.map(ExportSnapshot.java:146)
        …
        …
        …


So I presume the real exception is taking place on the target system. However, examining the namenode logs and a handful of
the datanode logs has not revealed any exceptions that correlate with those in the job log. Is there some other log I should be looking at?
I reduced the number of mappers to 6 and the target cluster has 10 datanodes, so it’s hard to believe its a capacity problem.

Thanks
Brian