Distributed cache errors running MR job with HFileOutputFormat

[Andrei Cojocaru <ma...@gmail.com>]

Good day,

I am running a MR job that takes some data on HDFS and using
HFileOutputFormat rewrites for bulk loading into HBase using the
completebulkload tool. I keep getting these errors on some of my
mappers:

java.io.IOException: The distributed cache object
hdfs://namenode:54310/user/hadoop/partitions_1319717349150#_partition.lst
changed during the job from 10/27/11 7:09 AM to 10/27/11 7:10 AM

I think it might be some race condition since the more heavily loaded
the nodes are, the more often this happens. Any idea what could be
wrong? I am running Hadoop 0.20.2-cdh3u2 and HBase 0.90.4-cdh3u2.

Thanks for any help.

-Andrei