You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Chris Mildebrandt <ch...@woodenrhino.com> on 2014/03/20 19:49:42 UTC

When are the checksums for local files generated?

Hello,

For the intermediate files the mapper generates (which will get sent to the
reducers), when are the checksums generated? Are they generated when a
request comes in for transfer, or are they generated when the file is
initially created? Also, are checksums checked/verified if the file is
accessed again by a local reducer (doesn't have to go over the network)?

Thanks,
-Chris