You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by "GOEKE, MATTHEW (AG/1000)" <ma...@monsanto.com> on 2011/08/15 23:01:10 UTC

Unit testing MR without dependency injection

Does anyone have any code examples for how they persist join data across multiple input splits and how they test it? Currently I populate a singleton in the setup method of my mapper (along with having jvm reuse turned on for this job) but with no way to have dependency injection into the mapper I am really having a hard time with wrapping a UT around the code. I could have a package scoped setter simply for testing purposes but that just feels dirty to be honest. Any help is greatly appreciated and I have both MRUnit and Mockito at my disposal.

  private BitPackedMarkerMap markerMap = BitPackedMarkerMapSingleton.getInstance().getMarkerMap();
  private int numberOfIndividuals = -999;
  private int numberOfAlleles = -999;

  @Override
  protected void setup(Context context) throws IOException, InterruptedException {
    LongPackedDoubleInteger inputSizes;
    if(markerMap.getSize() == 0){
    FileInputStream scoresInputStream = null;
      try{
        Path[] cacheFiles = DistributedCache.getLocalCacheFiles(context.getConfiguration());
        if (cacheFiles != null && cacheFiles.length > 0){
          scoresInputStream = new FileInputStream(cacheFiles[0].toString());
          inputSizes = markerMap.parse(scoresInputStream);
          numberOfIndividuals = inputSizes.getInt1();
          numberOfAlleles = inputSizes.getInt2();
        }
      } catch (IOException e){
        System.err.println("Exception reading DistributedCache: " + e);
        throw e;
      }finally {
        if(scoresInputStream != null){
          scoresInputStream.close();
        }
      }
    }
  }




Matt
This e-mail message may contain privileged and/or confidential information, and is intended to be received only by persons entitled
to receive such information. If you have received this e-mail in error, please notify the sender immediately. Please delete it and
all attachments from any servers, hard drives or any other media. Other use of this e-mail by you is strictly prohibited.

All e-mails and attachments sent and received are subject to monitoring, reading and archival by Monsanto, including its
subsidiaries. The recipient of this e-mail is solely responsible for checking for the presence of "Viruses" or other "Malware".
Monsanto, along with its subsidiaries, accepts no liability for any damage caused by any such code transmitted by or accompanying
this e-mail or any attachment.


The information contained in this email may be subject to the export control laws and regulations of the United States, potentially
including but not limited to the Export Administration Regulations (EAR) and sanctions regulations issued by the U.S. Department of
Treasury, Office of Foreign Asset Controls (OFAC).  As a recipient of this information you are obligated to comply with all
applicable U.S. export laws and regulations.