You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by Cameron Goodale <go...@apache.org> on 2012/09/20 16:51:43 UTC

Observation of the MetadataBasedProductMover

For anyone who needs to re-organize your OODT Archive I found this little
tool to be quite useful.

You can check out the code here:
https://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/tools/MetadataBasedProductMover.java


I have moved a lot of product using this tool, but I found a minor
inconsistency and I wanted to see if this is a bug.  You see in the index
output listed below the FileLocation doesn't match the newly updated
reference_data_store, but it does match the reference_orig path.

stored,indexed<CAS.ProductId:413e8ad6-9dd8-11e1-be66-9f8d45bb9efe>
stored,indexed<CAS.ProductName:MOD09GA.A2010046.h21v04.005.2010048120042.hdf>
stored,indexed<CAS.ProductReceivedTime:2012-05-14T08:19:53.095-07:00>
*stored,indexed<FileLocation:/usr/local/snow/data/archive/2010/46/MOD09GA>*
stored,indexed<Filename:MOD09GA.A2010046.h21v04.005.2010048120042.hdf>
stored,omitNorms<product_type_repoPath:file:///usr/local/snow/data/archive>
stored,omitNorms<product_type_versioner:gov.nasa.jpl.snow.versioning.ModisFileVersioner>
*
stored,omitNorms<reference_data_store:file:/usr/local/snow/data/archive/MOD09GA/2010/46/MOD09GA.A2010046.h21v04.005.2010048120042.hdf>
*
stored,omitNorms<reference_fileSize:90060629>
stored,indexed<reference_mimeType:application/x-hdf>
*
stored,omitNorms<reference_orig:file:/usr/local/snow/data/archive/2010/46/MOD09GA/MOD09GA.A2010046.h21v04.005.2010048120042.hdf>
*

So the question I am posing to the list is:  Should the FileLocation be
kept in-sync with the reference_data_store?  If this should happen, then I
think the MetadataBasedProductMover should be patched to make this update
at the time of the move.

Best Regards,


Cameron

Re: Observation of the MetadataBasedProductMover

Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Cam,

Yep the FileLocation should indeed be kept in sync. I think that the ProductMoverTool was constructed
without this in mind. 

Can you open a JIRA issue and we can fix?

Thanks!

Cheers,
Chris

On Sep 20, 2012, at 7:51 AM, Cameron Goodale wrote:

> For anyone who needs to re-organize your OODT Archive I found this little
> tool to be quite useful.
> 
> You can check out the code here:
> https://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/java/org/apache/oodt/cas/filemgr/tools/MetadataBasedProductMover.java
> 
> 
> I have moved a lot of product using this tool, but I found a minor
> inconsistency and I wanted to see if this is a bug.  You see in the index
> output listed below the FileLocation doesn't match the newly updated
> reference_data_store, but it does match the reference_orig path.
> 
> stored,indexed<CAS.ProductId:413e8ad6-9dd8-11e1-be66-9f8d45bb9efe>
> stored,indexed<CAS.ProductName:MOD09GA.A2010046.h21v04.005.2010048120042.hdf>
> stored,indexed<CAS.ProductReceivedTime:2012-05-14T08:19:53.095-07:00>
> *stored,indexed<FileLocation:/usr/local/snow/data/archive/2010/46/MOD09GA>*
> stored,indexed<Filename:MOD09GA.A2010046.h21v04.005.2010048120042.hdf>
> stored,omitNorms<product_type_repoPath:file:///usr/local/snow/data/archive>
> stored,omitNorms<product_type_versioner:gov.nasa.jpl.snow.versioning.ModisFileVersioner>
> *
> stored,omitNorms<reference_data_store:file:/usr/local/snow/data/archive/MOD09GA/2010/46/MOD09GA.A2010046.h21v04.005.2010048120042.hdf>
> *
> stored,omitNorms<reference_fileSize:90060629>
> stored,indexed<reference_mimeType:application/x-hdf>
> *
> stored,omitNorms<reference_orig:file:/usr/local/snow/data/archive/2010/46/MOD09GA/MOD09GA.A2010046.h21v04.005.2010048120042.hdf>
> *
> 
> So the question I am posing to the list is:  Should the FileLocation be
> kept in-sync with the reference_data_store?  If this should happen, then I
> think the MetadataBasedProductMover should be patched to make this update
> at the time of the move.
> 
> Best Regards,
> 
> 
> Cameron


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++