You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> on 2011/07/02 18:54:40 UTC
Re: Issues with ingesting Hierarchical products
Hey Tim,
Yep, I think it is a BasicVersioner problem.
Can you file an issue here [1] and report it as a bug? In the meanwhile, for a similar behavior, I'd
suggest using the DirectoryProductVersioner which was specifically built for Heirarchical products.
I'll take on the issue as soon as you report it and fix it (with a test case) in 0.4-SNAPSHOT.
Thanks!
Cheers,
Chris
[1] https://issues.apache.org/jira/browse/OODT
On Jun 29, 2011, at 3:12 PM, Stough, Timothy M (388F) wrote:
> Hello All,
>
> My product is a hierarchical product (directory) containing two files. It is called ALPSRP114390640-H1.0__A.
>
> The repository it should be stored in is:
> file:///proj/aria/repository/files/ALOSRawFrame.
>
> What's happening is that the product is being stored as
> file:///proj/aria/repository/files/ALOSRawFrameALPSRP114390640-H1.0__A
>
> It ought to be:
> file:///proj/aria/repository/files/ALOSRawFrame/ALPSRP114390640-H1.0__A
>
> My best guess is that it's a BasicVersioner problem. Logs and Policy files below... I'm running on a fresh checkout of tags/0.3.
>
> Thoughts, solutions, suggestions?
>
> Thanks,
> Tim.
>
> Running the log on FINER...
> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [ALOSRawFrameData]Jun 29, 2011 2:58:04 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [ALOSRawFrameData]Jun 29, 2011 2:58:04 PM org.apache.oodt.cas.filemgr.versioning.BasicVersioner createDataStoreReferences
> INFO: BasicVersioner: generated DataStore ref: file:/proj/aria/repository/files/ALOSRawFrameALPSRP114390640-H1.0__A/ from origRef: file:///proj/aria/archive/staging/raw-frame/ALPSRP114390640-H1.0__AJun 29, 2011 2:58:04 PM org.apache.oodt.cas.filemgr.versioning.VersioningUtils createBasicDataStoreRefsHierarchical
> FINER: tmpRef: ALPSRP114390640-H1.0__A/ALPSRP114390640-H1.0__A.rawJun 29, 2011 2:58:04 PM org.apache.oodt.cas.filemgr.versioning.VersioningUtils createBasicDataStoreRefsHierarchical
> FINE: VersioningUtils: Generated data store ref: file:/proj/aria/repository/files/ALOSRawFrameALPSRP114390640-H1.0__A/ALPSRP114390640-H1.0__A.raw from origRef: file:/proj/aria/archive/staging/raw-frame/ALPSRP114390640-H1.0__A/ALPSRP114390640-H1.0__A.raw
> Jun 29, 2011 2:58:04 PM org.apache.oodt.cas.filemgr.versioning.VersioningUtils createBasicDataStoreRefsHierarchical
>
> -----------------------------------------------------------------
> Tim Stough
> Jet Propulsion Lab
> Senior System Architect
> Data Understanding Group (Section 388)
> 818-393-5347 (office)
> 626-644-6574 (cell)
> GnuPG Public Key: http://www.ginkoleaf.net/stough-gpg-pub-key.txt
> -----------------------------------------------------------------
>
>
>
>
> <elements.xml><product-type-element-map.xml><product-types.xml>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Issues with ingesting Hierarchical products
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Tim,
Since I had time, I filed:
https://issues.apache.org/jira/browse/OODT-295
I'll attach a unit test shortly and put in a fix.
Cheers,
Chris
On Jul 2, 2011, at 9:54 AM, Mattmann, Chris A (388J) wrote:
> Hey Tim,
>
> Yep, I think it is a BasicVersioner problem.
>
> Can you file an issue here [1] and report it as a bug? In the meanwhile, for a similar behavior, I'd
> suggest using the DirectoryProductVersioner which was specifically built for Heirarchical products.
>
> I'll take on the issue as soon as you report it and fix it (with a test case) in 0.4-SNAPSHOT.
>
> Thanks!
>
> Cheers,
> Chris
>
> [1] https://issues.apache.org/jira/browse/OODT
>
> On Jun 29, 2011, at 3:12 PM, Stough, Timothy M (388F) wrote:
>
>> Hello All,
>>
>> My product is a hierarchical product (directory) containing two files. It is called ALPSRP114390640-H1.0__A.
>>
>> The repository it should be stored in is:
>> file:///proj/aria/repository/files/ALOSRawFrame.
>>
>> What's happening is that the product is being stored as
>> file:///proj/aria/repository/files/ALOSRawFrameALPSRP114390640-H1.0__A
>>
>> It ought to be:
>> file:///proj/aria/repository/files/ALOSRawFrame/ALPSRP114390640-H1.0__A
>>
>> My best guess is that it's a BasicVersioner problem. Logs and Policy files below... I'm running on a fresh checkout of tags/0.3.
>>
>> Thoughts, solutions, suggestions?
>>
>> Thanks,
>> Tim.
>>
>> Running the log on FINER...
>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor] for product type: [ALOSRawFrameData]Jun 29, 2011 2:58:04 PM org.apache.oodt.cas.filemgr.system.XmlRpcFileManager runExtractors
>> INFO: Running Met Extractor: [org.apache.oodt.cas.filemgr.metadata.extractors.examples.MimeTypeExtractor] for product type: [ALOSRawFrameData]Jun 29, 2011 2:58:04 PM org.apache.oodt.cas.filemgr.versioning.BasicVersioner createDataStoreReferences
>> INFO: BasicVersioner: generated DataStore ref: file:/proj/aria/repository/files/ALOSRawFrameALPSRP114390640-H1.0__A/ from origRef: file:///proj/aria/archive/staging/raw-frame/ALPSRP114390640-H1.0__AJun 29, 2011 2:58:04 PM org.apache.oodt.cas.filemgr.versioning.VersioningUtils createBasicDataStoreRefsHierarchical
>> FINER: tmpRef: ALPSRP114390640-H1.0__A/ALPSRP114390640-H1.0__A.rawJun 29, 2011 2:58:04 PM org.apache.oodt.cas.filemgr.versioning.VersioningUtils createBasicDataStoreRefsHierarchical
>> FINE: VersioningUtils: Generated data store ref: file:/proj/aria/repository/files/ALOSRawFrameALPSRP114390640-H1.0__A/ALPSRP114390640-H1.0__A.raw from origRef: file:/proj/aria/archive/staging/raw-frame/ALPSRP114390640-H1.0__A/ALPSRP114390640-H1.0__A.raw
>> Jun 29, 2011 2:58:04 PM org.apache.oodt.cas.filemgr.versioning.VersioningUtils createBasicDataStoreRefsHierarchical
>>
>> -----------------------------------------------------------------
>> Tim Stough
>> Jet Propulsion Lab
>> Senior System Architect
>> Data Understanding Group (Section 388)
>> 818-393-5347 (office)
>> 626-644-6574 (cell)
>> GnuPG Public Key: http://www.ginkoleaf.net/stough-gpg-pub-key.txt
>> -----------------------------------------------------------------
>>
>>
>>
>>
>> <elements.xml><product-type-element-map.xml><product-types.xml>
>
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++