You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by brian Foster <ho...@juno.com> on 2012/04/03 23:56:17 UTC
Review Request: Introduce a CAS-Metadata based renaming interface (CAS-PGE
Changes)
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/
-----------------------------------------------------------
Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
Summary
-------
CAS-PGE Changes to this issue...
- Renaming and Metadata extraction removed from CAS-PGE and instead CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
This addresses bug OODT-426.
https://issues.apache.org/jira/browse/OODT-426
Diffs
-----
trunk/pge/pom.xml 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java 1302648
trunk/pge/src/main/resources/examples/Crawler/action-beans.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/mime-types.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml PRE-CREATION
trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml 1302648
trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml 1302648
trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java 1302781
Diff: https://reviews.apache.org/r/4628/diff
Testing
-------
Several Unit-tests
Thanks,
brian
Re: Review Request: Introduce a CAS-Metadata based renaming interface
(CAS-PGE Changes)
Posted by brian Foster <ho...@juno.com>.
> On 2012-04-04 02:12:41, Paul Ramirez wrote:
> > trunk/pge/src/main/resources/examples/Crawler/action-beans.xml, lines 29-37
> > <https://reviews.apache.org/r/4628/diff/1/?file=98806#file98806line29>
> >
> > I'd define these properties in another file and then include them here. This is only a suggestion and not a just but I see the properties as something that could likely be changed or set to a fixed value and if we factor it out of here we can keep people from touching this file too much. I think this file just makes peoples heads spin at first but the properties don't (i.e. it hides the Spring goodness in a good way).
done
> On 2012-04-04 02:12:41, Paul Ramirez wrote:
> > trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml, lines 42-43
> > <https://reviews.apache.org/r/4628/diff/1/?file=98813#file98813line42>
> >
> > Put these examples inside comment tags as they wouldn't work as they existed anyhow. Also putting a longer description in the comment would help (i.e. one or more of these is not as helpful as what it does functionally. Why did we remove the files tag? Is this no longer supported? If it is then I recommend putting it back in but commented out.
> >
> > For instance, I'd expect that instead of metadata keys you want to set more of what will be done with that custom metadata would be of use. Also an example of multivalued metadata.
Added a TODO at the top of this file... The reader for this file still needs to be updated... so when i update it i'll make this file a working example when i write the unit-tests for it
- brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/#review6670
-----------------------------------------------------------
On 2012-04-03 21:56:17, brian Foster wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4628/
> -----------------------------------------------------------
>
> (Updated 2012-04-03 21:56:17)
>
>
> Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
>
>
> Summary
> -------
>
> CAS-PGE Changes to this issue...
> - Renaming and Metadata extraction removed from CAS-PGE and instead CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
>
>
> This addresses bug OODT-426.
> https://issues.apache.org/jira/browse/OODT-426
>
>
> Diffs
> -----
>
> trunk/pge/pom.xml 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java 1302648
> trunk/pge/src/main/resources/examples/Crawler/action-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-types.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml 1302648
> trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml 1302648
> trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java 1302781
>
> Diff: https://reviews.apache.org/r/4628/diff
>
>
> Testing
> -------
>
> Several Unit-tests
>
>
> Thanks,
>
> brian
>
>
Re: Review Request: Introduce a CAS-Metadata based renaming interface
(CAS-PGE Changes)
Posted by brian Foster <ho...@juno.com>.
> On 2012-04-04 02:12:41, Paul Ramirez wrote:
> > trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml, lines 42-43
> > <https://reviews.apache.org/r/4628/diff/1/?file=98813#file98813line42>
> >
> > Put these examples inside comment tags as they wouldn't work as they existed anyhow. Also putting a longer description in the comment would help (i.e. one or more of these is not as helpful as what it does functionally. Why did we remove the files tag? Is this no longer supported? If it is then I recommend putting it back in but commented out.
> >
> > For instance, I'd expect that instead of metadata keys you want to set more of what will be done with that custom metadata would be of use. Also an example of multivalued metadata.
>
> brian Foster wrote:
> Added a TODO at the top of this file... The reader for this file still needs to be updated... so when i update it i'll make this file a working example when i write the unit-tests for it
Also the file tags are no longer supported... use AutoDetectProductCrawler configuration now to specify which files in the outputDirs should be ingested
- brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/#review6670
-----------------------------------------------------------
On 2012-04-06 02:16:10, brian Foster wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4628/
> -----------------------------------------------------------
>
> (Updated 2012-04-06 02:16:10)
>
>
> Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
>
>
> Summary
> -------
>
> CAS-PGE Changes to this issue...
> - Renaming and Metadata extraction removed from CAS-PGE and instead CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
>
>
> This addresses bug OODT-426.
> https://issues.apache.org/jira/browse/OODT-426
>
>
> Diffs
> -----
>
> trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml 1302648
> trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml 1302648
> trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java 1302781
> trunk/pge/src/main/resources/examples/Crawler/mime-types.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/filename.extractor.config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/action-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/action-beans.properties PRE-CREATION
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java 1302648
> trunk/pge/pom.xml 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java 1302648
>
> Diff: https://reviews.apache.org/r/4628/diff
>
>
> Testing
> -------
>
> Several Unit-tests
>
>
> Thanks,
>
> brian
>
>
Re: Review Request: Introduce a CAS-Metadata based renaming interface
(CAS-PGE Changes)
Posted by Paul Ramirez <pr...@jpl.nasa.gov>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/#review6670
-----------------------------------------------------------
trunk/pge/src/main/resources/examples/Crawler/action-beans.xml
<https://reviews.apache.org/r/4628/#comment14432>
I'd define these properties in another file and then include them here. This is only a suggestion and not a just but I see the properties as something that could likely be changed or set to a fixed value and if we factor it out of here we can keep people from touching this file too much. I think this file just makes peoples heads spin at first but the properties don't (i.e. it hides the Spring goodness in a good way).
trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml
<https://reviews.apache.org/r/4628/#comment14431>
Put these examples inside comment tags as they wouldn't work as they existed anyhow. Also putting a longer description in the comment would help (i.e. one or more of these is not as helpful as what it does functionally. Why did we remove the files tag? Is this no longer supported? If it is then I recommend putting it back in but commented out.
For instance, I'd expect that instead of metadata keys you want to set more of what will be done with that custom metadata would be of use. Also an example of multivalued metadata.
- Paul
On 2012-04-03 21:56:17, brian Foster wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4628/
> -----------------------------------------------------------
>
> (Updated 2012-04-03 21:56:17)
>
>
> Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
>
>
> Summary
> -------
>
> CAS-PGE Changes to this issue...
> - Renaming and Metadata extraction removed from CAS-PGE and instead CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
>
>
> This addresses bug OODT-426.
> https://issues.apache.org/jira/browse/OODT-426
>
>
> Diffs
> -----
>
> trunk/pge/pom.xml 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java 1302648
> trunk/pge/src/main/resources/examples/Crawler/action-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-types.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml 1302648
> trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml 1302648
> trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java 1302781
>
> Diff: https://reviews.apache.org/r/4628/diff
>
>
> Testing
> -------
>
> Several Unit-tests
>
>
> Thanks,
>
> brian
>
>
Re: Review Request: Introduce a CAS-Metadata based renaming interface
(CAS-PGE Changes)
Posted by Chris Mattmann <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/#review6735
-----------------------------------------------------------
Ship it!
LGTM sounds good.
- Chris
On 2012-04-06 02:16:10, brian Foster wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4628/
> -----------------------------------------------------------
>
> (Updated 2012-04-06 02:16:10)
>
>
> Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
>
>
> Summary
> -------
>
> CAS-PGE Changes to this issue...
> - Renaming and Metadata extraction removed from CAS-PGE and instead CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
>
>
> This addresses bug OODT-426.
> https://issues.apache.org/jira/browse/OODT-426
>
>
> Diffs
> -----
>
> trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml 1302648
> trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml 1302648
> trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java 1302781
> trunk/pge/src/main/resources/examples/Crawler/mime-types.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/filename.extractor.config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/action-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/action-beans.properties PRE-CREATION
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java 1302648
> trunk/pge/pom.xml 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java 1302648
>
> Diff: https://reviews.apache.org/r/4628/diff
>
>
> Testing
> -------
>
> Several Unit-tests
>
>
> Thanks,
>
> brian
>
>
Re: Review Request: Introduce a CAS-Metadata based renaming interface
(CAS-PGE Changes)
Posted by Chris Mattmann <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/#review6734
-----------------------------------------------------------
Ship it!
- Chris
On 2012-04-06 02:16:10, brian Foster wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4628/
> -----------------------------------------------------------
>
> (Updated 2012-04-06 02:16:10)
>
>
> Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
>
>
> Summary
> -------
>
> CAS-PGE Changes to this issue...
> - Renaming and Metadata extraction removed from CAS-PGE and instead CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
>
>
> This addresses bug OODT-426.
> https://issues.apache.org/jira/browse/OODT-426
>
>
> Diffs
> -----
>
> trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml 1302648
> trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml 1302648
> trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java 1302781
> trunk/pge/src/main/resources/examples/Crawler/mime-types.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/filename.extractor.config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/action-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/action-beans.properties PRE-CREATION
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java 1302648
> trunk/pge/pom.xml 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java 1302648
>
> Diff: https://reviews.apache.org/r/4628/diff
>
>
> Testing
> -------
>
> Several Unit-tests
>
>
> Thanks,
>
> brian
>
>
Re: Review Request: Introduce a CAS-Metadata based renaming interface
(CAS-PGE Changes)
Posted by brian Foster <ho...@juno.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/
-----------------------------------------------------------
(Updated 2012-04-06 02:16:10.469275)
Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
Changes
-------
Updates per comments in reviews
Summary
-------
CAS-PGE Changes to this issue...
- Renaming and Metadata extraction removed from CAS-PGE and instead CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
This addresses bug OODT-426.
https://issues.apache.org/jira/browse/OODT-426
Diffs (updated)
-----
trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml PRE-CREATION
trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml 1302648
trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml 1302648
trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java 1302781
trunk/pge/src/main/resources/examples/Crawler/mime-types.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/filename.extractor.config.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/action-beans.xml PRE-CREATION
trunk/pge/src/main/resources/examples/Crawler/action-beans.properties PRE-CREATION
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java 1302648
trunk/pge/pom.xml 1302648
trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java 1302648
Diff: https://reviews.apache.org/r/4628/diff
Testing
-------
Several Unit-tests
Thanks,
brian
Re: Review Request: Introduce a CAS-Metadata based renaming interface
(CAS-PGE Changes)
Posted by Chris Mattmann <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/#review6694
-----------------------------------------------------------
Ship it!
LGTM, minor comments on my end. Great work. This will cause some user headache, but it's worth it and 0.4 is a game changing release.
- Chris
On 2012-04-03 21:56:17, brian Foster wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4628/
> -----------------------------------------------------------
>
> (Updated 2012-04-03 21:56:17)
>
>
> Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
>
>
> Summary
> -------
>
> CAS-PGE Changes to this issue...
> - Renaming and Metadata extraction removed from CAS-PGE and instead CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
>
>
> This addresses bug OODT-426.
> https://issues.apache.org/jira/browse/OODT-426
>
>
> Diffs
> -----
>
> trunk/pge/pom.xml 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java 1302648
> trunk/pge/src/main/resources/examples/Crawler/action-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-types.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml 1302648
> trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml 1302648
> trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java 1302781
>
> Diff: https://reviews.apache.org/r/4628/diff
>
>
> Testing
> -------
>
> Several Unit-tests
>
>
> Thanks,
>
> brian
>
>
Re: Review Request: Introduce a CAS-Metadata based renaming interface
(CAS-PGE Changes)
Posted by brian Foster <ho...@juno.com>.
> On 2012-04-04 18:34:56, Chris Mattmann wrote:
> > trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java, line 151
> > <https://reviews.apache.org/r/4628/diff/1/?file=98791#file98791line151>
> >
> > this seems like an ancillary change to this patch. However, it's a useful functionality so I don't feel strongly about separating it out. Just be wary of stuff like this (b/c as it grows) it can take away from the purpose of the patch ;)
ya... thought that too when i was making the change... but i was writing the unit-test for a method that was using it so i just fixed it right now so i don't have to rewrite the unit-test later
> On 2012-04-04 18:34:56, Chris Mattmann wrote:
> > trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java, line 49
> > <https://reviews.apache.org/r/4628/diff/1/?file=98792#file98792line49>
> >
> > +like
ack
- brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/#review6686
-----------------------------------------------------------
On 2012-04-06 02:16:10, brian Foster wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4628/
> -----------------------------------------------------------
>
> (Updated 2012-04-06 02:16:10)
>
>
> Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
>
>
> Summary
> -------
>
> CAS-PGE Changes to this issue...
> - Renaming and Metadata extraction removed from CAS-PGE and instead CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
>
>
> This addresses bug OODT-426.
> https://issues.apache.org/jira/browse/OODT-426
>
>
> Diffs
> -----
>
> trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml 1302648
> trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml 1302648
> trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java 1302781
> trunk/pge/src/main/resources/examples/Crawler/mime-types.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/filename.extractor.config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/action-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/action-beans.properties PRE-CREATION
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java 1302648
> trunk/pge/pom.xml 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java 1302648
>
> Diff: https://reviews.apache.org/r/4628/diff
>
>
> Testing
> -------
>
> Several Unit-tests
>
>
> Thanks,
>
> brian
>
>
Re: Review Request: Introduce a CAS-Metadata based renaming interface
(CAS-PGE Changes)
Posted by Chris Mattmann <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4628/#review6686
-----------------------------------------------------------
trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java
<https://reviews.apache.org/r/4628/#comment14489>
this seems like an ancillary change to this patch. However, it's a useful functionality so I don't feel strongly about separating it out. Just be wary of stuff like this (b/c as it grows) it can take away from the purpose of the patch ;)
trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java
<https://reviews.apache.org/r/4628/#comment14496>
+like
- Chris
On 2012-04-03 21:56:17, brian Foster wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/4628/
> -----------------------------------------------------------
>
> (Updated 2012-04-03 21:56:17)
>
>
> Review request for oodt, Chris Mattmann, Ricky Nguyen, Paul Ramirez, and Thomas Bennett.
>
>
> Summary
> -------
>
> CAS-PGE Changes to this issue...
> - Renaming and Metadata extraction removed from CAS-PGE and instead CAS-PGE now uses AutoDetectProductCrawler instead of StdProductCrawler
>
>
> This addresses bug OODT-426.
> https://issues.apache.org/jira/browse/OODT-426
>
>
> Diffs
> -----
>
> trunk/pge/pom.xml 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/OutputDir.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfig.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/PgeConfigMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RegExprOutputFiles.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/RenamingConv.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/config/XmlFilePgeConfigBuilder.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/metadata/PgeTaskMetKeys.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/ExternExtractorMetWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/FilenameExtractorWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/PcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/SciPgeConfigFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/metlist/MetadataListPcsMetFileWriter.java 1302648
> trunk/pge/src/main/java/org/apache/oodt/cas/pge/writers/xslt/XslTransformWriter.java 1302648
> trunk/pge/src/main/resources/examples/Crawler/action-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/crawler-config.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-extractor-map.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/mime-types.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/naming-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/Crawler/precondition-beans.xml PRE-CREATION
> trunk/pge/src/main/resources/examples/MetadataOutputFiles/metadata-output.xml 1302648
> trunk/pge/src/main/resources/examples/PgeConfigFiles/pge-config.xml 1302648
> trunk/pge/src/test/org/apache/oodt/cas/pge/TestPGETaskInstance.java 1302781
>
> Diff: https://reviews.apache.org/r/4628/diff
>
>
> Testing
> -------
>
> Several Unit-tests
>
>
> Thanks,
>
> brian
>
>