You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@oodt.apache.org by ma...@apache.org on 2010/07/17 05:13:59 UTC
svn commit: r965004 - in /incubator/oodt/trunk/metadata: ./
src/main/resources/ src/main/resources/examples/
src/test/org/apache/oodt/cas/metadata/extractors/ src/testdata/
Author: mattmann
Date: Sat Jul 17 03:13:59 2010
New Revision: 965004
URL: http://svn.apache.org/viewvc?rev=965004&view=rev
Log:
- OODT-15 WIP: fix problems with extractors in metadata; fix and enable all unit tests that were disabled per kelly; cleanup license headers on conf files; deploy conf files using Maven testResource bootstrap. Clean up deps. metadata fully functioning
Added:
incubator/oodt/trunk/metadata/src/testdata/copyandrewrite.test.conf
- copied, changed from r964942, incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/copyandrewrite.test.conf
incubator/oodt/trunk/metadata/src/testdata/extern-config.xml
incubator/oodt/trunk/metadata/src/testdata/samplemet.xml
- copied unchanged from r964942, incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/samplemet.xml
incubator/oodt/trunk/metadata/src/testdata/testExtractor
- copied unchanged from r964942, incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/testExtractor
incubator/oodt/trunk/metadata/src/testdata/testfile.txt
- copied unchanged from r964942, incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/testfile.txt
incubator/oodt/trunk/metadata/src/testdata/testfile2.txt
- copied unchanged from r964942, incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/testfile2.txt
incubator/oodt/trunk/metadata/src/testdata/testfile2.txt.met
- copied unchanged from r964942, incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/testfile2.txt.met
Removed:
incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/copyandrewrite.test.conf
incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/samplemet.xml
incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/testExtractor
incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/testfile.txt
incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/testfile2.txt
incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/testfile2.txt.met
Modified:
incubator/oodt/trunk/metadata/pom.xml
incubator/oodt/trunk/metadata/src/main/resources/examples/extern-config.xml
incubator/oodt/trunk/metadata/src/main/resources/examples/filename.extractor.config.xml
incubator/oodt/trunk/metadata/src/main/resources/examples/met_extr_preconditions.xml
incubator/oodt/trunk/metadata/src/main/resources/tika-mimetypes.xml
incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestCopyAndRewriteExtractor.java
incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestExternMetExtractor.java
incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestMetReader.java
incubator/oodt/trunk/metadata/src/testdata/met_extr_preconditions.xml
incubator/oodt/trunk/metadata/src/testdata/tika-mimetypes.xml
Modified: incubator/oodt/trunk/metadata/pom.xml
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/pom.xml?rev=965004&r1=965003&r2=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/pom.xml (original)
+++ incubator/oodt/trunk/metadata/pom.xml Sat Jul 17 03:13:59 2010
@@ -66,6 +66,24 @@ the License.
<include>tika-mimetypes.xml</include>
</includes>
</testResource>
+ <testResource>
+ <targetPath>org/apache/oodt/cas/metadata/extractors</targetPath>
+ <directory>${basedir}/src/testdata</directory>
+ <includes>
+ <include>copyandrewrite.test.conf</include>
+ <include>samplemet.xml</include>
+ <include>testfile.txt</include>
+ <include>testfile2.txt</include>
+ <include>testfile2.txt.met</include>
+ </includes>
+ </testResource>
+ <testResource>
+ <targetPath>org/apache/oodt/cas/metadata/extractors</targetPath>
+ <directory>${basedir}/src/testdata</directory>
+ <includes>
+ <include>extern-config.xml</include>
+ </includes>
+ </testResource>
</testResources>
<plugins>
<plugin>
@@ -74,19 +92,14 @@ the License.
<systemProperties>
<property>
<name>java.util.logging.config.file</name>
- <value>${basedir}/src/main/resources/logging.properties</value>
+ <value>${basedir}/src/main/resources/examples/logging.properties</value>
</property>
</systemProperties>
<forkMode>never</forkMode>
<includes>
<include>**/*Test*.java</include>
</includes>
- <excludes>
- <!-- FIXME: These all assume $CWD, but should use getResourceAsStream instead. Later. -->
- <exclude>org/apache/oodt/cas/metadata/extractors/TestMetReader.java</exclude>
- <exclude>org/apache/oodt/cas/metadata/extractors/TestExternMetExtractor.java</exclude>
- <exclude>org/apache/oodt/cas/metadata/extractors/TestCopyAndRewriteExtractor.java</exclude>
- </excludes>
+ <excludes/>
</configuration>
</plugin>
</plugins>
Modified: incubator/oodt/trunk/metadata/src/main/resources/examples/extern-config.xml
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/main/resources/examples/extern-config.xml?rev=965004&r1=965003&r2=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/src/main/resources/examples/extern-config.xml (original)
+++ incubator/oodt/trunk/metadata/src/main/resources/examples/extern-config.xml Sat Jul 17 03:13:59 2010
@@ -1,14 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
-<!--
- Copyright 2008 California Institute of Technology. ALL RIGHTS
- RESERVED. U.S. Government Sponsorship acknowledged.
-
- $Id$
-
- Author: Chris A. Mattmann
- Author: Brian M. Foster
- Description: XML Configuration file describing external command-line metadata extractor
- to be run through ExternMetExtractor facade.
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more contributor
+license agreements. See the NOTICE.txt file distributed with this work for
+additional information regarding copyright ownership. The ASF licenses this
+file to you under the Apache License, Version 2.0 (the "License"); you may not
+use this file except in compliance with the License. You may obtain a copy of
+the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations under
+the License.
-->
<!-- FIXME: change namespace URI? -->
<cas:externextractor xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
Modified: incubator/oodt/trunk/metadata/src/main/resources/examples/filename.extractor.config.xml
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/main/resources/examples/filename.extractor.config.xml?rev=965004&r1=965003&r2=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/src/main/resources/examples/filename.extractor.config.xml (original)
+++ incubator/oodt/trunk/metadata/src/main/resources/examples/filename.extractor.config.xml Sat Jul 17 03:13:59 2010
@@ -1,12 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
-<!--
- Copyright (c) 2009, California Institute of Technology.
- ALL RIGHTS RESERVED. U.S. Government sponsorship acknowledged.
-
- $Id$
-
- Author: Chris A. Mattmann
- Description: The configuration file for the Filename Token Met Extractor.
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more contributor
+license agreements. See the NOTICE.txt file distributed with this work for
+additional information regarding copyright ownership. The ASF licenses this
+file to you under the Apache License, Version 2.0 (the "License"); you may not
+use this file except in compliance with the License. You may obtain a copy of
+the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations under
+the License.
-->
<input>
<group name="SubstringOffsetGroup">
Modified: incubator/oodt/trunk/metadata/src/main/resources/examples/met_extr_preconditions.xml
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/main/resources/examples/met_extr_preconditions.xml?rev=965004&r1=965003&r2=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/src/main/resources/examples/met_extr_preconditions.xml (original)
+++ incubator/oodt/trunk/metadata/src/main/resources/examples/met_extr_preconditions.xml Sat Jul 17 03:13:59 2010
@@ -1,13 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
-<!--
- Copyright (c) 2008, California Institute of Technology.
- ALL RIGHTS RESERVED. U.S. Government sponsorship acknowledged.
-
- $Id$
-
- Author: bfoster, mattmann
- Description: Describes pre-conditions that should be evaluated before
- running a particular MetExtractor.
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more contributor
+license agreements. See the NOTICE.txt file distributed with this work for
+additional information regarding copyright ownership. The ASF licenses this
+file to you under the Apache License, Version 2.0 (the "License"); you may not
+use this file except in compliance with the License. You may obtain a copy of
+the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations under
+the License.
-->
<beans xmlns="http://www.springframework.org/schema/beans"
Modified: incubator/oodt/trunk/metadata/src/main/resources/tika-mimetypes.xml
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/main/resources/tika-mimetypes.xml?rev=965004&r1=965003&r2=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/src/main/resources/tika-mimetypes.xml (original)
+++ incubator/oodt/trunk/metadata/src/main/resources/tika-mimetypes.xml Sat Jul 17 03:13:59 2010
@@ -1,10 +1,4 @@
<?xml version="1.0" encoding="UTF-8"?>
-<!--
- Copyright (c) 2008, California Institute of Technology.
- ALL RIGHTS RESERVED. U.S. Government sponsorship acknowledged.
-
- $Id$
- -->
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
Modified: incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestCopyAndRewriteExtractor.java
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestCopyAndRewriteExtractor.java?rev=965004&r1=965003&r2=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestCopyAndRewriteExtractor.java (original)
+++ incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestCopyAndRewriteExtractor.java Sat Jul 17 03:13:59 2010
@@ -1,4 +1,4 @@
-/*
+/**
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
@@ -15,7 +15,6 @@
* limitations under the License.
*/
-
package org.apache.oodt.cas.metadata.extractors;
//OODT imports
@@ -23,93 +22,89 @@ import org.apache.oodt.cas.metadata.Meta
//JDK imports
import java.io.File;
-import java.io.FileInputStream;
//Junit imports
import junit.framework.TestCase;
/**
- * @author mattmann
- * @version $Revision$
- *
- * <p>
- * Describe your class here
- * </p>.
+ * Tests the CopyAndRewriteExtractor.
*/
public class TestCopyAndRewriteExtractor extends TestCase {
- private CopyAndRewriteExtractor extractor;
+ private CopyAndRewriteExtractor extractor;
- private static final String FILENAME = "Filename";
+ private static final String FILENAME = "Filename";
- private static final String FILE_LOCATION = "FileLocation";
+ private static final String FILE_LOCATION = "FileLocation";
- private static final String PRODUCT_TYPE = "ProductType";
+ private static final String PRODUCT_TYPE = "ProductType";
- private static final String confFilePath = "./src/test/gov/nasa/jpl/oodt/cas/metadata/extractors/copyandrewrite.test.conf";
+ private static final String confFilePath = "copyandrewrite.test.conf";
- private static final String extractFilePath = "./src/test/gov/nasa/jpl/oodt/cas/metadata/extractors/testfile.txt";
+ private static final String extractFilePath = "testfile.txt";
- private static final String expectedFilename = "testfile.txt";
+ private static final String expectedFilename = "testfile.txt";
- private static final String expectedProductType = "NewProductTypeGenericFile";
+ private static final String expectedProductType = "NewProductTypeGenericFile";
- private static String expectedFileLocation = null;
+ private static String expectedFileLocation = null;
- static {
- try {
- expectedFileLocation = "/new/loc/"
- + new File(extractFilePath).getParentFile()
- .getCanonicalPath();
- } catch (Exception ignore) {
- }
+ static {
+ try {
+ expectedFileLocation = "/new/loc/"
+ + new File(TestCopyAndRewriteExtractor.class.getResource(
+ extractFilePath).getFile()).getParentFile().getCanonicalPath();
+ } catch (Exception ignore) {
}
+ }
- public TestCopyAndRewriteExtractor() {
- CopyAndRewriteConfig config = new CopyAndRewriteConfig();
- try {
- config.load(new FileInputStream(confFilePath));
- } catch (Exception e) {
- fail(e.getMessage());
- }
-
- extractor = new CopyAndRewriteExtractor();
- extractor.setConfigFile(config);
+ public TestCopyAndRewriteExtractor() {
+ CopyAndRewriteConfig config = new CopyAndRewriteConfig();
+ try {
+ config.load(getClass().getResourceAsStream(confFilePath));
+ } catch (Exception e) {
+ fail(e.getMessage());
}
- public void testExtractMetadata() {
- Metadata met = null;
-
- try {
- met = extractor.extractMetadata(extractFilePath);
- } catch (Exception e) {
- fail(e.getMessage());
- }
-
- assertNotNull(met);
- assertNotNull(met.getHashtable());
- assertNotNull(met.getHashtable().keySet());
- assertEquals(3, met.getHashtable().keySet().size());
- assertTrue(met.containsKey(FILENAME));
- assertEquals(expectedFilename, met.getMetadata(FILENAME));
- assertTrue(met.containsKey(PRODUCT_TYPE));
- assertEquals(expectedProductType, met.getMetadata(PRODUCT_TYPE));
- assertTrue(met.containsKey(FILE_LOCATION));
- assertEquals("The expected file location: [" + expectedFileLocation
- + "] does not match " + "the obtained file location: ["
- + met.getMetadata(FILE_LOCATION) + "]", expectedFileLocation,
- met.getMetadata(FILE_LOCATION));
+ extractor = new CopyAndRewriteExtractor();
+ extractor.setConfigFile(config);
+ }
+
+ public void testExtractMetadata() {
+ Metadata met = null;
+
+ try {
+ met = extractor.extractMetadata(getClass().getResource(extractFilePath)
+ .getFile());
+ } catch (Exception e) {
+ fail(e.getMessage());
}
- public void testReplaceOrigMetFilePath() {
- Metadata met = null;
-
- try {
- met = extractor.extractMetadata(extractFilePath);
- } catch (Exception e) {
- fail(e.getMessage());
- }
-
- assertNotNull(met);
+ assertNotNull(met);
+ assertNotNull(met.getHashtable());
+ assertNotNull(met.getHashtable().keySet());
+ assertEquals(3, met.getHashtable().keySet().size());
+ assertTrue(met.containsKey(FILENAME));
+ assertEquals(expectedFilename, met.getMetadata(FILENAME));
+ assertTrue(met.containsKey(PRODUCT_TYPE));
+ assertEquals(expectedProductType, met.getMetadata(PRODUCT_TYPE));
+ assertTrue(met.containsKey(FILE_LOCATION));
+ assertEquals("The expected file location: [" + expectedFileLocation
+ + "] does not match " + "the obtained file location: ["
+ + met.getMetadata(FILE_LOCATION) + "]", expectedFileLocation, met
+ .getMetadata(FILE_LOCATION));
+ }
+
+ public void testReplaceOrigMetFilePath() {
+ Metadata met = null;
+
+ try {
+ met = extractor.extractMetadata(getClass().getResource(extractFilePath)
+ .getFile());
+ } catch (Exception e) {
+ fail(e.getMessage());
}
+
+ assertNotNull(met);
+ }
}
Modified: incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestExternMetExtractor.java
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestExternMetExtractor.java?rev=965004&r1=965003&r2=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestExternMetExtractor.java (original)
+++ incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestExternMetExtractor.java Sat Jul 17 03:13:59 2010
@@ -44,9 +44,9 @@ public class TestExternMetExtractor exte
private static final String PRODUCT_TYPE = "ProductType";
- private static final String configFilePath = "./src/main/resources/examples/extern-config.xml";
+ private static final String configFilePath = "extern-config.xml";
- private static final String extractFilePath = "./src/test/gov/nasa/jpl/oodt/cas/metadata/extractors/testfile.txt";
+ private static final String extractFilePath = "testfile.txt";
private static final String expectedFilename = "testfile.txt";
@@ -62,16 +62,16 @@ public class TestExternMetExtractor exte
}
Metadata met = null;
+
try {
- met = extractor.extractMetadata(new File(extractFilePath),
- new File(configFilePath));
+ met = extractor.extractMetadata(new File(getClass().getResource(extractFilePath).getFile()),
+ new File(getClass().getResource(configFilePath).getFile()));
} catch (MetExtractionException e) {
fail(e.getMessage());
}
assertNotNull(met);
- assertTrue(new File(
- "./src/test/gov/nasa/jpl/oodt/cas/metadata/extractors/testfile.txt.met").exists());
+ assertTrue(new File(getClass().getResource("testfile.txt.met").getFile()).exists());
assertTrue(met.containsKey(FILENAME));
assertTrue(met.containsKey(FILE_LOCATION));
assertTrue(met.containsKey(PRODUCT_TYPE));
@@ -88,8 +88,7 @@ public class TestExternMetExtractor exte
* @see junit.framework.TestCase#tearDown()
*/
protected void tearDown() throws Exception {
- File generatedMetFile = new File(
- "./src/test/gov/nasa/jpl/oodt/cas/metadata/extractors/testfile.txt.met");
+ File generatedMetFile = new File(getClass().getResource("testfile.txt.met").getFile());
if (generatedMetFile.exists()) {
generatedMetFile.delete();
}
Modified: incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestMetReader.java
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestMetReader.java?rev=965004&r1=965003&r2=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestMetReader.java (original)
+++ incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/TestMetReader.java Sat Jul 17 03:13:59 2010
@@ -47,7 +47,7 @@ public class TestMetReader extends TestC
private static final String expectedFileLocation = ".";
- private static final String testFile = "./src/test/gov/nasa/jpl/oodt/cas/metadata/extractors/testfile2.txt";
+ private static final String testFile = "testfile2.txt";
private static final String FILENAME = "Filename";
@@ -63,7 +63,7 @@ public class TestMetReader extends TestC
Metadata met = null;
try {
- met = extractor.extractMetadata(new File(testFile));
+ met = extractor.extractMetadata(getClass().getResource(testFile).getFile());
} catch (MetExtractionException e) {
fail(e.getMessage());
}
Copied: incubator/oodt/trunk/metadata/src/testdata/copyandrewrite.test.conf (from r964942, incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/copyandrewrite.test.conf)
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/testdata/copyandrewrite.test.conf?p2=incubator/oodt/trunk/metadata/src/testdata/copyandrewrite.test.conf&p1=incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/copyandrewrite.test.conf&r1=964942&r2=965004&rev=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/src/test/org/apache/oodt/cas/metadata/extractors/copyandrewrite.test.conf (original)
+++ incubator/oodt/trunk/metadata/src/testdata/copyandrewrite.test.conf Sat Jul 17 03:13:59 2010
@@ -1,6 +1,6 @@
numRewriteFields=2
rewriteField1=ProductType
rewriteField2=FileLocation
-orig.met.file.path=[PWD]/src/test/gov/nasa/jpl/oodt/cas/metadata/extractors/samplemet.xml
+orig.met.file.path=[PWD]/target/test-classes/org/apache/oodt/cas/metadata/extractors/samplemet.xml
ProductType.pattern=NewProductType[ProductType]
FileLocation.pattern=/new/loc/[FileLocation]
Added: incubator/oodt/trunk/metadata/src/testdata/extern-config.xml
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/testdata/extern-config.xml?rev=965004&view=auto
==============================================================================
--- incubator/oodt/trunk/metadata/src/testdata/extern-config.xml (added)
+++ incubator/oodt/trunk/metadata/src/testdata/extern-config.xml Sat Jul 17 03:13:59 2010
@@ -0,0 +1,69 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more contributor
+license agreements. See the NOTICE.txt file distributed with this work for
+additional information regarding copyright ownership. The ASF licenses this
+file to you under the Apache License, Version 2.0 (the "License"); you may not
+use this file except in compliance with the License. You may obtain a copy of
+the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations under
+the License.
+-->
+<!-- FIXME: change namespace URI? -->
+<cas:externextractor xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas">
+
+ <!--
+ workingDir: specifies the directory that the external command line met extractor will
+ be run in.
+
+ if workingDir is not specified, and if the external command line met extractor honors
+ the workingDir contract (generating .met file in directory that it is called from), then
+ the .met file will be generated in /path/to/file's parent directory. Otherwise, if the
+ external command let met extractor doesn't honor working dir, then all bets
+ are off.
+ -->
+ <exec workingDir="">
+ <!-- this path can be absolute or relative -->
+ <!-- you can optionally specify envReplace tag (=true)
+ to turn on environment variable replacement
+ -->
+ <extractorBinPath envReplace="true">[PWD]/src/testdata/testExtractor</extractorBinPath>
+
+ <!-- make sure to use character entities if there are any weird
+ parameters here that would screw up the XML.
+
+ args come after providing the file to this met extractor. So, in other
+ words, we assume that extractorBinPath >file< <args...>
+
+ where the first parameter is always the file to extract metadata from,
+ and the remaining parameters are optional to provide to the underlying
+ met extractor.
+ -->
+ <args>
+ <arg isDataFile="true"/>
+ <arg isMetFile="true"/>
+ <arg>-Dtrue=always</arg>
+ <arg>foo</arg>
+ <arg>bar</arg>
+ <!-- you can optionally specify that you would like environment
+ variable replacement on a particular argument
+
+ You can also optionally specify that the argument is a path
+ and that you would like it to be treated as such using the
+ isPath="true". If not specified, the path is treated as if
+ isPath was specified as "false", and String.replaceAll("\\s", "\\\\ ")
+ is not called. If called, all white spaces are replaced from the path
+ (using the above regex), and replaced with the literal string
+ "\ ".
+ -->
+ <arg envReplace="true" isPath="true">[HOME]/test boo</arg>
+ </args>
+ </exec>
+
+</cas:externextractor>
\ No newline at end of file
Modified: incubator/oodt/trunk/metadata/src/testdata/met_extr_preconditions.xml
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/testdata/met_extr_preconditions.xml?rev=965004&r1=965003&r2=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/src/testdata/met_extr_preconditions.xml (original)
+++ incubator/oodt/trunk/metadata/src/testdata/met_extr_preconditions.xml Sat Jul 17 03:13:59 2010
@@ -1,13 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
-<!--
- Copyright (c) 2008, California Institute of Technology.
- ALL RIGHTS RESERVED. U.S. Government sponsorship acknowledged.
-
- $Id$
-
- Author: bfoster, mattmann
- Description: Describes pre-conditions that should be evaluated before
- running a particular MetExtractor.
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more contributor
+license agreements. See the NOTICE.txt file distributed with this work for
+additional information regarding copyright ownership. The ASF licenses this
+file to you under the Apache License, Version 2.0 (the "License"); you may not
+use this file except in compliance with the License. You may obtain a copy of
+the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations under
+the License.
-->
<beans xmlns="http://www.springframework.org/schema/beans"
Modified: incubator/oodt/trunk/metadata/src/testdata/tika-mimetypes.xml
URL: http://svn.apache.org/viewvc/incubator/oodt/trunk/metadata/src/testdata/tika-mimetypes.xml?rev=965004&r1=965003&r2=965004&view=diff
==============================================================================
--- incubator/oodt/trunk/metadata/src/testdata/tika-mimetypes.xml (original)
+++ incubator/oodt/trunk/metadata/src/testdata/tika-mimetypes.xml Sat Jul 17 03:13:59 2010
@@ -1,10 +1,4 @@
<?xml version="1.0" encoding="UTF-8"?>
-<!--
- Copyright (c) 2008, California Institute of Technology.
- ALL RIGHTS RESERVED. U.S. Government sponsorship acknowledged.
-
- $Id$
- -->
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -25,7 +19,6 @@
The mime types within this file are based on the types in the mime-types.xml
file available in Apache Nutch.
-->
-
<mime-info>
<mime-type type="text/plain">
Re: svn commit: r965004 - in /incubator/oodt/trunk/metadata: ./
src/main/resources/ src/main/resources/examples/
src/test/org/apache/oodt/cas/metadata/extractors/ src/testdata/
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Ack, I was testing them from within the metadata project, where they still
work:
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec
Running org.apache.oodt.cas.metadata.extractors.TestCopyAndRewriteExtractor
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.39 sec
Results :
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 29 seconds
[INFO] Finished at: Sat Jul 17 08:39:51 PDT 2010
[INFO] Final Memory: 28M/81M
[INFO]
------------------------------------------------------------------------
[chipotle:oodt/trunk/metadata] mattmann% pwd
/Users/mattmann/src/oodt/trunk/metadata
[chipotle:oodt/trunk/metadata] mattmann%
The problem is, even though I changed all of the conf files to load using
getClass().getResource(name).getFile(), the file paths referenced inside of
the config files is relative still in some cases. Ack. So it's not going to
be easy...which is why I'm glad you volunteered! LOL
Cheers,
Chris
On 7/17/10 7:57 AM, "Sean Kelly" <ke...@apache.org> wrote:
> One other note:
>
> 1. Time permitting I'll adjust the code to do just that today
>
> and:
> 2. I need to remember to fix the "From:" header to read from my apache.org
> account!
>
> OK, that's two other notes in total:
>
> 1. Time permitting I'll adjust the code to do just that today
> 2. I need to remember to fix the "From:" header to read from my apache.org
> account!
>
> and:
> 3. No one expects the Spanish Inquisition
>
>
> On 2010.Jul.17, at 9.52a, Sean Kelly wrote:
>
>>> - OODT-15 WIP: fix problems with extractors in metadata; fix and enable all
>>> unit tests that were disabled per kelly; cleanup license headers on conf
>>> files; deploy conf files using Maven testResource bootstrap. Clean up deps.
>>> metadata fully functioning
>>
>>
>> Those tests are still, sadly, broken. They all assume that $CWD (current
>> working directory) is "..../metadata" and then create File objects relative
>> to $CWD:
>>
>> -----------------------------------------------------
>> % svn co https://svn.apache.org/repos/asf/incubator/oodt/trunk oodt
>> % cd oodt
>> % mvn test
>> [INFO] Scanning for projects...
>> [INFO] Reactor build order:
>> ...
>> -------------------------------------------------------
>> T E S T S
>> -------------------------------------------------------
>> Running org.apache.oodt.cas.metadata.extractors.TestMetReader
>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.204 sec
>> Running
>> org.apache.oodt.cas.metadata.extractors.TestExternMetExtractorConfigReader
>> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec <<<
>> FAILURE!
>> ...
>> Failed tests:
>>
>> testReadConfig(org.apache.oodt.cas.metadata.extractors.TestExternMetExtractor
>> ConfigReader)
>> testEval(org.apache.oodt.cas.metadata.preconditions.TestPreCondEvalUtils)
>>
>> testExtractor(org.apache.oodt.cas.metadata.extractors.TestExternMetExtractor)
>>
>> testExtractMetadata(org.apache.oodt.cas.metadata.extractors.TestCopyAndRewrit
>> eExtractor)
>>
>> testReplaceOrigMetFilePath(org.apache.oodt.cas.metadata.extractors.TestCopyAn
>> dRewriteExtractor)
>> ...
>> % echo bummer
>> bummer
>> -----------------------------------------------------
>>
>> The proper way to fetch resources, including test resources, is with
>> Class.getResource* methods. If the API being tested absolutely has to File
>> objects as input (instead of URLs, InputStreams, or Readers), then create
>> temporary files by reading the file data from with Class.getResourceAsStream,
>> and pass those temporary files to the APIs being tested.
>>
>>
>>
>> On 2010.Jul.16, at 10.13p, mattmann@apache.org wrote:
>>
>>> Author: mattmann
>>> Date: Sat Jul 17 03:13:59 2010
>>> New Revision: 965004
>>>
>>> URL: http://svn.apache.org/viewvc?rev=965004&view=rev
>>> Log:
>>
>
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: svn commit: r965004 - in /incubator/oodt/trunk/metadata: ./ src/main/resources/ src/main/resources/examples/ src/test/org/apache/oodt/cas/metadata/extractors/ src/testdata/
Posted by Sean Kelly <ke...@apache.org>.
One other note:
1. Time permitting I'll adjust the code to do just that today
and:
2. I need to remember to fix the "From:" header to read from my apache.org account!
OK, that's two other notes in total:
1. Time permitting I'll adjust the code to do just that today
2. I need to remember to fix the "From:" header to read from my apache.org account!
and:
3. No one expects the Spanish Inquisition
On 2010.Jul.17, at 9.52a, Sean Kelly wrote:
>> - OODT-15 WIP: fix problems with extractors in metadata; fix and enable all unit tests that were disabled per kelly; cleanup license headers on conf files; deploy conf files using Maven testResource bootstrap. Clean up deps. metadata fully functioning
>
>
> Those tests are still, sadly, broken. They all assume that $CWD (current working directory) is "..../metadata" and then create File objects relative to $CWD:
>
> -----------------------------------------------------
> % svn co https://svn.apache.org/repos/asf/incubator/oodt/trunk oodt
> % cd oodt
> % mvn test
> [INFO] Scanning for projects...
> [INFO] Reactor build order:
> ...
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running org.apache.oodt.cas.metadata.extractors.TestMetReader
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.204 sec
> Running org.apache.oodt.cas.metadata.extractors.TestExternMetExtractorConfigReader
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec <<< FAILURE!
> ...
> Failed tests:
> testReadConfig(org.apache.oodt.cas.metadata.extractors.TestExternMetExtractorConfigReader)
> testEval(org.apache.oodt.cas.metadata.preconditions.TestPreCondEvalUtils)
> testExtractor(org.apache.oodt.cas.metadata.extractors.TestExternMetExtractor)
> testExtractMetadata(org.apache.oodt.cas.metadata.extractors.TestCopyAndRewriteExtractor)
> testReplaceOrigMetFilePath(org.apache.oodt.cas.metadata.extractors.TestCopyAndRewriteExtractor)
> ...
> % echo bummer
> bummer
> -----------------------------------------------------
>
> The proper way to fetch resources, including test resources, is with Class.getResource* methods. If the API being tested absolutely has to File objects as input (instead of URLs, InputStreams, or Readers), then create temporary files by reading the file data from with Class.getResourceAsStream, and pass those temporary files to the APIs being tested.
>
>
>
> On 2010.Jul.16, at 10.13p, mattmann@apache.org wrote:
>
>> Author: mattmann
>> Date: Sat Jul 17 03:13:59 2010
>> New Revision: 965004
>>
>> URL: http://svn.apache.org/viewvc?rev=965004&view=rev
>> Log:
>
Re: svn commit: r965004 - in /incubator/oodt/trunk/metadata: ./ src/main/resources/ src/main/resources/examples/ src/test/org/apache/oodt/cas/metadata/extractors/ src/testdata/
Posted by Sean Kelly <se...@mac.com>.
> - OODT-15 WIP: fix problems with extractors in metadata; fix and enable all unit tests that were disabled per kelly; cleanup license headers on conf files; deploy conf files using Maven testResource bootstrap. Clean up deps. metadata fully functioning
Those tests are still, sadly, broken. They all assume that $CWD (current working directory) is "..../metadata" and then create File objects relative to $CWD:
-----------------------------------------------------
% svn co https://svn.apache.org/repos/asf/incubator/oodt/trunk oodt
% cd oodt
% mvn test
[INFO] Scanning for projects...
[INFO] Reactor build order:
...
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.oodt.cas.metadata.extractors.TestMetReader
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.204 sec
Running org.apache.oodt.cas.metadata.extractors.TestExternMetExtractorConfigReader
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec <<< FAILURE!
...
Failed tests:
testReadConfig(org.apache.oodt.cas.metadata.extractors.TestExternMetExtractorConfigReader)
testEval(org.apache.oodt.cas.metadata.preconditions.TestPreCondEvalUtils)
testExtractor(org.apache.oodt.cas.metadata.extractors.TestExternMetExtractor)
testExtractMetadata(org.apache.oodt.cas.metadata.extractors.TestCopyAndRewriteExtractor)
testReplaceOrigMetFilePath(org.apache.oodt.cas.metadata.extractors.TestCopyAndRewriteExtractor)
...
% echo bummer
bummer
-----------------------------------------------------
The proper way to fetch resources, including test resources, is with Class.getResource* methods. If the API being tested absolutely has to File objects as input (instead of URLs, InputStreams, or Readers), then create temporary files by reading the file data from with Class.getResourceAsStream, and pass those temporary files to the APIs being tested.
On 2010.Jul.16, at 10.13p, mattmann@apache.org wrote:
> Author: mattmann
> Date: Sat Jul 17 03:13:59 2010
> New Revision: 965004
>
> URL: http://svn.apache.org/viewvc?rev=965004&view=rev
> Log: