You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ctakes.apache.org by se...@apache.org on 2016/10/14 06:25:36 UTC
svn commit: r1764816 - in /ctakes/trunk:
ctakes-chunker-res/src/main/resources/org/apache/ctakes/chunker/pipeline/
ctakes-clinical-pipeline-res/ ctakes-clinical-pipeline-res/src/
ctakes-clinical-pipeline-res/src/main/ ctakes-clinical-pipeline-res/src/m...
Author: seanfinan
Date: Fri Oct 14 06:25:36 2016
New Revision: 1764816
URL: http://svn.apache.org/viewvc?rev=1764816&view=rev
Log:
Adding Clinical-Pipeline-res
Adding getReader, getAeNames to PipelineBuilder
named simple pipeline configuration files ".piper"
Adding piper file runner for easy cli
refactoring
Added:
ctakes/trunk/ctakes-chunker-res/src/main/resources/org/apache/ctakes/chunker/pipeline/
ctakes/trunk/ctakes-chunker-res/src/main/resources/org/apache/ctakes/chunker/pipeline/DefaultChunkerSubPipe.piper
ctakes/trunk/ctakes-clinical-pipeline-res/
ctakes/trunk/ctakes-clinical-pipeline-res/pom.xml
ctakes/trunk/ctakes-clinical-pipeline-res/src/
ctakes/trunk/ctakes-clinical-pipeline-res/src/main/
ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/
ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/org/
ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/org/apache/
ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/org/apache/ctakes/
ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/org/apache/ctakes/clinical/
ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/org/apache/ctakes/clinical/pipeline/
ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/org/apache/ctakes/clinical/pipeline/DefaultFastPipeline.piper
ctakes/trunk/ctakes-core-res/src/main/resources/org/apache/ctakes/core/pipeline/
ctakes/trunk/ctakes-core-res/src/main/resources/org/apache/ctakes/core/pipeline/DefaultTokenizerPipeline.piper
ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PiperFileReader.java
- copied, changed from r1764579, ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipelineReader.java
ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PiperFileRunner.java
ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/util/PropertyAeFactory.java
- copied, changed from r1764514, ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/PropertyAeFactory.java
ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldBuilderRunner.java
- copied, changed from r1764579, ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldBuildPipeRunner.java
ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldCuiPiperRunner.java
- copied, changed from r1764580, ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldCuiReadPipeRunner.java
ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPiperRunner.java
- copied, changed from r1764579, ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldReadPipeRunner.java
ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPropsPiperRunner.java
- copied, changed from r1764580, ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPropsReadPipeRunner.java
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.piper
- copied, changed from r1764580, ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.txt
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldCuiPipeline.piper
- copied unchanged from r1764580, ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldCuiPipeline.txt
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPipeline.piper
- copied unchanged from r1764579, ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPipeline.txt
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.piper
- copied, changed from r1764580, ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.txt
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldTkPropsPipeline.piper
- copied, changed from r1764580, ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldTkPropsPipeline.txt
Removed:
ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/PropertyAeFactory.java
ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipelineReader.java
ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldBuildPipeRunner.java
ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldCuiReadPipeRunner.java
ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPropsReadPipeRunner.java
ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldReadPipeRunner.java
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.txt
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldCuiPipeline.txt
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPipeline.txt
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.txt
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldTkPropsPipeline.txt
Modified:
ctakes/trunk/ctakes-clinical-pipeline/pom.xml
ctakes/trunk/ctakes-core/pom.xml
ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/StartFinishLogger.java
ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipelineBuilder.java
ctakes/trunk/ctakes-dictionary-lookup-fast/src/main/java/org/apache/ctakes/dictionary/lookup2/consumer/SemanticCleanupTermConsumer.java
ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/AssertionDefaults.properties
Added: ctakes/trunk/ctakes-chunker-res/src/main/resources/org/apache/ctakes/chunker/pipeline/DefaultChunkerSubPipe.piper
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-chunker-res/src/main/resources/org/apache/ctakes/chunker/pipeline/DefaultChunkerSubPipe.piper?rev=1764816&view=auto
==============================================================================
--- ctakes/trunk/ctakes-chunker-res/src/main/resources/org/apache/ctakes/chunker/pipeline/DefaultChunkerSubPipe.piper (added)
+++ ctakes/trunk/ctakes-chunker-res/src/main/resources/org/apache/ctakes/chunker/pipeline/DefaultChunkerSubPipe.piper Fri Oct 14 06:25:36 2016
@@ -0,0 +1,5 @@
+// Commands and parameters to create a default chunker processing sub-pipeline. This is not a full pipeline.
+
+add Chunker
+addDescription adjuster.ChunkAdjuster NP,NP 1
+addDescription adjuster.ChunkAdjuster NP,PP,NP 2
Added: ctakes/trunk/ctakes-clinical-pipeline-res/pom.xml
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-clinical-pipeline-res/pom.xml?rev=1764816&view=auto
==============================================================================
--- ctakes/trunk/ctakes-clinical-pipeline-res/pom.xml (added)
+++ ctakes/trunk/ctakes-clinical-pipeline-res/pom.xml Fri Oct 14 06:25:36 2016
@@ -0,0 +1,11 @@
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.ctakes</groupId>
+ <artifactId>ctakes</artifactId>
+ <version>3.2.3-SNAPSHOT</version>
+ </parent>
+ <artifactId>ctakes-clinical-pipeline-res</artifactId>
+ <name>ctakes-clinical-pipeline-res</name>
+ <description>Resources for clinical pipeline</description>
+</project>
\ No newline at end of file
Added: ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/org/apache/ctakes/clinical/pipeline/DefaultFastPipeline.piper
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/org/apache/ctakes/clinical/pipeline/DefaultFastPipeline.piper?rev=1764816&view=auto
==============================================================================
--- ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/org/apache/ctakes/clinical/pipeline/DefaultFastPipeline.piper (added)
+++ ctakes/trunk/ctakes-clinical-pipeline-res/src/main/resources/org/apache/ctakes/clinical/pipeline/DefaultFastPipeline.piper Fri Oct 14 06:25:36 2016
@@ -0,0 +1,24 @@
+// Commands and parameters to create a default plaintext document processing pipeline with UMLS lookup
+
+// Load a simple token processing pipeline from another pipeline file
+load org/apache/ctakes/core/pipeline/DefaultTokenizerPipeline.piper
+
+// Add non-core annotators
+add ContextDependentTokenizerAnnotator
+addDescription POSTagger
+
+// Default fast dictionary lookup
+add DefaultJCasTermAnnotator
+
+// Add the Dependency parser for use by cleartk
+addDescription ClearNLPDependencyParserAE
+
+// Add the cleartk package for cleartk class lookups
+addPackage org.apache.ctakes.assertion.medfacts.cleartk
+// Add the cleartk attribute engines
+addDescription PolarityCleartkAnalysisEngine
+addDescription UncertaintyCleartkAnalysisEngine
+addDescription HistoryCleartkAnalysisEngine
+addDescription ConditionalCleartkAnalysisEngine
+addDescription GenericCleartkAnalysisEngine
+addDescription SubjectCleartkAnalysisEngine
Modified: ctakes/trunk/ctakes-clinical-pipeline/pom.xml
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-clinical-pipeline/pom.xml?rev=1764816&r1=1764815&r2=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-clinical-pipeline/pom.xml (original)
+++ ctakes/trunk/ctakes-clinical-pipeline/pom.xml Fri Oct 14 06:25:36 2016
@@ -106,6 +106,10 @@
<artifactId>cleartk-ml</artifactId>
<version>2.0.0</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.ctakes</groupId>
+ <artifactId>ctakes-clinical-pipeline-res</artifactId>
+ </dependency>
</dependencies>
<profiles>
<profile>
Added: ctakes/trunk/ctakes-core-res/src/main/resources/org/apache/ctakes/core/pipeline/DefaultTokenizerPipeline.piper
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-core-res/src/main/resources/org/apache/ctakes/core/pipeline/DefaultTokenizerPipeline.piper?rev=1764816&view=auto
==============================================================================
--- ctakes/trunk/ctakes-core-res/src/main/resources/org/apache/ctakes/core/pipeline/DefaultTokenizerPipeline.piper (added)
+++ ctakes/trunk/ctakes-core-res/src/main/resources/org/apache/ctakes/core/pipeline/DefaultTokenizerPipeline.piper Fri Oct 14 06:25:36 2016
@@ -0,0 +1,5 @@
+// Commands and parameters to create a default token processing pipeline
+
+add SimpleSegmentAnnotator
+add SentenceDetector
+add TokenizerAnnotatorPTB
Modified: ctakes/trunk/ctakes-core/pom.xml
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-core/pom.xml?rev=1764816&r1=1764815&r2=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-core/pom.xml (original)
+++ ctakes/trunk/ctakes-core/pom.xml Fri Oct 14 06:25:36 2016
@@ -103,6 +103,10 @@
<groupId>org.apache.uima</groupId>
<artifactId>uimaj-examples</artifactId> <!-- for XCasWriterCasConsumer -->
</dependency>
+ <dependency>
+ <groupId>com.lexicalscope.jewelcli</groupId>
+ <artifactId>jewelcli</artifactId>
+ </dependency>
<dependency>
Modified: ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/StartFinishLogger.java
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/StartFinishLogger.java?rev=1764816&r1=1764815&r2=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/StartFinishLogger.java (original)
+++ ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/StartFinishLogger.java Fri Oct 14 06:25:36 2016
@@ -130,7 +130,7 @@ public class StartFinishLogger extends J
public static AnalysisEngineDescription createLoggedDescription( final AnalysisEngineDescription mainDescription )
throws ResourceInitializationException {
final String name = mainDescription.getAnnotatorImplementationName();
- return org.apache.uima.fit.factory.AnalysisEngineFactory.createEngineDescription(
+ return AnalysisEngineFactory.createEngineDescription(
createDescription( name, true ),
mainDescription,
createDescription( name, false ) );
Modified: ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipelineBuilder.java
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipelineBuilder.java?rev=1764816&r1=1764815&r2=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipelineBuilder.java (original)
+++ ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipelineBuilder.java Fri Oct 14 06:25:36 2016
@@ -1,9 +1,9 @@
package org.apache.ctakes.core.pipeline;
-import org.apache.ctakes.core.ae.PropertyAeFactory;
import org.apache.ctakes.core.cc.XmiWriterCasConsumerCtakes;
import org.apache.ctakes.core.cr.FilesInDirectoryCollectionReader;
+import org.apache.ctakes.core.util.PropertyAeFactory;
import org.apache.log4j.Logger;
import org.apache.uima.UIMAException;
import org.apache.uima.analysis_component.AnalysisComponent;
@@ -18,6 +18,7 @@ import org.apache.uima.resource.Resource
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.List;
/**
@@ -34,11 +35,13 @@ final public class PipelineBuilder {
static private final Logger LOGGER = Logger.getLogger( "PipelineBuilder" );
- private final List<AnalysisEngineDescription> _aeList;
+ private final List<String> _aeNameList;
+ private final List<AnalysisEngineDescription> _descList;
private CollectionReader _reader;
public PipelineBuilder() {
- _aeList = new ArrayList<>();
+ _aeNameList = new ArrayList<>();
+ _descList = new ArrayList<>();
}
/**
@@ -103,6 +106,14 @@ final public class PipelineBuilder {
}
/**
+ *
+ * @return the CollectionReader for the pipeline or null if none has been specified
+ */
+ public CollectionReader getReader() {
+ return _reader;
+ }
+
+ /**
* Use of this method is order-specific.
*
* @param component ae or cc component class to add to the pipeline
@@ -112,7 +123,8 @@ final public class PipelineBuilder {
*/
public PipelineBuilder add( final Class<? extends AnalysisComponent> component,
final Object... parameters ) throws ResourceInitializationException {
- _aeList.add( PropertyAeFactory.getInstance().createDescription( component, parameters ) );
+ _aeNameList.add( component.getName() );
+ _descList.add( PropertyAeFactory.getInstance().createDescription( component, parameters ) );
return this;
}
@@ -127,7 +139,8 @@ final public class PipelineBuilder {
*/
public PipelineBuilder addLogged( final Class<? extends AnalysisComponent> component,
final Object... parameters ) throws ResourceInitializationException {
- _aeList.add( PropertyAeFactory.getInstance().createLoggedDescription( component, parameters ) );
+ _aeNameList.add( component.getName() );
+ _descList.add( PropertyAeFactory.getInstance().createLoggedDescription( component, parameters ) );
return this;
}
@@ -138,11 +151,20 @@ final public class PipelineBuilder {
* @return this PipelineBuilder
*/
public PipelineBuilder addDescription( final AnalysisEngineDescription description ) {
- _aeList.add( description );
+ _aeNameList.add( description.getAnnotatorImplementationName() );
+ _descList.add( description );
return this;
}
/**
+ *
+ * @return an ordered list of the annotation engines in the pipeline
+ */
+ public List<String> getAeNames() {
+ return Collections.unmodifiableList( _aeNameList );
+ }
+
+ /**
* Adds ae that maintains CUI information throughout the run.
* CUI information can later be accessed using the {@link CuiCollector} singleton
* Use of this method is order-specific.
@@ -204,7 +226,7 @@ final public class PipelineBuilder {
return this;
}
final AggregateBuilder builder = new AggregateBuilder();
- _aeList.forEach( builder::add );
+ _descList.forEach( builder::add );
final AnalysisEngineDescription desc = builder.createAggregateDescription();
SimplePipeline.runPipeline( _reader, desc );
return this;
@@ -227,7 +249,7 @@ final public class PipelineBuilder {
final JCas jcas = JCasFactory.createJCas();
jcas.setDocumentText( text );
final AggregateBuilder builder = new AggregateBuilder();
- _aeList.forEach( builder::add );
+ _descList.forEach( builder::add );
final AnalysisEngineDescription desc = builder.createAggregateDescription();
SimplePipeline.runPipeline( jcas, desc );
return this;
Copied: ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PiperFileReader.java (from r1764579, ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipelineReader.java)
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PiperFileReader.java?p2=ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PiperFileReader.java&p1=ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipelineReader.java&r1=1764579&r2=1764816&rev=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipelineReader.java (original)
+++ ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PiperFileReader.java Fri Oct 14 06:25:36 2016
@@ -50,9 +50,9 @@ import java.util.regex.Pattern;
* @version %I%
* @since 10/10/2016
*/
-final public class PipelineReader {
+final public class PiperFileReader {
- static private final Logger LOGGER = Logger.getLogger( "PipelineReader" );
+ static private final Logger LOGGER = Logger.getLogger( "PiperFileReader" );
static private final String[] CTAKES_PACKAGES
= { "core",
@@ -90,7 +90,7 @@ final public class PipelineReader {
/**
* Create and empty PipelineReader
*/
- public PipelineReader() {
+ public PiperFileReader() {
_builder = new PipelineBuilder();
_userPackages = new ArrayList<>();
}
@@ -101,7 +101,7 @@ final public class PipelineReader {
* @param filePath path to the pipeline command file
* @throws UIMAException if the pipeline cannot be loaded
*/
- public PipelineReader( final String filePath ) throws UIMAException {
+ public PiperFileReader( final String filePath ) throws UIMAException {
_builder = new PipelineBuilder();
_userPackages = new ArrayList<>();
loadPipelineFile( filePath );
@@ -114,7 +114,8 @@ final public class PipelineReader {
*/
public void loadPipelineFile( final String filePath ) throws UIMAException {
try ( final BufferedReader reader
- = new BufferedReader( new InputStreamReader( FileLocator.getAsStream( filePath ) ) ) ) {
+ = new BufferedReader( new InputStreamReader(
+ FileLocator.getAsStream( FileLocator.getFullPath( filePath ) ) ) ) ) {
String line = reader.readLine();
while ( line != null ) {
line = line.trim();
@@ -131,7 +132,7 @@ final public class PipelineReader {
line = reader.readLine();
}
} catch ( IOException ioE ) {
- LOGGER.error( "Property File not found: " + filePath );
+ LOGGER.error( "Piper File not found: " + filePath );
}
}
Added: ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PiperFileRunner.java
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PiperFileRunner.java?rev=1764816&view=auto
==============================================================================
--- ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PiperFileRunner.java (added)
+++ ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PiperFileRunner.java Fri Oct 14 06:25:36 2016
@@ -0,0 +1,120 @@
+package org.apache.ctakes.core.pipeline;
+
+
+import com.lexicalscope.jewel.cli.CliFactory;
+import com.lexicalscope.jewel.cli.Option;
+import org.apache.log4j.Logger;
+import org.apache.uima.UIMAException;
+
+import java.io.IOException;
+
+/**
+ * @author SPF , chip-nlp
+ * @version %I%
+ * @since 10/13/2016
+ */
+public class PiperFileRunner {
+
+ static private final Logger LOGGER = Logger.getLogger( "PiperFileRunner" );
+
+ interface RunOptions {
+ @Option(
+ shortName = "p",
+ longName = "piper",
+ description = "path to the piper file containing commands and parameters for pipeline configuration." )
+ String getPiperPath();
+
+ @Option(
+ shortName = "i",
+ longName = "inputDir",
+ description = "path to the directory containing the clinical notes to be processed.",
+ defaultValue = "" )
+ String getInputDirectory();
+
+ @Option(
+ shortName = "o",
+ longName = "outputDir",
+ description = "path to the directory where the output files are to be written.",
+ defaultValue = "" )
+ String getOutputDirectory();
+
+ @Option(
+ longName = "xmiOut",
+ description = "path to the directory where xmi files are to be written. Adds XmiWriter to pipeline.",
+ defaultValue = "" )
+ String getXmiOutDirectory();
+
+ @Option(
+ longName = "user",
+ description = "UMLS username.",
+ defaultValue = "" )
+ String getUmlsUserName();
+
+ @Option(
+ longName = "pass",
+ description = "UMLS user password.",
+ defaultValue = "" )
+ String getUmlsPassword();
+
+ @Option(
+ shortName = { "?", "h" },
+ longName = "help",
+ description = "print usage",
+ helpRequest = true )
+ boolean isHelpWanted();
+
+ }
+
+ /**
+ * @param args general run options
+ */
+ public static void main( final String... args ) {
+ final RunOptions options = CliFactory.parseArguments( RunOptions.class, args );
+ try {
+ final PiperFileReader reader = new PiperFileReader();
+ final PipelineBuilder builder = reader.getBuilder();
+ // set the input directory parameter if needed
+ final String inputDir = options.getInputDirectory();
+ if ( !inputDir.isEmpty() ) {
+ builder.addParameters( "InputDirectory", inputDir );
+ }
+ // set the output directory parameter if needed
+ final String outputDir = options.getOutputDirectory();
+ final String xmiOutDir = options.getXmiOutDirectory();
+ if ( !outputDir.isEmpty() ) {
+ builder.addParameters( "OutputDirectory", outputDir );
+ } else if ( !xmiOutDir.isEmpty() ) {
+ builder.addParameters( "OutputDirectory", xmiOutDir );
+ }
+ // set the umls user and password parameters if needed
+ final String umlsUser = options.getUmlsUserName();
+ if ( !umlsUser.isEmpty() ) {
+ builder.addParameters( "umlsUser", umlsUser );
+ }
+ final String umlsPass = options.getUmlsPassword();
+ if ( !umlsPass.isEmpty() ) {
+ builder.addParameters( "umlsPass", umlsPass );
+ }
+ // load the piper file
+ reader.loadPipelineFile( options.getPiperPath() );
+ // if an input directory was specified but the piper didn't add a collection reader, add the default reader
+ if ( !inputDir.isEmpty() && builder.getReader() == null ) {
+ builder.readFiles( inputDir );
+ }
+ // if an xmi output directory was specified but the piper didn't add the xmi writer, add the
+ if ( !xmiOutDir.isEmpty() ) {
+ if ( !builder.getAeNames().stream().map( String::toLowerCase )
+ .anyMatch( n -> n.contains( "xmiwriter" ) ) ) {
+ builder.writeXMIs( xmiOutDir );
+ }
+ }
+ // run the pipeline
+ builder.run();
+ } catch ( UIMAException | IOException multE ) {
+ LOGGER.error( multE.getMessage() );
+ System.exit( 1 );
+ }
+ }
+
+
+}
Copied: ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/util/PropertyAeFactory.java (from r1764514, ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/PropertyAeFactory.java)
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/util/PropertyAeFactory.java?p2=ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/util/PropertyAeFactory.java&p1=ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/PropertyAeFactory.java&r1=1764514&r2=1764816&rev=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/ae/PropertyAeFactory.java (original)
+++ ctakes/trunk/ctakes-core/src/main/java/org/apache/ctakes/core/util/PropertyAeFactory.java Fri Oct 14 06:25:36 2016
@@ -1,5 +1,6 @@
-package org.apache.ctakes.core.ae;
+package org.apache.ctakes.core.util;
+import org.apache.ctakes.core.ae.StartFinishLogger;
import org.apache.ctakes.core.resource.FileLocator;
import org.apache.log4j.Logger;
import org.apache.uima.analysis_component.AnalysisComponent;
Modified: ctakes/trunk/ctakes-dictionary-lookup-fast/src/main/java/org/apache/ctakes/dictionary/lookup2/consumer/SemanticCleanupTermConsumer.java
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-dictionary-lookup-fast/src/main/java/org/apache/ctakes/dictionary/lookup2/consumer/SemanticCleanupTermConsumer.java?rev=1764816&r1=1764815&r2=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-dictionary-lookup-fast/src/main/java/org/apache/ctakes/dictionary/lookup2/consumer/SemanticCleanupTermConsumer.java (original)
+++ ctakes/trunk/ctakes-dictionary-lookup-fast/src/main/java/org/apache/ctakes/dictionary/lookup2/consumer/SemanticCleanupTermConsumer.java Fri Oct 14 06:25:36 2016
@@ -27,7 +27,7 @@ import java.util.*;
*/
public class SemanticCleanupTermConsumer extends AbstractTermConsumer {
- static private final Logger LOGGER = Logger.getLogger( "MetaWsdTermConsumer" );
+ static private final Logger LOGGER = Logger.getLogger( "SemanticCleanupTermConsumer" );
private final TermConsumer _idHitConsumer;
Copied: ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldBuilderRunner.java (from r1764579, ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldBuildPipeRunner.java)
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldBuilderRunner.java?p2=ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldBuilderRunner.java&p1=ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldBuildPipeRunner.java&r1=1764579&r2=1764816&rev=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldBuildPipeRunner.java (original)
+++ ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldBuilderRunner.java Fri Oct 14 06:25:36 2016
@@ -23,13 +23,13 @@ import java.io.IOException;
* @version %I%
* @since 10/10/2016
*/
-final public class HelloWorldBuildPipeRunner {
+final public class HelloWorldBuilderRunner {
- static private final Logger LOGGER = Logger.getLogger( "HelloWorldBuildPipeRunner" );
+ static private final Logger LOGGER = Logger.getLogger( "HelloWorldBuilderRunner" );
static private final String DOC_TEXT = "Hello World!";
- private HelloWorldBuildPipeRunner() {
+ private HelloWorldBuilderRunner() {
}
/**
Copied: ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldCuiPiperRunner.java (from r1764580, ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldCuiReadPipeRunner.java)
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldCuiPiperRunner.java?p2=ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldCuiPiperRunner.java&p1=ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldCuiReadPipeRunner.java&r1=1764580&r2=1764816&rev=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldCuiReadPipeRunner.java (original)
+++ ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldCuiPiperRunner.java Fri Oct 14 06:25:36 2016
@@ -3,14 +3,14 @@ package org.apache.ctakes.examples.pipel
import org.apache.ctakes.core.pipeline.CuiCollector;
import org.apache.ctakes.core.pipeline.PipelineBuilder;
-import org.apache.ctakes.core.pipeline.PipelineReader;
+import org.apache.ctakes.core.pipeline.PiperFileReader;
import org.apache.log4j.Logger;
import org.apache.uima.UIMAException;
import java.io.IOException;
/**
- * Build and run a pipeline using a {@link PipelineReader} and a {@link PipelineBuilder}.
+ * Build and run a pipeline using a {@link PiperFileReader} and a {@link PipelineBuilder}.
* <p>
* Example of a running a pipeline programatically w/o uima xml descriptor xml files
* Adds the default Tokenization pipeline and adding the Example HelloWorld Annotator
@@ -19,15 +19,15 @@ import java.io.IOException;
* @version %I%
* @since 10/10/2016
*/
-final public class HelloWorldCuiReadPipeRunner {
+final public class HelloWorldCuiPiperRunner {
- static private final Logger LOGGER = Logger.getLogger( "HelloWorldReadPipeRunner" );
+ static private final Logger LOGGER = Logger.getLogger( "HelloWorldCuiPiperRunner" );
- static private final String PIPELINE_FILE_PATH = "org/apache/ctakes/examples/pipeline/HelloWorldCuiPipeline.txt";
+ static private final String PIPELINE_FILE_PATH = "org/apache/ctakes/examples/pipeline/HelloWorldCuiPipeline.piper";
static private final String DOC_TEXT = "Hello World! I have allergies to nuts, bee stings, and shark bites.";
- private HelloWorldCuiReadPipeRunner() {
+ private HelloWorldCuiPiperRunner() {
}
/**
@@ -36,7 +36,7 @@ final public class HelloWorldCuiReadPipe
public static void main( final String... args ) {
try {
// Add a simple pre-defined existing pipeline for Tokenization from file
- final PipelineReader reader = new PipelineReader( PIPELINE_FILE_PATH );
+ final PiperFileReader reader = new PiperFileReader( PIPELINE_FILE_PATH );
PipelineBuilder builder = reader.getBuilder();
if ( args.length > 0 ) {
// Example to save the Aggregate descriptor to an xml file for external use such as the UIMA CVD
Copied: ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPiperRunner.java (from r1764579, ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldReadPipeRunner.java)
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPiperRunner.java?p2=ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPiperRunner.java&p1=ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldReadPipeRunner.java&r1=1764579&r2=1764816&rev=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldReadPipeRunner.java (original)
+++ ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPiperRunner.java Fri Oct 14 06:25:36 2016
@@ -2,14 +2,14 @@ package org.apache.ctakes.examples.pipel
import org.apache.ctakes.core.pipeline.PipelineBuilder;
-import org.apache.ctakes.core.pipeline.PipelineReader;
+import org.apache.ctakes.core.pipeline.PiperFileReader;
import org.apache.log4j.Logger;
import org.apache.uima.UIMAException;
import java.io.IOException;
/**
- * Build and run a pipeline using a {@link PipelineReader} and a {@link PipelineBuilder}.
+ * Build and run a pipeline using a {@link PiperFileReader} and a {@link PipelineBuilder}.
* <p>
* Example of a running a pipeline programatically w/o uima xml descriptor xml files
* Adds the default Tokenization pipeline and adding the Example HelloWorld Annotator
@@ -18,15 +18,15 @@ import java.io.IOException;
* @version %I%
* @since 10/10/2016
*/
-final public class HelloWorldReadPipeRunner {
+final public class HelloWorldPiperRunner {
- static private final Logger LOGGER = Logger.getLogger( "HelloWorldReadPipeRunner" );
+ static private final Logger LOGGER = Logger.getLogger( "HelloWorldPiperRunner" );
- static private final String PIPELINE_FILE_PATH = "org/apache/ctakes/examples/pipeline/HelloWorldPipeline.txt";
+ static private final String PIPELINE_FILE_PATH = "org/apache/ctakes/examples/pipeline/HelloWorldPipeline.piper";
static private final String DOC_TEXT = "Hello World!";
- private HelloWorldReadPipeRunner() {
+ private HelloWorldPiperRunner() {
}
/**
@@ -35,7 +35,7 @@ final public class HelloWorldReadPipeRun
public static void main( final String... args ) {
try {
// Add a simple pre-defined existing pipeline for Tokenization from file
- final PipelineReader reader = new PipelineReader( PIPELINE_FILE_PATH );
+ final PiperFileReader reader = new PiperFileReader( PIPELINE_FILE_PATH );
PipelineBuilder builder = reader.getBuilder();
if ( args.length > 0 ) {
// Example to save the Aggregate descriptor to an xml file for external use such as the UIMA CVD
Copied: ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPropsPiperRunner.java (from r1764580, ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPropsReadPipeRunner.java)
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPropsPiperRunner.java?p2=ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPropsPiperRunner.java&p1=ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPropsReadPipeRunner.java&r1=1764580&r2=1764816&rev=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPropsReadPipeRunner.java (original)
+++ ctakes/trunk/ctakes-examples/src/main/java/org/apache/ctakes/examples/pipeline/HelloWorldPropsPiperRunner.java Fri Oct 14 06:25:36 2016
@@ -3,14 +3,14 @@ package org.apache.ctakes.examples.pipel
import org.apache.ctakes.core.pipeline.EntityCollector;
import org.apache.ctakes.core.pipeline.PipelineBuilder;
-import org.apache.ctakes.core.pipeline.PipelineReader;
+import org.apache.ctakes.core.pipeline.PiperFileReader;
import org.apache.log4j.Logger;
import org.apache.uima.UIMAException;
import java.io.IOException;
/**
- * Build and run a pipeline using a {@link PipelineReader} and a {@link PipelineBuilder}.
+ * Build and run a pipeline using a {@link PiperFileReader} and a {@link PipelineBuilder}.
* <p>
* Example of a running a pipeline programatically w/o uima xml descriptor xml files
* Adds the default Tokenization pipeline and adding the Example HelloWorld Annotator
@@ -19,17 +19,17 @@ import java.io.IOException;
* @version %I%
* @since 10/10/2016
*/
-final public class HelloWorldPropsReadPipeRunner {
+final public class HelloWorldPropsPiperRunner {
- static private final Logger LOGGER = Logger.getLogger( "HelloWorldReadPipeRunner" );
+ static private final Logger LOGGER = Logger.getLogger( "HelloWorldPropsPiperRunner" );
static private final String PROPS_PIPELINE_FILE_PATH
- = "org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.txt";
+ = "org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.piper";
static private final String DOC_TEXT
= "Hello World! I feel no pain. My father takes aspirin. My sister might have a headache.";
- private HelloWorldPropsReadPipeRunner() {
+ private HelloWorldPropsPiperRunner() {
}
/**
@@ -38,7 +38,7 @@ final public class HelloWorldPropsReadPi
public static void main( final String... args ) {
try {
// Add a simple pre-defined existing pipeline for Tokenization from file
- final PipelineReader reader = new PipelineReader( PROPS_PIPELINE_FILE_PATH );
+ final PiperFileReader reader = new PiperFileReader( PROPS_PIPELINE_FILE_PATH );
// Add the property extraction pipeline
PipelineBuilder builder = reader.getBuilder();
if ( args.length > 0 ) {
Modified: ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/AssertionDefaults.properties
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/AssertionDefaults.properties?rev=1764816&r1=1764815&r2=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/AssertionDefaults.properties (original)
+++ ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/AssertionDefaults.properties Fri Oct 14 06:25:36 2016
@@ -1,6 +1,6 @@
# Parameters for AssertionAnalysisEngine and ConceptConverterAnalysisEngine
-assertionModelResourceImpl=file:org/apache/ctakes/assertion/models/i2b2.model
-scopeModelResourceImpl=file:org/apache/ctakes/assertion/models/scope.model
-cueModelResourceImpl=file:org/apache/ctakes/assertion/models/cue.model
-enabledFeaturesResourceImpl=file:org/apache/ctakes/assertion/models/featureFile11b
-posModelResourceImpl=file:org/apache/ctakes/assertion/models/pos.model
+assertionModelResource=file:org/apache/ctakes/assertion/models/i2b2.model
+scopeModelResource=file:org/apache/ctakes/assertion/models/scope.model
+cueModelResource=file:org/apache/ctakes/assertion/models/cue.model
+enabledFeaturesResource=file:org/apache/ctakes/assertion/models/featureFile11b
+posModelResource=file:org/apache/ctakes/assertion/models/pos.model
Copied: ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.piper (from r1764580, ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.txt)
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.piper?p2=ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.piper&p1=ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.txt&r1=1764580&r2=1764816&rev=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.txt (original)
+++ ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertPropsPipeline.piper Fri Oct 14 06:25:36 2016
@@ -2,7 +2,7 @@
// with Entity Property output
// Load a simple token processing pipeline from another file
-load org/apache/ctakes/examples/pipeline/HelloWorldPipeline.txt
+load org/apache/ctakes/examples/pipeline/HelloWorldPipeline.piper
// Assertion engines require dependencies
addDescription ClearNLPDependencyParserAE
Copied: ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.piper (from r1764580, ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.txt)
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.piper?p2=ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.piper&p1=ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.txt&r1=1764580&r2=1764816&rev=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.txt (original)
+++ ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldPropsPipeline.piper Fri Oct 14 06:25:36 2016
@@ -2,7 +2,7 @@
// with Entity Property output
// Load a simple token processing pipeline from another file
-load org/apache/ctakes/examples/pipeline/HelloWorldPipeline.txt
+load org/apache/ctakes/examples/pipeline/HelloWorldPipeline.piper
// Add the context annotator for simple annotation properties
// The default call sets it to detect negation
Copied: ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldTkPropsPipeline.piper (from r1764580, ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldTkPropsPipeline.txt)
URL: http://svn.apache.org/viewvc/ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldTkPropsPipeline.piper?p2=ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldTkPropsPipeline.piper&p1=ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldTkPropsPipeline.txt&r1=1764580&r2=1764816&rev=1764816&view=diff
==============================================================================
--- ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldTkPropsPipeline.txt (original)
+++ ctakes/trunk/ctakes-examples/src/main/resources/org/apache/ctakes/examples/pipeline/HelloWorldTkPropsPipeline.piper Fri Oct 14 06:25:36 2016
@@ -2,7 +2,7 @@
// with Entity Property output
// Load a simple token processing pipeline from another file
-load org/apache/ctakes/examples/pipeline/HelloWorldPipeline.txt
+load org/apache/ctakes/examples/pipeline/HelloWorldPipeline.piper
// I don't think that the chunker is strictly necessary, but here it is ...
add Chunker