You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ctakes.apache.org by se...@apache.org on 2022/12/22 00:32:32 UTC
[ctakes] branch main updated: PropertyAeFactory : much faster AE Description creation Added "initializing" and "processing" log messages to AEs, switched addLogged to add in their pipers Changed comment in runPiperFile scripts to --key instead of --user and --pass Information added to web-rest readme.
This is an automated email from the ASF dual-hosted git repository.
seanfinan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ctakes.git
The following commit(s) were added to refs/heads/main by this push:
new b3e1d20 PropertyAeFactory : much faster AE Description creation Added "initializing" and "processing" log messages to AEs, switched addLogged to add in their pipers Changed comment in runPiperFile scripts to --key instead of --user and --pass Information added to web-rest readme.
b3e1d20 is described below
commit b3e1d20bcb5e29b640689bed559ed61b454699b6
Author: Sean Finan <se...@childrens.harvard.edu>
AuthorDate: Wed Dec 21 19:32:20 2022 -0500
PropertyAeFactory : much faster AE Description creation
Added "initializing" and "processing" log messages to AEs, switched addLogged to add in their pipers
Changed comment in runPiperFile scripts to --key instead of --user and --pass
Information added to web-rest readme.
---
.../cleartk/AssertionCleartkAnalysisEngine.java | 1 +
.../assertion/pipeline/AssertionSubPipe.piper | 2 +-
.../pipeline/AttributeCleartkSubPipe.piper | 8 +----
.../pipeline/WindowedAttributeCleartkSubPipe.piper | 2 +-
.../core/ae/SentenceDetectorAnnotatorBIO.java | 2 ++
.../ctakes/core/pipeline/PipeBitLocator.java | 6 ++--
.../apache/ctakes/core/util/PropertyAeFactory.java | 16 ++++++++--
.../core/pipeline/FullTokenizerPipeline.piper | 2 +-
.../parser/ae/ClearNLPSemanticRoleLabelerAE.java | 3 +-
ctakes-distribution/src/main/bin/runPiperFile.bat | 3 +-
ctakes-distribution/src/main/bin/runPiperFile.sh | 3 +-
.../examples/pipeline/ApacheConDemoRelation.piper | 2 +-
.../examples/pipeline/ApacheConDemoSection.piper | 2 +-
.../examples/pipeline/HelloWorldAssertProps.piper | 2 +-
ctakes-web-rest/README | 37 ++++++++++++++++++----
15 files changed, 62 insertions(+), 29 deletions(-)
diff --git a/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/cleartk/AssertionCleartkAnalysisEngine.java b/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/cleartk/AssertionCleartkAnalysisEngine.java
index 70fa871..7dddafd 100644
--- a/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/cleartk/AssertionCleartkAnalysisEngine.java
+++ b/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/cleartk/AssertionCleartkAnalysisEngine.java
@@ -309,6 +309,7 @@ public abstract class AssertionCleartkAnalysisEngine extends
@Override
public void process( JCas jCas ) throws AnalysisEngineProcessException {
+ logger.info( "Processing ..." );
String documentId = DocIdUtil.getDocumentID( jCas );
String domainId = "";
String domainFeature = null;
diff --git a/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/AssertionSubPipe.piper b/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/AssertionSubPipe.piper
index 5e4d62c..8b7f889 100644
--- a/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/AssertionSubPipe.piper
+++ b/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/AssertionSubPipe.piper
@@ -3,7 +3,7 @@
// Add the Dependency parser for use by assertion
addDescription ClearNLPDependencyParserAE
// Add the Semantic Role Labeler parser for use by assertion
-addLogged ClearNLPSemanticRoleLabelerAE
+add ClearNLPSemanticRoleLabelerAE
// Add the assertion packages for class lookups
package org.apache.ctakes.assertion.medfacts
diff --git a/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/AttributeCleartkSubPipe.piper b/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/AttributeCleartkSubPipe.piper
index 832be75..64d4ad0 100644
--- a/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/AttributeCleartkSubPipe.piper
+++ b/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/AttributeCleartkSubPipe.piper
@@ -5,17 +5,11 @@
addDescription ClearNLPDependencyParserAE
// Add the Semantic Role Labeler parser for use by cleartk.
// Not necessary for assertion anymore, but it is used for temporal, coref, and others so be careful removing this.
-addLogged ClearNLPSemanticRoleLabelerAE
+add ClearNLPSemanticRoleLabelerAE
-// Add the cleartk package for cleartk class lookups
-package org.apache.ctakes.assertion.medfacts.cleartk
-
-// Add the cleartk attribute engines, logging for them as a block
-add StartFinishLogger LOGGER_NAME=CleartkAnalysisEngine LOGGER_TASK="Assigning Attributes" IS_START=true
addDescription PolarityCleartkAnalysisEngine
addDescription UncertaintyCleartkAnalysisEngine
addDescription HistoryCleartkAnalysisEngine
addDescription ConditionalCleartkAnalysisEngine
addDescription GenericCleartkAnalysisEngine
addDescription SubjectCleartkAnalysisEngine
-add StartFinishLogger LOGGER_NAME=CleartkAnalysisEngine LOGGER_TASK="Assigning Attributes"
diff --git a/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/WindowedAttributeCleartkSubPipe.piper b/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/WindowedAttributeCleartkSubPipe.piper
index 7c4ffd4..e33a6e2 100644
--- a/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/WindowedAttributeCleartkSubPipe.piper
+++ b/ctakes-assertion/src/user/resources/org/apache/ctakes/assertion/pipeline/WindowedAttributeCleartkSubPipe.piper
@@ -3,7 +3,7 @@
// Add the Dependency parser for use by cleartk
addDescription ClearNLPDependencyParserAE
// Add the Semantic Role Labeler parser for use by cleartk
-addLogged ClearNLPSemanticRoleLabelerAE
+add ClearNLPSemanticRoleLabelerAE
// Add the cleartk package for cleartk class lookups
package org.apache.ctakes.assertion.medfacts.cleartk
diff --git a/ctakes-core/src/main/java/org/apache/ctakes/core/ae/SentenceDetectorAnnotatorBIO.java b/ctakes-core/src/main/java/org/apache/ctakes/core/ae/SentenceDetectorAnnotatorBIO.java
index 98029a4..9542764 100644
--- a/ctakes-core/src/main/java/org/apache/ctakes/core/ae/SentenceDetectorAnnotatorBIO.java
+++ b/ctakes-core/src/main/java/org/apache/ctakes/core/ae/SentenceDetectorAnnotatorBIO.java
@@ -78,6 +78,7 @@ public class SentenceDetectorAnnotatorBIO extends CleartkAnnotator<String>{
@Override
public void initialize(UimaContext context)
throws ResourceInitializationException {
+ logger.info( "Initializing ..." );
super.initialize(context);
try{
Scanner scanner = new Scanner(FileLocator.getAsStream(tokenCountFile));
@@ -95,6 +96,7 @@ public class SentenceDetectorAnnotatorBIO extends CleartkAnnotator<String>{
@Override
public void process(JCas jcas) throws AnalysisEngineProcessException {
+ logger.info( "Processing ..." );
String uri=null;
try{
uri = ViewUriUtil.getURI(jcas).toString();
diff --git a/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipeBitLocator.java b/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipeBitLocator.java
index b0ced01..fb2e36c 100644
--- a/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipeBitLocator.java
+++ b/ctakes-core/src/main/java/org/apache/ctakes/core/pipeline/PipeBitLocator.java
@@ -36,20 +36,21 @@ public enum PipeBitLocator {
"contexttokenizer",
"postagger",
"chunker",
+ "dictionary.lookup2",
"dictionary.lookup.fast",
"dictionary.lookup.cased",
"dictionary.cased",
"assertion",
- "dictionary.lookup2",
+ "assertion.medfacts.cleartk",
"clinicalpipeline",
"clinical.pipeline",
"constituency.parser",
- "lvg",
"relationextractor",
"relation.extractor",
"coreference",
"dependency.parser",
"temporal",
+ "pbj",
"drug-ner",
"necontexts",
"ne.contexts",
@@ -59,6 +60,7 @@ public enum PipeBitLocator {
"smoking.status",
"dictionary.lookup",
"template.filler",
+ "lvg",
"examples" };
diff --git a/ctakes-core/src/main/java/org/apache/ctakes/core/util/PropertyAeFactory.java b/ctakes-core/src/main/java/org/apache/ctakes/core/util/PropertyAeFactory.java
index 13df636..6cebcc9 100644
--- a/ctakes-core/src/main/java/org/apache/ctakes/core/util/PropertyAeFactory.java
+++ b/ctakes-core/src/main/java/org/apache/ctakes/core/util/PropertyAeFactory.java
@@ -9,7 +9,9 @@ import org.apache.uima.collection.CollectionReader;
import org.apache.uima.collection.CollectionReaderDescription;
import org.apache.uima.fit.factory.AnalysisEngineFactory;
import org.apache.uima.fit.factory.CollectionReaderFactory;
+import org.apache.uima.fit.factory.TypeSystemDescriptionFactory;
import org.apache.uima.resource.ResourceInitializationException;
+import org.apache.uima.resource.metadata.TypeSystemDescription;
import java.util.HashMap;
import java.util.Map;
@@ -34,7 +36,17 @@ public enum PropertyAeFactory {
// Use a single hashmap so that multiple properties files can be used
final private Map<String, Object> _properties = new HashMap<>();
-
+ private TypeSystemDescription _typeSystemDescription;
+
+ PropertyAeFactory() {
+ try {
+ _typeSystemDescription = TypeSystemDescriptionFactory.createTypeSystemDescription();
+ } catch ( ResourceInitializationException riE ) {
+ Logger.getLogger( "PropertyAeFactory" )
+ .error( "Could not initialize cTAKES Type System\n" + riE.getMessage() );
+ System.exit( -1 );
+ }
+ }
/**
* Add key value pairs to the stored properties
@@ -158,7 +170,7 @@ public enum PropertyAeFactory {
final Object... parameters )
throws ResourceInitializationException {
final Object[] allParameters = getAllParameters( parameters );
- return AnalysisEngineFactory.createEngineDescription( classType, allParameters );
+ return AnalysisEngineFactory.createEngineDescription( classType, _typeSystemDescription, allParameters );
}
/**
diff --git a/ctakes-core/src/user/resources/org/apache/ctakes/core/pipeline/FullTokenizerPipeline.piper b/ctakes-core/src/user/resources/org/apache/ctakes/core/pipeline/FullTokenizerPipeline.piper
index 30952cd..2e15c1f 100644
--- a/ctakes-core/src/user/resources/org/apache/ctakes/core/pipeline/FullTokenizerPipeline.piper
+++ b/ctakes-core/src/user/resources/org/apache/ctakes/core/pipeline/FullTokenizerPipeline.piper
@@ -4,7 +4,7 @@
add BsvRegexSectionizer
// The sentence detector needs our custom model path, otherwise default values are used.
-//addLogged SentenceDetectorAnnotatorBIO classifierJarPath=/org/apache/ctakes/core/sentdetect/model.jar
+//add SentenceDetectorAnnotatorBIO classifierJarPath=/org/apache/ctakes/core/sentdetect/model.jar
// The SentenceDetectorAnnotatorBIO is a "lumper" that works well for notes in which end of line does not indicate a sentence.
// If that is not your case, then you may get better results using the more standard SentenceDetector
diff --git a/ctakes-dependency-parser/src/main/java/org/apache/ctakes/dependency/parser/ae/ClearNLPSemanticRoleLabelerAE.java b/ctakes-dependency-parser/src/main/java/org/apache/ctakes/dependency/parser/ae/ClearNLPSemanticRoleLabelerAE.java
index 4cf19b2..73cf99d 100644
--- a/ctakes-dependency-parser/src/main/java/org/apache/ctakes/dependency/parser/ae/ClearNLPSemanticRoleLabelerAE.java
+++ b/ctakes-dependency-parser/src/main/java/org/apache/ctakes/dependency/parser/ae/ClearNLPSemanticRoleLabelerAE.java
@@ -118,9 +118,9 @@ public class ClearNLPSemanticRoleLabelerAE extends JCasAnnotator_ImplBase {
@Override
public void initialize( UimaContext context ) throws ResourceInitializationException {
+ logger.info("Initializing ...");
super.initialize( context );
- logger.info("Initializing ClearNLP semantic role labeler");
try {
if(this.predModel == null){
this.identifier = SRLSharedPredictionModel.getDefaultModel();
@@ -144,6 +144,7 @@ public class ClearNLPSemanticRoleLabelerAE extends JCasAnnotator_ImplBase {
@Override
public void process( JCas jCas ) throws AnalysisEngineProcessException {
+ logger.info("Processing ...");
for ( Sentence sentence : JCasUtil.select( jCas, Sentence.class ) ) {
List<BaseToken> printableTokens = new ArrayList<>();
for ( BaseToken token : JCasUtil.selectCovered( jCas, BaseToken.class, sentence ) ) {
diff --git a/ctakes-distribution/src/main/bin/runPiperFile.bat b/ctakes-distribution/src/main/bin/runPiperFile.bat
index 70497be..3b05bc7 100644
--- a/ctakes-distribution/src/main/bin/runPiperFile.bat
+++ b/ctakes-distribution/src/main/bin/runPiperFile.bat
@@ -25,8 +25,7 @@
:: -s , --subDir {subDirectory} (for i/o)
:: --xmiOut {xmiOutputDirectory} (if different from -o)
:: -l , --lookupXml {dictionaryConfigFile} (fast only)
-:: --user {umlsUsername}
-:: --pass {umlsPassword}
+:: --key {umlsKey}
:: -? , --help
::
:: Other parameters may be declared in the piper file using the cli command:
diff --git a/ctakes-distribution/src/main/bin/runPiperFile.sh b/ctakes-distribution/src/main/bin/runPiperFile.sh
index 1e59649..bc05395 100644
--- a/ctakes-distribution/src/main/bin/runPiperFile.sh
+++ b/ctakes-distribution/src/main/bin/runPiperFile.sh
@@ -25,8 +25,7 @@
# -s , --subDir {subDirectory} (for i/o)
# --xmiOut {xmiOutputDirectory} (if different from -o)
# -l , --lookupXml {dictionaryConfigFile} (fast only)
-# --user {umlsUsername}
-# --pass {umlsPassword}
+# --key {umlsKey}
# -? , --help
#
# Other parameters may be declared in the piper file using the cli command:
diff --git a/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/ApacheConDemoRelation.piper b/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/ApacheConDemoRelation.piper
index 9aab561..8310a73 100644
--- a/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/ApacheConDemoRelation.piper
+++ b/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/ApacheConDemoRelation.piper
@@ -16,7 +16,7 @@ load DictionarySubPipe
load AttributeCleartkSubPipe
// Location.
-//addLogged LocationOfRelationExtractorAnnotator classifierJarPath=/org/apache/ctakes/relationextractor/models/location_of/model.jar
+//add LocationOfRelationExtractorAnnotator classifierJarPath=/org/apache/ctakes/relationextractor/models/location_of/model.jar
// Temporal (event, time, dtr, tlink)
//load TemporalSubPipe
diff --git a/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/ApacheConDemoSection.piper b/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/ApacheConDemoSection.piper
index 50f4e17..c8925bb 100644
--- a/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/ApacheConDemoSection.piper
+++ b/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/ApacheConDemoSection.piper
@@ -5,7 +5,7 @@ add SimpleSegmentAnnotator
//add ParagraphAnnotator
//add SentenceDetector
-addLogged SentenceDetectorAnnotatorBIO classifierJarPath=/org/apache/ctakes/core/models/sentdetect/model.jar
+add SentenceDetectorAnnotatorBIO classifierJarPath=/org/apache/ctakes/core/models/sentdetect/model.jar
// Save simple information
add ApacheConSentenceWriter
diff --git a/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertProps.piper b/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertProps.piper
index fb6173f..3368352 100644
--- a/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertProps.piper
+++ b/ctakes-examples/src/user/resources/org/apache/ctakes/examples/pipeline/HelloWorldAssertProps.piper
@@ -7,7 +7,7 @@ load org/apache/ctakes/examples/pipeline/HelloWorld.piper
// Assertion engines require dependencies
addDescription ClearNLPDependencyParserAE
// Add the Semantic Role Labeler parser for use by assertion
-addLogged ClearNLPSemanticRoleLabelerAE
+add ClearNLPSemanticRoleLabelerAE
// Use the assertion mini pipeline
// load parameters used by the following engines
diff --git a/ctakes-web-rest/README b/ctakes-web-rest/README
index 115d93b..ed6731f 100644
--- a/ctakes-web-rest/README
+++ b/ctakes-web-rest/README
@@ -1,12 +1,35 @@
-This is the README for the REST module in the cTakes project.
+* Project Apache cTAKES Web Rest
+* module: ctakes-web-rest
+
+ctakes-web-rest is not compiled or built when the main Apache cTAKES code is built with maven.
+ctakes-web-rest is not contained within the main Apache cTAKES binary distributable zipped files.
+ctakes-web-rest is a standalone project that is part of the larger Apache cTAKES Project, but that is not the same as being part of the main maven project.
+
+You must build the ctakes-web-rest module alone. Doing so will create its own self-contained Web Application Resource (aka Web ARchive) (WAR) file.
+The WAR file contains a build of the main Apache cTAKES binary distributable.
+
+If you are running maven by the command line then you must run in the ctakes-web-rest directory or direct maven to the ctakes-web-rest pom.
+If you are running maven in an IDE, then you must add ctakes-web-rest as its own maven project and execute maven commands on that project.
+Building the main ctakes project will not build ctakes-web-rest.
+
+This configuration exists because:
+- building ctakes-web-rest with the main ctakes maven project adds to the build time.
+- building ctakes-web-rest with the main ctakes maven project adds to the disk footprint of the build. i.e. it has its own huge target/ directory.
+- including ctakes-web-rest in the main Apache cTAKES binary distributable essentially puts two copies of cTAKES in the distributable.
+
+
+
This module performs natural language processing of input payload using REST API
endpoint and extracts out clicnical information in JSON format.
Installation:
-1) Build all the cTAKES modules as this module contains a number of references
-to other cTAKES modules.
+[comment]: <> (1) Build all the cTAKES modules as this module contains a number of references )
+
+[comment]: <> (to other cTAKES modules.)
+
+1) Build this ctakes-web-rest module.
2) Modify 'src\main\resources\org\apache\ctakes\dictionary\lookup\fast\customDictionary.xml'
to refer to the respective database where dictionary is loaded or use UMLS database.
@@ -14,14 +37,14 @@ to refer to the respective database where dictionary is loaded or use UMLS datab
3) Build ctakes-web-rest module and deploy ctakes-web-rest.war available
under 'target' folder in Apache Tomcat.
-4) Access the following URL to peform text analysis using cTAKES web application:
+4) Access the following URL to perform text analysis using cTAKES web application:
- http://<host-name>:<port>/ctakes-web-rest/index.jsp
+ http://<host-name>:<port>/ctakes-web-rest/index.jsp
5) Access the following REST API endpoint to perform text analysis using default pipeline:
- http://<host-name>:<port>/ctakes-web-rest/service/analyze?pipeline=Default
+ http://<host-name>:<port>/ctakes-web-rest/service/analyze?pipeline=Default
6) Access the following REST API endpoint to perform text analysis using full pipeline:
- http://<host-name>:<port>/ctakes-web-rest/service/analyze?pipeline=Full
\ No newline at end of file
+ http://<host-name>:<port>/ctakes-web-rest/service/analyze?pipeline=Full
\ No newline at end of file