RE: SentenceDetector [EXTERNAL] [SUSPICIOUS]

["Finan, Sean" <Se...@childrens.harvard.edu>]

Hi Ewan,

We use Tim's SentenceDetectorAnnotatorBIO in a project that has run hundreds of notes that contain newline-spanning sentences and it does work very well.  As Tim wrote, it does sometimes lump lines together if they aren't prose.  One example is lines of text in a list.  There are a few ctakes annotators that can help correct this: ParagraphSentenceFixer and ListSentenceFixer.  

If you are running the default clinical pipeline, you can use the SectionedFastPipeline.piper in ctakes-clinical-pipeline-res instead of the DefaultFastPipeline.piper
The difference between the two is that SectionedFastPipeline loads FullTokenizerPipeline.piper in ctakes-core-res instead of DefaultTokenizerPipeline.piper.  The FullTokenizer... has a reference to the Sentence...BIO, and you can enable it by swapping comment specifiers to:

// The sentence detector needs our custom model path, otherwise default values are used.
addLogged SentenceDetectorAnnotatorBIO classifierJarPath=/org/apache/ctakes/core/sentdetect/model.jar

// The SentenceDetectorAnnotatorBIO is a "lumper" that works well for notes in which end of line does not indicate a sentence.
// If that is not your case, then you may get better results using the more standard SentenceDetector
//add SentenceDetector

Sean

-----Original Message-----
From: Miller, Timothy [mailto:Timothy.Miller@childrens.harvard.edu] 
Sent: Friday, April 06, 2018 9:46 AM
To: dev@ctakes.apache.org
Subject: Re: SentenceDetector [EXTERNAL] [SUSPICIOUS]

The changes were mainly meant to adapt the OpenNLP model to idiosyncrasies of clinical text, but you're right that they have some shortcomings.

The newline thing is in the data sources used originally to build the model, there were frequent cases of headings/sentence fragments by themselves on a line, and _no_ cases of mid-sentence newlines. That, combined with the fact that OpenNLP's train file format (at the time) itself used newlines as a separator, led to the creation of that simple rule rather than trying to retrain with newline as a candidate sentence splitter. I created a different training file format and annotator that does what you suggest, and built an alternative sentence splitter model, here:
org/apache/ctakes/core/ae/SentenceDetectorAnnotatorBIO.java

it operates at the character level and splits a document into sentences. For some people it works better. For data where there are potentially mid-sentence newlines (like MIMIC), it is probably the only model with usable results. It's typical failure mode is to lump two sentences together, while the default annotator does the opposite.

Tim


On Fri, 2018-04-06 at 02:11 +0000, Ewan Mellor wrote:
> I'm looking at SentenceDetector from ctakes-core.  It has a surprising 
> idea of what counts as a "sentence".  Before I delve any deeper, I 
> wanted to ask whether there is a reason for what it's doing, in 
> particular whether there's anything in the clinical pipeline that's 
> depending on its behavior specifically.
> 
> The main problem I have is that it's splitting on characters like 
> colon and semicolon, which aren't usually considered sentence 
> separators, with the result that it often ends up tagging phrases 
> rather than whole sentences.
> 
> It's using SentenceDetectorCtakes and EndOfSentenceScannerImpl, which 
> seem to be derived from equivalents in OpenNLP, but with changes that 
> I can't track (they date from the original edu.mayo import as far as I 
> can tell).
> Other than the additional separator characters, I can't tell whether 
> these classes are doing anything important that you wouldn't equally 
> get from OpenNLP's SentenceDetectorME, so I don't know why they're 
> being used.
> 
> SentenceDetector is also splitting on newlines after passing the text 
> through the max entropy sentence model.  I don't see the point in this 
> -- if you're going to split on newlines anyway, then why not do that 
> before passing through the entropy model?  Or just have newline as one 
> of the potential EOS characters and treat it as a possible break point 
> rather than a definite one?
> 
> Any insight would be welcome.
> 
> Thanks,
> 
> Ewan.

Re: SentenceDetector [EXTERNAL] [SUSPICIOUS]

[Ewan Mellor <ct...@ewanmellor.org.uk>]

Thanks for the replies, both of you.  I have filed
https://issues.apache.org/jira/browse/CTAKES-507 with a patch to put
those details in the class comments.

I'm using double-newline as a paragraph separator, with mid-sentence single
newlines (the text is coming from an OCR phase so it's broken into lines).
I'm getting reasonable results now, by adding a custom annotator first that
adds Segment annotations for each of the paragraphs, and then using
SentenceDetectorAnnotatorBIO.  I then don't need ParagraphSentenceFixer
because my sentences can't span paragraphs as SentenceDetectorAnnotatorBIO
only sees one paragraph at a time.

I did find one place where it failed -- using the default tokenCounts.txt
it's breaking sentences at "Dr." which is a bit sad given the subject matter.
I'm hacking around that for now but if it proves to be an issue I may try
finding a tagged corpus to retrain with (I don't have a decent tagged
corpus myself, which is why I'm using the default models).

Thanks,

Ewan.

On Fri, Apr 06, 2018 at 02:06:11PM +0000, Finan, Sean wrote:

> Hi Ewan,
> 
> We use Tim's SentenceDetectorAnnotatorBIO in a project that has run hundreds of notes that contain newline-spanning sentences and it does work very well.  As Tim wrote, it does sometimes lump lines together if they aren't prose.  One example is lines of text in a list.  There are a few ctakes annotators that can help correct this: ParagraphSentenceFixer and ListSentenceFixer.  
> 
> If you are running the default clinical pipeline, you can use the SectionedFastPipeline.piper in ctakes-clinical-pipeline-res instead of the DefaultFastPipeline.piper
> The difference between the two is that SectionedFastPipeline loads FullTokenizerPipeline.piper in ctakes-core-res instead of DefaultTokenizerPipeline.piper.  The FullTokenizer... has a reference to the Sentence...BIO, and you can enable it by swapping comment specifiers to:
> 
> // The sentence detector needs our custom model path, otherwise default values are used.
> addLogged SentenceDetectorAnnotatorBIO classifierJarPath=/org/apache/ctakes/core/sentdetect/model.jar
> 
> // The SentenceDetectorAnnotatorBIO is a "lumper" that works well for notes in which end of line does not indicate a sentence.
> // If that is not your case, then you may get better results using the more standard SentenceDetector
> //add SentenceDetector
> 
> Sean
> 
> -----Original Message-----
> From: Miller, Timothy [mailto:Timothy.Miller@childrens.harvard.edu] 
> Sent: Friday, April 06, 2018 9:46 AM
> To: dev@ctakes.apache.org
> Subject: Re: SentenceDetector [EXTERNAL] [SUSPICIOUS]
> 
> The changes were mainly meant to adapt the OpenNLP model to idiosyncrasies of clinical text, but you're right that they have some shortcomings.
> 
> The newline thing is in the data sources used originally to build the model, there were frequent cases of headings/sentence fragments by themselves on a line, and _no_ cases of mid-sentence newlines. That, combined with the fact that OpenNLP's train file format (at the time) itself used newlines as a separator, led to the creation of that simple rule rather than trying to retrain with newline as a candidate sentence splitter. I created a different training file format and annotator that does what you suggest, and built an alternative sentence splitter model, here:
> org/apache/ctakes/core/ae/SentenceDetectorAnnotatorBIO.java
> 
> it operates at the character level and splits a document into sentences. For some people it works better. For data where there are potentially mid-sentence newlines (like MIMIC), it is probably the only model with usable results. It's typical failure mode is to lump two sentences together, while the default annotator does the opposite.
> 
> Tim
> 
> 
> On Fri, 2018-04-06 at 02:11 +0000, Ewan Mellor wrote:
> > I'm looking at SentenceDetector from ctakes-core.��It has a surprising 
> > idea of what counts as a "sentence".��Before I delve any deeper, I 
> > wanted to ask whether there is a reason for what it's doing, in 
> > particular whether there's anything in the clinical pipeline that's 
> > depending on its behavior specifically.
> > 
> > The main problem I have is that it's splitting on characters like 
> > colon and semicolon, which aren't usually considered sentence 
> > separators, with the result that it often ends up tagging phrases 
> > rather than whole sentences.
> > 
> > It's using SentenceDetectorCtakes and EndOfSentenceScannerImpl, which 
> > seem to be derived from equivalents in OpenNLP, but with changes that 
> > I can't track (they date from the original edu.mayo import as far as I 
> > can tell).
> > Other than the additional separator characters, I can't tell whether 
> > these classes are doing anything important that you wouldn't equally 
> > get from OpenNLP's SentenceDetectorME, so I don't know why they're 
> > being used.
> > 
> > SentenceDetector is also splitting on newlines after passing the text 
> > through the max entropy sentence model.��I don't see the point in this 
> > -- if you're going to split on newlines anyway, then why not do that 
> > before passing through the entropy model?��Or just have newline as one 
> > of the potential EOS characters and treat it as a possible break point 
> > rather than a definite one?
> > 
> > Any insight would be welcome.
> > 
> > Thanks,
> > 
> > Ewan.