You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by dw...@apache.org on 2022/09/23 10:59:52 UTC
[lucene] branch main updated (5b24a233bdf -> 3a04aa44c22)
This is an automated email from the ASF dual-hosted git repository.
dweiss pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/lucene.git
from 5b24a233bdf LUCENE-10425：speed up IndexSortSortedNumericDocValuesRangeQuery#BoundedDocSetIdIterator construction using bkd binary search (#687)
add 3a04aa44c22 Fix repeating token sentence boundary bug (#11734)
No new revisions were added by this update.
Summary of changes:
lucene/CHANGES.txt | 6 ++
.../analysis/opennlp/OpenNLPChunkerFilter.java | 63 +++++++----------
.../analysis/opennlp/OpenNLPLemmatizerFilter.java | 63 +++++++----------
.../lucene/analysis/opennlp/OpenNLPPOSFilter.java | 63 ++++++++---------
.../lucene/analysis/opennlp/OpenNLPTokenizer.java | 19 +++--
.../opennlp/SentenceAttributeExtractor.java | 81 +++++++++++++++++++++
.../analysis/opennlp/data/early-exit-bug-input.txt | 23 ++++++
.../opennlp/data/early-exit-bug-output.txt | 32 +++++++++
.../opennlp/TestOpenNLPChunkerFilterFactory.java | 12 ++++
.../TestOpenNLPLemmatizerFilterFactory.java | 82 ++++++++++++++++++++++
.../opennlp/TestOpenNLPPOSFilterFactory.java | 24 +++++++
.../{TypeAttribute.java => SentenceAttribute.java} | 24 ++++---
...tributeImpl.java => SentenceAttributeImpl.java} | 50 +++++++------
13 files changed, 391 insertions(+), 151 deletions(-)
create mode 100644 lucene/analysis/opennlp/src/java/org/apache/lucene/analysis/opennlp/SentenceAttributeExtractor.java
create mode 100644 lucene/analysis/opennlp/src/test-files/org/apache/lucene/analysis/opennlp/data/early-exit-bug-input.txt
create mode 100644 lucene/analysis/opennlp/src/test-files/org/apache/lucene/analysis/opennlp/data/early-exit-bug-output.txt
copy lucene/core/src/java/org/apache/lucene/analysis/tokenattributes/{TypeAttribute.java => SentenceAttribute.java} (66%)
copy lucene/core/src/java/org/apache/lucene/analysis/tokenattributes/{FlagsAttributeImpl.java => SentenceAttributeImpl.java} (58%)