You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Steve Rowe (JIRA)" <ji...@apache.org> on 2017/12/15 22:35:02 UTC
[jira] [Reopened] (LUCENE-2899) Add OpenNLP Analysis capabilities
as a module
[ https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Rowe reopened LUCENE-2899:
--------------------------------
My Jenkins found a reproducing seed for a {{TestOpenNLPPOSFilterFactory.testPos()}} failure:
{noformat}
[junit4] Suite: org.apache.lucene.analysis.opennlp.TestOpenNLPPOSFilterFactory
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestOpenNLPPOSFilterFactory -Dtests.method=testPOS -Dtests.seed=9CB6DAAD9AB4A5C3 -Dtests.slow=true -Dtests.locale=lv-LV -Dtests.timezone=America/La_Paz -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.02s J2 | TestOpenNLPPOSFilterFactory.testPOS <<<
[junit4] > Throwable #1: org.junit.ComparisonFailure: term 0 expected:<[Sentence]> but was:<[2]>
[junit4] > at __randomizedtesting.SeedInfo.seed([9CB6DAAD9AB4A5C3:E04A51BF1852C7E]:0)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:201)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:325)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:329)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:865)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:727)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertAnalyzesTo(BaseTokenStreamTestCase.java:390)
[junit4] > at org.apache.lucene.analysis.opennlp.TestOpenNLPPOSFilterFactory.testPOS(TestOpenNLPPOSFilterFactory.java:75)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@4f758cba), locale=lv-LV, timezone=America/La_Paz
[junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_151 (64-bit)/cpus=16,threads=1,free=412573592,total=514850816
[junit4] 2> NOTE: All tests run in this JVM: [TestOpenNLPPOSFilterFactory]
{noformat}
> Add OpenNLP Analysis capabilities as a module
> ---------------------------------------------
>
> Key: LUCENE-2899
> URL: https://issues.apache.org/jira/browse/LUCENE-2899
> Project: Lucene - Core
> Issue Type: New Feature
> Components: modules/analysis
> Reporter: Grant Ingersoll
> Assignee: Steve Rowe
> Priority: Minor
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-2899-6.1.0.patch, LUCENE-2899-RJN.patch, LUCENE-2899.patch, LUCENE-2899.patch, LUCENE-2899.patch, LUCENE-2899.patch, LUCENE-2899.patch, LUCENE-2899.patch, OpenNLPFilter.java, OpenNLPTokenizer.java
>
>
> Now that OpenNLP is an ASF project and has a nice license, it would be nice to have a submodule (under analysis) that exposed capabilities for it. Drew Farris, Tom Morton and I have code that does:
> * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it would have to change slightly to buffer tokens)
> * NamedEntity recognition as a TokenFilter
> We are also planning a Tokenizer/TokenFilter that can put parts of speech as either payloads (PartOfSpeechAttribute?) on a token or at the same position.
> I'd propose it go under:
> modules/analysis/opennlp
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org