[jira] [Assigned] (OPENNLP-1950) Fix ADChunkSampleStream fails with NPE on the FlorestaVirgem corpus

["Martin Wiesner (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OPENNLP-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Martin Wiesner reassigned OPENNLP-1950:
---------------------------------------

    Assignee: Kristian Rickert

> Fix ADChunkSampleStream fails with NPE on the FlorestaVirgem corpus
> -------------------------------------------------------------------
>
>                 Key: OPENNLP-1950
>                 URL: https://issues.apache.org/jira/browse/OPENNLP-1950
>             Project: OpenNLP
>          Issue Type: Bug
>    Affects Versions: 2.5.12, 3.0.0-M6
>            Reporter: Richard Zowalla
>            Assignee: Kristian Rickert
>            Priority: Minor
>             Fix For: 3.0.0
>
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> Reading {{FlorestaVirgem\_CF\_3.0\_ad.txt}} \(Linguateca, ISO\-8859\-1\) with {{ADChunkSampleStream}} fails after 4,641 samples:
> {noformat}
> java.lang.NullPointerException
>   at java.base/java.util.Objects.requireNonNull\(Objects.java:220\)
>   at java.base/java.util.List.copyOf\(List.java:1191\)
>   at opennlp.tools.chunker.ChunkSample.\(ChunkSample.java:68\)
>   at opennlp.tools.formats.ad.ADChunkSampleStream.read\(ADChunkSampleStream.java:116\)
> {noformat}
> One of the token, tag or chunk lists passed to {{ChunkSample}} contains a {{null}} element. {{ArvoresDeitadasEval}} only uses Bosque for chunking, so the evals don't catch it.
> _Fix:_ find the construct that yields the null tag, handle it or skip the sample with a warning, and add a test with that construct.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)