[jira] [Resolved] (OPENNLP-1888) Creating a Document annotation container: typed, offset-anchored layer s so every future analysis result is an additive 3.x feature

["Martin Wiesner (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OPENNLP-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Martin Wiesner resolved OPENNLP-1888.
-------------------------------------
    Resolution: Delivered

> Creating a Document annotation container: typed, offset-anchored layer s so every future analysis result is an additive 3.x feature
> -----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: OPENNLP-1888
>                 URL: https://issues.apache.org/jira/browse/OPENNLP-1888
>             Project: OpenNLP
>          Issue Type: Improvement
>    Affects Versions: 3.0.0-M4
>            Reporter: Kristian Rickert
>            Assignee: Kristian Rickert
>            Priority: Major
>             Fix For: 3.0.0-M6, 3.0.0
>
>          Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> Designing the document generic type in Java:
> To be explicit, because it is the first question this proposal should answer: *this ticket deprecates nothing and replaces nothing.* It is something I've brought up in the chats - the importance of now being the time to land API changes.
> h2. Motivation
> OpenNLP's Java API is tool-by-tool: a tokenizer returns spans, a tagger returns strings, a name finder returns spans, and the caller reassembles the pieces by hand, differently in every application. Each new capability adds another entry point and result type, so the cost compounds. I ran into this head-on writing the gRPC layer, which is mostly mapping tool outputs into a common document structure that the Java API itself never defines. Since 3.0 has not landed yet, this feels like the right time to discuss adding that layer.
> We already have two data points. {{Term}} validated the layered-container idea, but it is token-scoped, string-valued, and keyed by a closed enum, so it cannot hold anything structural. The emoji annotation layer (OPENNLP-1870) is the first structured result that did not fit, and it correctly became a separate accessor beside {{{}Term{}}}. Every future structured result (dependency arcs, coreference, relations, money, PII) would repeat that pattern, each growing its own parallel surface, while the combined-document view still would not exist.
> The fix is one small container in {{{}opennlp-api{}}}: typed annotation layers over the original text, identified by open-ended keys rather than an enum. Any future capability then becomes three additions that touch no existing type: a payload record, a layer key, and an annotator. Since anything published in 3.x is frozen until 4.0, shipping this in 3.0 is what keeps the parallel surfaces from ever accumulating.
> h2. Design
> Four public types, all in a new package in {{{}opennlp-api{}}}:
> {code:java}
> public interface Document {
>   CharSequence text();                            // the original text, never a normalized form
>   <T> List<Annotation<T>> get(LayerKey<T> layer); // empty list when the layer is absent
>   Set<LayerKey<?>> layers();
> }
> public record Annotation<T>(Span span, T value) { }
> public final class LayerKey<T> {
>   public static <T> LayerKey<T> of(String id, Class<T> type) { ... }
> }
> public interface DocumentAnnotator {
>   Document annotate(Document document);           // returns a new document with layers added
>   Set<LayerKey<?>> requires();
>   Set<LayerKey<?>> provides();
> }
> {code}
> Standard keys for today's results (sentences, tokens, tags, chunks, entities) ship with thin annotators wrapping the existing task interfaces, which stay exactly as they are. A {{DocumentAnalyzer}} builder assembles annotators into a pipeline ordered by {{{}requires(){}}}/{{{}provides(){}}}, mirroring {{{}TermAnalyzer{}}}.
> Rules that protect the additive property:
>  * Spans always point into the original text; normalization maps back through {{{}Alignment{}}}.
>  * Layer identity is an open key, never an enum; payload types live in the producing feature's package, and the container never imports or enumerates them.
>  * Annotations reference other annotations by layer and index (a dependency arc names token indices), keeping documents immutable and mappable to any encoding.
>  * No new dependencies, and no serialization framework in {{{}opennlp-api{}}}; encodings live where their dependencies already are.
>  * Evolution by default methods and new types only; nothing sealed, no abstract classes at the seam.
> h2. Coexistence with the task-level API
> The task interfaces are not a legacy layer. They remain the primary API for single-task use, with no deprecation and no planned removal; annotators delegate to them, so there is one behavior reachable two ways (the same relationship {{TermAnalyzer}} has to the normalizer rungs). Deprecation is reserved for the rare surface a layer supersedes outright, and even then removal waits for a major, so existing call sites compile and run for the life of 3.x. Models are unaffected: nothing about training or decoding changes.
> h2. Validation before freeze 
>  # The existing analyze pipeline (sentences, tokens, tags, entities) expressed as annotators, proving the adapters and ordering model.
>  # Dependency parsing as the first graph-shaped layer: its values reference other annotations, so it exercises the cross-layer rules that coreference and relation extraction will inherit. A transition-based parser over the existing maxent machinery, trained from CoNLL-U treebanks, suffices and is a wanted feature in its own right.
> h2. Out of scope
> Downstream features themselves (each is its own ticket), changes to any existing task interface or result type, and document serialization.
> h2. Acceptance criteria
>  * Container types in {{{}opennlp-api{}}}, no new dependencies, full javadoc; existing tools populate a document through adapters unchanged.
>  * Both validation consumers pass tests covering cross-layer references and span fidelity, including a normalization-shifted case.
>  * A new layer can be added in a test without touching the container's package.
>  * Binary compatibility from the previous release is verified mechanically in CI.
>  * No existing type is modified or deprecated; the task-level API remains fully supported.
> h4. Guardrail Note:
> The container's package must never grow knowledge of specific layers. The moment it imports a payload type or enumerates layer identities, the additive property is lost and every future feature becomes an API change again.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)