You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@opennlp.apache.org by "Martin Wiesner (Jira)" <ji...@apache.org> on 2026/07/06 19:44:00 UTC
[jira] [Updated] (OPENNLP-1870) Add emoji annotation layer wired to Sentiment, Name Finder, and Doccat
[ https://issues.apache.org/jira/browse/OPENNLP-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martin Wiesner updated OPENNLP-1870:
------------------------------------
Summary: Add emoji annotation layer wired to Sentiment, Name Finder, and Doccat (was: Emoji annotation layer wired to Sentiment, Name Finder, and Doccat)
> Add emoji annotation layer wired to Sentiment, Name Finder, and Doccat
> ----------------------------------------------------------------------
>
> Key: OPENNLP-1870
> URL: https://issues.apache.org/jira/browse/OPENNLP-1870
> Project: OpenNLP
> Issue Type: Task
> Components: dl, Sentiment Detection
> Affects Versions: 3.0.0-M4
> Reporter: Kristian Rickert
> Assignee: Kristian Rickert
> Priority: Major
>
> *Stacks on:* OPENNLP-1869.
> *Summary.* Extend the emoji lookup with a small, provenance-tagged annotation layer per symbol: human-readable name (CLDR short-name), sentiment (with its own {{sentiment_source}}), coarse entity type (VEHICLE, LANDMARK, ANIMAL, FLAG, FOOD, ...), and a document-category hint. Surface these as parallel layers on the {{Term}} model (original emoji, folded emoticon, name text, sentiment tag) so each consumer picks the layer it needs. Wire the annotations as optional input to Sentiment Detection, the Name Finder (name text plus entity-type as a gazetteer-like signal), and the Document Categorizer. Each annotation dimension is opt-in, so a consumer that only wants the folded text pays nothing for the rest.
> *Acceptance criteria.*
> * Optional annotation dimensions (name, sentiment, entity type, category), each with its own provenance/source field, surfaced through {{Term}}.
> * Wired as optional features into Sentiment Detection, Name Finder, and Document Categorizer.
> * Each dimension independently opt-in; default behavior unchanged.
> * Licensing cleared and documented for every bundled annotation source (see epic-level LEGAL item).
> h2. Out of scope (for the whole epic)
> * DL whitespace/dash offset-safety (done in 1850: {{NameFinderDL.findInOriginal}}, {{AbstractDL.normalizeInputAligned}}).
> * The {{Alignment}} model and {{andThen}} themselves (delivered in 1850; this epic consumes them).
> * Changing any default normalization behavior. This epic adds offset-aware capability and the provenance/annotation model; it does not turn expanding folds on by default.
> h2. Open questions to close in design review
> # The gate (NFC/NFKC/case-fold edits): confirm option (c) for the MVP, or take ICU4J. Gates OPENNLP-1867.
> # CSV vs annotated {{.txt}} for the lookups; one combined file or one per fold type.
> # Offset-aware method on {{CharSequenceNormalizer}} directly vs a parallel capability interface.
> # Default aggressiveness: full case folding and NFKC change tokens visibly, so likely opt-in, not default.
> # Whether {{UNSPECIFIED}} entries are allowed in a release build at all, or each must be reviewed and signed off.
> # Which dataset anchors each annotation dimension (emoji-sentiment ranking, CLDR for names, and what for entity type and category); same file or a separate keyed file.
> # Emoji/emoticon as punctuation vs a dedicated {{WordType}} for counting and feature generation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)