(opennlp) 01/02: Merge Apache main into OPENNLP-1951-ad-name-text-id

[kr...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

krickert pushed a commit to branch OPENNLP-1951-ad-name-text-id
in repository https://gitbox.apache.org/repos/asf/opennlp.git

commit 628cdfd2ac159ad1c6553b098e3d3e55d9be7691
Merge: 7def2ce9c 5bcbc29ee
Author: Kristian Rickert <kr...@apache.org>
AuthorDate: Thu Sep 17 02:32:57 2026 -0400

    Merge Apache main into OPENNLP-1951-ad-name-text-id
    
    # Conflicts:
    #       opennlp-docs/src/docbkx/corpora.xml

 .../tools/formats/ad/ADChunkSampleStream.java      | 13 +++-
 .../opennlp/tools/formats/ad/ADSentenceStream.java | 17 ++++-
 .../tools/formats/ad/ADChunkSampleStreamTest.java  | 47 ++++++++++++++
 .../tools/formats/ad/ADSentenceParserTest.java     | 74 ++++++++++++++++++++++
 opennlp-docs/src/docbkx/corpora.xml                | 12 ++++
 5 files changed, 161 insertions(+), 2 deletions(-)

diff --cc opennlp-docs/src/docbkx/corpora.xml
index 4681bba30,deeafe5a5..97c0f0a70
--- a/opennlp-docs/src/docbkx/corpora.xml
+++ b/opennlp-docs/src/docbkx/corpora.xml
@@@ -430,18 -430,17 +430,30 @@@ Execution time: 1,955 seconds]]
  			3.0.0, the empty open sentence no longer causes an endless read.
  		</para>
  	</section>
+ 	<section xml:id="tools.corpora.arvores-deitadas.leaf-tags">
+ 		<title>AD leaf tags with embedded equals signs</title>
+ 		<para>
+ 			Leaf tags may contain equals signs after the indentation, as in
+ 			<code>=H==CJT:num</code>. The reader takes the functional tag after the last
+ 			colon, so this example has the tag <code>num</code>. A chunk leaf without a
+ 			functional tag uses its lexeme as the tag instead of a null value. This permits
+ 			FlorestaVirgem chunk conversion to continue and changes the POS labels of the
+ 			affected tagged leaves to their functional tags, such as <code>num</code> or
+ 			<code>prop</code>. Re-evaluate models trained from affected samples.
+ 		</para>
++	</section>
 +	<section xml:id="tools.corpora.arvores-deitadas.text-boundaries">
 +		<title>Text boundaries in AD name samples</title>
 +		<para>
 +			During a pass over LIT or CIE input, consecutive sentences with the same text
 +			name share a text id. When the name changes, the first name sample of the next
 +			text sets <code>clearAdaptiveData</code>, so name-finder context does not carry
 +			across text boundaries. Resetting the stream clears this boundary state, so the
 +			first sample of a new pass also clears adaptive data. Amazonia input continues
 +			to use its numeric text id.
 +			This changes training and evaluation samples for LIT and CIE corpora; evaluate
 +			models with representative data from those corpora.
 +		</para>
  	</section>
  		<section xml:id="tools.corpora.arvores-deitadas.evaluation">
  			<title>Training and Evaluation</title>