You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucenenet.apache.org by "hidingmyname (via GitHub)" <gi...@apache.org> on 2026/10/12 16:39:03 UTC
[I] WordDelimiterFilter doc describes a nonexistent `combinations` parameter [lucenenet]
hidingmyname opened a new issue, #1467:
URL: https://github.com/apache/lucenenet/issues/1467
### Is there an existing issue for this?
- [x] I have searched the existing issues
### Describe the documentation issue
**Location** (lucenenet @ master b447bbf04): `src/Lucene.Net.Analysis.Common/Analysis/Miscellaneous/WordDelimiterFilter.cs:120-143`
**Current text**
```
The combinations parameter affects how subwords are combined: combinations="0" ... combinations="1" ...
```
**Problem**: There is no `combinations` parameter. Subword combining is controlled by the `CATENATE_*` flags.
**Suggested fix**: Rewrite the paragraph in terms of the `CATENATE_WORDS` / `CATENATE_NUMBERS` / `CATENATE_ALL` flags, as upstream did.
```diff
--- a/src/Lucene.Net.Analysis.Common/Analysis/Miscellaneous/WordDelimiterFilter.cs
+++ b/src/Lucene.Net.Analysis.Common/Analysis/Miscellaneous/WordDelimiterFilter.cs
@@ -118,9 +118,10 @@
/// </list>
/// <para/>
- /// The <b>combinations</b> parameter affects how subwords are combined:
+ /// The <b>GENERATE...</b> options affect how incoming tokens are broken into parts, and the
+ /// various <b>CATENATE_...</b> parameters affect how those parts are combined.
/// <list type="bullet">
- /// <item><description>combinations="0" causes no subword combinations: <code>"PowerShot"</code>
+ /// <item><description>If no CATENATE option is set, then no subword combinations are generated: <c>"PowerShot"</c>
/// → <c>0:"Power", 1:"Shot"</c> (0 and 1 are the token positions)</description></item>
- /// <item><description>combinations="1" means that in addition to the subwords, maximum runs of
+ /// <item><description>CATENATE_WORDS means that in addition to the subwords, maximum runs of
/// non-numeric subwords are catenated and produced at the same position of the
/// last subword in the run:
@@ -135,4 +136,6 @@
/// </ul>
/// </description></item>
+ /// <item><description>CATENATE_NUMBERS works like CATENATE_WORDS, but for adjacent digit sequences.</description></item>
+ /// <item><description>CATENATE_ALL smushes together all the token parts without distinguishing numbers and words.</description></item>
/// </list>
/// <para/>
@@ -140,6 +143,6 @@
/// subword delimiters. For example, if the source text contained "wi-fi" one may
/// want "wifi" "WiFi" "wi-fi" "wi+fi" queries to all match. One way of doing so
- /// is to specify combinations="1" in the analyzer used for indexing, and
- /// combinations="0" (the default) in the analyzer used for querying. Given that
+ /// is to specify CATENATE options in the analyzer used for indexing, and
+ /// not in the analyzer used for querying. Given that
/// the current <see cref="Standard.StandardTokenizer"/> immediately removes many intra-word
/// delimiters, it is recommended that this filter be used after a tokenizer that
```
### Additional context
**Fixed upstream in Lucene**: apache/lucene@0f53adbee49 (LUCENE-8238)
**How Lucene fixed it**: commit `0f53adbee49` (2018-04-06) "LUCENE-8238: improve javadocs for WordDelimiterFilter and WordDelimiterGraphFilter", file `lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/WordDelimiterFilter.java`
```diff
@@ -55,11 +55,14 @@ import org.apache.lucene.util.InPlaceMergeSorter;
* </li>
* </ul>
*
- * The <b>combinations</b> parameter affects how subwords are combined:
+ * The <b>GENERATE...</b> options affect how incoming tokens are broken into parts, and the
+ * various <b>CATENATE_...</b> parameters affect how those parts are combined.
+ *
* <ul>
- * <li>combinations="0" causes no subword combinations: <code>"PowerShot"</code>
- * → <code>0:"Power", 1:"Shot"</code> (0 and 1 are the token positions)</li>
- * <li>combinations="1" means that in addition to the subwords, maximum runs of
+ * <li>If no CATENATE option is set, then no subword combinations are generated:
+ * <code>"PowerShot"</code> → <code>0:"Power", 1:"Shot"</code> (0 and 1 are the token
+ * positions)</li>
+ * <li>CATENATE_WORDS means that in addition to the subwords, maximum runs of
* non-numeric subwords are catenated and produced at the same position of the
* last subword in the run:
* <ul>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] WordDelimiterFilter doc describes a nonexistent `combinations` parameter [lucenenet]
Posted by "paulirwin (via GitHub)" <gi...@apache.org>.
paulirwin closed issue #1467: WordDelimiterFilter doc describes a nonexistent `combinations` parameter
URL: https://github.com/apache/lucenenet/issues/1467
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org