[lucene-solr] branch branch_8_0 updated: Update language-analysis.adoc

[tf...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

tflobbe pushed a commit to branch branch_8_0
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git


The following commit(s) were added to refs/heads/branch_8_0 by this push:
     new 9d0dbe4  Update language-analysis.adoc
9d0dbe4 is described below

commit 9d0dbe43ede8801431dd61b377ea1e6b00b19450
Author: Konstantin Perikov <My...@users.noreply.github.com>
AuthorDate: Wed Jan 30 10:28:27 2019 +0000

    Update language-analysis.adoc
---
 solr/solr-ref-guide/src/language-analysis.adoc | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/solr/solr-ref-guide/src/language-analysis.adoc b/solr/solr-ref-guide/src/language-analysis.adoc
index d31d295..cca0387 100644
--- a/solr/solr-ref-guide/src/language-analysis.adoc
+++ b/solr/solr-ref-guide/src/language-analysis.adoc
@@ -577,6 +577,7 @@ Perform model-based lemmatization only, preserving the original token and emitti
 These factories are each designed to work with specific languages. The languages covered here are:
 
 * <<Arabic>>
+* <<Bengali>>
 * <<Brazilian Portuguese>>
 * <<Bulgarian>>
 * <<Catalan>>
@@ -633,6 +634,31 @@ This algorithm defines both character normalization and stemming, so these are s
 </analyzer>
 ----
 
+=== Bengali
+
+There are two filters written specifically for dealing with Bengali language. They use the Lucene classes `org.apache.lucene.analysis.bn.BengaliNormalizationFilter` and `org.apache.lucene.analysis.bn.BengaliStemFilter`.
+
+*Factory classes:* `solr.BengaliStemFilterFactory`, `solr.BengaliNormalizationFilterFactory`
+
+*Arguments:* None
+
+*Example:*
+
+[source,xml]
+----
+<analyzer> 
+  <tokenizer class="solr.StandardTokenizerFactory"/> 
+  <filter class="solr.BengaliNormalizationFilterFactory"/> 
+  <filter class="solr.BengaliStemFilterFactory"/>       
+</analyzer>
+
+----
+
+*Normalisation* - `মানুষ` -> `মানুস`
+
+*Stemming* - `সমস্ত` -> `সমস্` 
+
+
 === Brazilian Portuguese
 
 This is a Java filter written specifically for stemming the Brazilian dialect of the Portuguese language. It uses the Lucene class `org.apache.lucene.analysis.br.BrazilianStemmer`. Although that stemmer can be configured to use a list of protected words (which should not be stemmed), this factory does not accept any arguments to specify such a list.