Re: [I] Allow users to specify lucene analyzer when creating text index [pinot]

["0ZhangJc0 (via GitHub)" <gi...@apache.org>]

0ZhangJc0 commented on issue #9154:
URL: https://github.com/apache/pinot/issues/9154#issuecomment-2449030641

   > We have additional feature that we want to commit to OSS that allow user specify parameters to custom analyzer and pair it with custom query parser if needed.
   > 
   > ```
   > fieldConfigList: [
   >    {
   >         "name": "columnName",
   >         "indexType": "TEXT",
   >         "indexTypes": [
   >           "TEXT"
   >         ],
   >         "properties": {
   >           "luceneAnalyzerClass": "x.utils.lucene.analyzer.DelimiterAnalyzer",
   >           "luceneAnalyzerClassArgTypes": "java.lang.String, java.lang.String",
   >           "luceneAnalyzerClassArgs": " \\,.\n\t()[]{}\"':=-_$\\?@&|#+/,\\,.()[]{}\"':=-_$\\?@&|#+/",
   >           "luceneQueryParserClass": "x.utils.lucene.queryparser.AnalyzingQueryParser"
   >         },
   >       }
   >   ]
   > ```
   > 
   > We've been using these configs for better half of a year in production now and it seems to work well. The PR is [here](https://github.com/apache/pinot/pull/13003), but I need too add more unit tests so that it can be merged into OSS. We'll update the docs afterwards very soon.
   
   @jackluo923 
   I need to customize an analyzer. May I ask which module in the code should I add my customized analyzer to? Similar to your `x.utils.lucene.analyzer.DelimiterAnalyzer`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org