You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Nick Pentreath (JIRA)" <ji...@apache.org> on 2017/03/07 11:06:38 UTC
[jira] [Comment Edited] (SPARK-19848) Regex Support in
StopWordsRemover
[ https://issues.apache.org/jira/browse/SPARK-19848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899250#comment-15899250 ]
Nick Pentreath edited comment on SPARK-19848 at 3/7/17 11:06 AM:
-----------------------------------------------------------------
This seems more like a "token filter" in a sense. Perhaps the ML pipeline components mentioned [here|https://lucidworks.com/2016/04/13/spark-solr-lucenetextanalyzer/] may be of use?
was (Author: mlnick):
Perhaps the ML pipeline components mentioned [here|https://lucidworks.com/2016/04/13/spark-solr-lucenetextanalyzer/] may be of use?
> Regex Support in StopWordsRemover
> ---------------------------------
>
> Key: SPARK-19848
> URL: https://issues.apache.org/jira/browse/SPARK-19848
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Affects Versions: 2.1.0
> Reporter: Mohd Suaib Danish
> Priority: Minor
>
> Can we have regex feature in StopWordsRemover in addition to the provided list of stop words?
> Use cases can be following:
> 1. Remove all single or double letter words- [a-zA-Z]{1,2}
> 2. Remove anything starting with abc - ^abc
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org