You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Nick Pentreath (JIRA)" <ji...@apache.org> on 2017/03/07 11:06:38 UTC

[jira] [Commented] (SPARK-19848) Regex Support in StopWordsRemover

    [ https://issues.apache.org/jira/browse/SPARK-19848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899250#comment-15899250 ] 

Nick Pentreath commented on SPARK-19848:
----------------------------------------

Perhaps the ML pipeline components mentioned [here|https://lucidworks.com/2016/04/13/spark-solr-lucenetextanalyzer/] may be of use?

> Regex Support in StopWordsRemover
> ---------------------------------
>
>                 Key: SPARK-19848
>                 URL: https://issues.apache.org/jira/browse/SPARK-19848
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML
>    Affects Versions: 2.1.0
>            Reporter: Mohd Suaib Danish
>            Priority: Minor
>
> Can we have regex feature in StopWordsRemover in addition to the provided list of stop words?
> Use cases can be following:
> 1. Remove all single or double letter words- [a-zA-Z]{1,2}
> 2. Remove anything starting with abc - ^abc



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org