You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Noble Paul (JIRA)" <ji...@apache.org> on 2009/03/11 06:00:50 UTC

[jira] Created: (SOLR-1061) Improve regexTransformer to create multiple columns from regexGroups

Improve regexTransformer to create multiple columns from regexGroups
--------------------------------------------------------------------

                 Key: SOLR-1061
                 URL: https://issues.apache.org/jira/browse/SOLR-1061
             Project: Solr
          Issue Type: Improvement
          Components: contrib - DataImportHandler
            Reporter: Noble Paul
             Fix For: 1.4


example
{code:xml}
<field column="doesnotmatter" regex="Mr(\w*)\b(\w*)" sourceColName="full_name"  groupNames="1:firstName,2:secondName"/>
{code}

This is more efficient in extracting multiple values from a single String. In this case the column is redundant but it is ok

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (SOLR-1061) Improve RegexTransformer to create multiple columns from regex groups

Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shalin Shekhar Mangar reassigned SOLR-1061:
-------------------------------------------

    Assignee: Shalin Shekhar Mangar

> Improve RegexTransformer to create multiple columns from regex groups
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1061
>                 URL: https://issues.apache.org/jira/browse/SOLR-1061
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>            Reporter: Noble Paul
>            Assignee: Shalin Shekhar Mangar
>             Fix For: 1.4
>
>         Attachments: SOLR-1061.patch, SOLR-1061.patch, SOLR-1061.patch
>
>
> example
> {code:xml}
> <field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
> {code}
> This is more efficient in extracting multiple values from a single String. if some groups need to be omitted just leave it empty 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1061) Improve RegexTransformer to create multiple columns from regex groups

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-1061:
-----------------------------

    Attachment: SOLR-1061.patch

fix . w/ no testcase

> Improve RegexTransformer to create multiple columns from regex groups
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1061
>                 URL: https://issues.apache.org/jira/browse/SOLR-1061
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>            Reporter: Noble Paul
>             Fix For: 1.4
>
>         Attachments: SOLR-1061.patch
>
>
> example
> {code:xml}
> <field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
> {code}
> This is more efficient in extracting multiple values from a single String. if some groups need to be omitted just leave it empty 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1061) Improve RegexTransformer to create multiple columns from regex groups

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-1061:
-----------------------------

    Attachment: SOLR-1061.patch

with testcase

> Improve RegexTransformer to create multiple columns from regex groups
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1061
>                 URL: https://issues.apache.org/jira/browse/SOLR-1061
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>            Reporter: Noble Paul
>             Fix For: 1.4
>
>         Attachments: SOLR-1061.patch, SOLR-1061.patch, SOLR-1061.patch
>
>
> example
> {code:xml}
> <field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
> {code}
> This is more efficient in extracting multiple values from a single String. if some groups need to be omitted just leave it empty 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1061) Improve RegexTransformer to create multiple columns from regex groups

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-1061:
-----------------------------

    Attachment: SOLR-1061.patch

fix w/o testcase

> Improve RegexTransformer to create multiple columns from regex groups
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1061
>                 URL: https://issues.apache.org/jira/browse/SOLR-1061
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>            Reporter: Noble Paul
>             Fix For: 1.4
>
>         Attachments: SOLR-1061.patch
>
>
> example
> {code:xml}
> <field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
> {code}
> This is more efficient in extracting multiple values from a single String. if some groups need to be omitted just leave it empty 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1061) Improve RegexTransformer to create multiple columns from regex groups

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-1061:
-----------------------------

    Description: 
example
{code:xml}
<field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
{code}

This is more efficient in extracting multiple values from a single String. if some groups need to be omitted just leave it empty 

  was:
example
{code:xml}
<field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
{code}

This is more efficient in extracting multiple values from a single String. In this case the column is redundant but it is ok


> Improve RegexTransformer to create multiple columns from regex groups
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1061
>                 URL: https://issues.apache.org/jira/browse/SOLR-1061
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>            Reporter: Noble Paul
>             Fix For: 1.4
>
>
> example
> {code:xml}
> <field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
> {code}
> This is more efficient in extracting multiple values from a single String. if some groups need to be omitted just leave it empty 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (SOLR-1061) Improve RegexTransformer to create multiple columns from regex groups

Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shalin Shekhar Mangar resolved SOLR-1061.
-----------------------------------------

    Resolution: Fixed

Committed revision 755143.

Thanks Noble!

Fergus, can you open another issue for the other enhancements you mentioned? Also, I see only these flags in java.util.Pattern 
# CASE_INSENSITIVE
# MULTILINE
# DOTALL
# UNICODE_CASE
# CANON_EQ

What does global do exactly?

> Improve RegexTransformer to create multiple columns from regex groups
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1061
>                 URL: https://issues.apache.org/jira/browse/SOLR-1061
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>            Reporter: Noble Paul
>            Assignee: Shalin Shekhar Mangar
>             Fix For: 1.4
>
>         Attachments: SOLR-1061.patch, SOLR-1061.patch, SOLR-1061.patch
>
>
> example
> {code:xml}
> <field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
> {code}
> This is more efficient in extracting multiple values from a single String. if some groups need to be omitted just leave it empty 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1061) Improve RegexTransformer to create multiple columns from regex groups

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-1061:
-----------------------------

    Description: 
example
{code:xml}
<field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
{code}

This is more efficient in extracting multiple values from a single String. In this case the column is redundant but it is ok

  was:
example
{code:xml}
<field column="doesnotmatter" regex="Mr(\w*)\b(\w*)" sourceColName="full_name"  groupNames="1:firstName,2:secondName"/>
{code}

This is more efficient in extracting multiple values from a single String. In this case the column is redundant but it is ok

        Summary: Improve RegexTransformer to create multiple columns from regex groups  (was: Improve regexTransformer to create multiple columns from regexGroups)

> Improve RegexTransformer to create multiple columns from regex groups
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1061
>                 URL: https://issues.apache.org/jira/browse/SOLR-1061
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>            Reporter: Noble Paul
>             Fix For: 1.4
>
>
> example
> {code:xml}
> <field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
> {code}
> This is more efficient in extracting multiple values from a single String. In this case the column is redundant but it is ok

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1061) Improve RegexTransformer to create multiple columns from regex groups

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-1061:
-----------------------------

    Attachment:     (was: SOLR-1061.patch)

> Improve RegexTransformer to create multiple columns from regex groups
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1061
>                 URL: https://issues.apache.org/jira/browse/SOLR-1061
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>            Reporter: Noble Paul
>             Fix For: 1.4
>
>         Attachments: SOLR-1061.patch
>
>
> example
> {code:xml}
> <field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
> {code}
> This is more efficient in extracting multiple values from a single String. if some groups need to be omitted just leave it empty 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1061) Improve RegexTransformer to create multiple columns from regex groups

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-1061:
-----------------------------

    Attachment: SOLR-1061.patch

there was a bug in the last patch

> Improve RegexTransformer to create multiple columns from regex groups
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1061
>                 URL: https://issues.apache.org/jira/browse/SOLR-1061
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>            Reporter: Noble Paul
>             Fix For: 1.4
>
>         Attachments: SOLR-1061.patch, SOLR-1061.patch
>
>
> example
> {code:xml}
> <field column="full_name" regex="Mr(\w*)\b(\w*)" groupNames="firstName,secondName"/>
> {code}
> This is more efficient in extracting multiple values from a single String. if some groups need to be omitted just leave it empty 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (SOLR-1061) Improve regexTransformer to create multiple columns from regexGroups

Posted by "Fergus McMenemie (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SOLR-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680807#action_12680807 ] 

Fergus McMenemie commented on SOLR-1061:
----------------------------------------

Yes, yes. Another usecase I ran into a lot was having lat/long within the same XML field, this would have been really useful.  I guess if the matcher fails the fields/colums firstName and secondName are undefined?  However although the above is neat and clean it can of course now be done as follows:-
{code}
   <field column="firstName"       regex="Mr(\w*)\b\w*" replaceWith="$1"  sourceColName="full_name"/>
   <field column="secondName" regex="Mr\w*\b(\w*)" replaceWith="$1"  sourceColName="full_name"/>
{code}

Also I would think the following will be a related common usecase; imagine a field which listed an indeterminate number of aliases or alternate names for a person. This is bad data design but it happens. We need to expose regex's global feature

{code}
<firstName>josephine</firstname>
<aliases>jo,joe,jos<aliases>
{code}

{code}
   <field column="alias" regex="([^,]+)"  regex_options="global" sourceColName="aliases"/>
{code}

which would populate the column alias with multiple values. The attribute regex_options allows other regex options such as case insensitivity to be added as well.



> Improve regexTransformer to create multiple columns from regexGroups
> --------------------------------------------------------------------
>
>                 Key: SOLR-1061
>                 URL: https://issues.apache.org/jira/browse/SOLR-1061
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - DataImportHandler
>            Reporter: Noble Paul
>             Fix For: 1.4
>
>
> example
> {code:xml}
> <field column="doesnotmatter" regex="Mr(\w*)\b(\w*)" sourceColName="full_name"  groupNames="1:firstName,2:secondName"/>
> {code}
> This is more efficient in extracting multiple values from a single String. In this case the column is redundant but it is ok

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.