You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by ba3 <sb...@gmail.com> on 2009/07/26 15:52:37 UTC
Multiline Regex with Lucene
I was trying to do a regex search with the lucene and
JavaUtilRegexCapabilities.
The code used is :
RegexQuery query = new RegexQuery(new
Term("contents","(?m)hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#"));
query.setRegexImplementation(new JavaUtilRegexCapabilities());
I verified the regex in : http://www.gskinner.com/RegExr/ [with the multi
line checked]
In lucene though there are no hits. Can you please point me in the right
direction
-- Rgds
Ba3
Regex :
hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
Content :
hello world
This is to be searched
#
Test line should not be selected
hello
This should not work
some other lines
#
Not to be selected
hello world
Some lines
This is to be searched
Some lines
#
hello earth
some lines
#
--
View this message in context: http://www.nabble.com/Multiline-Regex-with-Lucene-tp24667109p24667109.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Multiline Regex with Lucene
Posted by Steven A Rowe <sa...@syr.edu>.
Hi Ba3,
I think what you're asking is how to use SpanNearQuery's without specifying the number of positions between the Span's. This is answered in the article I linked to - here's the quote:
> In certain situations, it can be convenient to have a SpanAndQuery.
> You can easily simulate this using a SpanNearQuery with a distance of
> Integer.MAX_VALUE.
That is, you can remove the distance restriction by saying that it can be (effectively) infinity. AFAIK, an individual document field can contain no more than Integer.MAX_VALUE terms, so this limitation is not due to the Spans implementation.
Steve
> -----Original Message-----
> From: ba3 [mailto:sbadhrinath@gmail.com]
> Sent: Wednesday, July 29, 2009 12:56 PM
> To: java-user@lucene.apache.org
> Subject: RE: Multiline Regex with Lucene
>
>
> Hi Steve,
>
> I went through the article. Thanks for the link. The span query
> mentions the
> position i.e n positions from the terms. The problem was like this :
>
> Lucene was <some more text> made by Doug <some more text> cutting
>
> If Doug is found between the words Lucene and cutting then it is a hit.
> [there can be any number of positions which is unknown]. If the number
> of
> positions are known then the spans could be used.
>
> I came across qsol where in the paragraphseperator and sentence
> seperator
> can be specified and string can be searched within the paragraph.
>
> Can you give your comments.
>
> - Rgds
> Ba3
>
>
>
>
>
> Steven A Rowe wrote:
> >
> > Hi ba3,
> >
> > Did you read the Lucid Imagination article I linked to?:
> >
> > http://www.lucidimagination.com/blog/2009/07/18/the-spanquery/
> >
> >
> > It has examples, including specifying the term indicating the end of
> the
> > span.
> >
> > If the article doesn't do it for you, I need more information to be
> able
> > to help. Can you give an example of what you want to do?
> >
> > Thanks,
> > Steve
> >
> >> -----Original Message-----
> >> From: ba3 [mailto:sbadhrinath@gmail.com]
> >> Sent: Tuesday, July 28, 2009 10:39 PM
> >> To: java-user@lucene.apache.org
> >> Subject: RE: Multiline Regex with Lucene
> >>
> >>
> >> Hi Steve,
> >>
> >> In case of span queries, the span first query can specify the start
> of
> >> the
> >> span, is it possible to specify the term [not the position]
> indicating
> >> the
> >> end of the span ?
> >>
> >> -- Regards
> >> Ba3
> >>
> >>
> >> Steven A Rowe wrote:
> >> >
> >> > Hi ba3,
> >> >
> >> > Check out the list of "Direct Known Subclasses" from the SpanQuery
> >> > javadocs to see what's available:
> >> >
> >> >
> >>
> http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/spans/
> >> SpanQuery.html
> >> >
> >> > SpanRegexQuery may be what you're looking for:
> >> >
> >> >
> >>
> http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/regex/
> >> SpanRegexQuery.html
> >> >
> >> >
> >> > Steve
> >> >
> >> >> -----Original Message-----
> >> >> From: ba3 [mailto:sbadhrinath@gmail.com]
> >> >> Sent: Tuesday, July 28, 2009 12:53 PM
> >> >> To: java-user@lucene.apache.org
> >> >> Subject: Re: Multiline Regex with Lucene
> >> >>
> >> >>
> >> >> Hi,
> >> >>
> >> >> Thanks for the pointers. I will try the span queries.
> >> >> But can span query support regexp as a term ?
> >> >>
> >> >> Also for more details in the problem :
> >> >> The problem is like this:
> >> >> find a search string inside a block of statements.
> >> >> The block starts with a string and ends with a character.
> >> >>
> >> >> -- Regards
> >> >> Ba3
> >> >>
> >> >>
> >> >>
> >> >> Erick Erickson wrote:
> >> >> >
> >> >> > I doubt you're thinking in terms of tokens. Your inputstream is
> >> broken
> >> >> up
> >> >> > into tokens (think of them as words,
> >> >> > depending upon the analyzer) and regex searchers are
> >> >> > confined to those *tokens*. So the concept of a multi-line
> >> >> > regex in a search is kind of ...odd...
> >> >> >
> >> >> > You could possibly index your input as UN_TOKENIZED, but
> >> >> > I really have no clue what Lucene would do with that. I think
> >> >> > you're off in uncharted territory here.
> >> >> >
> >> >> > Perhaps a better thing would be for you to explain *why* you
> >> >> > want to do this and perhaps folks can come up with some
> >> >> > suggestions, I suspect this may be an XY problem, see
> >> >> > http://www.perlmonks.org/index.pl?node_id=542341
> >> >> >
> >> >> > Best
> >> >> > Erick
> >> >> >
> >> >> > On Sun, Jul 26, 2009 at 9:52 AM, ba3 <sb...@gmail.com>
> >> wrote:
> >> >> >
> >> >> >>
> >> >> >> I was trying to do a regex search with the lucene and
> >> >> >> JavaUtilRegexCapabilities.
> >> >> >> The code used is :
> >> >> >> RegexQuery query = new RegexQuery(new
> >> >> >> Term("contents","(?m)hello.*(\r[^#]*)This is to be
> >> >> >> searched.*(\r[^#]*)#"));
> >> >> >> query.setRegexImplementation(new JavaUtilRegexCapabilities());
> >> >> >>
> >> >> >> I verified the regex in : http://www.gskinner.com/RegExr/
> [with
> >> the
> >> >> >> multi
> >> >> >> line checked]
> >> >> >> In lucene though there are no hits. Can you please point me in
> >> the
> >> >> right
> >> >> >> direction
> >> >> >>
> >> >> >> -- Rgds
> >> >> >> Ba3
> >> >> >>
> >> >> >> Regex :
> >> >> >> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
> >> >> >>
> >> >> >> Content :
> >> >> >> hello world
> >> >> >> This is to be searched
> >> >> >> #
> >> >> >> Test line should not be selected
> >> >> >> hello
> >> >> >> This should not work
> >> >> >> some other lines
> >> >> >> #
> >> >> >> Not to be selected
> >> >> >> hello world
> >> >> >> Some lines
> >> >> >> This is to be searched
> >> >> >> Some lines
> >> >> >> #
> >> >> >> hello earth
> >> >> >> some lines
> >> >> >> #
> >> >> >> --
> >> >> >> View this message in context:
> >> >> >>
> >> >> http://www.nabble.com/Multiline-Regex-with-Lucene-
> >> tp24667109p24667109.html
> >> >> >> Sent from the Lucene - Java Users mailing list archive at
> >> Nabble.com.
> >> >> >>
> >> >> >>
> >> >> >> --------------------------------------------------------------
> ---
> >> ----
> >> >> >> To unsubscribe, e-mail: java-user-
> unsubscribe@lucene.apache.org
> >> >> >> For additional commands, e-mail: java-user-
> help@lucene.apache.org
> >> >> >>
> >> >> >>
> >> >> >
> >> >> >
> >> >>
> >> >> --
> >> >> View this message in context: http://www.nabble.com/Multiline-
> Regex-
> >> with-
> >> >> Lucene-tp24667109p24703547.html
> >> >> Sent from the Lucene - Java Users mailing list archive at
> >> Nabble.com.
> >> >>
> >> >>
> >> >> -----------------------------------------------------------------
> ---
> >> -
> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >
> >> >
> >> > ------------------------------------------------------------------
> ---
> >> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >
> >> >
> >> >
> >>
> >> --
> >> View this message in context: http://www.nabble.com/Multiline-Regex-
> >> with-Lucene-tp24667109p24711445.html
> >> Sent from the Lucene - Java Users mailing list archive at
> Nabble.com.
> >>
> >>
> >> --------------------------------------------------------------------
> -
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Multiline-Regex-
> with-Lucene-tp24667109p24723404.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Multiline Regex with Lucene
Posted by Mark Miller <ma...@gmail.com>.
>>I came across qsol where in the paragraphseperator and sentence seperator
>>can be specified and string can be searched within the paragraph.
Qsol does this by using SpanQuerys. First you inject special marker tokens
as your paragraph/sentence markers, then you use a SpanNotQuery that looks
for a SpanNearQuery that doesn't intersect with a SpanTermQuery containing
the special marker term.
--
- Mark
http://www.lucidimagination.com
On Wed, Jul 29, 2009 at 4:56 PM, ba3 <sb...@gmail.com> wrote:
>
> Hi Steve,
>
> I went through the article. Thanks for the link. The span query mentions
> the
> position i.e n positions from the terms. The problem was like this :
>
> Lucene was <some more text> made by Doug <some more text> cutting
>
> If Doug is found between the words Lucene and cutting then it is a hit.
> [there can be any number of positions which is unknown]. If the number of
> positions are known then the spans could be used.
>
> I came across qsol where in the paragraphseperator and sentence seperator
> can be specified and string can be searched within the paragraph.
>
> Can you give your comments.
>
> - Rgds
> Ba3
>
>
>
>
>
> Steven A Rowe wrote:
> >
> > Hi ba3,
> >
> > Did you read the Lucid Imagination article I linked to?:
> >
> > http://www.lucidimagination.com/blog/2009/07/18/the-spanquery/
> >
> >
> > It has examples, including specifying the term indicating the end of the
> > span.
> >
> > If the article doesn't do it for you, I need more information to be able
> > to help. Can you give an example of what you want to do?
> >
> > Thanks,
> > Steve
> >
> >> -----Original Message-----
> >> From: ba3 [mailto:sbadhrinath@gmail.com]
> >> Sent: Tuesday, July 28, 2009 10:39 PM
> >> To: java-user@lucene.apache.org
> >> Subject: RE: Multiline Regex with Lucene
> >>
> >>
> >> Hi Steve,
> >>
> >> In case of span queries, the span first query can specify the start of
> >> the
> >> span, is it possible to specify the term [not the position] indicating
> >> the
> >> end of the span ?
> >>
> >> -- Regards
> >> Ba3
> >>
> >>
> >> Steven A Rowe wrote:
> >> >
> >> > Hi ba3,
> >> >
> >> > Check out the list of "Direct Known Subclasses" from the SpanQuery
> >> > javadocs to see what's available:
> >> >
> >> >
> >> http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/spans/
> >> SpanQuery.html
> >> >
> >> > SpanRegexQuery may be what you're looking for:
> >> >
> >> >
> >> http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/regex/
> >> SpanRegexQuery.html
> >> >
> >> >
> >> > Steve
> >> >
> >> >> -----Original Message-----
> >> >> From: ba3 [mailto:sbadhrinath@gmail.com]
> >> >> Sent: Tuesday, July 28, 2009 12:53 PM
> >> >> To: java-user@lucene.apache.org
> >> >> Subject: Re: Multiline Regex with Lucene
> >> >>
> >> >>
> >> >> Hi,
> >> >>
> >> >> Thanks for the pointers. I will try the span queries.
> >> >> But can span query support regexp as a term ?
> >> >>
> >> >> Also for more details in the problem :
> >> >> The problem is like this:
> >> >> find a search string inside a block of statements.
> >> >> The block starts with a string and ends with a character.
> >> >>
> >> >> -- Regards
> >> >> Ba3
> >> >>
> >> >>
> >> >>
> >> >> Erick Erickson wrote:
> >> >> >
> >> >> > I doubt you're thinking in terms of tokens. Your inputstream is
> >> broken
> >> >> up
> >> >> > into tokens (think of them as words,
> >> >> > depending upon the analyzer) and regex searchers are
> >> >> > confined to those *tokens*. So the concept of a multi-line
> >> >> > regex in a search is kind of ...odd...
> >> >> >
> >> >> > You could possibly index your input as UN_TOKENIZED, but
> >> >> > I really have no clue what Lucene would do with that. I think
> >> >> > you're off in uncharted territory here.
> >> >> >
> >> >> > Perhaps a better thing would be for you to explain *why* you
> >> >> > want to do this and perhaps folks can come up with some
> >> >> > suggestions, I suspect this may be an XY problem, see
> >> >> > http://www.perlmonks.org/index.pl?node_id=542341
> >> >> >
> >> >> > Best
> >> >> > Erick
> >> >> >
> >> >> > On Sun, Jul 26, 2009 at 9:52 AM, ba3 <sb...@gmail.com>
> >> wrote:
> >> >> >
> >> >> >>
> >> >> >> I was trying to do a regex search with the lucene and
> >> >> >> JavaUtilRegexCapabilities.
> >> >> >> The code used is :
> >> >> >> RegexQuery query = new RegexQuery(new
> >> >> >> Term("contents","(?m)hello.*(\r[^#]*)This is to be
> >> >> >> searched.*(\r[^#]*)#"));
> >> >> >> query.setRegexImplementation(new JavaUtilRegexCapabilities());
> >> >> >>
> >> >> >> I verified the regex in : http://www.gskinner.com/RegExr/ [with
> >> the
> >> >> >> multi
> >> >> >> line checked]
> >> >> >> In lucene though there are no hits. Can you please point me in
> >> the
> >> >> right
> >> >> >> direction
> >> >> >>
> >> >> >> -- Rgds
> >> >> >> Ba3
> >> >> >>
> >> >> >> Regex :
> >> >> >> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
> >> >> >>
> >> >> >> Content :
> >> >> >> hello world
> >> >> >> This is to be searched
> >> >> >> #
> >> >> >> Test line should not be selected
> >> >> >> hello
> >> >> >> This should not work
> >> >> >> some other lines
> >> >> >> #
> >> >> >> Not to be selected
> >> >> >> hello world
> >> >> >> Some lines
> >> >> >> This is to be searched
> >> >> >> Some lines
> >> >> >> #
> >> >> >> hello earth
> >> >> >> some lines
> >> >> >> #
> >> >> >> --
> >> >> >> View this message in context:
> >> >> >>
> >> >> http://www.nabble.com/Multiline-Regex-with-Lucene-
> >> tp24667109p24667109.html
> >> >> >> Sent from the Lucene - Java Users mailing list archive at
> >> Nabble.com.
> >> >> >>
> >> >> >>
> >> >> >> -----------------------------------------------------------------
> >> ----
> >> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >> >>
> >> >> >>
> >> >> >
> >> >> >
> >> >>
> >> >> --
> >> >> View this message in context: http://www.nabble.com/Multiline-Regex-
> >> with-
> >> >> Lucene-tp24667109p24703547.html
> >> >> Sent from the Lucene - Java Users mailing list archive at
> >> Nabble.com.
> >> >>
> >> >>
> >> >> --------------------------------------------------------------------
> >> -
> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >
> >> >
> >> >
> >>
> >> --
> >> View this message in context: http://www.nabble.com/Multiline-Regex-
> >> with-Lucene-tp24667109p24711445.html
> >> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Multiline-Regex-with-Lucene-tp24667109p24723404.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
RE: Multiline Regex with Lucene
Posted by ba3 <sb...@gmail.com>.
Hi Steve,
I went through the article. Thanks for the link. The span query mentions the
position i.e n positions from the terms. The problem was like this :
Lucene was <some more text> made by Doug <some more text> cutting
If Doug is found between the words Lucene and cutting then it is a hit.
[there can be any number of positions which is unknown]. If the number of
positions are known then the spans could be used.
I came across qsol where in the paragraphseperator and sentence seperator
can be specified and string can be searched within the paragraph.
Can you give your comments.
- Rgds
Ba3
Steven A Rowe wrote:
>
> Hi ba3,
>
> Did you read the Lucid Imagination article I linked to?:
>
> http://www.lucidimagination.com/blog/2009/07/18/the-spanquery/
>
>
> It has examples, including specifying the term indicating the end of the
> span.
>
> If the article doesn't do it for you, I need more information to be able
> to help. Can you give an example of what you want to do?
>
> Thanks,
> Steve
>
>> -----Original Message-----
>> From: ba3 [mailto:sbadhrinath@gmail.com]
>> Sent: Tuesday, July 28, 2009 10:39 PM
>> To: java-user@lucene.apache.org
>> Subject: RE: Multiline Regex with Lucene
>>
>>
>> Hi Steve,
>>
>> In case of span queries, the span first query can specify the start of
>> the
>> span, is it possible to specify the term [not the position] indicating
>> the
>> end of the span ?
>>
>> -- Regards
>> Ba3
>>
>>
>> Steven A Rowe wrote:
>> >
>> > Hi ba3,
>> >
>> > Check out the list of "Direct Known Subclasses" from the SpanQuery
>> > javadocs to see what's available:
>> >
>> >
>> http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/spans/
>> SpanQuery.html
>> >
>> > SpanRegexQuery may be what you're looking for:
>> >
>> >
>> http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/regex/
>> SpanRegexQuery.html
>> >
>> >
>> > Steve
>> >
>> >> -----Original Message-----
>> >> From: ba3 [mailto:sbadhrinath@gmail.com]
>> >> Sent: Tuesday, July 28, 2009 12:53 PM
>> >> To: java-user@lucene.apache.org
>> >> Subject: Re: Multiline Regex with Lucene
>> >>
>> >>
>> >> Hi,
>> >>
>> >> Thanks for the pointers. I will try the span queries.
>> >> But can span query support regexp as a term ?
>> >>
>> >> Also for more details in the problem :
>> >> The problem is like this:
>> >> find a search string inside a block of statements.
>> >> The block starts with a string and ends with a character.
>> >>
>> >> -- Regards
>> >> Ba3
>> >>
>> >>
>> >>
>> >> Erick Erickson wrote:
>> >> >
>> >> > I doubt you're thinking in terms of tokens. Your inputstream is
>> broken
>> >> up
>> >> > into tokens (think of them as words,
>> >> > depending upon the analyzer) and regex searchers are
>> >> > confined to those *tokens*. So the concept of a multi-line
>> >> > regex in a search is kind of ...odd...
>> >> >
>> >> > You could possibly index your input as UN_TOKENIZED, but
>> >> > I really have no clue what Lucene would do with that. I think
>> >> > you're off in uncharted territory here.
>> >> >
>> >> > Perhaps a better thing would be for you to explain *why* you
>> >> > want to do this and perhaps folks can come up with some
>> >> > suggestions, I suspect this may be an XY problem, see
>> >> > http://www.perlmonks.org/index.pl?node_id=542341
>> >> >
>> >> > Best
>> >> > Erick
>> >> >
>> >> > On Sun, Jul 26, 2009 at 9:52 AM, ba3 <sb...@gmail.com>
>> wrote:
>> >> >
>> >> >>
>> >> >> I was trying to do a regex search with the lucene and
>> >> >> JavaUtilRegexCapabilities.
>> >> >> The code used is :
>> >> >> RegexQuery query = new RegexQuery(new
>> >> >> Term("contents","(?m)hello.*(\r[^#]*)This is to be
>> >> >> searched.*(\r[^#]*)#"));
>> >> >> query.setRegexImplementation(new JavaUtilRegexCapabilities());
>> >> >>
>> >> >> I verified the regex in : http://www.gskinner.com/RegExr/ [with
>> the
>> >> >> multi
>> >> >> line checked]
>> >> >> In lucene though there are no hits. Can you please point me in
>> the
>> >> right
>> >> >> direction
>> >> >>
>> >> >> -- Rgds
>> >> >> Ba3
>> >> >>
>> >> >> Regex :
>> >> >> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
>> >> >>
>> >> >> Content :
>> >> >> hello world
>> >> >> This is to be searched
>> >> >> #
>> >> >> Test line should not be selected
>> >> >> hello
>> >> >> This should not work
>> >> >> some other lines
>> >> >> #
>> >> >> Not to be selected
>> >> >> hello world
>> >> >> Some lines
>> >> >> This is to be searched
>> >> >> Some lines
>> >> >> #
>> >> >> hello earth
>> >> >> some lines
>> >> >> #
>> >> >> --
>> >> >> View this message in context:
>> >> >>
>> >> http://www.nabble.com/Multiline-Regex-with-Lucene-
>> tp24667109p24667109.html
>> >> >> Sent from the Lucene - Java Users mailing list archive at
>> Nabble.com.
>> >> >>
>> >> >>
>> >> >> -----------------------------------------------------------------
>> ----
>> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >>
>> >> --
>> >> View this message in context: http://www.nabble.com/Multiline-Regex-
>> with-
>> >> Lucene-tp24667109p24703547.html
>> >> Sent from the Lucene - Java Users mailing list archive at
>> Nabble.com.
>> >>
>> >>
>> >> --------------------------------------------------------------------
>> -
>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>> >
>> >
>>
>> --
>> View this message in context: http://www.nabble.com/Multiline-Regex-
>> with-Lucene-tp24667109p24711445.html
>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/Multiline-Regex-with-Lucene-tp24667109p24723404.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Multiline Regex with Lucene
Posted by Steven A Rowe <sa...@syr.edu>.
Hi ba3,
Did you read the Lucid Imagination article I linked to?:
http://www.lucidimagination.com/blog/2009/07/18/the-spanquery/
It has examples, including specifying the term indicating the end of the span.
If the article doesn't do it for you, I need more information to be able to help. Can you give an example of what you want to do?
Thanks,
Steve
> -----Original Message-----
> From: ba3 [mailto:sbadhrinath@gmail.com]
> Sent: Tuesday, July 28, 2009 10:39 PM
> To: java-user@lucene.apache.org
> Subject: RE: Multiline Regex with Lucene
>
>
> Hi Steve,
>
> In case of span queries, the span first query can specify the start of
> the
> span, is it possible to specify the term [not the position] indicating
> the
> end of the span ?
>
> -- Regards
> Ba3
>
>
> Steven A Rowe wrote:
> >
> > Hi ba3,
> >
> > Check out the list of "Direct Known Subclasses" from the SpanQuery
> > javadocs to see what's available:
> >
> >
> http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/spans/
> SpanQuery.html
> >
> > SpanRegexQuery may be what you're looking for:
> >
> >
> http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/regex/
> SpanRegexQuery.html
> >
> >
> > Steve
> >
> >> -----Original Message-----
> >> From: ba3 [mailto:sbadhrinath@gmail.com]
> >> Sent: Tuesday, July 28, 2009 12:53 PM
> >> To: java-user@lucene.apache.org
> >> Subject: Re: Multiline Regex with Lucene
> >>
> >>
> >> Hi,
> >>
> >> Thanks for the pointers. I will try the span queries.
> >> But can span query support regexp as a term ?
> >>
> >> Also for more details in the problem :
> >> The problem is like this:
> >> find a search string inside a block of statements.
> >> The block starts with a string and ends with a character.
> >>
> >> -- Regards
> >> Ba3
> >>
> >>
> >>
> >> Erick Erickson wrote:
> >> >
> >> > I doubt you're thinking in terms of tokens. Your inputstream is
> broken
> >> up
> >> > into tokens (think of them as words,
> >> > depending upon the analyzer) and regex searchers are
> >> > confined to those *tokens*. So the concept of a multi-line
> >> > regex in a search is kind of ...odd...
> >> >
> >> > You could possibly index your input as UN_TOKENIZED, but
> >> > I really have no clue what Lucene would do with that. I think
> >> > you're off in uncharted territory here.
> >> >
> >> > Perhaps a better thing would be for you to explain *why* you
> >> > want to do this and perhaps folks can come up with some
> >> > suggestions, I suspect this may be an XY problem, see
> >> > http://www.perlmonks.org/index.pl?node_id=542341
> >> >
> >> > Best
> >> > Erick
> >> >
> >> > On Sun, Jul 26, 2009 at 9:52 AM, ba3 <sb...@gmail.com>
> wrote:
> >> >
> >> >>
> >> >> I was trying to do a regex search with the lucene and
> >> >> JavaUtilRegexCapabilities.
> >> >> The code used is :
> >> >> RegexQuery query = new RegexQuery(new
> >> >> Term("contents","(?m)hello.*(\r[^#]*)This is to be
> >> >> searched.*(\r[^#]*)#"));
> >> >> query.setRegexImplementation(new JavaUtilRegexCapabilities());
> >> >>
> >> >> I verified the regex in : http://www.gskinner.com/RegExr/ [with
> the
> >> >> multi
> >> >> line checked]
> >> >> In lucene though there are no hits. Can you please point me in
> the
> >> right
> >> >> direction
> >> >>
> >> >> -- Rgds
> >> >> Ba3
> >> >>
> >> >> Regex :
> >> >> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
> >> >>
> >> >> Content :
> >> >> hello world
> >> >> This is to be searched
> >> >> #
> >> >> Test line should not be selected
> >> >> hello
> >> >> This should not work
> >> >> some other lines
> >> >> #
> >> >> Not to be selected
> >> >> hello world
> >> >> Some lines
> >> >> This is to be searched
> >> >> Some lines
> >> >> #
> >> >> hello earth
> >> >> some lines
> >> >> #
> >> >> --
> >> >> View this message in context:
> >> >>
> >> http://www.nabble.com/Multiline-Regex-with-Lucene-
> tp24667109p24667109.html
> >> >> Sent from the Lucene - Java Users mailing list archive at
> Nabble.com.
> >> >>
> >> >>
> >> >> -----------------------------------------------------------------
> ----
> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >>
> >> >>
> >> >
> >> >
> >>
> >> --
> >> View this message in context: http://www.nabble.com/Multiline-Regex-
> with-
> >> Lucene-tp24667109p24703547.html
> >> Sent from the Lucene - Java Users mailing list archive at
> Nabble.com.
> >>
> >>
> >> --------------------------------------------------------------------
> -
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Multiline-Regex-
> with-Lucene-tp24667109p24711445.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Multiline Regex with Lucene
Posted by ba3 <sb...@gmail.com>.
Hi Steve,
In case of span queries, the span first query can specify the start of the
span, is it possible to specify the term [not the position] indicating the
end of the span ?
-- Regards
Ba3
Steven A Rowe wrote:
>
> Hi ba3,
>
> Check out the list of "Direct Known Subclasses" from the SpanQuery
> javadocs to see what's available:
>
> http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/spans/SpanQuery.html
>
> SpanRegexQuery may be what you're looking for:
>
> http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/regex/SpanRegexQuery.html
>
>
> Steve
>
>> -----Original Message-----
>> From: ba3 [mailto:sbadhrinath@gmail.com]
>> Sent: Tuesday, July 28, 2009 12:53 PM
>> To: java-user@lucene.apache.org
>> Subject: Re: Multiline Regex with Lucene
>>
>>
>> Hi,
>>
>> Thanks for the pointers. I will try the span queries.
>> But can span query support regexp as a term ?
>>
>> Also for more details in the problem :
>> The problem is like this:
>> find a search string inside a block of statements.
>> The block starts with a string and ends with a character.
>>
>> -- Regards
>> Ba3
>>
>>
>>
>> Erick Erickson wrote:
>> >
>> > I doubt you're thinking in terms of tokens. Your inputstream is broken
>> up
>> > into tokens (think of them as words,
>> > depending upon the analyzer) and regex searchers are
>> > confined to those *tokens*. So the concept of a multi-line
>> > regex in a search is kind of ...odd...
>> >
>> > You could possibly index your input as UN_TOKENIZED, but
>> > I really have no clue what Lucene would do with that. I think
>> > you're off in uncharted territory here.
>> >
>> > Perhaps a better thing would be for you to explain *why* you
>> > want to do this and perhaps folks can come up with some
>> > suggestions, I suspect this may be an XY problem, see
>> > http://www.perlmonks.org/index.pl?node_id=542341
>> >
>> > Best
>> > Erick
>> >
>> > On Sun, Jul 26, 2009 at 9:52 AM, ba3 <sb...@gmail.com> wrote:
>> >
>> >>
>> >> I was trying to do a regex search with the lucene and
>> >> JavaUtilRegexCapabilities.
>> >> The code used is :
>> >> RegexQuery query = new RegexQuery(new
>> >> Term("contents","(?m)hello.*(\r[^#]*)This is to be
>> >> searched.*(\r[^#]*)#"));
>> >> query.setRegexImplementation(new JavaUtilRegexCapabilities());
>> >>
>> >> I verified the regex in : http://www.gskinner.com/RegExr/ [with the
>> >> multi
>> >> line checked]
>> >> In lucene though there are no hits. Can you please point me in the
>> right
>> >> direction
>> >>
>> >> -- Rgds
>> >> Ba3
>> >>
>> >> Regex :
>> >> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
>> >>
>> >> Content :
>> >> hello world
>> >> This is to be searched
>> >> #
>> >> Test line should not be selected
>> >> hello
>> >> This should not work
>> >> some other lines
>> >> #
>> >> Not to be selected
>> >> hello world
>> >> Some lines
>> >> This is to be searched
>> >> Some lines
>> >> #
>> >> hello earth
>> >> some lines
>> >> #
>> >> --
>> >> View this message in context:
>> >>
>> http://www.nabble.com/Multiline-Regex-with-Lucene-tp24667109p24667109.html
>> >> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>
>> >>
>> >
>> >
>>
>> --
>> View this message in context: http://www.nabble.com/Multiline-Regex-with-
>> Lucene-tp24667109p24703547.html
>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/Multiline-Regex-with-Lucene-tp24667109p24711445.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Multiline Regex with Lucene
Posted by Steven A Rowe <sa...@syr.edu>.
Hi ba3,
Check out the list of "Direct Known Subclasses" from the SpanQuery javadocs to see what's available:
http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/spans/SpanQuery.html
SpanRegexQuery may be what you're looking for:
http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/regex/SpanRegexQuery.html
Steve
> -----Original Message-----
> From: ba3 [mailto:sbadhrinath@gmail.com]
> Sent: Tuesday, July 28, 2009 12:53 PM
> To: java-user@lucene.apache.org
> Subject: Re: Multiline Regex with Lucene
>
>
> Hi,
>
> Thanks for the pointers. I will try the span queries.
> But can span query support regexp as a term ?
>
> Also for more details in the problem :
> The problem is like this:
> find a search string inside a block of statements.
> The block starts with a string and ends with a character.
>
> -- Regards
> Ba3
>
>
>
> Erick Erickson wrote:
> >
> > I doubt you're thinking in terms of tokens. Your inputstream is broken up
> > into tokens (think of them as words,
> > depending upon the analyzer) and regex searchers are
> > confined to those *tokens*. So the concept of a multi-line
> > regex in a search is kind of ...odd...
> >
> > You could possibly index your input as UN_TOKENIZED, but
> > I really have no clue what Lucene would do with that. I think
> > you're off in uncharted territory here.
> >
> > Perhaps a better thing would be for you to explain *why* you
> > want to do this and perhaps folks can come up with some
> > suggestions, I suspect this may be an XY problem, see
> > http://www.perlmonks.org/index.pl?node_id=542341
> >
> > Best
> > Erick
> >
> > On Sun, Jul 26, 2009 at 9:52 AM, ba3 <sb...@gmail.com> wrote:
> >
> >>
> >> I was trying to do a regex search with the lucene and
> >> JavaUtilRegexCapabilities.
> >> The code used is :
> >> RegexQuery query = new RegexQuery(new
> >> Term("contents","(?m)hello.*(\r[^#]*)This is to be
> >> searched.*(\r[^#]*)#"));
> >> query.setRegexImplementation(new JavaUtilRegexCapabilities());
> >>
> >> I verified the regex in : http://www.gskinner.com/RegExr/ [with the
> >> multi
> >> line checked]
> >> In lucene though there are no hits. Can you please point me in the right
> >> direction
> >>
> >> -- Rgds
> >> Ba3
> >>
> >> Regex :
> >> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
> >>
> >> Content :
> >> hello world
> >> This is to be searched
> >> #
> >> Test line should not be selected
> >> hello
> >> This should not work
> >> some other lines
> >> #
> >> Not to be selected
> >> hello world
> >> Some lines
> >> This is to be searched
> >> Some lines
> >> #
> >> hello earth
> >> some lines
> >> #
> >> --
> >> View this message in context:
> >> http://www.nabble.com/Multiline-Regex-with-Lucene-tp24667109p24667109.html
> >> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Multiline-Regex-with-
> Lucene-tp24667109p24703547.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Multiline Regex with Lucene
Posted by Uwe Schindler <uw...@thetaphi.de>.
Yes, SpanRegExQuery
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: ba3 [mailto:sbadhrinath@gmail.com]
> Sent: Tuesday, July 28, 2009 6:53 PM
> To: java-user@lucene.apache.org
> Subject: Re: Multiline Regex with Lucene
>
>
> Hi,
>
> Thanks for the pointers. I will try the span queries.
> But can span query support regexp as a term ?
>
> Also for more details in the problem :
> The problem is like this:
> find a search string inside a block of statements.
> The block starts with a string and ends with a character.
>
> -- Regards
> Ba3
>
>
>
> Erick Erickson wrote:
> >
> > I doubt you're thinking in terms of tokens. Your inputstream is broken
> up
> > into tokens (think of them as words,
> > depending upon the analyzer) and regex searchers are
> > confined to those *tokens*. So the concept of a multi-line
> > regex in a search is kind of ...odd...
> >
> > You could possibly index your input as UN_TOKENIZED, but
> > I really have no clue what Lucene would do with that. I think
> > you're off in uncharted territory here.
> >
> > Perhaps a better thing would be for you to explain *why* you
> > want to do this and perhaps folks can come up with some
> > suggestions, I suspect this may be an XY problem, see
> > http://www.perlmonks.org/index.pl?node_id=542341
> >
> > Best
> > Erick
> >
> > On Sun, Jul 26, 2009 at 9:52 AM, ba3 <sb...@gmail.com> wrote:
> >
> >>
> >> I was trying to do a regex search with the lucene and
> >> JavaUtilRegexCapabilities.
> >> The code used is :
> >> RegexQuery query = new RegexQuery(new
> >> Term("contents","(?m)hello.*(\r[^#]*)This is to be
> >> searched.*(\r[^#]*)#"));
> >> query.setRegexImplementation(new JavaUtilRegexCapabilities());
> >>
> >> I verified the regex in : http://www.gskinner.com/RegExr/ [with the
> >> multi
> >> line checked]
> >> In lucene though there are no hits. Can you please point me in the
> right
> >> direction
> >>
> >> -- Rgds
> >> Ba3
> >>
> >> Regex :
> >> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
> >>
> >> Content :
> >> hello world
> >> This is to be searched
> >> #
> >> Test line should not be selected
> >> hello
> >> This should not work
> >> some other lines
> >> #
> >> Not to be selected
> >> hello world
> >> Some lines
> >> This is to be searched
> >> Some lines
> >> #
> >> hello earth
> >> some lines
> >> #
> >> --
> >> View this message in context:
> >> http://www.nabble.com/Multiline-Regex-with-Lucene-
> tp24667109p24667109.html
> >> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Multiline-Regex-with-
> Lucene-tp24667109p24703547.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Multiline Regex with Lucene
Posted by ba3 <sb...@gmail.com>.
Hi,
Thanks for the pointers. I will try the span queries.
But can span query support regexp as a term ?
Also for more details in the problem :
The problem is like this:
find a search string inside a block of statements.
The block starts with a string and ends with a character.
-- Regards
Ba3
Erick Erickson wrote:
>
> I doubt you're thinking in terms of tokens. Your inputstream is broken up
> into tokens (think of them as words,
> depending upon the analyzer) and regex searchers are
> confined to those *tokens*. So the concept of a multi-line
> regex in a search is kind of ...odd...
>
> You could possibly index your input as UN_TOKENIZED, but
> I really have no clue what Lucene would do with that. I think
> you're off in uncharted territory here.
>
> Perhaps a better thing would be for you to explain *why* you
> want to do this and perhaps folks can come up with some
> suggestions, I suspect this may be an XY problem, see
> http://www.perlmonks.org/index.pl?node_id=542341
>
> Best
> Erick
>
> On Sun, Jul 26, 2009 at 9:52 AM, ba3 <sb...@gmail.com> wrote:
>
>>
>> I was trying to do a regex search with the lucene and
>> JavaUtilRegexCapabilities.
>> The code used is :
>> RegexQuery query = new RegexQuery(new
>> Term("contents","(?m)hello.*(\r[^#]*)This is to be
>> searched.*(\r[^#]*)#"));
>> query.setRegexImplementation(new JavaUtilRegexCapabilities());
>>
>> I verified the regex in : http://www.gskinner.com/RegExr/ [with the
>> multi
>> line checked]
>> In lucene though there are no hits. Can you please point me in the right
>> direction
>>
>> -- Rgds
>> Ba3
>>
>> Regex :
>> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
>>
>> Content :
>> hello world
>> This is to be searched
>> #
>> Test line should not be selected
>> hello
>> This should not work
>> some other lines
>> #
>> Not to be selected
>> hello world
>> Some lines
>> This is to be searched
>> Some lines
>> #
>> hello earth
>> some lines
>> #
>> --
>> View this message in context:
>> http://www.nabble.com/Multiline-Regex-with-Lucene-tp24667109p24667109.html
>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
--
View this message in context: http://www.nabble.com/Multiline-Regex-with-Lucene-tp24667109p24703547.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Multiline Regex with Lucene
Posted by Erick Erickson <er...@gmail.com>.
I doubt you're thinking in terms of tokens. Your inputstream is broken up
into tokens (think of them as words,
depending upon the analyzer) and regex searchers are
confined to those *tokens*. So the concept of a multi-line
regex in a search is kind of ...odd...
You could possibly index your input as UN_TOKENIZED, but
I really have no clue what Lucene would do with that. I think
you're off in uncharted territory here.
Perhaps a better thing would be for you to explain *why* you
want to do this and perhaps folks can come up with some
suggestions, I suspect this may be an XY problem, see
http://www.perlmonks.org/index.pl?node_id=542341
Best
Erick
On Sun, Jul 26, 2009 at 9:52 AM, ba3 <sb...@gmail.com> wrote:
>
> I was trying to do a regex search with the lucene and
> JavaUtilRegexCapabilities.
> The code used is :
> RegexQuery query = new RegexQuery(new
> Term("contents","(?m)hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#"));
> query.setRegexImplementation(new JavaUtilRegexCapabilities());
>
> I verified the regex in : http://www.gskinner.com/RegExr/ [with the multi
> line checked]
> In lucene though there are no hits. Can you please point me in the right
> direction
>
> -- Rgds
> Ba3
>
> Regex :
> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
>
> Content :
> hello world
> This is to be searched
> #
> Test line should not be selected
> hello
> This should not work
> some other lines
> #
> Not to be selected
> hello world
> Some lines
> This is to be searched
> Some lines
> #
> hello earth
> some lines
> #
> --
> View this message in context:
> http://www.nabble.com/Multiline-Regex-with-Lucene-tp24667109p24667109.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
RE: Multiline Regex with Lucene
Posted by Steven A Rowe <sa...@syr.edu>.
Hi ba3,
StandardAnalyzer breaks text into individual terms, removes most punctuation, downcases, removes stopwords, etc. Your example text becomes the following sequence of terms:
1. hello
2. world
3. searched (this,is,to,be are all in the default stopword set)
4. test ('#', like most other punctuation, is removed)
...
The regular expression you gave is matched against the text of each of the individual tokens produced by the Analyzer (first "hello", then "world", etc.), so you'll never get a match.
There are two routes I can think of that you could take:
1. Use KeywordAnalyzer instead of StandardAnalyzer when you index the contents field. That would place the entire text in a single field value, and would allow your regex to match the full text, rather than pieces of it. This is not scalable, though -- if your index size is non-trivial, I don't think you want to go this route.
2. Use SpanQuery's. See the nice article by Mark Miller on the subject over at Lucid Imagination: <http://www.lucidimagination.com/blog/2009/07/18/the-spanquery/>.
If you continue to use StandardAnalyzer, you may want to customize the stopword set it uses, if you have not already done so, so that you don't lose terms you want to match against.
Since in your example you're trying to match against punctuation ('#'), you may want to switch to a less-restrictive Analyzer, e.g. WhitespaceAnalyzer, which just breaks text at whitespace sequences, and doesn't modify the resulting terms any further.
Steve
> -----Original Message-----
> From: ba3 [mailto:sbadhrinath@gmail.com]
> Sent: Monday, July 27, 2009 9:29 PM
> To: java-user@lucene.apache.org
> Subject: RE: Multiline Regex with Lucene
>
>
> Hi Steve,
>
> I had used the standardanalyzer. Should a different one be used ?
>
> --
> Ba3
>
>
> Steven A Rowe wrote:
> >
> > Hi ba3,
> >
> > What analyzer did you use when indexing the content field?
> >
> > Steve
> >
> >> -----Original Message-----
> >> From: ba3
> >> Sent: Sunday, July 26, 2009 9:53 AM
> >> To: java-user@lucene.apache.org
> >> Subject: Multiline Regex with Lucene
> >>
> >>
> >> I was trying to do a regex search with the lucene and
> >> JavaUtilRegexCapabilities.
> >> The code used is :
> >> RegexQuery query = new RegexQuery(new
> >> Term("contents","(?m)hello.*(\r[^#]*)This is to be
> >> searched.*(\r[^#]*)#"));
> >> query.setRegexImplementation(new JavaUtilRegexCapabilities());
> >>
> >> I verified the regex in : http://www.gskinner.com/RegExr/ [with the
> >> multi line checked]
> >> In lucene though there are no hits. Can you please point me in the
> >> right direction
> >>
> >> -- Rgds
> >> Ba3
> >>
> >> Regex :
> >> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
> >>
> >> Content :
> >> hello world
> >> This is to be searched
> >> #
> >> Test line should not be selected
> >> hello
> >> This should not work
> >> some other lines
> >> #
> >> Not to be selected
> >> hello world
> >> Some lines
> >> This is to be searched
> >> Some lines
> >> #
> >> hello earth
> >> some lines
> >> #
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Multiline-Regex-with-
> Lucene-tp24667109p24691023.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Multiline Regex with Lucene
Posted by ba3 <sb...@gmail.com>.
Hi Steve,
I had used the standardanalyzer. Should a different one be used ?
--
Ba3
Steven A Rowe wrote:
>
> Hi ba3,
>
> What analyzer did you use when indexing the content field?
>
> Steve
>
>> -----Original Message-----
>> From: ba3
>> Sent: Sunday, July 26, 2009 9:53 AM
>> To: java-user@lucene.apache.org
>> Subject: Multiline Regex with Lucene
>>
>>
>> I was trying to do a regex search with the lucene and
>> JavaUtilRegexCapabilities.
>> The code used is :
>> RegexQuery query = new RegexQuery(new
>> Term("contents","(?m)hello.*(\r[^#]*)This is to be
>> searched.*(\r[^#]*)#"));
>> query.setRegexImplementation(new JavaUtilRegexCapabilities());
>>
>> I verified the regex in : http://www.gskinner.com/RegExr/ [with the
>> multi line checked]
>> In lucene though there are no hits. Can you please point me in the
>> right direction
>>
>> -- Rgds
>> Ba3
>>
>> Regex :
>> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
>>
>> Content :
>> hello world
>> This is to be searched
>> #
>> Test line should not be selected
>> hello
>> This should not work
>> some other lines
>> #
>> Not to be selected
>> hello world
>> Some lines
>> This is to be searched
>> Some lines
>> #
>> hello earth
>> some lines
>> #
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/Multiline-Regex-with-Lucene-tp24667109p24691023.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Multiline Regex with Lucene
Posted by Steven A Rowe <sa...@syr.edu>.
Hi ba3,
What analyzer did you use when indexing the content field?
Steve
> -----Original Message-----
> From: ba3 [mailto:sbadhrinath@gmail.com]
> Sent: Sunday, July 26, 2009 9:53 AM
> To: java-user@lucene.apache.org
> Subject: Multiline Regex with Lucene
>
>
> I was trying to do a regex search with the lucene and
> JavaUtilRegexCapabilities.
> The code used is :
> RegexQuery query = new RegexQuery(new
> Term("contents","(?m)hello.*(\r[^#]*)This is to be
> searched.*(\r[^#]*)#"));
> query.setRegexImplementation(new JavaUtilRegexCapabilities());
>
> I verified the regex in : http://www.gskinner.com/RegExr/ [with the
> multi line checked]
> In lucene though there are no hits. Can you please point me in the
> right direction
>
> -- Rgds
> Ba3
>
> Regex :
> hello.*(\r[^#]*)This is to be searched.*(\r[^#]*)#
>
> Content :
> hello world
> This is to be searched
> #
> Test line should not be selected
> hello
> This should not work
> some other lines
> #
> Not to be selected
> hello world
> Some lines
> This is to be searched
> Some lines
> #
> hello earth
> some lines
> #
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org