The new Contrib QueryParser should not be slated to replace the old one yet

[Mark Miller <ma...@gmail.com>]

I don't think we should stick with the current path of replacing the 
current QueryParser with the new contrib QueryParser in Lucene 3.0.

The new QueryParser has not been used much at all yet. Its interfaces 
(which will need to abide by back compat in core) have not been vetted 
enough.

The new parser appears to add complication to some of things that were 
very simple with the old parser.

The main benefits of the new parser are claimed to be the ability to 
plug and play many syntaxes and QueryBuilders. This is not an end user 
benefit though and I'm not even sure how much of a benefit it is to us. 
There is currently only one impl. It seems to me, once you start another 
impl, its a long shot that the exact same query tree representation is 
going to work with a completely different syntax. Sure, if you are just 
doing postfix rather than prefix, it will be fine – but the stuff that 
would likely be done – actual new syntaxes – are not likely to be very 
pluggable. If a syntax can map to the same query tree, I think we would 
likely stick to a single syntax – else suffer the confusion and 
maintenance headaches for syntactic sugar. More than a well factored 
QueryParser that can more easily allow different syntaxes to map to the 
same query tree representation, I think we just want a single solid 
syntax for core Lucene that supports Spans to some degree. We basically 
have that now, sans the spans support. Other, more exotic QueryParsers 
should live in contrib, as they do now.

Which isn't to say this QueryParser should not one day rule the roost – 
but I don't think its earned the right yet. And I don't think there is a 
hurry to toss the old parser.

Personally, I think that the old parser should not be deprecated. Lets 
let the new parser breath in contrib for a bit. Lets see if anyone 
actually adds any other syntaxes. Lets see if the pluggability results 
in any improvements. Lets see if some of the harder things to do 
(overriding query build methods?) become easier or keep people from 
using the new parser.

Lets just see if the new parser draws users without us forcing them to 
it. And lets also wait and see what other committers say – not many have 
gotten much time to deal with the new parser, or deal with user list 
questions on it.

I just think its premature to start moving people to this new parser. It 
didn't even really get in until right before release – the paint on the 
thing still reeks. There is no rush. I saw we undeprecate the current 
QueryParser and remove the wording in the new QueryParser about it 
replacing the new in 3.0. Later, if we think it should replace it (after 
having some experience to judge from), we can reinstate the current 
plan. Anyone agree?

-- 
- Mark

http://www.lucidimagination.com




---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Erik Hatcher <eh...@apache.org>]

Agreed, don't deprecate our beloved QueryParser.

	Erik

On Aug 11, 2009, at 1:54 PM, Mark Miller wrote:

> I don't think we should stick with the current path of replacing the  
> current QueryParser with the new contrib QueryParser in Lucene 3.0.
>
> The new QueryParser has not been used much at all yet. Its  
> interfaces (which will need to abide by back compat in core) have  
> not been vetted enough.
>
> The new parser appears to add complication to some of things that  
> were very simple with the old parser.
>
> The main benefits of the new parser are claimed to be the ability to  
> plug and play many syntaxes and QueryBuilders. This is not an end  
> user benefit though and I'm not even sure how much of a benefit it  
> is to us. There is currently only one impl. It seems to me, once you  
> start another impl, its a long shot that the exact same query tree  
> representation is going to work with a completely different syntax.  
> Sure, if you are just doing postfix rather than prefix, it will be  
> fine – but the stuff that would likely be done – actual new syntaxes  
> – are not likely to be very pluggable. If a syntax can map to the  
> same query tree, I think we would likely stick to a single syntax –  
> else suffer the confusion and maintenance headaches for syntactic  
> sugar. More than a well factored QueryParser that can more easily  
> allow different syntaxes to map to the same query tree  
> representation, I think we just want a single solid syntax for core  
> Lucene that supports Spans to some degree. We basically have that  
> now, sans the spans support. Other, more exotic QueryParsers should  
> live in contrib, as they do now.
>
> Which isn't to say this QueryParser should not one day rule the  
> roost – but I don't think its earned the right yet. And I don't  
> think there is a hurry to toss the old parser.
>
> Personally, I think that the old parser should not be deprecated.  
> Lets let the new parser breath in contrib for a bit. Lets see if  
> anyone actually adds any other syntaxes. Lets see if the  
> pluggability results in any improvements. Lets see if some of the  
> harder things to do (overriding query build methods?) become easier  
> or keep people from using the new parser.
>
> Lets just see if the new parser draws users without us forcing them  
> to it. And lets also wait and see what other committers say – not  
> many have gotten much time to deal with the new parser, or deal with  
> user list questions on it.
>
> I just think its premature to start moving people to this new  
> parser. It didn't even really get in until right before release –  
> the paint on the thing still reeks. There is no rush. I saw we  
> undeprecate the current QueryParser and remove the wording in the  
> new QueryParser about it replacing the new in 3.0. Later, if we  
> think it should replace it (after having some experience to judge  
> from), we can reinstate the current plan. Anyone agree?
>
> -- 
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Mark Miller <ma...@gmail.com>]

Michael Busch wrote:
>
> We should also realize that - thanks to Luis and Adriano - we now have 
> actual code that can be the basis of discussions and that we can take 
> and improve. No matter if this new QP is going to replace the old one 
> or not, I'm very thankful that the two went through the effort of 
> creating it. This framework has been very successful internally and we 
> wanted to share something good with the Lucene community.
>
>  Michael
>
Agreed! I'd also like to extend my thanks to Luis and Adriano! And to 
IBM for donating the code! I am certainly not looking a gift horse in 
the mouth.

And I think its still very likely this parser will replace the old. 
Despite my rant to not deprecate the current QP yet, I do think its a 
nice design, and I do think it has a lot of value going forward. I just 
think its a big enough deal that we should let it sit for a release in 
contrib while everyone has a chance to take stalk of it. If you look at 
the Qsol parser I used to play around with, it also has an intermediate 
abstract query tree (its just way uglier and less plugable - dont go 
look ;) ) - I think that makes a lot of sense, and I do know that it 
will bring benefits to many developers in the future.

I just think it makes sense to wait a bit and see how things shake out. 
I think that will give us more freedom in terms of addressing the 
shortcomings we/users may find.

Giving users such a large framework to extends is a back compat 
nightmare ! Lets just see how things go for a bit before we start really 
locking in back compat on the thing (which we should presumably do if it 
were to become the new QP).

-- 
- Mark

http://www.lucidimagination.com




---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Michael Busch <bu...@gmail.com>]

Thanks, Jason! Glad the new QP is useful for you.

I'd like to explain a bit the (IBM internal) history of this new QP: A 
few years ago we wanted to change/extend Lucene's query syntax. We did 
similar things as you mention, like no stemming for quoted terms, 
additional syntax features, etc. Very soon we found out that it wasn't 
possible to extend the current QP without changing the javacc code. We 
maintained our own copy of the QP for a long time, which had several 
thousand lines of code. Fixes in Lucene's QP were hard to merge into our 
copy sometimes. Also, everyone who worked on it had to have javacc skills.

After managing our own copy for almost two years we decided to develop a 
new QP framework that allows to change/extend different parts of the QP 
individually. We had the goal of keeping the javacc part as small as 
possible and separated from the rest. That's the reason for the first 
layer of the new QP (SyntaxParser).

The second goal was to isolate building Query objects. This gives us the 
flexibility of quickly changing which Query objects to instantiate 
(QueryBuilders). Only the QueryBuilders need to have dependencies on 
Lucene. The result is that we can share most of the QP code with another 
team internally, that doesn't even use Lucene. They can simply switch 
the QueryBuilder layer to build their own Query objects and can use most 
of the other parts.

That leaves semantic and linguistic features to the middle layer: 
QueryProcessors. We have several of them, they do the dirty work, and we 
share them internally across teams as well.

So IBM internally sharing the QP code was successful so far - three 
separate products are using it. That was the reason we thought that it 
would be useful for Lucene as well. There are currently different QP 
implementations in contrib. We thought it would be nice to switch them 
all to the same framework and share as much code as possible between 
them. This has the big advantage of increasing maintainability. It will 
also be much easier to handle query syntax backwards-compatibility. E.g. 
if we want to change the RangeQuery syntax from [], {} to <=, >= (a 
change Lucene users asked for), we could simply create a new 
SyntaxParser implementation. Users would have to change one line of code 
to switch to the new syntax (instantiating the new SyntaxParser). Other 
users, who need to keep the old syntax, or maybe have a lot of saved 
queries with unescaped '<' and '=' characters, could keep the old 
SyntaxParser. How would you do that with the current QP without copying 
the entire thing? How would we maintain the copies?

Of course it would have been preferable to develop the whole thing 
transparently in public. However, it takes time to get approvals to open 
source code, so we decided to continue with the implementation in 
parallel to save time. We thought the easiest way to help users 
switching over to the new QP would be to create an implementation that 
behaves 100% like the current QP. So Luis and Adriano worked hard on 
creating a new, 100% Lucene-compatible implementation, with a wrapper 
class that allows using the new QP exactly like the old one and even 
running all old unit tests.

The major concern about the new QP now is its complexity. I don't 
disagree: the learning curve of a component that has dozens of classes 
is higher compared to a single class. Luis and Adriano did the 
implementation in this very structured way based on the experience they 
gathered internally in the past. Of course we could have implemented 
everything less generic in fewer classes. Maybe that is better for 
Lucene. We can still change and improve that - and those are the 
discussions we have to have now. Discuss which abstraction make sense, 
where we can condense the code. For these discussions it would be 
helpful to have an understanding of what the difference between core 
classes of the framework (not very many!) and the Lucene compatibility 
implementation are.

We should also realize that - thanks to Luis and Adriano - we now have 
actual code that can be the basis of discussions and that we can take 
and improve. No matter if this new QP is going to replace the old one or 
not, I'm very thankful that the two went through the effort of creating 
it. This framework has been very successful internally and we wanted to 
share something good with the Lucene community.

  Michael

On 8/11/09 10:44 PM, Jason Rutherglen wrote:
> I'm starting to use the new parser to emulate Google's queries
> (i.e. a phrase query with a single term means no-stemming,
> something the current QP doesn't allow because it converts the
> quoted query into a term query inside the JavaCC portion). It's
> been very straightforward and logical to use (so far).
>
> Thanks to the contrib query parser team!
>
> On Tue, Aug 11, 2009 at 10:54 AM, Mark Miller<ma...@gmail.com>  wrote:
>    
>> I don't think we should stick with the current path of replacing the current
>> QueryParser with the new contrib QueryParser in Lucene 3.0.
>>
>> The new QueryParser has not been used much at all yet. Its interfaces (which
>> will need to abide by back compat in core) have not been vetted enough.
>>
>> The new parser appears to add complication to some of things that were very
>> simple with the old parser.
>>
>> The main benefits of the new parser are claimed to be the ability to plug
>> and play many syntaxes and QueryBuilders. This is not an end user benefit
>> though and I'm not even sure how much of a benefit it is to us. There is
>> currently only one impl. It seems to me, once you start another impl, its a
>> long shot that the exact same query tree representation is going to work
>> with a completely different syntax. Sure, if you are just doing postfix
>> rather than prefix, it will be fine – but the stuff that would likely be
>> done – actual new syntaxes – are not likely to be very pluggable. If a
>> syntax can map to the same query tree, I think we would likely stick to a
>> single syntax – else suffer the confusion and maintenance headaches for
>> syntactic sugar. More than a well factored QueryParser that can more easily
>> allow different syntaxes to map to the same query tree representation, I
>> think we just want a single solid syntax for core Lucene that supports Spans
>> to some degree. We basically have that now, sans the spans support. Other,
>> more exotic QueryParsers should live in contrib, as they do now.
>>
>> Which isn't to say this QueryParser should not one day rule the roost – but
>> I don't think its earned the right yet. And I don't think there is a hurry
>> to toss the old parser.
>>
>> Personally, I think that the old parser should not be deprecated. Lets let
>> the new parser breath in contrib for a bit. Lets see if anyone actually adds
>> any other syntaxes. Lets see if the pluggability results in any
>> improvements. Lets see if some of the harder things to do (overriding query
>> build methods?) become easier or keep people from using the new parser.
>>
>> Lets just see if the new parser draws users without us forcing them to it.
>> And lets also wait and see what other committers say – not many have gotten
>> much time to deal with the new parser, or deal with user list questions on
>> it.
>>
>> I just think its premature to start moving people to this new parser. It
>> didn't even really get in until right before release – the paint on the
>> thing still reeks. There is no rush. I saw we undeprecate the current
>> QueryParser and remove the wording in the new QueryParser about it replacing
>> the new in 3.0. Later, if we think it should replace it (after having some
>> experience to judge from), we can reinstate the current plan. Anyone agree?
>>
>> --
>> - Mark
>>
>> http://www.lucidimagination.com
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>>      
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
>    


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Michael Busch <bu...@gmail.com>]

I think opaque terms is a good and useful feature and we have discussed 
that several times and experimentally implemented in the past.

However I think that should be separate discussion/feature request. It 
solves a different problem.

  Michael

On 8/12/09 1:51 AM, Shai Erera wrote:
>
>     Is there any example when you cannot use the processing phase for
>     that?
>
>
> I actually meant that w/ the old QP I can also do it, by extending 
> QueryParser and overriding "newWildcardQuery(Term)". I'm sure this can 
> be done w/ the new QP as well. I just gave an example to something the 
> new QP does not allow me to do more easily.
>
> About the opaque clauses and '@' - usually I'd think it's not the user 
> who writes such queries, but the application developer. Therefore the 
> '@' does not really matter.
>
> Without opaque clauses, if I want to add some ability, like Spatial 
> search together w/ the other Lucene syntax, I will have a problem. I 
> will need to copy the SyntaxParser and add Spatial syntax to it. And 
> with that I lose whatever improvements that will be done on the 
> default SyntaxParser.
>
> We can not have the '@', but field::'some query', i.e., double colon 
> (::) and query string surrounded w/ '. Maybe that will look more 
> native to the user. We can perhaps have one colon (:) and ' to 
> surround the query and change the field handling to recognize this is 
> an opaque field (because of the '), but I don't know if this breaks 
> the current syntax/parser.
>
> Shai
>
> On Wed, Aug 12, 2009 at 11:08 AM, Adriano Crestani 
> <adrianocrestani@gmail.com <ma...@gmail.com>> wrote:
>
>     If I want to control how Wildcard clauses are handled, I can do it
>     w/ today's QP as well, just extend it and override the appropriate
>     getter method.
>
>     The SyntaxParser can produce WildcardQueryNode object which can
>     further be processed on the processing phase. Is there any example
>     when you cannot use the processing phase for that?
>
>
>     In conclusion, I think that if we want to have a truly extensible
>     QP, we should start w/ the query syntax first, and my proposal are
>     those opaque terms.
>
>     Agree, I also think we need to improve a lot the syntax parsing
>     phase. It's really simple and not extensible yet. Opaque terms are
>     interesting, I just don't think users will like to type '@' before
>     the field names, actually the user has no idea why he's typing
>     that @, so there is no need for that. I think we could do a
>     mapping from field name to parser directly. Anyway, this approach
>     would only work for field:term syntaxes, any other different
>     syntax, like xml syntax, will need a different approach. I cannot
>     think about a generic API yet for this approach, any suggestions?
>
>
>     On Wed, Aug 12, 2009 at 12:54 AM, Shai Erera <serera@gmail.com
>     <ma...@gmail.com>> wrote:
>
>         If I want to control how Wildcard clauses are handled, I can
>         do it w/ today's QP as well, just extend it and override the
>         appropriate getter method.
>
>
>

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Luis Alves <la...@gmail.com>]

I like the idea of having syntax extensions,
and proposed something similar for "ComplexPhraseQueryParser" JIRA issue.

But we should create a new jira issue for this using the new queryparser,
this is a new feature an is not backward compatible with the current 
lucene syntax.

For the syntax I would like to propose something similar to calling 
functions
::complexPhrase('other syntax')

but I also like
complePhrase:'other syntax'

and in case the user configure just one extension he could use just
::('other syntax')
or
'other syntax'


Adriano Crestani wrote:
> We can perhaps have one colon (:) and ' to surround the query and 
> change the field handling to recognize this is an opaque field 
> (because of the '), but I don't know if this breaks the current 
> syntax/parser.
>
> I think this way is cleaner :)
>
> On Wed, Aug 12, 2009 at 1:51 AM, Shai Erera <serera@gmail.com 
> <ma...@gmail.com>> wrote:
>
>     We can perhaps have one colon (:) and ' to surround the query and
>     change the field handling to recognize this is an opaque field
>     (because of the '), but I don't know if this breaks the current
>     syntax/parser.
>
>


-- 
-Lafa



---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Adriano Crestani <ad...@gmail.com>]

We can perhaps have one colon (:) and ' to surround the query and change the
field handling to recognize this is an opaque field (because of the '), but
I don't know if this breaks the current syntax/parser.

I think this way is cleaner :)

On Wed, Aug 12, 2009 at 1:51 AM, Shai Erera <se...@gmail.com> wrote:

> We can perhaps have one colon (:) and ' to surround the query and change
> the field handling to recognize this is an opaque field (because of the '),
> but I don't know if this breaks the current syntax/parser.
>

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Shai Erera <se...@gmail.com>]

>
> Is there any example when you cannot use the processing phase for that?
>

I actually meant that w/ the old QP I can also do it, by extending
QueryParser and overriding "newWildcardQuery(Term)". I'm sure this can be
done w/ the new QP as well. I just gave an example to something the new QP
does not allow me to do more easily.

About the opaque clauses and '@' - usually I'd think it's not the user who
writes such queries, but the application developer. Therefore the '@' does
not really matter.

Without opaque clauses, if I want to add some ability, like Spatial search
together w/ the other Lucene syntax, I will have a problem. I will need to
copy the SyntaxParser and add Spatial syntax to it. And with that I lose
whatever improvements that will be done on the default SyntaxParser.

We can not have the '@', but field::'some query', i.e., double colon (::)
and query string surrounded w/ '. Maybe that will look more native to the
user. We can perhaps have one colon (:) and ' to surround the query and
change the field handling to recognize this is an opaque field (because of
the '), but I don't know if this breaks the current syntax/parser.

Shai

On Wed, Aug 12, 2009 at 11:08 AM, Adriano Crestani <
adrianocrestani@gmail.com> wrote:

> If I want to control how Wildcard clauses are handled, I can do it w/
> today's QP as well, just extend it and override the appropriate getter
> method.
>
> The SyntaxParser can produce WildcardQueryNode object which can further be
> processed on the processing phase. Is there any example when you cannot use
> the processing phase for that?
>
> In conclusion, I think that if we want to have a truly extensible QP, we
> should start w/ the query syntax first, and my proposal are those opaque
> terms.
>
> Agree, I also think we need to improve a lot the syntax parsing phase. It's
> really simple and not extensible yet. Opaque terms are interesting, I just
> don't think users will like to type '@' before the field names, actually the
> user has no idea why he's typing that @, so there is no need for that. I
> think we could do a mapping from field name to parser directly. Anyway, this
> approach would only work for field:term syntaxes, any other different
> syntax, like xml syntax, will need a different approach. I cannot think
> about a generic API yet for this approach, any suggestions?
>
>
> On Wed, Aug 12, 2009 at 12:54 AM, Shai Erera <se...@gmail.com> wrote:
>
>> If I want to control how Wildcard clauses are handled, I can do it w/
>> today's QP as well, just extend it and override the appropriate getter
>> method.
>>
>
>

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Adriano Crestani <ad...@gmail.com>]

If I want to control how Wildcard clauses are handled, I can do it w/
today's QP as well, just extend it and override the appropriate getter
method.

The SyntaxParser can produce WildcardQueryNode object which can further be
processed on the processing phase. Is there any example when you cannot use
the processing phase for that?

In conclusion, I think that if we want to have a truly extensible QP, we
should start w/ the query syntax first, and my proposal are those opaque
terms.

Agree, I also think we need to improve a lot the syntax parsing phase. It's
really simple and not extensible yet. Opaque terms are interesting, I just
don't think users will like to type '@' before the field names, actually the
user has no idea why he's typing that @, so there is no need for that. I
think we could do a mapping from field name to parser directly. Anyway, this
approach would only work for field:term syntaxes, any other different
syntax, like xml syntax, will need a different approach. I cannot think
about a generic API yet for this approach, any suggestions?

On Wed, Aug 12, 2009 at 12:54 AM, Shai Erera <se...@gmail.com> wrote:

> If I want to control how Wildcard clauses are handled, I can do it w/
> today's QP as well, just extend it and override the appropriate getter
> method.
>

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Shai Erera <se...@gmail.com>]

Michael, I wrote the above reply before I noticed you already replied.
Thanks for the explanation.

I guess that the way I see it, being able to extend a SyntaxParser is more
important than building my final Query object. If I want to enhance the
query syntax by replacing [] {} w/ <= and >=. How do I do that? I should
still copy the entire SyntaxParser logic, and modify these two places,
right?

By copying, I also copy existing bugs. If say someone fixes a bug which is
not related at all to the change above, how do I merge it? By default I
won't inherit it, and I'll need to manually apply the patch to my now
private version of SyntaxParser, even though 90% of it is still the original
Lucene parser.

And if someone augments the current syntax or parser w/ better parsing of
boolean queries. How do I take advantage of that?

W/ the opaque terms above, I should not have any problem w/ that. If I want
to have a Spatial syntax, I write a Spatial parser. If I want to have a
different range queries syntax, I write my own Range parser. If someone
fixes a bug in core parser, or improves how other sections of the syntax are
handled, I get those for free, because I never touch the parser's logic.

The new QP, AFAIU, does not help me in this case. All it gives me are some
helper classes (Builders, Processors maybe). But my core problem is the
syntax. If I want to control how Wildcard clauses are handled, I can do it
w/ today's QP as well, just extend it and override the appropriate getter
method.

In conclusion, I think that if we want to have a truly extensible QP, we
should start w/ the query syntax first, and my proposal are those opaque
terms. Then, we can have a discussion about whether the new QP allows us to
support it more easily or not. If say we have an interface QueryParser w/ a
single parse(String) method which returns a Query object. Do we really care
how this QP was written? Whether it uses the new QP framework or something
else?

Shai

On Wed, Aug 12, 2009 at 10:43 AM, Shai Erera <se...@gmail.com> wrote:

> With the new QP we can build out a syntax that's compatible with
>> GData and be able to embed location/spatial queries directly
>> into the query string. (i.e. @+40.75-074.00 + 5mi)
>>
>
> What do you mean "with the new QP"? What prevents you from doing that w/o
> the new QP, as in writing your own QP? What are the benefits the new QP has
> when you come to deal w/ such terms? Unless you're talking about extending
> the Lucene syntax w/ spatial clauses. Just for my education, how do you
> extend the new QP w/ this information? Can you extend the Tokenizer, or do
> you need to write a new one?
>
> I'm trying to separate between the query syntax and a QP. The new QP is
> more of a framework for how to parse queries. It's well architected and
> designed. It allows to build different QPs for different syntaxes easily.
>
> As for the query syntax, what if we had augmented Lucene query syntax w/
> opaque clauses support. Something like @qpname::'query string'. Then, we can
> add to a QP a QP mapping from qpname to QP instance. That would allow anyone
> to use Lucene's QP and write new QPs (however they want) to match different
> opaque clauses.
>
> For the example above, I could write this query: "restaurants @spatial::'@+40.75-074.00
> + 5mi' " (quotes are not part of the query string) and instantiate the QP as
> follows:
> QueryParser qp = new QueryParser();
> qp.addQueryParser("spatial", new SpatialQueryParser());
> qp.parse(queryString);
>
> Upon parsing, the default QP would hit the opaque clause and defer parsing
> of the text in between ' to SpatialQueryParser. We'd need to come up w/ a
> simple QP interface, with a parse() method or something that it can call.
> Nothing too fancy.
>
> SpatialQueryParser could be implemented however we choose. Not necessarily
> using the new QP framework.
>
> Maybe we should add this to Lucene anyway, and the new QP would just make
> the implementations easier.
>
> BTW, in case I managed to make a wrong impression - I'm not against the new
> QP :).
>
> Shai
>
>
> On Wed, Aug 12, 2009 at 8:53 AM, Jason Rutherglen <
> jason.rutherglen@gmail.com> wrote:
>
>> With the new QP we can build out a syntax that's compatible with
>> GData and be able to embed location/spatial queries directly
>> into the query string. (i.e. @+40.75-074.00 + 5mi)
>>
>> SQL like range queries (i.e. [megapixel >= 3.0])
>>
>> On Tue, Aug 11, 2009 at 10:44 PM, Jason
>> Rutherglen<ja...@gmail.com> wrote:
>> > I'm starting to use the new parser to emulate Google's queries
>> > (i.e. a phrase query with a single term means no-stemming,
>> > something the current QP doesn't allow because it converts the
>> > quoted query into a term query inside the JavaCC portion). It's
>> > been very straightforward and logical to use (so far).
>> >
>> > Thanks to the contrib query parser team!
>> >
>> > On Tue, Aug 11, 2009 at 10:54 AM, Mark Miller<ma...@gmail.com>
>> wrote:
>> >> I don't think we should stick with the current path of replacing the
>> current
>> >> QueryParser with the new contrib QueryParser in Lucene 3.0.
>> >>
>> >> The new QueryParser has not been used much at all yet. Its interfaces
>> (which
>> >> will need to abide by back compat in core) have not been vetted enough.
>> >>
>> >> The new parser appears to add complication to some of things that were
>> very
>> >> simple with the old parser.
>> >>
>> >> The main benefits of the new parser are claimed to be the ability to
>> plug
>> >> and play many syntaxes and QueryBuilders. This is not an end user
>> benefit
>> >> though and I'm not even sure how much of a benefit it is to us. There
>> is
>> >> currently only one impl. It seems to me, once you start another impl,
>> its a
>> >> long shot that the exact same query tree representation is going to
>> work
>> >> with a completely different syntax. Sure, if you are just doing postfix
>> >> rather than prefix, it will be fine – but the stuff that would likely
>> be
>> >> done – actual new syntaxes – are not likely to be very pluggable. If a
>> >> syntax can map to the same query tree, I think we would likely stick to
>> a
>> >> single syntax – else suffer the confusion and maintenance headaches for
>> >> syntactic sugar. More than a well factored QueryParser that can more
>> easily
>> >> allow different syntaxes to map to the same query tree representation,
>> I
>> >> think we just want a single solid syntax for core Lucene that supports
>> Spans
>> >> to some degree. We basically have that now, sans the spans support.
>> Other,
>> >> more exotic QueryParsers should live in contrib, as they do now.
>> >>
>> >> Which isn't to say this QueryParser should not one day rule the roost –
>> but
>> >> I don't think its earned the right yet. And I don't think there is a
>> hurry
>> >> to toss the old parser.
>> >>
>> >> Personally, I think that the old parser should not be deprecated. Lets
>> let
>> >> the new parser breath in contrib for a bit. Lets see if anyone actually
>> adds
>> >> any other syntaxes. Lets see if the pluggability results in any
>> >> improvements. Lets see if some of the harder things to do (overriding
>> query
>> >> build methods?) become easier or keep people from using the new parser.
>> >>
>> >> Lets just see if the new parser draws users without us forcing them to
>> it.
>> >> And lets also wait and see what other committers say – not many have
>> gotten
>> >> much time to deal with the new parser, or deal with user list questions
>> on
>> >> it.
>> >>
>> >> I just think its premature to start moving people to this new parser.
>> It
>> >> didn't even really get in until right before release – the paint on the
>> >> thing still reeks. There is no rush. I saw we undeprecate the current
>> >> QueryParser and remove the wording in the new QueryParser about it
>> replacing
>> >> the new in 3.0. Later, if we think it should replace it (after having
>> some
>> >> experience to judge from), we can reinstate the current plan. Anyone
>> agree?
>> >>
>> >> --
>> >> - Mark
>> >>
>> >> http://www.lucidimagination.com
>> >>
>> >>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: java-dev-help@lucene.apache.org
>> >>
>> >>
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Shai Erera <se...@gmail.com>]

>
> With the new QP we can build out a syntax that's compatible with
> GData and be able to embed location/spatial queries directly
> into the query string. (i.e. @+40.75-074.00 + 5mi)
>

What do you mean "with the new QP"? What prevents you from doing that w/o
the new QP, as in writing your own QP? What are the benefits the new QP has
when you come to deal w/ such terms? Unless you're talking about extending
the Lucene syntax w/ spatial clauses. Just for my education, how do you
extend the new QP w/ this information? Can you extend the Tokenizer, or do
you need to write a new one?

I'm trying to separate between the query syntax and a QP. The new QP is more
of a framework for how to parse queries. It's well architected and designed.
It allows to build different QPs for different syntaxes easily.

As for the query syntax, what if we had augmented Lucene query syntax w/
opaque clauses support. Something like @qpname::'query string'. Then, we can
add to a QP a QP mapping from qpname to QP instance. That would allow anyone
to use Lucene's QP and write new QPs (however they want) to match different
opaque clauses.

For the example above, I could write this query: "restaurants
@spatial::'@+40.75-074.00
+ 5mi' " (quotes are not part of the query string) and instantiate the QP as
follows:
QueryParser qp = new QueryParser();
qp.addQueryParser("spatial", new SpatialQueryParser());
qp.parse(queryString);

Upon parsing, the default QP would hit the opaque clause and defer parsing
of the text in between ' to SpatialQueryParser. We'd need to come up w/ a
simple QP interface, with a parse() method or something that it can call.
Nothing too fancy.

SpatialQueryParser could be implemented however we choose. Not necessarily
using the new QP framework.

Maybe we should add this to Lucene anyway, and the new QP would just make
the implementations easier.

BTW, in case I managed to make a wrong impression - I'm not against the new
QP :).

Shai

On Wed, Aug 12, 2009 at 8:53 AM, Jason Rutherglen <
jason.rutherglen@gmail.com> wrote:

> With the new QP we can build out a syntax that's compatible with
> GData and be able to embed location/spatial queries directly
> into the query string. (i.e. @+40.75-074.00 + 5mi)
>
> SQL like range queries (i.e. [megapixel >= 3.0])
>
> On Tue, Aug 11, 2009 at 10:44 PM, Jason
> Rutherglen<ja...@gmail.com> wrote:
> > I'm starting to use the new parser to emulate Google's queries
> > (i.e. a phrase query with a single term means no-stemming,
> > something the current QP doesn't allow because it converts the
> > quoted query into a term query inside the JavaCC portion). It's
> > been very straightforward and logical to use (so far).
> >
> > Thanks to the contrib query parser team!
> >
> > On Tue, Aug 11, 2009 at 10:54 AM, Mark Miller<ma...@gmail.com>
> wrote:
> >> I don't think we should stick with the current path of replacing the
> current
> >> QueryParser with the new contrib QueryParser in Lucene 3.0.
> >>
> >> The new QueryParser has not been used much at all yet. Its interfaces
> (which
> >> will need to abide by back compat in core) have not been vetted enough.
> >>
> >> The new parser appears to add complication to some of things that were
> very
> >> simple with the old parser.
> >>
> >> The main benefits of the new parser are claimed to be the ability to
> plug
> >> and play many syntaxes and QueryBuilders. This is not an end user
> benefit
> >> though and I'm not even sure how much of a benefit it is to us. There is
> >> currently only one impl. It seems to me, once you start another impl,
> its a
> >> long shot that the exact same query tree representation is going to work
> >> with a completely different syntax. Sure, if you are just doing postfix
> >> rather than prefix, it will be fine – but the stuff that would likely be
> >> done – actual new syntaxes – are not likely to be very pluggable. If a
> >> syntax can map to the same query tree, I think we would likely stick to
> a
> >> single syntax – else suffer the confusion and maintenance headaches for
> >> syntactic sugar. More than a well factored QueryParser that can more
> easily
> >> allow different syntaxes to map to the same query tree representation, I
> >> think we just want a single solid syntax for core Lucene that supports
> Spans
> >> to some degree. We basically have that now, sans the spans support.
> Other,
> >> more exotic QueryParsers should live in contrib, as they do now.
> >>
> >> Which isn't to say this QueryParser should not one day rule the roost –
> but
> >> I don't think its earned the right yet. And I don't think there is a
> hurry
> >> to toss the old parser.
> >>
> >> Personally, I think that the old parser should not be deprecated. Lets
> let
> >> the new parser breath in contrib for a bit. Lets see if anyone actually
> adds
> >> any other syntaxes. Lets see if the pluggability results in any
> >> improvements. Lets see if some of the harder things to do (overriding
> query
> >> build methods?) become easier or keep people from using the new parser.
> >>
> >> Lets just see if the new parser draws users without us forcing them to
> it.
> >> And lets also wait and see what other committers say – not many have
> gotten
> >> much time to deal with the new parser, or deal with user list questions
> on
> >> it.
> >>
> >> I just think its premature to start moving people to this new parser. It
> >> didn't even really get in until right before release – the paint on the
> >> thing still reeks. There is no rush. I saw we undeprecate the current
> >> QueryParser and remove the wording in the new QueryParser about it
> replacing
> >> the new in 3.0. Later, if we think it should replace it (after having
> some
> >> experience to judge from), we can reinstate the current plan. Anyone
> agree?
> >>
> >> --
> >> - Mark
> >>
> >> http://www.lucidimagination.com
> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-dev-help@lucene.apache.org
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Adriano Crestani <ad...@gmail.com>]

Some comments in line:

The new QueryParser has not been used much at all yet. Its interfaces (which
will need to abide by back compat in core) have not been vetted enough.

Agreed, I still think some points must still be discussed about the API, and
to start discussing about it, the contributors must have a deeper understand
about the main points of the new QP, so further we can discuss what must be
adjusted. So, I think it should stay longer on contrib as experimental and
as people start (and they already started [?] ) using it we will have a
feedback from them.

The new parser appears to add complication to some of things that were very
simple with the old parser.

Some things could be easily done with the old QP, because it was designed
specificily for that. For example, you can easily change how RangeQuery
objects are created, because the old QP provides a method for that.
Otherwise, it becomes very difficult to maintain, add extra processing, plug
and unplug functionality, separate syntax from semantic. On the company I
work for, we used to use the old one, just extending it. There was always a
new requirement and the code had to be changed, there was IF statements and
control variables all over the place, I wonder if other companies are
running into the same problems using the old QP. My conclusion here is:
things that are simple are usually not very powerfull, flexible and
mantainable.

This new QP that was contributed to Lucene can be split in 2 parts: core and
lucene QP implementation.

The core contains the QP framework classes, the framework tries to define
the best way to implement and QP, so it's easily mantainable and
extensible/flexible if you follow the rules. Of course the user can just
ignore what the framework suggest, like to do processing at buiding time, or
parsing at processing time and etc, it does not prohibit that. Maybe the
framework classes are not finished yet, they might require more work to get
it in a better shape, at the same time I think it's ready for a release,
since you can already write a complete QP with it. I would like to suggest
that the core classes to be included in 3.0 lucene core.

The lucene QP implementation is the old QP implementation applied against
the new framework rules. The syntax and semantic were separated and now on
any new functionality or change can be easily perfomed. OK, maybe this new
implementation should be in contrib and whether the user thinks it's better
than the old one.

On Tue, Aug 11, 2009 at 10:53 PM, Jason Rutherglen <
jason.rutherglen@gmail.com> wrote:

> With the new QP we can build out a syntax that's compatible with
> GData and be able to embed location/spatial queries directly
> into the query string. (i.e. @+40.75-074.00 + 5mi)
>
> SQL like range queries (i.e. [megapixel >= 3.0])
>
> On Tue, Aug 11, 2009 at 10:44 PM, Jason
> Rutherglen<ja...@gmail.com> wrote:
> > I'm starting to use the new parser to emulate Google's queries
> > (i.e. a phrase query with a single term means no-stemming,
> > something the current QP doesn't allow because it converts the
> > quoted query into a term query inside the JavaCC portion). It's
> > been very straightforward and logical to use (so far).
> >
> > Thanks to the contrib query parser team!
> >
> > On Tue, Aug 11, 2009 at 10:54 AM, Mark Miller<ma...@gmail.com>
> wrote:
> >> I don't think we should stick with the current path of replacing the
> current
> >> QueryParser with the new contrib QueryParser in Lucene 3.0.
> >>
> >> The new QueryParser has not been used much at all yet. Its interfaces
> (which
> >> will need to abide by back compat in core) have not been vetted enough.
> >>
> >> The new parser appears to add complication to some of things that were
> very
> >> simple with the old parser.
> >>
> >> The main benefits of the new parser are claimed to be the ability to
> plug
> >> and play many syntaxes and QueryBuilders. This is not an end user
> benefit
> >> though and I'm not even sure how much of a benefit it is to us. There is
> >> currently only one impl. It seems to me, once you start another impl,
> its a
> >> long shot that the exact same query tree representation is going to work
> >> with a completely different syntax. Sure, if you are just doing postfix
> >> rather than prefix, it will be fine – but the stuff that would likely be
> >> done – actual new syntaxes – are not likely to be very pluggable. If a
> >> syntax can map to the same query tree, I think we would likely stick to
> a
> >> single syntax – else suffer the confusion and maintenance headaches for
> >> syntactic sugar. More than a well factored QueryParser that can more
> easily
> >> allow different syntaxes to map to the same query tree representation, I
> >> think we just want a single solid syntax for core Lucene that supports
> Spans
> >> to some degree. We basically have that now, sans the spans support.
> Other,
> >> more exotic QueryParsers should live in contrib, as they do now.
> >>
> >> Which isn't to say this QueryParser should not one day rule the roost –
> but
> >> I don't think its earned the right yet. And I don't think there is a
> hurry
> >> to toss the old parser.
> >>
> >> Personally, I think that the old parser should not be deprecated. Lets
> let
> >> the new parser breath in contrib for a bit. Lets see if anyone actually
> adds
> >> any other syntaxes. Lets see if the pluggability results in any
> >> improvements. Lets see if some of the harder things to do (overriding
> query
> >> build methods?) become easier or keep people from using the new parser.
> >>
> >> Lets just see if the new parser draws users without us forcing them to
> it.
> >> And lets also wait and see what other committers say – not many have
> gotten
> >> much time to deal with the new parser, or deal with user list questions
> on
> >> it.
> >>
> >> I just think its premature to start moving people to this new parser. It
> >> didn't even really get in until right before release – the paint on the
> >> thing still reeks. There is no rush. I saw we undeprecate the current
> >> QueryParser and remove the wording in the new QueryParser about it
> replacing
> >> the new in 3.0. Later, if we think it should replace it (after having
> some
> >> experience to judge from), we can reinstate the current plan. Anyone
> agree?
> >>
> >> --
> >> - Mark
> >>
> >> http://www.lucidimagination.com
> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-dev-help@lucene.apache.org
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Jason Rutherglen <ja...@gmail.com>]

With the new QP we can build out a syntax that's compatible with
GData and be able to embed location/spatial queries directly
into the query string. (i.e. @+40.75-074.00 + 5mi)

SQL like range queries (i.e. [megapixel >= 3.0])

On Tue, Aug 11, 2009 at 10:44 PM, Jason
Rutherglen<ja...@gmail.com> wrote:
> I'm starting to use the new parser to emulate Google's queries
> (i.e. a phrase query with a single term means no-stemming,
> something the current QP doesn't allow because it converts the
> quoted query into a term query inside the JavaCC portion). It's
> been very straightforward and logical to use (so far).
>
> Thanks to the contrib query parser team!
>
> On Tue, Aug 11, 2009 at 10:54 AM, Mark Miller<ma...@gmail.com> wrote:
>> I don't think we should stick with the current path of replacing the current
>> QueryParser with the new contrib QueryParser in Lucene 3.0.
>>
>> The new QueryParser has not been used much at all yet. Its interfaces (which
>> will need to abide by back compat in core) have not been vetted enough.
>>
>> The new parser appears to add complication to some of things that were very
>> simple with the old parser.
>>
>> The main benefits of the new parser are claimed to be the ability to plug
>> and play many syntaxes and QueryBuilders. This is not an end user benefit
>> though and I'm not even sure how much of a benefit it is to us. There is
>> currently only one impl. It seems to me, once you start another impl, its a
>> long shot that the exact same query tree representation is going to work
>> with a completely different syntax. Sure, if you are just doing postfix
>> rather than prefix, it will be fine – but the stuff that would likely be
>> done – actual new syntaxes – are not likely to be very pluggable. If a
>> syntax can map to the same query tree, I think we would likely stick to a
>> single syntax – else suffer the confusion and maintenance headaches for
>> syntactic sugar. More than a well factored QueryParser that can more easily
>> allow different syntaxes to map to the same query tree representation, I
>> think we just want a single solid syntax for core Lucene that supports Spans
>> to some degree. We basically have that now, sans the spans support. Other,
>> more exotic QueryParsers should live in contrib, as they do now.
>>
>> Which isn't to say this QueryParser should not one day rule the roost – but
>> I don't think its earned the right yet. And I don't think there is a hurry
>> to toss the old parser.
>>
>> Personally, I think that the old parser should not be deprecated. Lets let
>> the new parser breath in contrib for a bit. Lets see if anyone actually adds
>> any other syntaxes. Lets see if the pluggability results in any
>> improvements. Lets see if some of the harder things to do (overriding query
>> build methods?) become easier or keep people from using the new parser.
>>
>> Lets just see if the new parser draws users without us forcing them to it.
>> And lets also wait and see what other committers say – not many have gotten
>> much time to deal with the new parser, or deal with user list questions on
>> it.
>>
>> I just think its premature to start moving people to this new parser. It
>> didn't even really get in until right before release – the paint on the
>> thing still reeks. There is no rush. I saw we undeprecate the current
>> QueryParser and remove the wording in the new QueryParser about it replacing
>> the new in 3.0. Later, if we think it should replace it (after having some
>> experience to judge from), we can reinstate the current plan. Anyone agree?
>>
>> --
>> - Mark
>>
>> http://www.lucidimagination.com
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Jason Rutherglen <ja...@gmail.com>]

I'm starting to use the new parser to emulate Google's queries
(i.e. a phrase query with a single term means no-stemming,
something the current QP doesn't allow because it converts the
quoted query into a term query inside the JavaCC portion). It's
been very straightforward and logical to use (so far).

Thanks to the contrib query parser team!

On Tue, Aug 11, 2009 at 10:54 AM, Mark Miller<ma...@gmail.com> wrote:
> I don't think we should stick with the current path of replacing the current
> QueryParser with the new contrib QueryParser in Lucene 3.0.
>
> The new QueryParser has not been used much at all yet. Its interfaces (which
> will need to abide by back compat in core) have not been vetted enough.
>
> The new parser appears to add complication to some of things that were very
> simple with the old parser.
>
> The main benefits of the new parser are claimed to be the ability to plug
> and play many syntaxes and QueryBuilders. This is not an end user benefit
> though and I'm not even sure how much of a benefit it is to us. There is
> currently only one impl. It seems to me, once you start another impl, its a
> long shot that the exact same query tree representation is going to work
> with a completely different syntax. Sure, if you are just doing postfix
> rather than prefix, it will be fine – but the stuff that would likely be
> done – actual new syntaxes – are not likely to be very pluggable. If a
> syntax can map to the same query tree, I think we would likely stick to a
> single syntax – else suffer the confusion and maintenance headaches for
> syntactic sugar. More than a well factored QueryParser that can more easily
> allow different syntaxes to map to the same query tree representation, I
> think we just want a single solid syntax for core Lucene that supports Spans
> to some degree. We basically have that now, sans the spans support. Other,
> more exotic QueryParsers should live in contrib, as they do now.
>
> Which isn't to say this QueryParser should not one day rule the roost – but
> I don't think its earned the right yet. And I don't think there is a hurry
> to toss the old parser.
>
> Personally, I think that the old parser should not be deprecated. Lets let
> the new parser breath in contrib for a bit. Lets see if anyone actually adds
> any other syntaxes. Lets see if the pluggability results in any
> improvements. Lets see if some of the harder things to do (overriding query
> build methods?) become easier or keep people from using the new parser.
>
> Lets just see if the new parser draws users without us forcing them to it.
> And lets also wait and see what other committers say – not many have gotten
> much time to deal with the new parser, or deal with user list questions on
> it.
>
> I just think its premature to start moving people to this new parser. It
> didn't even really get in until right before release – the paint on the
> thing still reeks. There is no rush. I saw we undeprecate the current
> QueryParser and remove the wording in the new QueryParser about it replacing
> the new in 3.0. Later, if we think it should replace it (after having some
> experience to judge from), we can reinstate the current plan. Anyone agree?
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Mark Miller <ma...@gmail.com>]

Hey Shai - I'm not saying if new syntax's come lets use it. Sorry if it 
came off that way - I'm basically saying - lets see it get used - lets 
see if the things that it offers are taken advantage of. A new syntax is 
not a plus to me necessarily (though it is nice) - personally, I just 
want a solid core syntax for Lucene, and I think the rest is gravy. But 
because the new QP is billed as easy to develop new syntaxs for (in the 
package.html I think), I'm just saying, lets see how the thing turns 
out. I don't mean to really pin point any one thing in that regard - we 
should just let it breath, and then take stalk again.

Basically I just think we should give it a little time. I only wrote out 
so much, and tried to come up with points like that, because an early 
short comment went ignored ;)

- Mark

Shai Erera wrote:
> Mark,
>
> I support not deprecating the current QP.
>
> But I just wanted to comment on "let's wait 'till people add more 
> syntaxes". I don't think that that's the issue here. The new QP is 
> indeed useful for plugging in different search syntaxes, but I 
> personally don't believe that in an application more than one search 
> syntax is used. If there are such, then I'd think their number is very 
> small. And, I agree w/ you - two different syntaxes are not that 
> likely to be able to reuse the same Query tree etc.
>
> However, the new QP, AFAIU, allows one to extend the Lucene syntax 
> more easily. And if some extension to Lucene's syntax is useful, why 
> contribute it as a contrib module and not augment the default QP?
>
> So just contributing a new query syntax as a contrib module doesn't 
> mean the new QP should be used. In fact, I wrote a QP for a different 
> syntax than Lucene's, and I didn't use the new QP as base and it works 
> just great. In fact, my QP is quite simple, and does not involve 
> building a query tree, using builders etc.
>
> And in general I think, writing your own QP for your own query syntax 
> is a super advanced thing, which only few do. So this QP will benefit 
> the minority of Lucene users / developers, IMO.
>
> So I'm not sure that waiting for users to contribute more syntaxes is 
> what we need in order to decide whether this QP should replace the old 
> one. We're more likely to see users experiencing problems w/ it (just 
> because it's new and hasn't been used in the field much yet) in the 
> near future.
>
> This QP currently looks like an OOD exercise. If there will be more 
> syntaxes contributed, then it wins. Otherwise, it's just s rewrite of 
> the old QP, and we need to be sure that the rewrite is worth it.
>
> Shai
>
> On Wed, Aug 12, 2009 at 1:03 AM, Michael McCandless 
> <lucene@mikemccandless.com <ma...@mikemccandless.com>> wrote:
>
>     +1
>
>     Mike
>
>     On Tue, Aug 11, 2009 at 5:43 PM, Michael Busch<buschmic@gmail.com
>     <ma...@gmail.com>> wrote:
>     > I agree we should not remove the old one in 3.0. That's way too
>     early.
>     > If we change the bw-policy we can replace it maybe in 3.1.
>     >
>     > On 8/11/09 11:40 AM, Uwe Schindler wrote:
>     >>
>     >> Yes, we should not deprecate the old one!
>     >>
>     >> -----
>     >> Uwe Schindler
>     >> H.-H.-Meier-Allee 63, D-28213 Bremen
>     >> http://www.thetaphi.de
>     >> eMail: uwe@thetaphi.de <ma...@thetaphi.de>
>     >>
>     >>
>     >>>
>     >>> -----Original Message-----
>     >>> From: Grant Ingersoll [mailto:gsingers@apache.org
>     <ma...@apache.org>]
>     >>> Sent: Tuesday, August 11, 2009 8:32 PM
>     >>> To: java-dev@lucene.apache.org <ma...@lucene.apache.org>
>     >>> Subject: Re: The new Contrib QueryParser should not be slated
>     to replace
>     >>> the old one yet
>     >>>
>     >>> +1, old QP should not be deprecated.  Since the new one is in
>     contrib,
>     >>> it should just be stated that it doesn't necessarily have the same
>     >>> back compat. issues as core, either that or it is marked as
>     >>> experimental.
>     >>>
>     >>> -Grant
>     >>>
>     >>> On Aug 11, 2009, at 1:54 PM, Mark Miller wrote:
>     >>>
>     >>>
>     >>>>
>     >>>> I don't think we should stick with the current path of
>     replacing the
>     >>>> current QueryParser with the new contrib QueryParser in
>     Lucene 3.0.
>     >>>>
>     >>>> The new QueryParser has not been used much at all yet. Its
>     >>>> interfaces (which will need to abide by back compat in core) have
>     >>>> not been vetted enough.
>     >>>>
>     >>>> The new parser appears to add complication to some of things that
>     >>>> were very simple with the old parser.
>     >>>>
>     >>>> The main benefits of the new parser are claimed to be the
>     ability to
>     >>>> plug and play many syntaxes and QueryBuilders. This is not an end
>     >>>> user benefit though and I'm not even sure how much of a
>     benefit it
>     >>>> is to us. There is currently only one impl. It seems to me,
>     once you
>     >>>> start another impl, its a long shot that the exact same query
>     tree
>     >>>> representation is going to work with a completely different
>     syntax.
>     >>>> Sure, if you are just doing postfix rather than prefix, it
>     will be
>     >>>> fine - but the stuff that would likely be done - actual new
>     syntaxes
>     >>>> - are not likely to be very pluggable. If a syntax can map to the
>     >>>> same query tree, I think we would likely stick to a single
>     syntax -
>     >>>> else suffer the confusion and maintenance headaches for syntactic
>     >>>> sugar. More than a well factored QueryParser that can more easily
>     >>>> allow different syntaxes to map to the same query tree
>     >>>> representation, I think we just want a single solid syntax
>     for core
>     >>>> Lucene that supports Spans to some degree. We basically have that
>     >>>> now, sans the spans support. Other, more exotic QueryParsers
>     should
>     >>>> live in contrib, as they do now.
>     >>>>
>     >>>> Which isn't to say this QueryParser should not one day rule the
>     >>>> roost - but I don't think its earned the right yet. And I don't
>     >>>> think there is a hurry to toss the old parser.
>     >>>>
>     >>>> Personally, I think that the old parser should not be deprecated.
>     >>>> Lets let the new parser breath in contrib for a bit. Lets see if
>     >>>> anyone actually adds any other syntaxes. Lets see if the
>     >>>> pluggability results in any improvements. Lets see if some of the
>     >>>> harder things to do (overriding query build methods?) become
>     easier
>     >>>> or keep people from using the new parser.
>     >>>>
>     >>>> Lets just see if the new parser draws users without us
>     forcing them
>     >>>> to it. And lets also wait and see what other committers say - not
>     >>>> many have gotten much time to deal with the new parser, or
>     deal with
>     >>>> user list questions on it.
>     >>>>
>     >>>> I just think its premature to start moving people to this new
>     >>>> parser. It didn't even really get in until right before release -
>     >>>> the paint on the thing still reeks. There is no rush. I saw we
>     >>>> undeprecate the current QueryParser and remove the wording in the
>     >>>> new QueryParser about it replacing the new in 3.0. Later, if we
>     >>>> think it should replace it (after having some experience to judge
>     >>>> from), we can reinstate the current plan. Anyone agree?
>     >>>>
>     >>>> --
>     >>>> - Mark
>     >>>>
>     >>>> http://www.lucidimagination.com
>     >>>>
>     >>>>
>     >>>>
>     >>>>
>     >>>>
>     ---------------------------------------------------------------------
>     >>>> To unsubscribe, e-mail:
>     java-dev-unsubscribe@lucene.apache.org
>     <ma...@lucene.apache.org>
>     >>>> For additional commands, e-mail:
>     java-dev-help@lucene.apache.org
>     <ma...@lucene.apache.org>
>     >>>>
>     >>>>
>     >>>
>     >>>
>     ---------------------------------------------------------------------
>     >>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>     <ma...@lucene.apache.org>
>     >>> For additional commands, e-mail:
>     java-dev-help@lucene.apache.org
>     <ma...@lucene.apache.org>
>     >>>
>     >>
>     >>
>     >>
>     ---------------------------------------------------------------------
>     >> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>     <ma...@lucene.apache.org>
>     >> For additional commands, e-mail:
>     java-dev-help@lucene.apache.org
>     <ma...@lucene.apache.org>
>     >>
>     >>
>     >>
>     >
>     >
>     >
>     ---------------------------------------------------------------------
>     > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>     <ma...@lucene.apache.org>
>     > For additional commands, e-mail: java-dev-help@lucene.apache.org
>     <ma...@lucene.apache.org>
>     >
>     >
>
>     ---------------------------------------------------------------------
>     To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>     <ma...@lucene.apache.org>
>     For additional commands, e-mail: java-dev-help@lucene.apache.org
>     <ma...@lucene.apache.org>
>
>


-- 
- Mark

http://www.lucidimagination.com




---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Shai Erera <se...@gmail.com>]

Mark,

I support not deprecating the current QP.

But I just wanted to comment on "let's wait 'till people add more syntaxes".
I don't think that that's the issue here. The new QP is indeed useful for
plugging in different search syntaxes, but I personally don't believe that
in an application more than one search syntax is used. If there are such,
then I'd think their number is very small. And, I agree w/ you - two
different syntaxes are not that likely to be able to reuse the same Query
tree etc.

However, the new QP, AFAIU, allows one to extend the Lucene syntax more
easily. And if some extension to Lucene's syntax is useful, why contribute
it as a contrib module and not augment the default QP?

So just contributing a new query syntax as a contrib module doesn't mean the
new QP should be used. In fact, I wrote a QP for a different syntax than
Lucene's, and I didn't use the new QP as base and it works just great. In
fact, my QP is quite simple, and does not involve building a query tree,
using builders etc.

And in general I think, writing your own QP for your own query syntax is a
super advanced thing, which only few do. So this QP will benefit the
minority of Lucene users / developers, IMO.

So I'm not sure that waiting for users to contribute more syntaxes is what
we need in order to decide whether this QP should replace the old one. We're
more likely to see users experiencing problems w/ it (just because it's new
and hasn't been used in the field much yet) in the near future.

This QP currently looks like an OOD exercise. If there will be more syntaxes
contributed, then it wins. Otherwise, it's just s rewrite of the old QP, and
we need to be sure that the rewrite is worth it.

Shai

On Wed, Aug 12, 2009 at 1:03 AM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> +1
>
> Mike
>
> On Tue, Aug 11, 2009 at 5:43 PM, Michael Busch<bu...@gmail.com> wrote:
> > I agree we should not remove the old one in 3.0. That's way too early.
> > If we change the bw-policy we can replace it maybe in 3.1.
> >
> > On 8/11/09 11:40 AM, Uwe Schindler wrote:
> >>
> >> Yes, we should not deprecate the old one!
> >>
> >> -----
> >> Uwe Schindler
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> http://www.thetaphi.de
> >> eMail: uwe@thetaphi.de
> >>
> >>
> >>>
> >>> -----Original Message-----
> >>> From: Grant Ingersoll [mailto:gsingers@apache.org]
> >>> Sent: Tuesday, August 11, 2009 8:32 PM
> >>> To: java-dev@lucene.apache.org
> >>> Subject: Re: The new Contrib QueryParser should not be slated to
> replace
> >>> the old one yet
> >>>
> >>> +1, old QP should not be deprecated.  Since the new one is in contrib,
> >>> it should just be stated that it doesn't necessarily have the same
> >>> back compat. issues as core, either that or it is marked as
> >>> experimental.
> >>>
> >>> -Grant
> >>>
> >>> On Aug 11, 2009, at 1:54 PM, Mark Miller wrote:
> >>>
> >>>
> >>>>
> >>>> I don't think we should stick with the current path of replacing the
> >>>> current QueryParser with the new contrib QueryParser in Lucene 3.0.
> >>>>
> >>>> The new QueryParser has not been used much at all yet. Its
> >>>> interfaces (which will need to abide by back compat in core) have
> >>>> not been vetted enough.
> >>>>
> >>>> The new parser appears to add complication to some of things that
> >>>> were very simple with the old parser.
> >>>>
> >>>> The main benefits of the new parser are claimed to be the ability to
> >>>> plug and play many syntaxes and QueryBuilders. This is not an end
> >>>> user benefit though and I'm not even sure how much of a benefit it
> >>>> is to us. There is currently only one impl. It seems to me, once you
> >>>> start another impl, its a long shot that the exact same query tree
> >>>> representation is going to work with a completely different syntax.
> >>>> Sure, if you are just doing postfix rather than prefix, it will be
> >>>> fine - but the stuff that would likely be done - actual new syntaxes
> >>>> - are not likely to be very pluggable. If a syntax can map to the
> >>>> same query tree, I think we would likely stick to a single syntax -
> >>>> else suffer the confusion and maintenance headaches for syntactic
> >>>> sugar. More than a well factored QueryParser that can more easily
> >>>> allow different syntaxes to map to the same query tree
> >>>> representation, I think we just want a single solid syntax for core
> >>>> Lucene that supports Spans to some degree. We basically have that
> >>>> now, sans the spans support. Other, more exotic QueryParsers should
> >>>> live in contrib, as they do now.
> >>>>
> >>>> Which isn't to say this QueryParser should not one day rule the
> >>>> roost - but I don't think its earned the right yet. And I don't
> >>>> think there is a hurry to toss the old parser.
> >>>>
> >>>> Personally, I think that the old parser should not be deprecated.
> >>>> Lets let the new parser breath in contrib for a bit. Lets see if
> >>>> anyone actually adds any other syntaxes. Lets see if the
> >>>> pluggability results in any improvements. Lets see if some of the
> >>>> harder things to do (overriding query build methods?) become easier
> >>>> or keep people from using the new parser.
> >>>>
> >>>> Lets just see if the new parser draws users without us forcing them
> >>>> to it. And lets also wait and see what other committers say - not
> >>>> many have gotten much time to deal with the new parser, or deal with
> >>>> user list questions on it.
> >>>>
> >>>> I just think its premature to start moving people to this new
> >>>> parser. It didn't even really get in until right before release -
> >>>> the paint on the thing still reeks. There is no rush. I saw we
> >>>> undeprecate the current QueryParser and remove the wording in the
> >>>> new QueryParser about it replacing the new in 3.0. Later, if we
> >>>> think it should replace it (after having some experience to judge
> >>>> from), we can reinstate the current plan. Anyone agree?
> >>>>
> >>>> --
> >>>> - Mark
> >>>>
> >>>> http://www.lucidimagination.com
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> >>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: java-dev-help@lucene.apache.org
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-dev-help@lucene.apache.org
> >>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-dev-help@lucene.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Michael McCandless <lu...@mikemccandless.com>]

+1

Mike

On Tue, Aug 11, 2009 at 5:43 PM, Michael Busch<bu...@gmail.com> wrote:
> I agree we should not remove the old one in 3.0. That's way too early.
> If we change the bw-policy we can replace it maybe in 3.1.
>
> On 8/11/09 11:40 AM, Uwe Schindler wrote:
>>
>> Yes, we should not deprecate the old one!
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>>>
>>> -----Original Message-----
>>> From: Grant Ingersoll [mailto:gsingers@apache.org]
>>> Sent: Tuesday, August 11, 2009 8:32 PM
>>> To: java-dev@lucene.apache.org
>>> Subject: Re: The new Contrib QueryParser should not be slated to replace
>>> the old one yet
>>>
>>> +1, old QP should not be deprecated.  Since the new one is in contrib,
>>> it should just be stated that it doesn't necessarily have the same
>>> back compat. issues as core, either that or it is marked as
>>> experimental.
>>>
>>> -Grant
>>>
>>> On Aug 11, 2009, at 1:54 PM, Mark Miller wrote:
>>>
>>>
>>>>
>>>> I don't think we should stick with the current path of replacing the
>>>> current QueryParser with the new contrib QueryParser in Lucene 3.0.
>>>>
>>>> The new QueryParser has not been used much at all yet. Its
>>>> interfaces (which will need to abide by back compat in core) have
>>>> not been vetted enough.
>>>>
>>>> The new parser appears to add complication to some of things that
>>>> were very simple with the old parser.
>>>>
>>>> The main benefits of the new parser are claimed to be the ability to
>>>> plug and play many syntaxes and QueryBuilders. This is not an end
>>>> user benefit though and I'm not even sure how much of a benefit it
>>>> is to us. There is currently only one impl. It seems to me, once you
>>>> start another impl, its a long shot that the exact same query tree
>>>> representation is going to work with a completely different syntax.
>>>> Sure, if you are just doing postfix rather than prefix, it will be
>>>> fine - but the stuff that would likely be done - actual new syntaxes
>>>> - are not likely to be very pluggable. If a syntax can map to the
>>>> same query tree, I think we would likely stick to a single syntax -
>>>> else suffer the confusion and maintenance headaches for syntactic
>>>> sugar. More than a well factored QueryParser that can more easily
>>>> allow different syntaxes to map to the same query tree
>>>> representation, I think we just want a single solid syntax for core
>>>> Lucene that supports Spans to some degree. We basically have that
>>>> now, sans the spans support. Other, more exotic QueryParsers should
>>>> live in contrib, as they do now.
>>>>
>>>> Which isn't to say this QueryParser should not one day rule the
>>>> roost - but I don't think its earned the right yet. And I don't
>>>> think there is a hurry to toss the old parser.
>>>>
>>>> Personally, I think that the old parser should not be deprecated.
>>>> Lets let the new parser breath in contrib for a bit. Lets see if
>>>> anyone actually adds any other syntaxes. Lets see if the
>>>> pluggability results in any improvements. Lets see if some of the
>>>> harder things to do (overriding query build methods?) become easier
>>>> or keep people from using the new parser.
>>>>
>>>> Lets just see if the new parser draws users without us forcing them
>>>> to it. And lets also wait and see what other committers say - not
>>>> many have gotten much time to deal with the new parser, or deal with
>>>> user list questions on it.
>>>>
>>>> I just think its premature to start moving people to this new
>>>> parser. It didn't even really get in until right before release -
>>>> the paint on the thing still reeks. There is no rush. I saw we
>>>> undeprecate the current QueryParser and remove the wording in the
>>>> new QueryParser about it replacing the new in 3.0. Later, if we
>>>> think it should replace it (after having some experience to judge
>>>> from), we can reinstate the current plan. Anyone agree?
>>>>
>>>> --
>>>> - Mark
>>>>
>>>> http://www.lucidimagination.com
>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Michael Busch <bu...@gmail.com>]

I agree we should not remove the old one in 3.0. That's way too early.
If we change the bw-policy we can replace it maybe in 3.1.

On 8/11/09 11:40 AM, Uwe Schindler wrote:
> Yes, we should not deprecate the old one!
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>    
>> -----Original Message-----
>> From: Grant Ingersoll [mailto:gsingers@apache.org]
>> Sent: Tuesday, August 11, 2009 8:32 PM
>> To: java-dev@lucene.apache.org
>> Subject: Re: The new Contrib QueryParser should not be slated to replace
>> the old one yet
>>
>> +1, old QP should not be deprecated.  Since the new one is in contrib,
>> it should just be stated that it doesn't necessarily have the same
>> back compat. issues as core, either that or it is marked as
>> experimental.
>>
>> -Grant
>>
>> On Aug 11, 2009, at 1:54 PM, Mark Miller wrote:
>>
>>      
>>> I don't think we should stick with the current path of replacing the
>>> current QueryParser with the new contrib QueryParser in Lucene 3.0.
>>>
>>> The new QueryParser has not been used much at all yet. Its
>>> interfaces (which will need to abide by back compat in core) have
>>> not been vetted enough.
>>>
>>> The new parser appears to add complication to some of things that
>>> were very simple with the old parser.
>>>
>>> The main benefits of the new parser are claimed to be the ability to
>>> plug and play many syntaxes and QueryBuilders. This is not an end
>>> user benefit though and I'm not even sure how much of a benefit it
>>> is to us. There is currently only one impl. It seems to me, once you
>>> start another impl, its a long shot that the exact same query tree
>>> representation is going to work with a completely different syntax.
>>> Sure, if you are just doing postfix rather than prefix, it will be
>>> fine - but the stuff that would likely be done - actual new syntaxes
>>> - are not likely to be very pluggable. If a syntax can map to the
>>> same query tree, I think we would likely stick to a single syntax -
>>> else suffer the confusion and maintenance headaches for syntactic
>>> sugar. More than a well factored QueryParser that can more easily
>>> allow different syntaxes to map to the same query tree
>>> representation, I think we just want a single solid syntax for core
>>> Lucene that supports Spans to some degree. We basically have that
>>> now, sans the spans support. Other, more exotic QueryParsers should
>>> live in contrib, as they do now.
>>>
>>> Which isn't to say this QueryParser should not one day rule the
>>> roost - but I don't think its earned the right yet. And I don't
>>> think there is a hurry to toss the old parser.
>>>
>>> Personally, I think that the old parser should not be deprecated.
>>> Lets let the new parser breath in contrib for a bit. Lets see if
>>> anyone actually adds any other syntaxes. Lets see if the
>>> pluggability results in any improvements. Lets see if some of the
>>> harder things to do (overriding query build methods?) become easier
>>> or keep people from using the new parser.
>>>
>>> Lets just see if the new parser draws users without us forcing them
>>> to it. And lets also wait and see what other committers say - not
>>> many have gotten much time to deal with the new parser, or deal with
>>> user list questions on it.
>>>
>>> I just think its premature to start moving people to this new
>>> parser. It didn't even really get in until right before release -
>>> the paint on the thing still reeks. There is no rush. I saw we
>>> undeprecate the current QueryParser and remove the wording in the
>>> new QueryParser about it replacing the new in 3.0. Later, if we
>>> think it should replace it (after having some experience to judge
>>> from), we can reinstate the current plan. Anyone agree?
>>>
>>> --
>>> - Mark
>>>
>>> http://www.lucidimagination.com
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>>        
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>      
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
>    


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

RE: The new Contrib QueryParser should not be slated to replace the old one yet

[Uwe Schindler <uw...@thetaphi.de>]

Yes, we should not deprecate the old one!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Grant Ingersoll [mailto:gsingers@apache.org]
> Sent: Tuesday, August 11, 2009 8:32 PM
> To: java-dev@lucene.apache.org
> Subject: Re: The new Contrib QueryParser should not be slated to replace
> the old one yet
> 
> +1, old QP should not be deprecated.  Since the new one is in contrib,
> it should just be stated that it doesn't necessarily have the same
> back compat. issues as core, either that or it is marked as
> experimental.
> 
> -Grant
> 
> On Aug 11, 2009, at 1:54 PM, Mark Miller wrote:
> 
> > I don't think we should stick with the current path of replacing the
> > current QueryParser with the new contrib QueryParser in Lucene 3.0.
> >
> > The new QueryParser has not been used much at all yet. Its
> > interfaces (which will need to abide by back compat in core) have
> > not been vetted enough.
> >
> > The new parser appears to add complication to some of things that
> > were very simple with the old parser.
> >
> > The main benefits of the new parser are claimed to be the ability to
> > plug and play many syntaxes and QueryBuilders. This is not an end
> > user benefit though and I'm not even sure how much of a benefit it
> > is to us. There is currently only one impl. It seems to me, once you
> > start another impl, its a long shot that the exact same query tree
> > representation is going to work with a completely different syntax.
> > Sure, if you are just doing postfix rather than prefix, it will be
> > fine - but the stuff that would likely be done - actual new syntaxes
> > - are not likely to be very pluggable. If a syntax can map to the
> > same query tree, I think we would likely stick to a single syntax -
> > else suffer the confusion and maintenance headaches for syntactic
> > sugar. More than a well factored QueryParser that can more easily
> > allow different syntaxes to map to the same query tree
> > representation, I think we just want a single solid syntax for core
> > Lucene that supports Spans to some degree. We basically have that
> > now, sans the spans support. Other, more exotic QueryParsers should
> > live in contrib, as they do now.
> >
> > Which isn't to say this QueryParser should not one day rule the
> > roost - but I don't think its earned the right yet. And I don't
> > think there is a hurry to toss the old parser.
> >
> > Personally, I think that the old parser should not be deprecated.
> > Lets let the new parser breath in contrib for a bit. Lets see if
> > anyone actually adds any other syntaxes. Lets see if the
> > pluggability results in any improvements. Lets see if some of the
> > harder things to do (overriding query build methods?) become easier
> > or keep people from using the new parser.
> >
> > Lets just see if the new parser draws users without us forcing them
> > to it. And lets also wait and see what other committers say - not
> > many have gotten much time to deal with the new parser, or deal with
> > user list questions on it.
> >
> > I just think its premature to start moving people to this new
> > parser. It didn't even really get in until right before release -
> > the paint on the thing still reeks. There is no rush. I saw we
> > undeprecate the current QueryParser and remove the wording in the
> > new QueryParser about it replacing the new in 3.0. Later, if we
> > think it should replace it (after having some experience to judge
> > from), we can reinstate the current plan. Anyone agree?
> >
> > --
> > - Mark
> >
> > http://www.lucidimagination.com
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-dev-help@lucene.apache.org
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Grant Ingersoll <gs...@apache.org>]

+1, old QP should not be deprecated.  Since the new one is in contrib,  
it should just be stated that it doesn't necessarily have the same  
back compat. issues as core, either that or it is marked as  
experimental.

-Grant

On Aug 11, 2009, at 1:54 PM, Mark Miller wrote:

> I don't think we should stick with the current path of replacing the  
> current QueryParser with the new contrib QueryParser in Lucene 3.0.
>
> The new QueryParser has not been used much at all yet. Its  
> interfaces (which will need to abide by back compat in core) have  
> not been vetted enough.
>
> The new parser appears to add complication to some of things that  
> were very simple with the old parser.
>
> The main benefits of the new parser are claimed to be the ability to  
> plug and play many syntaxes and QueryBuilders. This is not an end  
> user benefit though and I'm not even sure how much of a benefit it  
> is to us. There is currently only one impl. It seems to me, once you  
> start another impl, its a long shot that the exact same query tree  
> representation is going to work with a completely different syntax.  
> Sure, if you are just doing postfix rather than prefix, it will be  
> fine – but the stuff that would likely be done – actual new syntaxes  
> – are not likely to be very pluggable. If a syntax can map to the  
> same query tree, I think we would likely stick to a single syntax –  
> else suffer the confusion and maintenance headaches for syntactic  
> sugar. More than a well factored QueryParser that can more easily  
> allow different syntaxes to map to the same query tree  
> representation, I think we just want a single solid syntax for core  
> Lucene that supports Spans to some degree. We basically have that  
> now, sans the spans support. Other, more exotic QueryParsers should  
> live in contrib, as they do now.
>
> Which isn't to say this QueryParser should not one day rule the  
> roost – but I don't think its earned the right yet. And I don't  
> think there is a hurry to toss the old parser.
>
> Personally, I think that the old parser should not be deprecated.  
> Lets let the new parser breath in contrib for a bit. Lets see if  
> anyone actually adds any other syntaxes. Lets see if the  
> pluggability results in any improvements. Lets see if some of the  
> harder things to do (overriding query build methods?) become easier  
> or keep people from using the new parser.
>
> Lets just see if the new parser draws users without us forcing them  
> to it. And lets also wait and see what other committers say – not  
> many have gotten much time to deal with the new parser, or deal with  
> user list questions on it.
>
> I just think its premature to start moving people to this new  
> parser. It didn't even really get in until right before release –  
> the paint on the thing still reeks. There is no rush. I saw we  
> undeprecate the current QueryParser and remove the wording in the  
> new QueryParser about it replacing the new in 3.0. Later, if we  
> think it should replace it (after having some experience to judge  
> from), we can reinstate the current plan. Anyone agree?
>
> -- 
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Re: The new Contrib QueryParser should not be slated to replace the old one yet

[Erik Hatcher <eh...@apache.org>]

+1

In other words: undeprecate our good friend QueryParser.

	Erik

On Aug 11, 2009, at 1:54 PM, Mark Miller wrote:

> I don't think we should stick with the current path of replacing the  
> current QueryParser with the new contrib QueryParser in Lucene 3.0.
>
> The new QueryParser has not been used much at all yet. Its  
> interfaces (which will need to abide by back compat in core) have  
> not been vetted enough.
>
> The new parser appears to add complication to some of things that  
> were very simple with the old parser.
>
> The main benefits of the new parser are claimed to be the ability to  
> plug and play many syntaxes and QueryBuilders. This is not an end  
> user benefit though and I'm not even sure how much of a benefit it  
> is to us. There is currently only one impl. It seems to me, once you  
> start another impl, its a long shot that the exact same query tree  
> representation is going to work with a completely different syntax.  
> Sure, if you are just doing postfix rather than prefix, it will be  
> fine – but the stuff that would likely be done – actual new syntaxes  
> – are not likely to be very pluggable. If a syntax can map to the  
> same query tree, I think we would likely stick to a single syntax –  
> else suffer the confusion and maintenance headaches for syntactic  
> sugar. More than a well factored QueryParser that can more easily  
> allow different syntaxes to map to the same query tree  
> representation, I think we just want a single solid syntax for core  
> Lucene that supports Spans to some degree. We basically have that  
> now, sans the spans support. Other, more exotic QueryParsers should  
> live in contrib, as they do now.
>
> Which isn't to say this QueryParser should not one day rule the  
> roost – but I don't think its earned the right yet. And I don't  
> think there is a hurry to toss the old parser.
>
> Personally, I think that the old parser should not be deprecated.  
> Lets let the new parser breath in contrib for a bit. Lets see if  
> anyone actually adds any other syntaxes. Lets see if the  
> pluggability results in any improvements. Lets see if some of the  
> harder things to do (overriding query build methods?) become easier  
> or keep people from using the new parser.
>
> Lets just see if the new parser draws users without us forcing them  
> to it. And lets also wait and see what other committers say – not  
> many have gotten much time to deal with the new parser, or deal with  
> user list questions on it.
>
> I just think its premature to start moving people to this new  
> parser. It didn't even really get in until right before release –  
> the paint on the thing still reeks. There is no rush. I saw we  
> undeprecate the current QueryParser and remove the wording in the  
> new QueryParser about it replacing the new in 3.0. Later, if we  
> think it should replace it (after having some experience to judge  
> from), we can reinstate the current plan. Anyone agree?
>
> -- 
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org