You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (Created) (JIRA)" <ji...@apache.org> on 2012/02/16 11:20:59 UTC
[jira] [Created] (LUCENE-3792) Remove StringField
Remove StringField
------------------
Key: LUCENE-3792
URL: https://issues.apache.org/jira/browse/LUCENE-3792
Project: Lucene - Java
Issue Type: Task
Reporter: Robert Muir
Often on the mailing list there is confusion about NOT_ANALYZED.
Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
not being consistent at query time (you really need to configure KeywordAnalyzer for the field
on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
once you've done that, you dont need NOT_ANALYZED).
So I think StringField is a trap too for the same reasons, just under a
different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209430#comment-13209430 ]
Robert Muir commented on LUCENE-3792:
-------------------------------------
OK, i think seriously it would take major work to do something here that would make everyone happy.
I still don't like the situation, but unless there are serious objections, I'd like to commit the javadocs,
just to hopefully reduce the amount of time this trap gets answered on the user list.
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3792_javadocs_3x.patch, LUCENE-3792_javadocs_3x.patch
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209267#comment-13209267 ]
Robert Muir commented on LUCENE-3792:
-------------------------------------
{quote}
If you don't run those through a query analyzer (e.g. generally produce TermQuery directly) then you have lots of additional work.
{quote}
Thats not true, because keywordanalyzer does nothing to the terms, you can continue to produce termquery directly and it will work.
So expert users are fine.
This issue isnt about expert users, its about how our API traps people that are not expert users.
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Priority: Blocker
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209258#comment-13209258 ]
Robert Muir commented on LUCENE-3792:
-------------------------------------
Well we are at a standstill. We constantly get these problems on the users list from NOT_ANALYZED
and I don't like reintroducing the trap again.
So I'm -1 to StringField in 4.0
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Priority: Blocker
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209290#comment-13209290 ]
Uwe Schindler commented on LUCENE-3792:
---------------------------------------
I am fine with that. As I said: We need correct documentation to prevent traps. But removing traps by removing possibilities is the wrong way to go. It will force users to have crazy code for simple things, thats my problem. I don't use QueryParser, so I don't want solutions prefering QueryParser over anything else. QueryParser is just a convenience API around new BQ()...new TQ... E.g., I have my own query parser (and I suggest everybody else to have the same for user-facing queries) that simply analyzes the text and creates a BQ out of all anaylzed terms. No syntax, nothing. And I only execute that one on the field in question. Thats very simplistic but another way to look on things.
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3792_javadocs_3x.patch, LUCENE-3792_javadocs_3x.patch
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3792:
--------------------------------
Attachment: LUCENE-3792_javadocs_3x.patch
Its obvious Uwe and I aren't going to agree here immediately, so here is a patch adding a big warning to 3.x javadocs.
For now I'd like to apply the same warning to StringField in trunk (I just made the patch against 3.x)
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3792_javadocs_3x.patch, LUCENE-3792_javadocs_3x.patch
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209697#comment-13209697 ]
Robert Muir commented on LUCENE-3792:
-------------------------------------
Hossman I think KEYWORD_ANALYZED is the ideal name for 3.x actually. I think in combination with the javadocs it would be more clear.
This still leaves the question for trunk (currently StringField):
positives are that its actually a "nice" name, concise and to the point.
another positive is that StringField omits things like positions, and in trunk we don't silently fail if you form a phrase from this.
one negative is that both StringField and TextField confusingly take String in their ctors, (I've chosen the wrong one myself before on accident).
Basically to me, this is a combination of traps. Trunk is somewhat better because it throws exceptions for positional queries if
you actually excluded positions...
in all cases in 3.x, the wrong 'configuration' here creates a situation where the user just 'does not get results' and they have
no idea why... despite the fact they used the same Analyzer at query-time and index-time like a good user. thats what I find so frustrating.
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3792_javadocs_3x.patch, LUCENE-3792_javadocs_3x.patch
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209260#comment-13209260 ]
Uwe Schindler commented on LUCENE-3792:
---------------------------------------
I said call it different.
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Priority: Blocker
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Chris Male (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209973#comment-13209973 ]
Chris Male commented on LUCENE-3792:
------------------------------------
{quote}
one negative is that both StringField and TextField confusingly take String in their ctors, (I've chosen the wrong one myself before on accident).
{quote}
Would renaming StringField, as suggested earlier, alleviate this? KeywordAnalyzedField?
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3792_javadocs_3x.patch, LUCENE-3792_javadocs_3x.patch
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209256#comment-13209256 ]
Uwe Schindler commented on LUCENE-3792:
---------------------------------------
The backside of this is now, that you need to explicitely use a KeywordAnalyzer now for Primary Key fields. If you don't run those through a query analyzer (e.g. generally produce TermQuery directly) then you have lots of additional work. For simple "lookup" queries and indexing a PK key, this is a no go.
-1 on removing that completely, it should simply called different. We should maybe add PKQuery and PKField to have a symmetry.
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Priority: Blocker
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210135#comment-13210135 ]
Uwe Schindler commented on LUCENE-3792:
---------------------------------------
+1 for KeywordField
For symmetry i would propose to add a KeywordQuery, too (that rewrites to new ConstantScoreQuery(new TermQuery())).
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3792_javadocs_3x.patch, LUCENE-3792_javadocs_3x.patch
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209272#comment-13209272 ]
Robert Muir commented on LUCENE-3792:
-------------------------------------
{quote}
If you remove StringField completely, pleaese also remove NumericField and force users to use PerFieldAnalyzerWrapper with a NumericTokenStream.
{quote}
I actually am not sure this is such a bad idea?
If we were to enforce such a thing, it would also be possible to add a modification to the queryparser (instanceof NumericTokenStream)
so that numeric fields then work out of the box with the query parser nicely?
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3792) Remove StringField
Posted by "Uwe Schindler (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-3792:
----------------------------------
Priority: Major (was: Blocker)
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Hoss Man (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209630#comment-13209630 ]
Hoss Man commented on LUCENE-3792:
----------------------------------
StrawMan suggestion off the top of my head:
* rename NOT_ANALYZED to something like KEYWORD_ANALYZED
* document KEYWORD_ANALYZED as being a convenience flag (and/or optimization?) for achieving equivalent behavior as using PerFieldAnalyzer with KeywordAnalyzer for this field, and keep using / re-word rmuir's patch warning to make it clear that if you use this at index time, any attempts to construct queries against it using the QueryParser will need KeywordAnalyzer.
...would that flag name == analyzer name equivalence help people remember not to get trapped by this?
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3792_javadocs_3x.patch, LUCENE-3792_javadocs_3x.patch
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3792:
--------------------------------
Attachment: LUCENE-3792_javadocs_3x.patch
Sorry, incomplete wording (I forgot to save before svn diff).
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3792_javadocs_3x.patch, LUCENE-3792_javadocs_3x.patch
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Issue Comment Edited] (LUCENE-3792) Remove StringField
Posted by "Uwe Schindler (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209256#comment-13209256 ]
Uwe Schindler edited comment on LUCENE-3792 at 2/16/12 10:46 AM:
-----------------------------------------------------------------
The backside of this is now, that you need to explicitely use a KeywordAnalyzer now for Primary Key fields. If you don't run those through a query analyzer (e.g. generally produce TermQuery directly) then you have lots of additional work. For simple "lookup" queries and indexing a PK key, this is a no go.
-1 on removing that completely, it should simply called different. We should maybe add PKQuery (a ConstantScore TermQuery) and PKField to have a symmetry.
was (Author: thetaphi):
The backside of this is now, that you need to explicitely use a KeywordAnalyzer now for Primary Key fields. If you don't run those through a query analyzer (e.g. generally produce TermQuery directly) then you have lots of additional work. For simple "lookup" queries and indexing a PK key, this is a no go.
-1 on removing that completely, it should simply called different. We should maybe add PKQuery and PKField to have a symmetry.
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Priority: Blocker
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Uwe Schindler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209269#comment-13209269 ]
Uwe Schindler commented on LUCENE-3792:
---------------------------------------
bq. Well we are at a standstill. We constantly get these problems on the users list from NOT_ANALYZED
You cannot prevent users from doing the wrong thing. If you remove StringField completely, pleaese also remove NumericField and force users to use PerFieldAnalyzerWrapper with a NumericTokenStream. If you add a numeric field you cannot ask for it with query parser. If you add a StringField, you cann ask with QueryParser. Simple rule. It must just be clearly documented. And possible StringField renamed.
People using primary keys or other untokenized values should simply not use QueryParser. Use a ComstantScoreTermyQuery and you are fine.
This is all just a documentation problem, so I am completely against removing that. Not everybody is using Lucene purely as a full-text engine.
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Priority: Blocker
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Shai Erera (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210062#comment-13210062 ]
Shai Erera commented on LUCENE-3792:
------------------------------------
A couple of comments:
* NOT_ANALYZED has two variants - with and without norms. I always uses NOT_ANALYZED_NO_NORMS. Is the KWD_ANALYZED suggestion meant to create two variants too? If not, we should somehow let the user be able to express a KWD analyzed field with and without norms.
* How about renaming StringField to KeywordField? Same as Chris's idea, but removing Analyzed ... shorter name and still captures the essence of it.
* And maybe TextField -> TokenizedField?
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3792_javadocs_3x.patch, LUCENE-3792_javadocs_3x.patch
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3792:
--------------------------------
Priority: Blocker (was: Major)
Affects Version/s: 4.0
Fix Version/s: 4.0
Setting this as blocker (sorry).
Its a huge trap when someone sets the same Analyzer on IndexWriter and QueryParser
but the analysis isn't "actually the same".
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Priority: Blocker
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209273#comment-13209273 ]
Robert Muir commented on LUCENE-3792:
-------------------------------------
{quote}
Not everybody is using Lucene purely as a full-text engine.
{quote}
But we cannot let "non-fulltext" uses break the design for the intended use case (full-text).
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210105#comment-13210105 ]
Robert Muir commented on LUCENE-3792:
-------------------------------------
{quote}
NOT_ANALYZED has two variants - with and without norms.
{quote}
You are right, I forgot about this. For NOT_ANALYZED with norms, we should probably just throw CoderMalfunctionError()
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Fix For: 4.0
>
> Attachments: LUCENE-3792_javadocs_3x.patch, LUCENE-3792_javadocs_3x.patch
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3792) Remove StringField
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209254#comment-13209254 ]
Robert Muir commented on LUCENE-3792:
-------------------------------------
On 3.x, I'd like to deprecate NOT_ANALYZED for the same reasons. This at
least discourages people from running into that trap there and using
KeywordAnalyzer instead.
> Remove StringField
> ------------------
>
> Key: LUCENE-3792
> URL: https://issues.apache.org/jira/browse/LUCENE-3792
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Priority: Blocker
> Fix For: 4.0
>
>
> Often on the mailing list there is confusion about NOT_ANALYZED.
> Besides being useless (Just use KeywordAnalyzer instead), people trip up on this
> not being consistent at query time (you really need to configure KeywordAnalyzer for the field
> on your PerFieldAnalyzerWrapper so it will do the same thing at query time... oh wait
> once you've done that, you dont need NOT_ANALYZED).
> So I think StringField is a trap too for the same reasons, just under a
> different name, lets remove it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org