You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Hoss Man (JIRA)" <ji...@apache.org> on 2007/02/23 19:52:05 UTC
[jira] Created: (SOLR-172) add doc without uniqueKey field causes
NPE
add doc without uniqueKey field causes NPE
------------------------------------------
Key: SOLR-172
URL: https://issues.apache.org/jira/browse/SOLR-172
Project: Solr
Issue Type: Bug
Reporter: Hoss Man
as mentioned in email by otis...
SimplePostTool: WARNING: Unexpected response from Solr: '<result
status="1">java.lang.NullPointerException
at org.apache.solr.schema.FieldType.storedToIndexed(FieldType.java:248)
at org.apache.solr.update.UpdateHandler.getIndexedId(UpdateHandler.java:134)
at
org.apache.solr.update.DirectUpdateHandler2.overwriteBoth(DirectUpdateHandler2.java:380)
...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field causes NPE
Posted by Ryan McKinley <ry...@gmail.com>.
> to sneak in docs that don't match the mold/ don't have a unique id
> (not sure if that's good or not).
Probably not a good idea, but I don't think we should change the
functionality with this patch, just the error message. SOLR-139,
requires a new add document command - that would be a good place to
introduce more rigorous constraints.
> I'm pretty sure there are some tests that do this though.
>
I don't think so. All the tests pass if you throw an exception with a
missing ID.
Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field
causes NPE
Posted by Chris Hostetter <ho...@fucit.org>.
: > 1) if there is a <uniqueKey> delcaration in your schema, then any
: > <add><doc> *MUST* have that field ... regardless of what overwrite or
: > allowDup options are specified.
:
: That's not currently true. allowDups means no overwriting, and no
: need for the id field in the doc being added. It provides a mechanism
Hmmm... i guess i hadn't thought of it that way .... i thought a uniqueKey
declaration required that the field be specified for all docs, and
allDups=true just garunteed that he duplicate would be allowed.
: (not sure if that's good or not). I'm pretty sure there are some
i'm not sure either.
-Hoss
Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field causes NPE
Posted by Yonik Seeley <yo...@apache.org>.
On 2/26/07, Chris Hostetter <ho...@fucit.org> wrote:
> 1) if there is a <uniqueKey> delcaration in your schema, then any
> <add><doc> *MUST* have that field ... regardless of what overwrite or
> allowDup options are specified.
That's not currently true. allowDups means no overwriting, and no
need for the id field in the doc being added. It provides a mechanism
to sneak in docs that don't match the mold/ don't have a unique id
(not sure if that's good or not). I'm pretty sure there are some
tests that do this though.
-Yonik
Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field
causes NPE
Posted by Chris Hostetter <ho...@fucit.org>.
I'm a little lost by your whole question ...
1) if there is a <uniqueKey> delcaration in your schema, then any
<add><doc> *MUST* have that field ... regardless of what overwrite or
allowDup options are specified.
2) overwrite and allowDups options *only* make sense if there is a
<uniqueKey> field defined in the schema ... i there is no uniqueKey those
options should be ignored (or trigger an error ... but i'd vote for
ignoring so that client libraries can be configured to set them to one
thing by defualt without needing to know what type of schema the user
has)
: Date: Mon, 26 Feb 2007 12:49:06 -0800
: From: Ryan McKinley <ry...@gmail.com>
: Reply-To: solr-dev@lucene.apache.org
: To: solr-dev@lucene.apache.org
: Subject: Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field
: causes NPE
:
: Thinking about this a bit more (I should do that *before* hitting the
: send button!)
:
: scratch the required="true" suggestion. If you send a request with
: add.overwriteX and no id, it gives you an error (#2) if you want to
: get behavior #1, you should have to send add.allowDups = true
:
-Hoss
Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field causes NPE
Posted by Ryan McKinley <ry...@gmail.com>.
Thinking about this a bit more (I should do that *before* hitting the
send button!)
scratch the required="true" suggestion. If you send a request with
add.overwriteX and no id, it gives you an error (#2) if you want to
get behavior #1, you should have to send add.allowDups = true
Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field causes NPE
Posted by Ryan McKinley <ry...@gmail.com>.
On 2/26/07, Chris Hostetter <ho...@fucit.org> wrote:
>
> : It probably would simplify some logic to assume unique keys are
> : required, but we should think a little on it first. Would it make it
> : harder to model existing random lucene indicies?
>
> yeah ... because then people would be force client side to generate an
> arbitrary unique key for each doc ... if we really wnat to go this route
> 9and i don't see any reason why we should) we should make sure we support
> an autoincrment counter as a default field value so people who don't care
> about document uniqueness don't have to do anything new to keep using Solr
> (or to migrate from Lucene to Solr)
>
The problem with SOLR-172 is when you send an add command with
"overwrite*". If this happens, the update handler could:
1. quietly use allowDups() instead
2. send an error saying "missing id"
In the applications i'm working on, 2 is the better option - that may
not always be the case. I hate to suggest schema bloat, but maybe we
should have:
<uniqueKey required="true">
Also, if no <uniqueKey> is defined in schema.xml, the add command
should always use allowDups() even if the command said overwrite.
ryan
Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field
causes NPE
Posted by Chris Hostetter <ho...@fucit.org>.
: It probably would simplify some logic to assume unique keys are
: required, but we should think a little on it first. Would it make it
: harder to model existing random lucene indicies?
yeah ... because then people would be force client side to generate an
arbitrary unique key for each doc ... if we really wnat to go this route
9and i don't see any reason why we should) we should make sure we support
an autoincrment counter as a default field value so people who don't care
about document uniqueness don't have to do anything new to keep using Solr
(or to migrate from Lucene to Solr)
-Hoss
Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field causes NPE
Posted by Yonik Seeley <yo...@apache.org>.
On 2/24/07, Ryan McKinley (JIRA) <ji...@apache.org> wrote:
> Is it ever ok to send a document without a unique Key? does/should solr require a unique key field in the schema?
Solr doesn't currently require a unique key field in the schema, and
if there isn't one, then it's OK to send docs without one.
> It seems like not much works properly if one is not specified - maybe it should be an explicit requirement.
Well, overwriting documents doesn't work, but everything else should.
I've seen a case where the users had no unique key and did rebuilds
from scratch all the time (not that I think it's a good idea).
It probably would simplify some logic to assume unique keys are
required, but we should think a little on it first. Would it make it
harder to model existing random lucene indicies?
-Yonik
Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field causes NPE
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Feb 25, 2007, at 1:19 AM, Erik Hatcher wrote:
> I have a hard envisioning a case where a document would be indexed
> without a unique identifier. While it
^ time
> *can* be done with just Lucene, I've never seen a case where a
> unique key wasn't just a great idea but a real necessity.
In other words, I'm fine with Solr enforcing a unique identifier per
document if it helps streamline things. If there isn't a compelling
reason other than this pesky exception handling thing, then when the
schema says there is a unique id and one isn't sent, we send a
logical exception back, not an NPE. No biggie.
>
> Erik
>
>
> On Feb 24, 2007, at 6:10 PM, Ryan McKinley (JIRA) wrote:
>
>>
>> [ https://issues.apache.org/jira/browse/SOLR-172?
>> page=com.atlassian.jira.plugin.system.issuetabpanels:comment-
>> tabpanel#action_12475663 ]
>>
>> Ryan McKinley commented on SOLR-172:
>> ------------------------------------
>>
>> Is it ever ok to send a document without a unique Key? does/
>> should solr require a unique key field in the schema?
>>
>> It seems like not much works properly if one is not specified -
>> maybe it should be an explicit requirement.
>>
>>> add doc without uniqueKey field causes NPE
>>> ------------------------------------------
>>>
>>> Key: SOLR-172
>>> URL: https://issues.apache.org/jira/browse/SOLR-172
>>> Project: Solr
>>> Issue Type: Bug
>>> Reporter: Hoss Man
>>>
>>> as mentioned in email by otis...
>>> SimplePostTool: WARNING: Unexpected response from Solr: '<result
>>> status="1">java.lang.NullPointerException
>>> at org.apache.solr.schema.FieldType.storedToIndexed
>>> (FieldType.java:248)
>>> at org.apache.solr.update.UpdateHandler.getIndexedId
>>> (UpdateHandler.java:134)
>>> at
>>> org.apache.solr.update.DirectUpdateHandler2.overwriteBoth
>>> (DirectUpdateHandler2.java:380)
>>> ...
>>
>> --
>> This message is automatically generated by JIRA.
>> -
>> You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (SOLR-172) add doc without uniqueKey field causes NPE
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
I have a hard envisioning a case where a document would be indexed
without a unique identifier. While it *can* be done with just
Lucene, I've never seen a case where a unique key wasn't just a great
idea but a real necessity.
Erik
On Feb 24, 2007, at 6:10 PM, Ryan McKinley (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/SOLR-172?
> page=com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel#action_12475663 ]
>
> Ryan McKinley commented on SOLR-172:
> ------------------------------------
>
> Is it ever ok to send a document without a unique Key? does/should
> solr require a unique key field in the schema?
>
> It seems like not much works properly if one is not specified -
> maybe it should be an explicit requirement.
>
>> add doc without uniqueKey field causes NPE
>> ------------------------------------------
>>
>> Key: SOLR-172
>> URL: https://issues.apache.org/jira/browse/SOLR-172
>> Project: Solr
>> Issue Type: Bug
>> Reporter: Hoss Man
>>
>> as mentioned in email by otis...
>> SimplePostTool: WARNING: Unexpected response from Solr: '<result
>> status="1">java.lang.NullPointerException
>> at org.apache.solr.schema.FieldType.storedToIndexed
>> (FieldType.java:248)
>> at org.apache.solr.update.UpdateHandler.getIndexedId
>> (UpdateHandler.java:134)
>> at
>> org.apache.solr.update.DirectUpdateHandler2.overwriteBoth
>> (DirectUpdateHandler2.java:380)
>> ...
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-172) add doc without uniqueKey field causes
NPE
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12475663 ]
Ryan McKinley commented on SOLR-172:
------------------------------------
Is it ever ok to send a document without a unique Key? does/should solr require a unique key field in the schema?
It seems like not much works properly if one is not specified - maybe it should be an explicit requirement.
> add doc without uniqueKey field causes NPE
> ------------------------------------------
>
> Key: SOLR-172
> URL: https://issues.apache.org/jira/browse/SOLR-172
> Project: Solr
> Issue Type: Bug
> Reporter: Hoss Man
>
> as mentioned in email by otis...
> SimplePostTool: WARNING: Unexpected response from Solr: '<result
> status="1">java.lang.NullPointerException
> at org.apache.solr.schema.FieldType.storedToIndexed(FieldType.java:248)
> at org.apache.solr.update.UpdateHandler.getIndexedId(UpdateHandler.java:134)
> at
> org.apache.solr.update.DirectUpdateHandler2.overwriteBoth(DirectUpdateHandler2.java:380)
> ...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-172) add doc without uniqueKey field causes
NPE
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan McKinley updated SOLR-172:
-------------------------------
Attachment: SOLR-172-uniqueKey-NPE.patch
Updated patch in responce to discussion. This version:
1. Throws a nice exception if the add command is overwrite and the document does not contain a key.
2. Does *not* throw an exception if there you call addDups()
3. rewrites all 'addDocs' commands where there is no unique key in the schema to be 'allowDups'
> add doc without uniqueKey field causes NPE
> ------------------------------------------
>
> Key: SOLR-172
> URL: https://issues.apache.org/jira/browse/SOLR-172
> Project: Solr
> Issue Type: Bug
> Reporter: Hoss Man
> Attachments: SOLR-172-uniqueKey-NPE.patch, SOLR-172-uniqueKey-NPE.patch
>
>
> as mentioned in email by otis...
> SimplePostTool: WARNING: Unexpected response from Solr: '<result
> status="1">java.lang.NullPointerException
> at org.apache.solr.schema.FieldType.storedToIndexed(FieldType.java:248)
> at org.apache.solr.update.UpdateHandler.getIndexedId(UpdateHandler.java:134)
> at
> org.apache.solr.update.DirectUpdateHandler2.overwriteBoth(DirectUpdateHandler2.java:380)
> ...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-172) add doc without uniqueKey field causes
NPE
Posted by "Ryan McKinley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan McKinley updated SOLR-172:
-------------------------------
Attachment: SOLR-172-uniqueKey-NPE.patch
This patch adds a check to UpdateHandler.addDoc( ) that makes sure the document id is specified *if* the schema declares one.
I had to change some fields in ConvertedLegacyTest from "id_i" to "id"
> add doc without uniqueKey field causes NPE
> ------------------------------------------
>
> Key: SOLR-172
> URL: https://issues.apache.org/jira/browse/SOLR-172
> Project: Solr
> Issue Type: Bug
> Reporter: Hoss Man
> Attachments: SOLR-172-uniqueKey-NPE.patch
>
>
> as mentioned in email by otis...
> SimplePostTool: WARNING: Unexpected response from Solr: '<result
> status="1">java.lang.NullPointerException
> at org.apache.solr.schema.FieldType.storedToIndexed(FieldType.java:248)
> at org.apache.solr.update.UpdateHandler.getIndexedId(UpdateHandler.java:134)
> at
> org.apache.solr.update.DirectUpdateHandler2.overwriteBoth(DirectUpdateHandler2.java:380)
> ...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-172) add doc without uniqueKey field causes
NPE
Posted by "Mike Klaas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Klaas resolved SOLR-172.
-----------------------------
Resolution: Fixed
Fix Version/s: 1.2
Assignee: Mike Klaas
Committed in r520088. Thanks Ryan!
> add doc without uniqueKey field causes NPE
> ------------------------------------------
>
> Key: SOLR-172
> URL: https://issues.apache.org/jira/browse/SOLR-172
> Project: Solr
> Issue Type: Bug
> Reporter: Hoss Man
> Assigned To: Mike Klaas
> Fix For: 1.2
>
> Attachments: SOLR-172-uniqueKey-NPE.patch, SOLR-172-uniqueKey-NPE.patch
>
>
> as mentioned in email by otis...
> SimplePostTool: WARNING: Unexpected response from Solr: '<result
> status="1">java.lang.NullPointerException
> at org.apache.solr.schema.FieldType.storedToIndexed(FieldType.java:248)
> at org.apache.solr.update.UpdateHandler.getIndexedId(UpdateHandler.java:134)
> at
> org.apache.solr.update.DirectUpdateHandler2.overwriteBoth(DirectUpdateHandler2.java:380)
> ...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-172) add doc without uniqueKey field causes
NPE
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12475700 ]
Hoss Man commented on SOLR-172:
-------------------------------
solr is designed so that if you don't specify a <uniqueKey> field in your schema, then no uniqueness is enforced (just like lucene) the tradeoffs being that you can't replace a document by readding it, and and you (obviously) can't delete by id.
i've never used this functionality myself, but i can imagine it might be useful to some people.
this bug in particular is about the confusing error message when the schema says there is a uniqueKey, but no value was specified in the add ... if the schema didn't say there was a uniqueKey - there wouldn't be an error at all
> add doc without uniqueKey field causes NPE
> ------------------------------------------
>
> Key: SOLR-172
> URL: https://issues.apache.org/jira/browse/SOLR-172
> Project: Solr
> Issue Type: Bug
> Reporter: Hoss Man
>
> as mentioned in email by otis...
> SimplePostTool: WARNING: Unexpected response from Solr: '<result
> status="1">java.lang.NullPointerException
> at org.apache.solr.schema.FieldType.storedToIndexed(FieldType.java:248)
> at org.apache.solr.update.UpdateHandler.getIndexedId(UpdateHandler.java:134)
> at
> org.apache.solr.update.DirectUpdateHandler2.overwriteBoth(DirectUpdateHandler2.java:380)
> ...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.