You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Benson Margulies <bi...@gmail.com> on 2012/04/10 23:15:10 UTC
I've broken delete in SolrCloud and I'm a bit clueless as to how
In my cloud configuration, if I push
<delete>
<query>*:*</query>
</delete>
followed by:
<commit/>
I get no errors, the log looks happy enough, but the documents remain
in the index, visible to /query.
Here's what seems my relevant bit of solrconfig.xml. My URP only
implements processAdd.
<updateRequestProcessorChain name="RNI">
<!-- some day, add parameters when we have some -->
<processor class="com.basistech.rni.solr.NameIndexingUpdateRequestProcessorFactory"/>
<processor class="solr.LogUpdateProcessorFactory" />
<processor class="solr.DistributedUpdateProcessorFactory"/>
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
<!-- activate RNI processing by adding the RNI URP to the chain
for xml updates -->
<requestHandler name="/update"
class="solr.XmlUpdateRequestHandler">
<lst name="defaults">
<str name="update.chain">RNI</str>
</lst>
</requestHandler>
Re: I've broken delete in SolrCloud and I'm a bit clueless as to how
Posted by Benson Margulies <bi...@gmail.com>.
See https://issues.apache.org/jira/browse/SOLR-3347. I can replace the
solrconfig.xml with the vanilla solrconfig.xml and the problem
remains.
On Wed, Apr 11, 2012 at 6:35 AM, Darren Govoni <da...@ontrenet.com> wrote:
> Hard to say why its not working for you. Start with a fresh Solr and
> work forward from there or back out your configs and plugins until it
> works again.
>
> On Tue, 2012-04-10 at 17:15 -0400, Benson Margulies wrote:
>> In my cloud configuration, if I push
>>
>> <delete>
>> <query>*:*</query>
>> </delete>
>>
>> followed by:
>>
>> <commit/>
>>
>> I get no errors, the log looks happy enough, but the documents remain
>> in the index, visible to /query.
>>
>> Here's what seems my relevant bit of solrconfig.xml. My URP only
>> implements processAdd.
>>
>> <updateRequestProcessorChain name="RNI">
>> <!-- some day, add parameters when we have some -->
>> <processor class="com.basistech.rni.solr.NameIndexingUpdateRequestProcessorFactory"/>
>> <processor class="solr.LogUpdateProcessorFactory" />
>> <processor class="solr.DistributedUpdateProcessorFactory"/>
>> <processor class="solr.RunUpdateProcessorFactory" />
>> </updateRequestProcessorChain>
>>
>> <!-- activate RNI processing by adding the RNI URP to the chain
>> for xml updates -->
>> <requestHandler name="/update"
>> class="solr.XmlUpdateRequestHandler">
>> <lst name="defaults">
>> <str name="update.chain">RNI</str>
>> </lst>
>> </requestHandler>
>>
>
>
Re: I've broken delete in SolrCloud and I'm a bit clueless as to how
Posted by Benson Margulies <bi...@gmail.com>.
On Thu, Apr 12, 2012 at 2:14 PM, Mark Miller <ma...@gmail.com> wrote:
> google must not have found it - i put that in a month or so ago I believe -
> at least weeks. As you can see, there is still a bit to fill in, but it
> covers the high level. I'd like to add example snippets for the rest soon.
Mark, is it all true? I don't have an update log or a replication
handler, and neither does the default, and it all works fine in the
simple case from the top of that wiki page.
Re: I've broken delete in SolrCloud and I'm a bit clueless as to how
Posted by Mark Miller <ma...@gmail.com>.
google must not have found it - i put that in a month or so ago I believe -
at least weeks. As you can see, there is still a bit to fill in, but it
covers the high level. I'd like to add example snippets for the rest soon.
On Thu, Apr 12, 2012 at 12:04 PM, Benson Margulies <bi...@gmail.com>wrote:
> On Thu, Apr 12, 2012 at 11:56 AM, Mark Miller <ma...@gmail.com>
> wrote:
> > Please see the documentation:
> http://wiki.apache.org/solr/SolrCloud#Required_Config
>
> Did I fail to find this in google or did I just goad you into a writing
> job?
>
> I'm inclined to write a JIRA asking for _version_ to be configurable
> just like the uniqueKey in the schema.
>
>
>
> >
> > schema.xml
> >
> > You must have a _version_ field defined:
> >
> > <field name="_version_" type="long" indexed="true" stored="true"/>
> >
> > On Apr 11, 2012, at 9:10 AM, Benson Margulies wrote:
> >
> >> I didn't have a _version_ field, since nothing in the schema says that
> >> it's required!
> >>
> >> On Wed, Apr 11, 2012 at 6:35 AM, Darren Govoni <da...@ontrenet.com>
> wrote:
> >>> Hard to say why its not working for you. Start with a fresh Solr and
> >>> work forward from there or back out your configs and plugins until it
> >>> works again.
> >>>
> >>> On Tue, 2012-04-10 at 17:15 -0400, Benson Margulies wrote:
> >>>> In my cloud configuration, if I push
> >>>>
> >>>> <delete>
> >>>> <query>*:*</query>
> >>>> </delete>
> >>>>
> >>>> followed by:
> >>>>
> >>>> <commit/>
> >>>>
> >>>> I get no errors, the log looks happy enough, but the documents remain
> >>>> in the index, visible to /query.
> >>>>
> >>>> Here's what seems my relevant bit of solrconfig.xml. My URP only
> >>>> implements processAdd.
> >>>>
> >>>> <updateRequestProcessorChain name="RNI">
> >>>> <!-- some day, add parameters when we have some -->
> >>>> <processor
> class="com.basistech.rni.solr.NameIndexingUpdateRequestProcessorFactory"/>
> >>>> <processor class="solr.LogUpdateProcessorFactory" />
> >>>> <processor class="solr.DistributedUpdateProcessorFactory"/>
> >>>> <processor class="solr.RunUpdateProcessorFactory" />
> >>>> </updateRequestProcessorChain>
> >>>>
> >>>> <!-- activate RNI processing by adding the RNI URP to the chain
> >>>> for xml updates -->
> >>>> <requestHandler name="/update"
> >>>> class="solr.XmlUpdateRequestHandler">
> >>>> <lst name="defaults">
> >>>> <str name="update.chain">RNI</str>
> >>>> </lst>
> >>>> </requestHandler>
> >>>>
> >>>
> >>>
> >
> > - Mark Miller
> > lucidimagination.com
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
--
- Mark
http://www.lucidimagination.com
Re: I've broken delete in SolrCloud and I'm a bit clueless as to how
Posted by Benson Margulies <bi...@gmail.com>.
On Thu, Apr 12, 2012 at 11:56 AM, Mark Miller <ma...@gmail.com> wrote:
> Please see the documentation: http://wiki.apache.org/solr/SolrCloud#Required_Config
Did I fail to find this in google or did I just goad you into a writing job?
I'm inclined to write a JIRA asking for _version_ to be configurable
just like the uniqueKey in the schema.
>
> schema.xml
>
> You must have a _version_ field defined:
>
> <field name="_version_" type="long" indexed="true" stored="true"/>
>
> On Apr 11, 2012, at 9:10 AM, Benson Margulies wrote:
>
>> I didn't have a _version_ field, since nothing in the schema says that
>> it's required!
>>
>> On Wed, Apr 11, 2012 at 6:35 AM, Darren Govoni <da...@ontrenet.com> wrote:
>>> Hard to say why its not working for you. Start with a fresh Solr and
>>> work forward from there or back out your configs and plugins until it
>>> works again.
>>>
>>> On Tue, 2012-04-10 at 17:15 -0400, Benson Margulies wrote:
>>>> In my cloud configuration, if I push
>>>>
>>>> <delete>
>>>> <query>*:*</query>
>>>> </delete>
>>>>
>>>> followed by:
>>>>
>>>> <commit/>
>>>>
>>>> I get no errors, the log looks happy enough, but the documents remain
>>>> in the index, visible to /query.
>>>>
>>>> Here's what seems my relevant bit of solrconfig.xml. My URP only
>>>> implements processAdd.
>>>>
>>>> <updateRequestProcessorChain name="RNI">
>>>> <!-- some day, add parameters when we have some -->
>>>> <processor class="com.basistech.rni.solr.NameIndexingUpdateRequestProcessorFactory"/>
>>>> <processor class="solr.LogUpdateProcessorFactory" />
>>>> <processor class="solr.DistributedUpdateProcessorFactory"/>
>>>> <processor class="solr.RunUpdateProcessorFactory" />
>>>> </updateRequestProcessorChain>
>>>>
>>>> <!-- activate RNI processing by adding the RNI URP to the chain
>>>> for xml updates -->
>>>> <requestHandler name="/update"
>>>> class="solr.XmlUpdateRequestHandler">
>>>> <lst name="defaults">
>>>> <str name="update.chain">RNI</str>
>>>> </lst>
>>>> </requestHandler>
>>>>
>>>
>>>
>
> - Mark Miller
> lucidimagination.com
>
>
>
>
>
>
>
>
>
>
>
Re: I've broken delete in SolrCloud and I'm a bit clueless as to how
Posted by Benson Margulies <bi...@gmail.com>.
I'm probably confused, but it seems to me that the case I hit does not
meet any of Yonik's criteria.
I have no replicas. I'm running SolrCloud in the simple mode where
each doc ends up in exactly one place.
I think that it's just a bug that the code refuses to do the local
deletion when there's no version info.
However, if I am confused, it sure seems like a candidate for the 'at
least throw instead of failing silently' policy.
Re: I've broken delete in SolrCloud and I'm a bit clueless as to
how
Posted by Chris Hostetter <ho...@fucit.org>.
: Off the top of my head:
: _version_ is needed for solr cloud where a leader forwards updates to
: replicas, unless you're handing update distribution yourself or
: providing pre-built shards.
: _version_ is needed for realtime-get and optimistic locking
:
: We should document for sure... but at this point it's not clear what
: we should enforce. (not saying we shouldn't enforce anything... just
: that I haven't really thought about it)
well ... it may eventually make sense to global enforce it for
consistency, but in the meantime the individual components that dpeend on
it can certainly enforce it (just like my uniqueKey example; the search
components that require it check for themselves on init and fail fast)
(ie: sounds like the RealTimeGetHandler and the existing
DistributedUpdateProcessor should fail fast on init if the schema doesn't
have it)
-Hoss
Re: I've broken delete in SolrCloud and I'm a bit clueless as to how
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Thu, Apr 12, 2012 at 2:21 PM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> : Please see the documentation: http://wiki.apache.org/solr/SolrCloud#Required_Config> :
>
> : schema.xml
> :
> : You must have a _version_ field defined:
> :
> : <field name="_version_" type="long" indexed="true" stored="true"/>
>
> Seems like this is the kind of thing that should make Solr fail hard and
> fast on SolrCore init if it sees you are running in cloud mode and yet it
> doesn't find this -- similar to how some other features fail hard and fast
> if you don't have uniqueKey.
Off the top of my head:
_version_ is needed for solr cloud where a leader forwards updates to
replicas, unless you're handing update distribution yourself or
providing pre-built shards.
_version_ is needed for realtime-get and optimistic locking
We should document for sure... but at this point it's not clear what
we should enforce. (not saying we shouldn't enforce anything... just
that I haven't really thought about it)
-Yonik
lucenerevolution.com - Lucene/Solr Open Source Search Conference.
Boston May 7-10
Re: I've broken delete in SolrCloud and I'm a bit clueless as to how
Posted by Mark Miller <ma...@gmail.com>.
I think someone already made a JIRA issue like that. I think Yonik might
have had an opinion about it that I cannot remember right now.
On Thu, Apr 12, 2012 at 2:21 PM, Chris Hostetter
<ho...@fucit.org>wrote:
>
> : Please see the documentation:
> http://wiki.apache.org/solr/SolrCloud#Required_Config
> :
> : schema.xml
> :
> : You must have a _version_ field defined:
> :
> : <field name="_version_" type="long" indexed="true" stored="true"/>
>
> Seems like this is the kind of thing that should make Solr fail hard and
> fast on SolrCore init if it sees you are running in cloud mode and yet it
> doesn't find this -- similar to how some other features fail hard and fast
> if you don't have uniqueKey.
>
>
> -Hoss
>
--
- Mark
http://www.lucidimagination.com
Re: I've broken delete in SolrCloud and I'm a bit clueless as to
how
Posted by Chris Hostetter <ho...@fucit.org>.
: Please see the documentation: http://wiki.apache.org/solr/SolrCloud#Required_Config
:
: schema.xml
:
: You must have a _version_ field defined:
:
: <field name="_version_" type="long" indexed="true" stored="true"/>
Seems like this is the kind of thing that should make Solr fail hard and
fast on SolrCore init if it sees you are running in cloud mode and yet it
doesn't find this -- similar to how some other features fail hard and fast
if you don't have uniqueKey.
-Hoss
Re: I've broken delete in SolrCloud and I'm a bit clueless as to how
Posted by Mark Miller <ma...@gmail.com>.
Please see the documentation: http://wiki.apache.org/solr/SolrCloud#Required_Config
schema.xml
You must have a _version_ field defined:
<field name="_version_" type="long" indexed="true" stored="true"/>
On Apr 11, 2012, at 9:10 AM, Benson Margulies wrote:
> I didn't have a _version_ field, since nothing in the schema says that
> it's required!
>
> On Wed, Apr 11, 2012 at 6:35 AM, Darren Govoni <da...@ontrenet.com> wrote:
>> Hard to say why its not working for you. Start with a fresh Solr and
>> work forward from there or back out your configs and plugins until it
>> works again.
>>
>> On Tue, 2012-04-10 at 17:15 -0400, Benson Margulies wrote:
>>> In my cloud configuration, if I push
>>>
>>> <delete>
>>> <query>*:*</query>
>>> </delete>
>>>
>>> followed by:
>>>
>>> <commit/>
>>>
>>> I get no errors, the log looks happy enough, but the documents remain
>>> in the index, visible to /query.
>>>
>>> Here's what seems my relevant bit of solrconfig.xml. My URP only
>>> implements processAdd.
>>>
>>> <updateRequestProcessorChain name="RNI">
>>> <!-- some day, add parameters when we have some -->
>>> <processor class="com.basistech.rni.solr.NameIndexingUpdateRequestProcessorFactory"/>
>>> <processor class="solr.LogUpdateProcessorFactory" />
>>> <processor class="solr.DistributedUpdateProcessorFactory"/>
>>> <processor class="solr.RunUpdateProcessorFactory" />
>>> </updateRequestProcessorChain>
>>>
>>> <!-- activate RNI processing by adding the RNI URP to the chain
>>> for xml updates -->
>>> <requestHandler name="/update"
>>> class="solr.XmlUpdateRequestHandler">
>>> <lst name="defaults">
>>> <str name="update.chain">RNI</str>
>>> </lst>
>>> </requestHandler>
>>>
>>
>>
- Mark Miller
lucidimagination.com
Re: I've broken delete in SolrCloud and I'm a bit clueless as to how
Posted by Benson Margulies <bi...@gmail.com>.
I didn't have a _version_ field, since nothing in the schema says that
it's required!
On Wed, Apr 11, 2012 at 6:35 AM, Darren Govoni <da...@ontrenet.com> wrote:
> Hard to say why its not working for you. Start with a fresh Solr and
> work forward from there or back out your configs and plugins until it
> works again.
>
> On Tue, 2012-04-10 at 17:15 -0400, Benson Margulies wrote:
>> In my cloud configuration, if I push
>>
>> <delete>
>> <query>*:*</query>
>> </delete>
>>
>> followed by:
>>
>> <commit/>
>>
>> I get no errors, the log looks happy enough, but the documents remain
>> in the index, visible to /query.
>>
>> Here's what seems my relevant bit of solrconfig.xml. My URP only
>> implements processAdd.
>>
>> <updateRequestProcessorChain name="RNI">
>> <!-- some day, add parameters when we have some -->
>> <processor class="com.basistech.rni.solr.NameIndexingUpdateRequestProcessorFactory"/>
>> <processor class="solr.LogUpdateProcessorFactory" />
>> <processor class="solr.DistributedUpdateProcessorFactory"/>
>> <processor class="solr.RunUpdateProcessorFactory" />
>> </updateRequestProcessorChain>
>>
>> <!-- activate RNI processing by adding the RNI URP to the chain
>> for xml updates -->
>> <requestHandler name="/update"
>> class="solr.XmlUpdateRequestHandler">
>> <lst name="defaults">
>> <str name="update.chain">RNI</str>
>> </lst>
>> </requestHandler>
>>
>
>
Re: I've broken delete in SolrCloud and I'm a bit clueless as to
how
Posted by Darren Govoni <da...@ontrenet.com>.
Hard to say why its not working for you. Start with a fresh Solr and
work forward from there or back out your configs and plugins until it
works again.
On Tue, 2012-04-10 at 17:15 -0400, Benson Margulies wrote:
> In my cloud configuration, if I push
>
> <delete>
> <query>*:*</query>
> </delete>
>
> followed by:
>
> <commit/>
>
> I get no errors, the log looks happy enough, but the documents remain
> in the index, visible to /query.
>
> Here's what seems my relevant bit of solrconfig.xml. My URP only
> implements processAdd.
>
> <updateRequestProcessorChain name="RNI">
> <!-- some day, add parameters when we have some -->
> <processor class="com.basistech.rni.solr.NameIndexingUpdateRequestProcessorFactory"/>
> <processor class="solr.LogUpdateProcessorFactory" />
> <processor class="solr.DistributedUpdateProcessorFactory"/>
> <processor class="solr.RunUpdateProcessorFactory" />
> </updateRequestProcessorChain>
>
> <!-- activate RNI processing by adding the RNI URP to the chain
> for xml updates -->
> <requestHandler name="/update"
> class="solr.XmlUpdateRequestHandler">
> <lst name="defaults">
> <str name="update.chain">RNI</str>
> </lst>
> </requestHandler>
>