You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Martínez, Alfonso <al...@indra.es> on 2015/07/15 11:01:34 UTC

Does update field feature work in a schema with dynamic fields?

Hi,

i'm using Solr 4.10.3, and i'm trying update a doc field using atomic update (http://wiki.apache.org/solr/Atomic_Updates).

My schema.xml is like this:

<!--Fields-->
<field name="id" type="string" indexed="true" stored="true" required="true" />
<field name="name" type="string" indexed="true" stored="true" />
<field name="src_desc" type="string" indexed="true" stored="true" />
<field name="_version_" type="long" indexed="true" stored="true" /
<!--Dynamic fields definition-->
<dynamicField name="src_*" type="string" indexed="true" stored="true" multiValued="false" />
<dynamicField name="dinamic_*" type="string" indexed="true" stored="true" multiValued="false" />
<!--Copy fields-->
<copyField source="src_*" dest="dinamic_*"/>


I add a document with this command:



curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H "Content-Type: text/xml" --data-binary '<add><doc><field name="id">1</field><field name="name" >paco</field><field name="src_desc" >friend of mine</field></doc></add>'



And later I update the field 'name' with this command:



curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H "Content-Type: text/xml" --data-binary '<add><doc><field name="id">1</field><field name="name" update="set">paquico</field></doc></add>'



As I do so the doc i retrive from Solr is:



<result name="response" numFound="1" start="0" maxScore="1.0">
 <doc>
  <str name="id">1</str>
  <str name="name">paquico</str>
  <str name="src_desc">friend of mine</str>
  <arr name="dinamic_desc">
   <str>friend of mine</str>
   <str>friend of mine</str>
  </arr>
  <long name="_version_">1506750859550130176</long>
  <float name="score">1.0</float>
 </doc>
</result>



So I get a non multivalued field (dinamic_desc) with multiple values :(





________________________________
Este correo electrónico y, en su caso, cualquier fichero anexo al mismo, contiene información de carácter confidencial exclusivamente dirigida a su destinatario o destinatarios. Si no es vd. el destinatario indicado, queda notificado que la lectura, utilización, divulgación y/o copia sin autorización está prohibida en virtud de la legislación vigente. En el caso de haber recibido este correo electrónico por error, se ruega notificar inmediatamente esta circunstancia mediante reenvío a la dirección electrónica del remitente.
Evite imprimir este mensaje si no es estrictamente necesario.

This email and any file attached to it (when applicable) contain(s) confidential information that is exclusively addressed to its recipient(s). If you are not the indicated recipient, you are informed that reading, using, disseminating and/or copying it without authorisation is forbidden in accordance with the legislation in effect. If you have received this email by mistake, please immediately notify the sender of the situation by resending it to their email address.
Avoid printing this message if it is not absolutely necessary.

RE: Does update field feature work in a schema with dynamic fields?

Posted by Martínez, Alfonso <al...@indra.es>.
Ok, thank very much.

When I try a second atomic udpate is when I got the exception you mentioned "multiple values encountered
for non multiValued copy field". First time there is not exception but the non-multivalued field get indexed with 2 values.

Cheers.

________________________________________
From: Alessandro Benedetti [benedetti.alex85@gmail.com]
Sent: Wednesday, July 15, 2015 2:28 PM
To: solr-user@lucene.apache.org
Subject: Re: Does update field feature work in a schema with dynamic fields?

Going through the code in the RunUpdateRequestProcessor we call at one
point :

…

Document luceneDocument = cmd.getLuceneDocument();
// SolrCore.verbose("updateDocument",updateTerm,luceneDocument,writer);
writer.updateDocument(updateTerm, luceneDocument);

..


Inside that method we call :

public Document getLuceneDocument() {
  return DocumentBuilder.toDocument(getSolrInputDocument(), req.getSchema());
}


Then exploring the toDocument we find what we need :

org/apache/solr/update/DocumentBuilder.java:114

And looking into there we realise it is a bug :

…

if (!destinationField.multiValued() && destHasValues) {
  throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
          "ERROR: "+getID(doc, schema)+"multiple values encountered
for non multiValued copy field " +
                  destinationField.getName() + ": " + v);
}

...


Because the check is actually checking in already used fields, not
expecting  the copy field to be already valorised.

Hence the exception coming.

I will create an issue for that and provide a patch as soon as I have
time ( or anyone can provide the patch).


Cheers






2015-07-15 12:27 GMT+01:00 Alessandro Benedetti <be...@gmail.com>
:

> Ohhh!
> I didn't read it completely, so i missed the copy field.
> Ok now.
> This is the explanation :
> Copy fields are added at indexing time, when the document arrived to the
> RunUpdateRequest processor.
> If i remember well at this point , before we start the indexing the
> content of source field is added to the copy field as a value.
>
> The first time you indexed your document , the first copy was added.
>
> What you didn't know is the fact that actually atomic update works in this
> way :
>
> 1) I get the current Doc from the index ( the stored fields),
> 2) I do the update, and
> 3) then I send the document to the indexing processing chain *again*.
> So the value is copied a second time.
>
> This will produce the duplicate value.
> I can go in deep, but I think this is the cause.
>
> Cheers
>
>
> 2015-07-15 12:10 GMT+01:00 Martínez López, Alfonso <al...@indra.es>:
>
>> Hi, thanks for your help!
>>
>> Value for 'dinamic_desc' field come from 'src_desc' field. I copy the
>> value with:
>>
>> <copyField source="src_*" dest="dinamic_*"/>
>>
>> Seems like when I update a different field (field 'name') via atomic
>> update, the copyField directive copies the value again from 'src_desc' to
>> 'desc_field', instead of updating the value, like if 'desc_field' or
>> 'desc_*' where multivalued.
>>
>> Cheers.
>> ________________________________________
>> From: Alessandro Benedetti [benedetti.alex85@gmail.com]
>> Sent: Wednesday, July 15, 2015 12:56 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Does update field feature work in a schema with dynamic
>> fields?
>>
>> Just tried, on Solr 5.1 and I get the proper behaviour.
>>
>> Actually where is the value for the dinamic_desc coming from ?
>>
>> I can not see it in the updates and actually it is not in my index.
>> Are you sure you have not forgotten any detail ?
>>
>> Cheers
>>
>> 2015-07-15 11:48 GMT+01:00 Alessandro Benedetti <
>> benedetti.alex85@gmail.com>
>> :
>>
>> > This is kinda weird and looks a lot like a bug.
>> > Let me try to reproduce it locally!
>> > I let you know soon !
>> >
>> > Cheers
>> >
>> > 2015-07-15 10:01 GMT+01:00 Martínez López, Alfonso <al...@indra.es>:
>> >
>> >> Hi,
>> >>
>> >> i'm using Solr 4.10.3, and i'm trying update a doc field using atomic
>> >> update (http://wiki.apache.org/solr/Atomic_Updates).
>> >>
>> >> My schema.xml is like this:
>> >>
>> >> <!--Fields-->
>> >> <field name="id" type="string" indexed="true" stored="true"
>> >> required="true" />
>> >> <field name="name" type="string" indexed="true" stored="true" />
>> >> <field name="src_desc" type="string" indexed="true" stored="true" />
>> >> <field name="_version_" type="long" indexed="true" stored="true" /
>> >> <!--Dynamic fields definition-->
>> >> <dynamicField name="src_*" type="string" indexed="true" stored="true"
>> >> multiValued="false" />
>> >> <dynamicField name="dinamic_*" type="string" indexed="true"
>> stored="true"
>> >> multiValued="false" />
>> >> <!--Copy fields-->
>> >> <copyField source="src_*" dest="dinamic_*"/>
>> >>
>> >>
>> >> I add a document with this command:
>> >>
>> >>
>> >>
>> >> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
>> >> "Content-Type: text/xml" --data-binary '<add><doc><field
>> >> name="id">1</field><field name="name" >paco</field><field
>> name="src_desc"
>> >> >friend of mine</field></doc></add>'
>> >>
>> >>
>> >>
>> >> And later I update the field 'name' with this command:
>> >>
>> >>
>> >>
>> >> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
>> >> "Content-Type: text/xml" --data-binary '<add><doc><field
>> >> name="id">1</field><field name="name"
>> >> update="set">paquico</field></doc></add>'
>> >>
>> >>
>> >>
>> >> As I do so the doc i retrive from Solr is:
>> >>
>> >>
>> >>
>> >> <result name="response" numFound="1" start="0" maxScore="1.0">
>> >>  <doc>
>> >>   <str name="id">1</str>
>> >>   <str name="name">paquico</str>
>> >>   <str name="src_desc">friend of mine</str>
>> >>   <arr name="dinamic_desc">
>> >>    <str>friend of mine</str>
>> >>    <str>friend of mine</str>
>> >>   </arr>
>> >>   <long name="_version_">1506750859550130176</long>
>> >>   <float name="score">1.0</float>
>> >>  </doc>
>> >> </result>
>> >>
>> >>
>> >>
>> >> So I get a non multivalued field (dinamic_desc) with multiple values :(
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> ________________________________
>> >> Este correo electrónico y, en su caso, cualquier fichero anexo al
>> mismo,
>> >> contiene información de carácter confidencial exclusivamente dirigida
>> a su
>> >> destinatario o destinatarios. Si no es vd. el destinatario indicado,
>> queda
>> >> notificado que la lectura, utilización, divulgación y/o copia sin
>> >> autorización está prohibida en virtud de la legislación vigente. En el
>> caso
>> >> de haber recibido este correo electrónico por error, se ruega notificar
>> >> inmediatamente esta circunstancia mediante reenvío a la dirección
>> >> electrónica del remitente.
>> >> Evite imprimir este mensaje si no es estrictamente necesario.
>> >>
>> >> This email and any file attached to it (when applicable) contain(s)
>> >> confidential information that is exclusively addressed to its
>> recipient(s).
>> >> If you are not the indicated recipient, you are informed that reading,
>> >> using, disseminating and/or copying it without authorisation is
>> forbidden
>> >> in accordance with the legislation in effect. If you have received this
>> >> email by mistake, please immediately notify the sender of the
>> situation by
>> >> resending it to their email address.
>> >> Avoid printing this message if it is not absolutely necessary.
>> >>
>> >
>> >
>> >
>> > --
>> > --------------------------
>> >
>> > Benedetti Alessandro
>> > Visiting card - http://about.me/alessandro_benedetti
>> > Blog - http://alexbenedetti.blogspot.co.uk
>> >
>> > "Tyger, tyger burning bright
>> > In the forests of the night,
>> > What immortal hand or eye
>> > Could frame thy fearful symmetry?"
>> >
>> > William Blake - Songs of Experience -1794 England
>> >
>>
>>
>>
>> --
>> --------------------------
>>
>> Benedetti Alessandro
>> Visiting card - http://about.me/alessandro_benedetti
>> Blog - http://alexbenedetti.blogspot.co.uk
>>
>> "Tyger, tyger burning bright
>> In the forests of the night,
>> What immortal hand or eye
>> Could frame thy fearful symmetry?"
>>
>> William Blake - Songs of Experience -1794 England
>>
>> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo,
>> contiene información de carácter confidencial exclusivamente dirigida a su
>> destinatario o destinatarios. Si no es vd. el destinatario indicado, queda
>> notificado que la lectura, utilización, divulgación y/o copia sin
>> autorización está prohibida en virtud de la legislación vigente. En el caso
>> de haber recibido este correo electrónico por error, se ruega notificar
>> inmediatamente esta circunstancia mediante reenvío a la dirección
>> electrónica del remitente.
>> Evite imprimir este mensaje si no es estrictamente necesario.
>>
>> This email and any file attached to it (when applicable) contain(s)
>> confidential information that is exclusively addressed to its recipient(s).
>> If you are not the indicated recipient, you are informed that reading,
>> using, disseminating and/or copying it without authorisation is forbidden
>> in accordance with the legislation in effect. If you have received this
>> email by mistake, please immediately notify the sender of the situation by
>> resending it to their email address.
>> Avoid printing this message if it is not absolutely necessary.
>>
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>



--
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Este correo electrónico y, en su caso, cualquier fichero anexo al mismo, contiene información de carácter confidencial exclusivamente dirigida a su destinatario o destinatarios. Si no es vd. el destinatario indicado, queda notificado que la lectura, utilización, divulgación y/o copia sin autorización está prohibida en virtud de la legislación vigente. En el caso de haber recibido este correo electrónico por error, se ruega notificar inmediatamente esta circunstancia mediante reenvío a la dirección electrónica del remitente.
Evite imprimir este mensaje si no es estrictamente necesario.

This email and any file attached to it (when applicable) contain(s) confidential information that is exclusively addressed to its recipient(s). If you are not the indicated recipient, you are informed that reading, using, disseminating and/or copying it without authorisation is forbidden in accordance with the legislation in effect. If you have received this email by mistake, please immediately notify the sender of the situation by resending it to their email address.
Avoid printing this message if it is not absolutely necessary.

Re: Does update field feature work in a schema with dynamic fields?

Posted by Alessandro Benedetti <be...@gmail.com>.
Going through the code in the RunUpdateRequestProcessor we call at one
point :

…

Document luceneDocument = cmd.getLuceneDocument();
// SolrCore.verbose("updateDocument",updateTerm,luceneDocument,writer);
writer.updateDocument(updateTerm, luceneDocument);

..


Inside that method we call :

public Document getLuceneDocument() {
  return DocumentBuilder.toDocument(getSolrInputDocument(), req.getSchema());
}


Then exploring the toDocument we find what we need :

org/apache/solr/update/DocumentBuilder.java:114

And looking into there we realise it is a bug :

…

if (!destinationField.multiValued() && destHasValues) {
  throw new SolrException(SolrException.ErrorCode.BAD_REQUEST,
          "ERROR: "+getID(doc, schema)+"multiple values encountered
for non multiValued copy field " +
                  destinationField.getName() + ": " + v);
}

...


Because the check is actually checking in already used fields, not
expecting  the copy field to be already valorised.

Hence the exception coming.

I will create an issue for that and provide a patch as soon as I have
time ( or anyone can provide the patch).


Cheers






2015-07-15 12:27 GMT+01:00 Alessandro Benedetti <be...@gmail.com>
:

> Ohhh!
> I didn't read it completely, so i missed the copy field.
> Ok now.
> This is the explanation :
> Copy fields are added at indexing time, when the document arrived to the
> RunUpdateRequest processor.
> If i remember well at this point , before we start the indexing the
> content of source field is added to the copy field as a value.
>
> The first time you indexed your document , the first copy was added.
>
> What you didn't know is the fact that actually atomic update works in this
> way :
>
> 1) I get the current Doc from the index ( the stored fields),
> 2) I do the update, and
> 3) then I send the document to the indexing processing chain *again*.
> So the value is copied a second time.
>
> This will produce the duplicate value.
> I can go in deep, but I think this is the cause.
>
> Cheers
>
>
> 2015-07-15 12:10 GMT+01:00 Martínez López, Alfonso <al...@indra.es>:
>
>> Hi, thanks for your help!
>>
>> Value for 'dinamic_desc' field come from 'src_desc' field. I copy the
>> value with:
>>
>> <copyField source="src_*" dest="dinamic_*"/>
>>
>> Seems like when I update a different field (field 'name') via atomic
>> update, the copyField directive copies the value again from 'src_desc' to
>> 'desc_field', instead of updating the value, like if 'desc_field' or
>> 'desc_*' where multivalued.
>>
>> Cheers.
>> ________________________________________
>> From: Alessandro Benedetti [benedetti.alex85@gmail.com]
>> Sent: Wednesday, July 15, 2015 12:56 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Does update field feature work in a schema with dynamic
>> fields?
>>
>> Just tried, on Solr 5.1 and I get the proper behaviour.
>>
>> Actually where is the value for the dinamic_desc coming from ?
>>
>> I can not see it in the updates and actually it is not in my index.
>> Are you sure you have not forgotten any detail ?
>>
>> Cheers
>>
>> 2015-07-15 11:48 GMT+01:00 Alessandro Benedetti <
>> benedetti.alex85@gmail.com>
>> :
>>
>> > This is kinda weird and looks a lot like a bug.
>> > Let me try to reproduce it locally!
>> > I let you know soon !
>> >
>> > Cheers
>> >
>> > 2015-07-15 10:01 GMT+01:00 Martínez López, Alfonso <al...@indra.es>:
>> >
>> >> Hi,
>> >>
>> >> i'm using Solr 4.10.3, and i'm trying update a doc field using atomic
>> >> update (http://wiki.apache.org/solr/Atomic_Updates).
>> >>
>> >> My schema.xml is like this:
>> >>
>> >> <!--Fields-->
>> >> <field name="id" type="string" indexed="true" stored="true"
>> >> required="true" />
>> >> <field name="name" type="string" indexed="true" stored="true" />
>> >> <field name="src_desc" type="string" indexed="true" stored="true" />
>> >> <field name="_version_" type="long" indexed="true" stored="true" /
>> >> <!--Dynamic fields definition-->
>> >> <dynamicField name="src_*" type="string" indexed="true" stored="true"
>> >> multiValued="false" />
>> >> <dynamicField name="dinamic_*" type="string" indexed="true"
>> stored="true"
>> >> multiValued="false" />
>> >> <!--Copy fields-->
>> >> <copyField source="src_*" dest="dinamic_*"/>
>> >>
>> >>
>> >> I add a document with this command:
>> >>
>> >>
>> >>
>> >> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
>> >> "Content-Type: text/xml" --data-binary '<add><doc><field
>> >> name="id">1</field><field name="name" >paco</field><field
>> name="src_desc"
>> >> >friend of mine</field></doc></add>'
>> >>
>> >>
>> >>
>> >> And later I update the field 'name' with this command:
>> >>
>> >>
>> >>
>> >> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
>> >> "Content-Type: text/xml" --data-binary '<add><doc><field
>> >> name="id">1</field><field name="name"
>> >> update="set">paquico</field></doc></add>'
>> >>
>> >>
>> >>
>> >> As I do so the doc i retrive from Solr is:
>> >>
>> >>
>> >>
>> >> <result name="response" numFound="1" start="0" maxScore="1.0">
>> >>  <doc>
>> >>   <str name="id">1</str>
>> >>   <str name="name">paquico</str>
>> >>   <str name="src_desc">friend of mine</str>
>> >>   <arr name="dinamic_desc">
>> >>    <str>friend of mine</str>
>> >>    <str>friend of mine</str>
>> >>   </arr>
>> >>   <long name="_version_">1506750859550130176</long>
>> >>   <float name="score">1.0</float>
>> >>  </doc>
>> >> </result>
>> >>
>> >>
>> >>
>> >> So I get a non multivalued field (dinamic_desc) with multiple values :(
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> ________________________________
>> >> Este correo electrónico y, en su caso, cualquier fichero anexo al
>> mismo,
>> >> contiene información de carácter confidencial exclusivamente dirigida
>> a su
>> >> destinatario o destinatarios. Si no es vd. el destinatario indicado,
>> queda
>> >> notificado que la lectura, utilización, divulgación y/o copia sin
>> >> autorización está prohibida en virtud de la legislación vigente. En el
>> caso
>> >> de haber recibido este correo electrónico por error, se ruega notificar
>> >> inmediatamente esta circunstancia mediante reenvío a la dirección
>> >> electrónica del remitente.
>> >> Evite imprimir este mensaje si no es estrictamente necesario.
>> >>
>> >> This email and any file attached to it (when applicable) contain(s)
>> >> confidential information that is exclusively addressed to its
>> recipient(s).
>> >> If you are not the indicated recipient, you are informed that reading,
>> >> using, disseminating and/or copying it without authorisation is
>> forbidden
>> >> in accordance with the legislation in effect. If you have received this
>> >> email by mistake, please immediately notify the sender of the
>> situation by
>> >> resending it to their email address.
>> >> Avoid printing this message if it is not absolutely necessary.
>> >>
>> >
>> >
>> >
>> > --
>> > --------------------------
>> >
>> > Benedetti Alessandro
>> > Visiting card - http://about.me/alessandro_benedetti
>> > Blog - http://alexbenedetti.blogspot.co.uk
>> >
>> > "Tyger, tyger burning bright
>> > In the forests of the night,
>> > What immortal hand or eye
>> > Could frame thy fearful symmetry?"
>> >
>> > William Blake - Songs of Experience -1794 England
>> >
>>
>>
>>
>> --
>> --------------------------
>>
>> Benedetti Alessandro
>> Visiting card - http://about.me/alessandro_benedetti
>> Blog - http://alexbenedetti.blogspot.co.uk
>>
>> "Tyger, tyger burning bright
>> In the forests of the night,
>> What immortal hand or eye
>> Could frame thy fearful symmetry?"
>>
>> William Blake - Songs of Experience -1794 England
>>
>> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo,
>> contiene información de carácter confidencial exclusivamente dirigida a su
>> destinatario o destinatarios. Si no es vd. el destinatario indicado, queda
>> notificado que la lectura, utilización, divulgación y/o copia sin
>> autorización está prohibida en virtud de la legislación vigente. En el caso
>> de haber recibido este correo electrónico por error, se ruega notificar
>> inmediatamente esta circunstancia mediante reenvío a la dirección
>> electrónica del remitente.
>> Evite imprimir este mensaje si no es estrictamente necesario.
>>
>> This email and any file attached to it (when applicable) contain(s)
>> confidential information that is exclusively addressed to its recipient(s).
>> If you are not the indicated recipient, you are informed that reading,
>> using, disseminating and/or copying it without authorisation is forbidden
>> in accordance with the legislation in effect. If you have received this
>> email by mistake, please immediately notify the sender of the situation by
>> resending it to their email address.
>> Avoid printing this message if it is not absolutely necessary.
>>
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>



-- 
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Does update field feature work in a schema with dynamic fields?

Posted by Alessandro Benedetti <be...@gmail.com>.
Ohhh!
I didn't read it completely, so i missed the copy field.
Ok now.
This is the explanation :
Copy fields are added at indexing time, when the document arrived to the
RunUpdateRequest processor.
If i remember well at this point , before we start the indexing the content
of source field is added to the copy field as a value.

The first time you indexed your document , the first copy was added.

What you didn't know is the fact that actually atomic update works in this
way :

1) I get the current Doc from the index ( the stored fields),
2) I do the update, and
3) then I send the document to the indexing processing chain *again*.
So the value is copied a second time.

This will produce the duplicate value.
I can go in deep, but I think this is the cause.

Cheers


2015-07-15 12:10 GMT+01:00 Martínez López, Alfonso <al...@indra.es>:

> Hi, thanks for your help!
>
> Value for 'dinamic_desc' field come from 'src_desc' field. I copy the
> value with:
>
> <copyField source="src_*" dest="dinamic_*"/>
>
> Seems like when I update a different field (field 'name') via atomic
> update, the copyField directive copies the value again from 'src_desc' to
> 'desc_field', instead of updating the value, like if 'desc_field' or
> 'desc_*' where multivalued.
>
> Cheers.
> ________________________________________
> From: Alessandro Benedetti [benedetti.alex85@gmail.com]
> Sent: Wednesday, July 15, 2015 12:56 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Does update field feature work in a schema with dynamic
> fields?
>
> Just tried, on Solr 5.1 and I get the proper behaviour.
>
> Actually where is the value for the dinamic_desc coming from ?
>
> I can not see it in the updates and actually it is not in my index.
> Are you sure you have not forgotten any detail ?
>
> Cheers
>
> 2015-07-15 11:48 GMT+01:00 Alessandro Benedetti <
> benedetti.alex85@gmail.com>
> :
>
> > This is kinda weird and looks a lot like a bug.
> > Let me try to reproduce it locally!
> > I let you know soon !
> >
> > Cheers
> >
> > 2015-07-15 10:01 GMT+01:00 Martínez López, Alfonso <al...@indra.es>:
> >
> >> Hi,
> >>
> >> i'm using Solr 4.10.3, and i'm trying update a doc field using atomic
> >> update (http://wiki.apache.org/solr/Atomic_Updates).
> >>
> >> My schema.xml is like this:
> >>
> >> <!--Fields-->
> >> <field name="id" type="string" indexed="true" stored="true"
> >> required="true" />
> >> <field name="name" type="string" indexed="true" stored="true" />
> >> <field name="src_desc" type="string" indexed="true" stored="true" />
> >> <field name="_version_" type="long" indexed="true" stored="true" /
> >> <!--Dynamic fields definition-->
> >> <dynamicField name="src_*" type="string" indexed="true" stored="true"
> >> multiValued="false" />
> >> <dynamicField name="dinamic_*" type="string" indexed="true"
> stored="true"
> >> multiValued="false" />
> >> <!--Copy fields-->
> >> <copyField source="src_*" dest="dinamic_*"/>
> >>
> >>
> >> I add a document with this command:
> >>
> >>
> >>
> >> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
> >> "Content-Type: text/xml" --data-binary '<add><doc><field
> >> name="id">1</field><field name="name" >paco</field><field
> name="src_desc"
> >> >friend of mine</field></doc></add>'
> >>
> >>
> >>
> >> And later I update the field 'name' with this command:
> >>
> >>
> >>
> >> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
> >> "Content-Type: text/xml" --data-binary '<add><doc><field
> >> name="id">1</field><field name="name"
> >> update="set">paquico</field></doc></add>'
> >>
> >>
> >>
> >> As I do so the doc i retrive from Solr is:
> >>
> >>
> >>
> >> <result name="response" numFound="1" start="0" maxScore="1.0">
> >>  <doc>
> >>   <str name="id">1</str>
> >>   <str name="name">paquico</str>
> >>   <str name="src_desc">friend of mine</str>
> >>   <arr name="dinamic_desc">
> >>    <str>friend of mine</str>
> >>    <str>friend of mine</str>
> >>   </arr>
> >>   <long name="_version_">1506750859550130176</long>
> >>   <float name="score">1.0</float>
> >>  </doc>
> >> </result>
> >>
> >>
> >>
> >> So I get a non multivalued field (dinamic_desc) with multiple values :(
> >>
> >>
> >>
> >>
> >>
> >> ________________________________
> >> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo,
> >> contiene información de carácter confidencial exclusivamente dirigida a
> su
> >> destinatario o destinatarios. Si no es vd. el destinatario indicado,
> queda
> >> notificado que la lectura, utilización, divulgación y/o copia sin
> >> autorización está prohibida en virtud de la legislación vigente. En el
> caso
> >> de haber recibido este correo electrónico por error, se ruega notificar
> >> inmediatamente esta circunstancia mediante reenvío a la dirección
> >> electrónica del remitente.
> >> Evite imprimir este mensaje si no es estrictamente necesario.
> >>
> >> This email and any file attached to it (when applicable) contain(s)
> >> confidential information that is exclusively addressed to its
> recipient(s).
> >> If you are not the indicated recipient, you are informed that reading,
> >> using, disseminating and/or copying it without authorisation is
> forbidden
> >> in accordance with the legislation in effect. If you have received this
> >> email by mistake, please immediately notify the sender of the situation
> by
> >> resending it to their email address.
> >> Avoid printing this message if it is not absolutely necessary.
> >>
> >
> >
> >
> > --
> > --------------------------
> >
> > Benedetti Alessandro
> > Visiting card - http://about.me/alessandro_benedetti
> > Blog - http://alexbenedetti.blogspot.co.uk
> >
> > "Tyger, tyger burning bright
> > In the forests of the night,
> > What immortal hand or eye
> > Could frame thy fearful symmetry?"
> >
> > William Blake - Songs of Experience -1794 England
> >
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>
> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo,
> contiene información de carácter confidencial exclusivamente dirigida a su
> destinatario o destinatarios. Si no es vd. el destinatario indicado, queda
> notificado que la lectura, utilización, divulgación y/o copia sin
> autorización está prohibida en virtud de la legislación vigente. En el caso
> de haber recibido este correo electrónico por error, se ruega notificar
> inmediatamente esta circunstancia mediante reenvío a la dirección
> electrónica del remitente.
> Evite imprimir este mensaje si no es estrictamente necesario.
>
> This email and any file attached to it (when applicable) contain(s)
> confidential information that is exclusively addressed to its recipient(s).
> If you are not the indicated recipient, you are informed that reading,
> using, disseminating and/or copying it without authorisation is forbidden
> in accordance with the legislation in effect. If you have received this
> email by mistake, please immediately notify the sender of the situation by
> resending it to their email address.
> Avoid printing this message if it is not absolutely necessary.
>



-- 
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

RE: Does update field feature work in a schema with dynamic fields?

Posted by Martínez, Alfonso <al...@indra.es>.
Hi, thanks for your help!

Value for 'dinamic_desc' field come from 'src_desc' field. I copy the value with:

<copyField source="src_*" dest="dinamic_*"/>

Seems like when I update a different field (field 'name') via atomic update, the copyField directive copies the value again from 'src_desc' to 'desc_field', instead of updating the value, like if 'desc_field' or 'desc_*' where multivalued.

Cheers.
________________________________________
From: Alessandro Benedetti [benedetti.alex85@gmail.com]
Sent: Wednesday, July 15, 2015 12:56 PM
To: solr-user@lucene.apache.org
Subject: Re: Does update field feature work in a schema with dynamic fields?

Just tried, on Solr 5.1 and I get the proper behaviour.

Actually where is the value for the dinamic_desc coming from ?

I can not see it in the updates and actually it is not in my index.
Are you sure you have not forgotten any detail ?

Cheers

2015-07-15 11:48 GMT+01:00 Alessandro Benedetti <be...@gmail.com>
:

> This is kinda weird and looks a lot like a bug.
> Let me try to reproduce it locally!
> I let you know soon !
>
> Cheers
>
> 2015-07-15 10:01 GMT+01:00 Martínez López, Alfonso <al...@indra.es>:
>
>> Hi,
>>
>> i'm using Solr 4.10.3, and i'm trying update a doc field using atomic
>> update (http://wiki.apache.org/solr/Atomic_Updates).
>>
>> My schema.xml is like this:
>>
>> <!--Fields-->
>> <field name="id" type="string" indexed="true" stored="true"
>> required="true" />
>> <field name="name" type="string" indexed="true" stored="true" />
>> <field name="src_desc" type="string" indexed="true" stored="true" />
>> <field name="_version_" type="long" indexed="true" stored="true" /
>> <!--Dynamic fields definition-->
>> <dynamicField name="src_*" type="string" indexed="true" stored="true"
>> multiValued="false" />
>> <dynamicField name="dinamic_*" type="string" indexed="true" stored="true"
>> multiValued="false" />
>> <!--Copy fields-->
>> <copyField source="src_*" dest="dinamic_*"/>
>>
>>
>> I add a document with this command:
>>
>>
>>
>> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
>> "Content-Type: text/xml" --data-binary '<add><doc><field
>> name="id">1</field><field name="name" >paco</field><field name="src_desc"
>> >friend of mine</field></doc></add>'
>>
>>
>>
>> And later I update the field 'name' with this command:
>>
>>
>>
>> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
>> "Content-Type: text/xml" --data-binary '<add><doc><field
>> name="id">1</field><field name="name"
>> update="set">paquico</field></doc></add>'
>>
>>
>>
>> As I do so the doc i retrive from Solr is:
>>
>>
>>
>> <result name="response" numFound="1" start="0" maxScore="1.0">
>>  <doc>
>>   <str name="id">1</str>
>>   <str name="name">paquico</str>
>>   <str name="src_desc">friend of mine</str>
>>   <arr name="dinamic_desc">
>>    <str>friend of mine</str>
>>    <str>friend of mine</str>
>>   </arr>
>>   <long name="_version_">1506750859550130176</long>
>>   <float name="score">1.0</float>
>>  </doc>
>> </result>
>>
>>
>>
>> So I get a non multivalued field (dinamic_desc) with multiple values :(
>>
>>
>>
>>
>>
>> ________________________________
>> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo,
>> contiene información de carácter confidencial exclusivamente dirigida a su
>> destinatario o destinatarios. Si no es vd. el destinatario indicado, queda
>> notificado que la lectura, utilización, divulgación y/o copia sin
>> autorización está prohibida en virtud de la legislación vigente. En el caso
>> de haber recibido este correo electrónico por error, se ruega notificar
>> inmediatamente esta circunstancia mediante reenvío a la dirección
>> electrónica del remitente.
>> Evite imprimir este mensaje si no es estrictamente necesario.
>>
>> This email and any file attached to it (when applicable) contain(s)
>> confidential information that is exclusively addressed to its recipient(s).
>> If you are not the indicated recipient, you are informed that reading,
>> using, disseminating and/or copying it without authorisation is forbidden
>> in accordance with the legislation in effect. If you have received this
>> email by mistake, please immediately notify the sender of the situation by
>> resending it to their email address.
>> Avoid printing this message if it is not absolutely necessary.
>>
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>



--
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Este correo electrónico y, en su caso, cualquier fichero anexo al mismo, contiene información de carácter confidencial exclusivamente dirigida a su destinatario o destinatarios. Si no es vd. el destinatario indicado, queda notificado que la lectura, utilización, divulgación y/o copia sin autorización está prohibida en virtud de la legislación vigente. En el caso de haber recibido este correo electrónico por error, se ruega notificar inmediatamente esta circunstancia mediante reenvío a la dirección electrónica del remitente.
Evite imprimir este mensaje si no es estrictamente necesario.

This email and any file attached to it (when applicable) contain(s) confidential information that is exclusively addressed to its recipient(s). If you are not the indicated recipient, you are informed that reading, using, disseminating and/or copying it without authorisation is forbidden in accordance with the legislation in effect. If you have received this email by mistake, please immediately notify the sender of the situation by resending it to their email address.
Avoid printing this message if it is not absolutely necessary.

Re: Does update field feature work in a schema with dynamic fields?

Posted by Alessandro Benedetti <be...@gmail.com>.
Just tried, on Solr 5.1 and I get the proper behaviour.

Actually where is the value for the dinamic_desc coming from ?

I can not see it in the updates and actually it is not in my index.
Are you sure you have not forgotten any detail ?

Cheers

2015-07-15 11:48 GMT+01:00 Alessandro Benedetti <be...@gmail.com>
:

> This is kinda weird and looks a lot like a bug.
> Let me try to reproduce it locally!
> I let you know soon !
>
> Cheers
>
> 2015-07-15 10:01 GMT+01:00 Martínez López, Alfonso <al...@indra.es>:
>
>> Hi,
>>
>> i'm using Solr 4.10.3, and i'm trying update a doc field using atomic
>> update (http://wiki.apache.org/solr/Atomic_Updates).
>>
>> My schema.xml is like this:
>>
>> <!--Fields-->
>> <field name="id" type="string" indexed="true" stored="true"
>> required="true" />
>> <field name="name" type="string" indexed="true" stored="true" />
>> <field name="src_desc" type="string" indexed="true" stored="true" />
>> <field name="_version_" type="long" indexed="true" stored="true" /
>> <!--Dynamic fields definition-->
>> <dynamicField name="src_*" type="string" indexed="true" stored="true"
>> multiValued="false" />
>> <dynamicField name="dinamic_*" type="string" indexed="true" stored="true"
>> multiValued="false" />
>> <!--Copy fields-->
>> <copyField source="src_*" dest="dinamic_*"/>
>>
>>
>> I add a document with this command:
>>
>>
>>
>> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
>> "Content-Type: text/xml" --data-binary '<add><doc><field
>> name="id">1</field><field name="name" >paco</field><field name="src_desc"
>> >friend of mine</field></doc></add>'
>>
>>
>>
>> And later I update the field 'name' with this command:
>>
>>
>>
>> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
>> "Content-Type: text/xml" --data-binary '<add><doc><field
>> name="id">1</field><field name="name"
>> update="set">paquico</field></doc></add>'
>>
>>
>>
>> As I do so the doc i retrive from Solr is:
>>
>>
>>
>> <result name="response" numFound="1" start="0" maxScore="1.0">
>>  <doc>
>>   <str name="id">1</str>
>>   <str name="name">paquico</str>
>>   <str name="src_desc">friend of mine</str>
>>   <arr name="dinamic_desc">
>>    <str>friend of mine</str>
>>    <str>friend of mine</str>
>>   </arr>
>>   <long name="_version_">1506750859550130176</long>
>>   <float name="score">1.0</float>
>>  </doc>
>> </result>
>>
>>
>>
>> So I get a non multivalued field (dinamic_desc) with multiple values :(
>>
>>
>>
>>
>>
>> ________________________________
>> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo,
>> contiene información de carácter confidencial exclusivamente dirigida a su
>> destinatario o destinatarios. Si no es vd. el destinatario indicado, queda
>> notificado que la lectura, utilización, divulgación y/o copia sin
>> autorización está prohibida en virtud de la legislación vigente. En el caso
>> de haber recibido este correo electrónico por error, se ruega notificar
>> inmediatamente esta circunstancia mediante reenvío a la dirección
>> electrónica del remitente.
>> Evite imprimir este mensaje si no es estrictamente necesario.
>>
>> This email and any file attached to it (when applicable) contain(s)
>> confidential information that is exclusively addressed to its recipient(s).
>> If you are not the indicated recipient, you are informed that reading,
>> using, disseminating and/or copying it without authorisation is forbidden
>> in accordance with the legislation in effect. If you have received this
>> email by mistake, please immediately notify the sender of the situation by
>> resending it to their email address.
>> Avoid printing this message if it is not absolutely necessary.
>>
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card - http://about.me/alessandro_benedetti
> Blog - http://alexbenedetti.blogspot.co.uk
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>



-- 
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Does update field feature work in a schema with dynamic fields?

Posted by Alessandro Benedetti <be...@gmail.com>.
This is kinda weird and looks a lot like a bug.
Let me try to reproduce it locally!
I let you know soon !

Cheers

2015-07-15 10:01 GMT+01:00 Martínez López, Alfonso <al...@indra.es>:

> Hi,
>
> i'm using Solr 4.10.3, and i'm trying update a doc field using atomic
> update (http://wiki.apache.org/solr/Atomic_Updates).
>
> My schema.xml is like this:
>
> <!--Fields-->
> <field name="id" type="string" indexed="true" stored="true"
> required="true" />
> <field name="name" type="string" indexed="true" stored="true" />
> <field name="src_desc" type="string" indexed="true" stored="true" />
> <field name="_version_" type="long" indexed="true" stored="true" /
> <!--Dynamic fields definition-->
> <dynamicField name="src_*" type="string" indexed="true" stored="true"
> multiValued="false" />
> <dynamicField name="dinamic_*" type="string" indexed="true" stored="true"
> multiValued="false" />
> <!--Copy fields-->
> <copyField source="src_*" dest="dinamic_*"/>
>
>
> I add a document with this command:
>
>
>
> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
> "Content-Type: text/xml" --data-binary '<add><doc><field
> name="id">1</field><field name="name" >paco</field><field name="src_desc"
> >friend of mine</field></doc></add>'
>
>
>
> And later I update the field 'name' with this command:
>
>
>
> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
> "Content-Type: text/xml" --data-binary '<add><doc><field
> name="id">1</field><field name="name"
> update="set">paquico</field></doc></add>'
>
>
>
> As I do so the doc i retrive from Solr is:
>
>
>
> <result name="response" numFound="1" start="0" maxScore="1.0">
>  <doc>
>   <str name="id">1</str>
>   <str name="name">paquico</str>
>   <str name="src_desc">friend of mine</str>
>   <arr name="dinamic_desc">
>    <str>friend of mine</str>
>    <str>friend of mine</str>
>   </arr>
>   <long name="_version_">1506750859550130176</long>
>   <float name="score">1.0</float>
>  </doc>
> </result>
>
>
>
> So I get a non multivalued field (dinamic_desc) with multiple values :(
>
>
>
>
>
> ________________________________
> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo,
> contiene información de carácter confidencial exclusivamente dirigida a su
> destinatario o destinatarios. Si no es vd. el destinatario indicado, queda
> notificado que la lectura, utilización, divulgación y/o copia sin
> autorización está prohibida en virtud de la legislación vigente. En el caso
> de haber recibido este correo electrónico por error, se ruega notificar
> inmediatamente esta circunstancia mediante reenvío a la dirección
> electrónica del remitente.
> Evite imprimir este mensaje si no es estrictamente necesario.
>
> This email and any file attached to it (when applicable) contain(s)
> confidential information that is exclusively addressed to its recipient(s).
> If you are not the indicated recipient, you are informed that reading,
> using, disseminating and/or copying it without authorisation is forbidden
> in accordance with the legislation in effect. If you have received this
> email by mistake, please immediately notify the sender of the situation by
> resending it to their email address.
> Avoid printing this message if it is not absolutely necessary.
>



-- 
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Does update field feature work in a schema with dynamic fields?

Posted by Alessandro Benedetti <be...@gmail.com>.
Sorry Erick, I completely agree with you, I didn;'t specify in details what
I was thinking :

" copy fields must not be executed if the updated field is not a source
field ( in a copy field couple) "

furthermore I agree again with you, copy field should try to give a
different analysis to the new field, simply changing the name would be
matter of field analysis. Nothing required at indexing time !

Cheers


2015-07-15 16:26 GMT+01:00 Erick Erickson <er...@gmail.com>:

> Alfonso:
>
> Haven't worked with this myself, but could "field aliasing" handle your
> use-case
> _without_ the need for a copyField at all?
> See: https://issues.apache.org/jira/browse/SOLR-1205
>
> Again I need to emphasize that I HAVE NOT worked with this so it may be
> a really bad suggestion. Or it may not apply. Or.....
>
> Alessandro:
>
> You wrote: "The simple solution should be to not execute any copy field
> instruction in an updatedDocument."
>
> I don't think that would work as then legitimate copyFields wouldn't
> be reflected.
> Say you had a copyField from A to B. Now you atomically update A. That
> update
> should be reflected in B...
>
> Best,
> Erick
>
> On Wed, Jul 15, 2015 at 8:19 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> > On 7/15/2015 8:55 AM, Martínez López, Alfonso wrote:
> >> in some cases it can be necessary to have the copy field stored. My
> Solr instance is used by some legacy applications that need to retrive
> fields by some especific field names. That's why i need to mantain 2 copies
> of the same field: one with the old name and other for the new name (that
> is provided by others applications).
> >>
> >> In my case I could work this out at a higher lever but it would be very
> helpful it can be solve in the schema.xml.
> >
> > This is understandable ... but unless a new feature is created, stored
> > copyField destinations are incompatible with Atomic Updates.
> >
> > If we do build a new feature, we'd need to figure out exactly how Solr
> > should behave by default, and possibly whether there should be a
> > configuration option to choose old or new behavior.
> >
> > It might be a good idea to have DistributedUpdateProcessor (or possibly
> > the AtomicUpdateDocumentMerger that it uses) skip the import of any
> > field that's a copyField destination by default, unless a configuration
> > is present that restores the old behavior.  The information about which
> > fields are copyField destinations *is* available to those classes, so it
> > would probably be a very easy change.
> >
> > There's not really a STRONG need for that new feature ... if you store
> > both the source and destination fields, then you are storing exactly the
> > same information twice, which is wasteful and increases resource
> > requirements.  Most applications that use Solr have at least surface
> > knowledge of the schema, which should make it possible for the
> > application writer to just pull the information from the source field.
> > It seems that your application may not have that knowledge, though.
> >
> > Thanks,
> > Shawn
> >
>



-- 
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Does update field feature work in a schema with dynamic fields?

Posted by Erick Erickson <er...@gmail.com>.
Alfonso:

Haven't worked with this myself, but could "field aliasing" handle your use-case
_without_ the need for a copyField at all?
See: https://issues.apache.org/jira/browse/SOLR-1205

Again I need to emphasize that I HAVE NOT worked with this so it may be
a really bad suggestion. Or it may not apply. Or.....

Alessandro:

You wrote: "The simple solution should be to not execute any copy field
instruction in an updatedDocument."

I don't think that would work as then legitimate copyFields wouldn't
be reflected.
Say you had a copyField from A to B. Now you atomically update A. That update
should be reflected in B...

Best,
Erick

On Wed, Jul 15, 2015 at 8:19 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 7/15/2015 8:55 AM, Martínez López, Alfonso wrote:
>> in some cases it can be necessary to have the copy field stored. My Solr instance is used by some legacy applications that need to retrive fields by some especific field names. That's why i need to mantain 2 copies of the same field: one with the old name and other for the new name (that is provided by others applications).
>>
>> In my case I could work this out at a higher lever but it would be very helpful it can be solve in the schema.xml.
>
> This is understandable ... but unless a new feature is created, stored
> copyField destinations are incompatible with Atomic Updates.
>
> If we do build a new feature, we'd need to figure out exactly how Solr
> should behave by default, and possibly whether there should be a
> configuration option to choose old or new behavior.
>
> It might be a good idea to have DistributedUpdateProcessor (or possibly
> the AtomicUpdateDocumentMerger that it uses) skip the import of any
> field that's a copyField destination by default, unless a configuration
> is present that restores the old behavior.  The information about which
> fields are copyField destinations *is* available to those classes, so it
> would probably be a very easy change.
>
> There's not really a STRONG need for that new feature ... if you store
> both the source and destination fields, then you are storing exactly the
> same information twice, which is wasteful and increases resource
> requirements.  Most applications that use Solr have at least surface
> knowledge of the schema, which should make it possible for the
> application writer to just pull the information from the source field.
> It seems that your application may not have that knowledge, though.
>
> Thanks,
> Shawn
>

Re: Does update field feature work in a schema with dynamic fields?

Posted by Shawn Heisey <ap...@elyograg.org>.
On 7/15/2015 8:55 AM, Martínez López, Alfonso wrote:
> in some cases it can be necessary to have the copy field stored. My Solr instance is used by some legacy applications that need to retrive fields by some especific field names. That's why i need to mantain 2 copies of the same field: one with the old name and other for the new name (that is provided by others applications).
>
> In my case I could work this out at a higher lever but it would be very helpful it can be solve in the schema.xml.

This is understandable ... but unless a new feature is created, stored
copyField destinations are incompatible with Atomic Updates.

If we do build a new feature, we'd need to figure out exactly how Solr
should behave by default, and possibly whether there should be a
configuration option to choose old or new behavior.

It might be a good idea to have DistributedUpdateProcessor (or possibly
the AtomicUpdateDocumentMerger that it uses) skip the import of any
field that's a copyField destination by default, unless a configuration
is present that restores the old behavior.  The information about which
fields are copyField destinations *is* available to those classes, so it
would probably be a very easy change.

There's not really a STRONG need for that new feature ... if you store
both the source and destination fields, then you are storing exactly the
same information twice, which is wasteful and increases resource
requirements.  Most applications that use Solr have at least surface
knowledge of the schema, which should make it possible for the
application writer to just pull the information from the source field. 
It seems that your application may not have that knowledge, though.

Thanks,
Shawn


RE: Does update field feature work in a schema with dynamic fields?

Posted by Martínez, Alfonso <al...@indra.es>.
Hi,
in some cases it can be necessary to have the copy field stored. My Solr instance is used by some legacy applications that need to retrive fields by some especific field names. That's why i need to mantain 2 copies of the same field: one with the old name and other for the new name (that is provided by others applications).

In my case I could work this out at a higher lever but it would be very helpful it can be solve in the schema.xml.
________________________________________
From: Alessandro Benedetti [benedetti.alex85@gmail.com]
Sent: Wednesday, July 15, 2015 4:39 PM
To: solr-user@lucene.apache.org
Subject: Re: Does update field feature work in a schema with dynamic fields?

Hey Shawn, I was debugging a little bit,this is the problem :

When adding a field from the solr document, to the Lucene one, even if this
document was previously added by the execution of the copy field
instruction to the Lucene Document, this check is carried :

org/apache/solr/update/DocumentBuilder.java:89

// Make sure it has the correct number
if( sfield!=null && !sfield.multiValued() && field.getValueCount() > 1 ) {
  throw new SolrException( SolrException.ErrorCode.BAD_REQUEST,
      "ERROR: "+getID(doc, schema)+"multiple values encountered for
non multiValued field " +
        sfield.getName() + ": " +field.getValue() );
}


This will actually check the Solr Document field.getValueCount(),
ignoring the fact that the field was already added to the Lucene
Document to be indexed.

So the field will be added again.

I understand that doesn't make a lot of sense to store a copy field (
nothing will change from the source field stored content).

But I guess we should try to avoid this anomalies.

The simple solution should be to not execute any copy field
instruction in an updatedDocument.

Unfortunately we don't have any signal in the DocumentBuilder at that
point, that will let us know if the Document is a new one or an
updated one.

What do you think?

Will be easy to have the Document builder aware if it is an add or an
update, and react ?

If not we need to think something else.


Cheers





2015-07-15 15:25 GMT+01:00 Shawn Heisey <ap...@elyograg.org>:

> On 7/15/2015 3:01 AM, Martínez López, Alfonso wrote:
> > <!--Fields-->
> > <field name="id" type="string" indexed="true" stored="true"
> required="true" />
> > <field name="name" type="string" indexed="true" stored="true" />
> > <field name="src_desc" type="string" indexed="true" stored="true" />
> > <field name="_version_" type="long" indexed="true" stored="true" /
> > <!--Dynamic fields definition-->
> > <dynamicField name="src_*" type="string" indexed="true" stored="true"
> multiValued="false" />
> > <dynamicField name="dinamic_*" type="string" indexed="true"
> stored="true" multiValued="false" />
> > <!--Copy fields-->
> > <copyField source="src_*" dest="dinamic_*"/>
>
> <snip>
>
> > And later I update the field 'name' with this command:
> >
> > curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
> "Content-Type: text/xml" --data-binary '<add><doc><field
> name="id">1</field><field name="name"
> update="set">paquico</field></doc></add>'
> >
> > As I do so the doc i retrive from Solr is:
> >
> > <result name="response" numFound="1" start="0" maxScore="1.0">
> >  <doc>
> >   <str name="id">1</str>
> >   <str name="name">paquico</str>
> >   <str name="src_desc">friend of mine</str>
> >   <arr name="dinamic_desc">
> >    <str>friend of mine</str>
> >    <str>friend of mine</str>
> >   </arr>
> >   <long name="_version_">1506750859550130176</long>
> >   <float name="score">1.0</float>
> >  </doc>
> > </result>
>
> The problem here is that the copyField destination is stored, so you get
> the original value of the destination field plus another copy from
> src_desc.
>
> If you look carefully at the "caveats and limitations" for Atomic
> Updates, you will see that all copyField destinations must be unstored
> for proper operation:
>
> https://wiki.apache.org/solr/Atomic_Updates#Caveats_and_Limitations
>
> It appears that this information was *NOT* in the Solr Reference Guide,
> so I updated the reference guide to include it.
>
>
> https://cwiki.apache.org/confluence/display/solr/Updating+Parts+of+Documents
>
> Here's a question for those with more expertise than me:  If a copyField
> destination were stored, but not multiValued, and an atomic update was
> attempted, would the update fail entirely?  I suspect it would, and I'd
> like to make the ref guide info as accurate as I can.
>
> Thanks,
> Shawn
>
>


--
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Este correo electrónico y, en su caso, cualquier fichero anexo al mismo, contiene información de carácter confidencial exclusivamente dirigida a su destinatario o destinatarios. Si no es vd. el destinatario indicado, queda notificado que la lectura, utilización, divulgación y/o copia sin autorización está prohibida en virtud de la legislación vigente. En el caso de haber recibido este correo electrónico por error, se ruega notificar inmediatamente esta circunstancia mediante reenvío a la dirección electrónica del remitente.
Evite imprimir este mensaje si no es estrictamente necesario.

This email and any file attached to it (when applicable) contain(s) confidential information that is exclusively addressed to its recipient(s). If you are not the indicated recipient, you are informed that reading, using, disseminating and/or copying it without authorisation is forbidden in accordance with the legislation in effect. If you have received this email by mistake, please immediately notify the sender of the situation by resending it to their email address.
Avoid printing this message if it is not absolutely necessary.

Re: Does update field feature work in a schema with dynamic fields?

Posted by Alessandro Benedetti <be...@gmail.com>.
Hey Shawn, I was debugging a little bit,this is the problem :

When adding a field from the solr document, to the Lucene one, even if this
document was previously added by the execution of the copy field
instruction to the Lucene Document, this check is carried :

org/apache/solr/update/DocumentBuilder.java:89

// Make sure it has the correct number
if( sfield!=null && !sfield.multiValued() && field.getValueCount() > 1 ) {
  throw new SolrException( SolrException.ErrorCode.BAD_REQUEST,
      "ERROR: "+getID(doc, schema)+"multiple values encountered for
non multiValued field " +
        sfield.getName() + ": " +field.getValue() );
}


This will actually check the Solr Document field.getValueCount(),
ignoring the fact that the field was already added to the Lucene
Document to be indexed.

So the field will be added again.

I understand that doesn't make a lot of sense to store a copy field (
nothing will change from the source field stored content).

But I guess we should try to avoid this anomalies.

The simple solution should be to not execute any copy field
instruction in an updatedDocument.

Unfortunately we don't have any signal in the DocumentBuilder at that
point, that will let us know if the Document is a new one or an
updated one.

What do you think?

Will be easy to have the Document builder aware if it is an add or an
update, and react ?

If not we need to think something else.


Cheers





2015-07-15 15:25 GMT+01:00 Shawn Heisey <ap...@elyograg.org>:

> On 7/15/2015 3:01 AM, Martínez López, Alfonso wrote:
> > <!--Fields-->
> > <field name="id" type="string" indexed="true" stored="true"
> required="true" />
> > <field name="name" type="string" indexed="true" stored="true" />
> > <field name="src_desc" type="string" indexed="true" stored="true" />
> > <field name="_version_" type="long" indexed="true" stored="true" /
> > <!--Dynamic fields definition-->
> > <dynamicField name="src_*" type="string" indexed="true" stored="true"
> multiValued="false" />
> > <dynamicField name="dinamic_*" type="string" indexed="true"
> stored="true" multiValued="false" />
> > <!--Copy fields-->
> > <copyField source="src_*" dest="dinamic_*"/>
>
> <snip>
>
> > And later I update the field 'name' with this command:
> >
> > curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H
> "Content-Type: text/xml" --data-binary '<add><doc><field
> name="id">1</field><field name="name"
> update="set">paquico</field></doc></add>'
> >
> > As I do so the doc i retrive from Solr is:
> >
> > <result name="response" numFound="1" start="0" maxScore="1.0">
> >  <doc>
> >   <str name="id">1</str>
> >   <str name="name">paquico</str>
> >   <str name="src_desc">friend of mine</str>
> >   <arr name="dinamic_desc">
> >    <str>friend of mine</str>
> >    <str>friend of mine</str>
> >   </arr>
> >   <long name="_version_">1506750859550130176</long>
> >   <float name="score">1.0</float>
> >  </doc>
> > </result>
>
> The problem here is that the copyField destination is stored, so you get
> the original value of the destination field plus another copy from
> src_desc.
>
> If you look carefully at the "caveats and limitations" for Atomic
> Updates, you will see that all copyField destinations must be unstored
> for proper operation:
>
> https://wiki.apache.org/solr/Atomic_Updates#Caveats_and_Limitations
>
> It appears that this information was *NOT* in the Solr Reference Guide,
> so I updated the reference guide to include it.
>
>
> https://cwiki.apache.org/confluence/display/solr/Updating+Parts+of+Documents
>
> Here's a question for those with more expertise than me:  If a copyField
> destination were stored, but not multiValued, and an atomic update was
> attempted, would the update fail entirely?  I suspect it would, and I'd
> like to make the ref guide info as accurate as I can.
>
> Thanks,
> Shawn
>
>


-- 
--------------------------

Benedetti Alessandro
Visiting card - http://about.me/alessandro_benedetti
Blog - http://alexbenedetti.blogspot.co.uk

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England

Re: Does update field feature work in a schema with dynamic fields?

Posted by Shawn Heisey <ap...@elyograg.org>.
On 7/15/2015 3:01 AM, Martínez López, Alfonso wrote:
> <!--Fields-->
> <field name="id" type="string" indexed="true" stored="true" required="true" />
> <field name="name" type="string" indexed="true" stored="true" />
> <field name="src_desc" type="string" indexed="true" stored="true" />
> <field name="_version_" type="long" indexed="true" stored="true" /
> <!--Dynamic fields definition-->
> <dynamicField name="src_*" type="string" indexed="true" stored="true" multiValued="false" />
> <dynamicField name="dinamic_*" type="string" indexed="true" stored="true" multiValued="false" />
> <!--Copy fields-->
> <copyField source="src_*" dest="dinamic_*"/>

<snip>

> And later I update the field 'name' with this command:
>
> curl http://<solr_host>:<sorl_port>/solr/default/update?commit=true -H "Content-Type: text/xml" --data-binary '<add><doc><field name="id">1</field><field name="name" update="set">paquico</field></doc></add>'
>
> As I do so the doc i retrive from Solr is:
>
> <result name="response" numFound="1" start="0" maxScore="1.0">
>  <doc>
>   <str name="id">1</str>
>   <str name="name">paquico</str>
>   <str name="src_desc">friend of mine</str>
>   <arr name="dinamic_desc">
>    <str>friend of mine</str>
>    <str>friend of mine</str>
>   </arr>
>   <long name="_version_">1506750859550130176</long>
>   <float name="score">1.0</float>
>  </doc>
> </result>

The problem here is that the copyField destination is stored, so you get
the original value of the destination field plus another copy from src_desc.

If you look carefully at the "caveats and limitations" for Atomic
Updates, you will see that all copyField destinations must be unstored
for proper operation:

https://wiki.apache.org/solr/Atomic_Updates#Caveats_and_Limitations

It appears that this information was *NOT* in the Solr Reference Guide,
so I updated the reference guide to include it.

https://cwiki.apache.org/confluence/display/solr/Updating+Parts+of+Documents

Here's a question for those with more expertise than me:  If a copyField
destination were stored, but not multiValued, and an atomic update was
attempted, would the update fail entirely?  I suspect it would, and I'd
like to make the ref guide info as accurate as I can.

Thanks,
Shawn