You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Da...@coop.ch on 2026/07/29 12:53:26 UTC

Partial Updates unclear Documentation

Dear Solr Team

What happens when you send a partial document update like the following?


{
 "id":"mydoc",
 "price":{"set":99}
}

but no document with the id "mydoc" exists at this moment.

Or what happens if the partial document update is missing the id field?

Or what happens if you send a mixed request, containing fields with modifiers and fields containing only the value?

{
 "id":"mydoc",
 "name": "product",
 "price":{"set":99}
}


I personally know that in the first case it acts as an upsert. so inserting the document if its missing, and updating it if it already  exists.
And in the second case it always inserts a new document, with its id being generated.

For the third case I am not entirely sure, but I think if there is at least one field in the request with a modifier the whole request acts as a partial document update and fields without modifiers are handled as if they have the modifier "set". Thats what the behaviour on solr 9.10.1 looks like at least.

The Documentation does not really explain any of the above cases clearly. This also leads to AI making very questionable statements that are just wrong.

https://share.google/aimode/rZE2udrRajAUNDD6y
Google just casually tells us that it only acts as an upsert but ONLY if the document already exists in the collection, which is wrong and also does not make any sense as that would not be an upsert.

https://chatgpt.com/s/t_6a69f66582dc81919da8e03451e267f1
Also Chatgpt is just wrong about it.

As far as I understand it, partial document updates were introduced in solr 4 and the upsert behaviour was introduced in solr 4.1. but as its not clearly documented I am not 100% sure.
https://stackoverflow.com/questions/14886814/can-solr-perform-an-upsert#comment21025920_14888083

Kind Regards


Re: Partial Updates unclear Documentation

Posted by Chris Hostetter <ho...@fucit.org>.
First off: I want to make sure every one is very clear that the entire 
premise of this thread is asking about behavior that is not officially 
supported or recommened -- that's why it's under-specified in the 
documentation. (in the same way that asking something like "what happens 
if i send a JSON query inside of a JSON update?" isn't specified)

What Dmitri is describing is the "actual behavior" given un-supported 
input -- Solr's behavior of (essentially) "do the best we can 
with what you've given us" (as opposed to a hard failure) is largely just 
due to quirks of how atomic updates are implemented, with a dash of "let's 
avoid breaking things for people who might have come to rely on this 
behavior"

By all means, feel free to submit a patch with a suggested doc update that 
says something like: "NOTE: mixing atomic updates with plain field updates 
is not recommended and may have unexpected behavior"



Second: This comment here still seems to be completley overlooking 
Dmitri's point about *existing* fields in the index...

: > So in your example, say doc with id “foo” has 10 fields in the index, then
...
: > { "id" : "foo", "bar" : { "set" : "baz" }, "bleh" : "schmeh" }
: 
: If what you say is true, then this is equivalent to a full upsert and the
: "partial update" instruction is redundant. My reading of the fine manual is

The presence of an atomic update modifier (set in this example) on any 
field(s) in the add document command, instructs solr that the *entire*
command should be treated as an atomic update.  All existing fields in the 
document (unless explicitly mentioned in the add document command) should 
be left alone.

Claiming "the 'partial update' instruction is redundant" implies a 
missunderstanding of the end state of these two sequences...

{ "id" : "1", "aaa" : "xxx", "bbb" : "bbb" }
{ "id" : "1", "aaa" : { "set":"yyy"} }

{ "id" : "2", "aaa" : "xxx", "bbb" : "bbb" }
{ "id" : "2", "aaa" : "yyy" }

in both cases, the document will be updated to use "aaa=yyy" but only doc 
#1 will keep it's "bbb=bbb"


Effectively: mixing in "plain fields" with atomic updates makes the "plain 
fields" act like "set" commands -- but there must be at least one field 
with an atomic-update modifier for solr to reconginze that you don't want 
to *completely* replace the entire document.



Third: this comment...

: "partial update" instruction is redundant. My reading of the fine manual is
: that partial update only reindexes the single docvalues field, which is not
: the right thing to do in this case, so it has to be ignored.

...implies a confusion between "atomic updates" and "in-place updates"

Atomic Updates are only in-place updates if all of the specified fields to 
be modified cab be modified "in place" ...

https://solr.apache.org/guide/solr/latest/indexing-guide/partial-document-updates.html#in-place-updates


-Hoss
http://www.lucidworks.com/

Re: Partial Updates unclear Documentation

Posted by Dmitri Maziuk <dm...@gmail.com>.
On 7/29/26 20:32, Rahul Goswami wrote:
> I read my response again just to ensure I said the right thing, and I did
> say the right thing :)
> 
> You misinterpreted my response.

I don't think so.

> So in your example, say doc with id “foo” has 10 fields in the index, then
> the below doc sent in an update will indeed result in a partial update, and
> not an error (i.e values for “bar” and “bleh” will be updated and the
> remaining fields in the original doc in the index will be retained;
> ofcourse copy fields of bar and bleh if any will be updated too)  :
> 
> { "id" : "foo", "bar" : { "set" : "baz" }, "bleh" : "schmeh" }

If what you say is true, then this is equivalent to a full upsert and 
the "partial update" instruction is redundant. My reading of the fine 
manual is that partial update only reindexes the single docvalues field, 
which is not the right thing to do in this case, so it has to be ignored.

Personally I'd consider it an error and reject it with "make up your 
mind: do you want a partial update or a full one", but it's not my code.

At the very least the actual behaviour needs to be documented in TFM.

Dima


Re: Partial Updates unclear Documentation

Posted by Rahul Goswami <ra...@gmail.com>.
I read my response again just to ensure I said the right thing, and I did
say the right thing :)

You misinterpreted my response.

{ "id" : "foo", "bar" : "baz" }

is not equivalent to

{ "id" : "foo", "bar" : { "set" : "baz" }}

As you said, the first doc will result in a doc with just two fields, while
the second will result in just the “bar” field being updated in the
original doc (from the index)

When I say “if your actual doc has 10 fields” I mean the actual doc already
present in the index.

So in your example, say doc with id “foo” has 10 fields in the index, then
the below doc sent in an update will indeed result in a partial update, and
not an error (i.e values for “bar” and “bleh” will be updated and the
remaining fields in the original doc in the index will be retained;
ofcourse copy fields of bar and bleh if any will be updated too)  :

{ "id" : "foo", "bar" : { "set" : "baz" }, "bleh" : "schmeh" }

- Rahul


On Wed, Jul 29, 2026 at 7:32 PM Dmitri Maziuk <dm...@gmail.com>
wrote:

> On 7/29/26 18:12, Rahul Goswami wrote:
> ...
> > ==> if the atomic update logic sees a field with modifier in the supplied
> > doc, it treats the update as  atomic (partial) update. So if your actual
> > doc has 10 fields and in your above example in addition to “id” and
> “price”
> > (with “set” modifier) you have a third field say “product” (without
> > modifier), it will be treated as a partial update and all 10 fields
> should
> > still be present post update (ofcourse with the new values for “price”
> and
> > “product”).
>
> So are you saying
>
> { "id" : "foo", "bar" : "baz" }
>
> is equivalent to
>
> { "id" : "foo", "bar" : { "set" : "baz" }}
>
> ??
>
> If true, it's a bug.
>
> The 1st form should replace the original document with what's posted: 2
> fields.
>
> And if you mix full and partial update instructions in the same post, i.e.
> { "id" : "foo", "bar" : { "set" : "baz" }, "bleh" : "schmeh" }
> that should error out.
>
> Dima
>
>

Re: Partial Updates unclear Documentation

Posted by Dmitri Maziuk <dm...@gmail.com>.
On 7/29/26 18:12, Rahul Goswami wrote:
...
> ==> if the atomic update logic sees a field with modifier in the supplied
> doc, it treats the update as  atomic (partial) update. So if your actual
> doc has 10 fields and in your above example in addition to “id” and “price”
> (with “set” modifier) you have a third field say “product” (without
> modifier), it will be treated as a partial update and all 10 fields should
> still be present post update (ofcourse with the new values for “price” and
> “product”).

So are you saying

{ "id" : "foo", "bar" : "baz" }

is equivalent to

{ "id" : "foo", "bar" : { "set" : "baz" }}

??

If true, it's a bug.

The 1st form should replace the original document with what's posted: 2 
fields.

And if you mix full and partial update instructions in the same post, i.e.
{ "id" : "foo", "bar" : { "set" : "baz" }, "bleh" : "schmeh" }
that should error out.

Dima


Re: Partial Updates unclear Documentation

Posted by Rahul Goswami <ra...@gmail.com>.
1) if id “mydoc” doesnt exist
>
> {
>   "id":"mydoc",
>   "price":{"set":99}
> }

==> This will just add a new doc with two fields id:mydoc and price:99


2)
> Or what happens if the partial document update is missing the id field?

==> Update should fail as partial updates need unique key to be sent

3)
>Or what happens if you send a mixed request, containing fields with
modifiers and fields containing only the value?

==> if the atomic update logic sees a field with modifier in the supplied
doc, it treats the update as  atomic (partial) update. So if your actual
doc has 10 fields and in your above example in addition to “id” and “price”
(with “set” modifier) you have a third field say “product” (without
modifier), it will be treated as a partial update and all 10 fields should
still be present post update (ofcourse with the new values for “price” and
“product”).

- Rahul

On Wed, Jul 29, 2026 at 6:57 PM Dmitri Maziuk <dm...@gmail.com>
wrote:

> On 7/29/26 07:53, Dario.Viva@coop.ch wrote:
> > Dear Solr Team
> >
> > What happens when you send a partial document update like the following?
> >
> >
> > {
> >   "id":"mydoc",
> >   "price":{"set":99}
> > }
> >
> > but no document with the id "mydoc" exists at this moment.
> >
> > Or what happens if the partial document update is missing the id field?
> >
> > Or what happens if you send a mixed request, containing fields with
> modifiers and fields containing only the value?
> >
>
> What you would normally do for #1 is covered under "optimistic
> concurrency" and you will post
>
> }
>   "id":"mydoc",
>   "_version_": number,
>   "price":{"set":99}
> }
>
> #2 should not work, assuming id is the unique key. And I don't think
> partial updates work with any field other than the unique key.
>
> Dima
>
>

Re: Partial Updates unclear Documentation

Posted by Dmitri Maziuk <dm...@gmail.com>.
On 7/29/26 07:53, Dario.Viva@coop.ch wrote:
> Dear Solr Team
> 
> What happens when you send a partial document update like the following?
> 
> 
> {
>   "id":"mydoc",
>   "price":{"set":99}
> }
> 
> but no document with the id "mydoc" exists at this moment.
> 
> Or what happens if the partial document update is missing the id field?
> 
> Or what happens if you send a mixed request, containing fields with modifiers and fields containing only the value?
> 

What you would normally do for #1 is covered under "optimistic 
concurrency" and you will post

}
  "id":"mydoc",
  "_version_": number,
  "price":{"set":99}
}

#2 should not work, assuming id is the unique key. And I don't think 
partial updates work with any field other than the unique key.

Dima


Re: Partial Updates unclear Documentation

Posted by Renato <re...@onnet.ch>.
Hi Dario

I see your point. As you already know how it behaves in most of the cases you are outlining, I encourage you to update the doc by yourself and open a pull request. The doc-files are in the git repo, here the one you are referring to:
https://github.com/apache/solr/blob/main/solr/solr-ref-guide/modules/indexing-guide/pages/partial-document-updates.adoc

You are able to edit the file directly on Github and open a  PR without closing the repo, not sure how practical this is for adoc-files though.

Thanks,

Renato


> On 29 Jul 2026, at 14:53, <Da...@coop.ch> <Da...@coop.ch> wrote:
> 
> Dear Solr Team
> 
> What happens when you send a partial document update like the following?
> 
> 
> {
> "id":"mydoc",
> "price":{"set":99}
> }
> 
> but no document with the id "mydoc" exists at this moment.
> 
> Or what happens if the partial document update is missing the id field?
> 
> Or what happens if you send a mixed request, containing fields with modifiers and fields containing only the value?
> 
> {
> "id":"mydoc",
> "name": "product",
> "price":{"set":99}
> }
> 
> 
> I personally know that in the first case it acts as an upsert. so inserting the document if its missing, and updating it if it already  exists.
> And in the second case it always inserts a new document, with its id being generated.
> 
> For the third case I am not entirely sure, but I think if there is at least one field in the request with a modifier the whole request acts as a partial document update and fields without modifiers are handled as if they have the modifier "set". Thats what the behaviour on solr 9.10.1 looks like at least.
> 
> The Documentation does not really explain any of the above cases clearly. This also leads to AI making very questionable statements that are just wrong.
> 
> https://share.google/aimode/rZE2udrRajAUNDD6y
> Google just casually tells us that it only acts as an upsert but ONLY if the document already exists in the collection, which is wrong and also does not make any sense as that would not be an upsert.
> 
> https://chatgpt.com/s/t_6a69f66582dc81919da8e03451e267f1
> Also Chatgpt is just wrong about it.
> 
> As far as I understand it, partial document updates were introduced in solr 4 and the upsert behaviour was introduced in solr 4.1. but as its not clearly documented I am not 100% sure.
> https://stackoverflow.com/questions/14886814/can-solr-perform-an-upsert#comment21025920_14888083
> 
> Kind Regards
>