You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by ho...@apache.org on 2020/07/31 18:43:24 UTC
[lucene-solr] branch jira/SOLR-14383 created (now 7086d67)
This is an automated email from the ASF dual-hosted git repository.
hossman pushed a change to branch jira/SOLR-14383
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git.
at 7086d67 SOLR-14383: tweaks bassed on feedback from smiley
This branch includes the following new commits:
new 8423eb9 SOLR-14383.patch circa 2020-07-30
new 7086d67 SOLR-14383: tweaks bassed on feedback from smiley
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[lucene-solr] 02/02: SOLR-14383: tweaks bassed on feedback from
smiley
Posted by ho...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hossman pushed a commit to branch jira/SOLR-14383
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
commit 7086d6765215c1137ccc623468d4925a595d1fea
Author: Chris Hostetter <ho...@apache.org>
AuthorDate: Fri Jul 31 11:14:58 2020 -0700
SOLR-14383: tweaks bassed on feedback from smiley
- typo fixes
- using TIP box to draw attention to prefix routing
- switch from 'producs+skus+docs' to 'products+skus+manuals' for examples
---
.../src/indexing-nested-documents.adoc | 60 ++++++++++++----------
.../src/searching-nested-documents.adoc | 20 ++++----
.../src/transforming-result-documents.adoc | 8 +--
.../src/updating-parts-of-documents.adoc | 26 +++++-----
4 files changed, 59 insertions(+), 55 deletions(-)
diff --git a/solr/solr-ref-guide/src/indexing-nested-documents.adoc b/solr/solr-ref-guide/src/indexing-nested-documents.adoc
index 7b53600..85c6943 100644
--- a/solr/solr-ref-guide/src/indexing-nested-documents.adoc
+++ b/solr/solr-ref-guide/src/indexing-nested-documents.adoc
@@ -36,11 +36,11 @@ With the exception of in-place updates, <<#maintaining-integrity-with-updates-an
[#example-indexing-syntax]
== Example Indexing Syntax: Psuedo-Fields
-This example shows what it looks like to index two root level "product" documents, each containing two different types of child documents specified in "psuedo-fields": "skus" and "docs". Two of the "sku" type documents have their own nested child "docs" documents...
+This example shows what it looks like to index two root level "product" documents, each containing two different types of child documents specified in "psuedo-fields": "skus" and "manuals". Two of the "sku" type documents have their own nested child "manuals" documents...
[NOTE]
====
-Even though the child documents in these examples are provided syntactically as field values syntactically, this is simply a matter of syntax and as such `skus` and `docs` are not actual fields in the documents. Consequently, these field names need not be defined in the schema and probably shouldn't be as it would be confusing. There is no "child document" field type.
+Even though the child documents in these examples are provided syntactically as field values syntactically, this is simply a matter of syntax and as such `skus` and `manuals` are not actual fields in the documents. Consequently, these field names need not be defined in the schema and probably shouldn't be as it would be confusing. There is no "child document" field type.
====
//
@@ -61,23 +61,23 @@ Even though the child documents in these examples are provided syntactically as
"skus": [ { "id": "P11!S21",
"color_s": "RED",
"price_i": 42,
- "docs": [ { "id": "P11!D41",
- "name_s": "Red Swingline Brochure",
- "content_t": "..."
- } ]
+ "manuals": [ { "id": "P11!D41",
+ "name_s": "Red Swingline Brochure",
+ "content_t": "..."
+ } ]
},
{ "id": "P11!S31",
"color_s": "BLACK",
"price_i": 3
} ],
- "docs": [ { "id": "P11!D51",
- "name_s": "Quick Reference Guide",
- "content_t": "How to use your stapler ..."
- },
- { "id": "P11!D61",
- "name_s": "Warranty Details",
- "content_t": "... lifetime garuntee ..."
- } ]
+ "manuals": [ { "id": "P11!D51",
+ "name_s": "Quick Reference Guide",
+ "content_t": "How to use your stapler ..."
+ },
+ { "id": "P11!D61",
+ "name_s": "Warranty Details",
+ "content_t": "... lifetime guarantee ..."
+ } ]
},
{ "id": "P22!prod",
"name_s": "Mont Blanc Fountain Pen",
@@ -85,19 +85,19 @@ Even though the child documents in these examples are provided syntactically as
"skus": [ { "id": "P22!S22",
"color_s": "RED",
"price_i": 89,
- "docs": [ { "id": "P21!D41",
- "name_s": "Red Mont Blanc Brochure",
- "content_t": "..."
- } ]
+ "manuals": [ { "id": "P21!D41",
+ "name_s": "Red Mont Blanc Brochure",
+ "content_t": "..."
+ } ]
},
{ "id": "P22!S32",
"color_s": "BLACK",
"price_i": 67
} ],
- "docs": [ { "id": "P22!D52",
- "name_s": "How To Use A Pen",
- "content_t": "Start by removing the cap ..."
- } ]
+ "manuals": [ { "id": "P22!D52",
+ "name_s": "How To Use A Pen",
+ "content_t": "Start by removing the cap ..."
+ } ]
} ]
----
// end::sample-indexing-deeply-nested-documents[]
@@ -148,7 +148,6 @@ There are several additional schema considerations that should be considered for
** All field names in the schema can only be configured in one -- different types of child documents can not have the same field name configured in different ways.
** It may be infeasible to use `required` for any field names that aren't reqiured for all types of documents.
** Even child documents need a _globally_ unique `id`.
-*** When using Solr Cloud it is a _VERY_ good idea to use <<shards-and-indexing-data-in-solrcloud#document-routing,prefix based compositeIds>> with a common prefix for all documents in the block. This makes it much easier to apply <<updating-parts-of-documents#updating-child-documents,atomic updates to individual child documents>>
* `\_root_` must be configured to either be stored (`stored="true"`) or use doc values (`docValues="true"`) to enable <<updating-parts-of-documents#updating-child-documents,atomic updates of nested documents>>.
** Also, beware of `uniqueBlock(\_root_)` <<json-facet-api#stat-facet-functions,field type limitation>>, if you plan to use one.
* `\_nest_path_` is an optional field that (if definied) will be populated by Solr automatically with the ancestor path of each non-root document.
@@ -160,8 +159,8 @@ There are several additional schema considerations that should be considered for
----
** This field is neccessary if you wish to use <<updating-parts-of-documents#updating-child-documents,atomic updates of nested documents>>
** This field is neccessary in order for Solr to properly record & reconstruct the nested relationship of documents when using the `<<searching-nested-documents.adoc#child-doc-transformer,[child]>>` doc transformer.
-*** If this field does not exist, the `[child]` transformer will return all descendent child docs as a flattened list -- just as if they had been <<#indexing-anonymous-children,indexed as anonymous children>>.
-** If you do not use `\_nest_path_` it is strongly recomended that every document have some field that differentiates root docs from their nested children -- and differentiates different "types" of child documents. This is not strictly neccessary, so long as it's possible to write a "filter" query that can be used to isolate and select only parent documents for use in the <<other-parsers.adoc#block-join-query-parsers,block join query parsers>> and <<searching-nested-documents.adoc#child [...]
+*** If this field does not exist, the `[child]` transformer will return all descendent child documents as a flattened list -- just as if they had been <<#indexing-anonymous-children,indexed as anonymous children>>.
+** If you do not use `\_nest_path_` it is strongly recomended that every document have some field that differentiates root documents from their nested children -- and differentiates different "types" of child documents. This is not strictly neccessary, so long as it's possible to write a "filter" query that can be used to isolate and select only parent documents for use in the <<other-parsers.adoc#block-join-query-parsers,block join query parsers>> and <<searching-nested-documents.adoc# [...]
* `\_nest_parent_` is an optional field that (if defined) will be populated by Solr automatically to store the `id` of each document's _immediate_ parent document (if there is one).
+
[sourece,xml]
@@ -169,6 +168,11 @@ There are several additional schema considerations that should be considered for
<field name="_nest_parent_" type="string" indexed="true" stored="true" />
----
+[TIP]
+====
+When using Solr Cloud it is a _VERY_ good idea to use <<shards-and-indexing-data-in-solrcloud#document-routing,prefix based compositeIds>> with a common prefix for all documents in the block. This makes it much easier to apply <<updating-parts-of-documents#updating-child-documents,atomic updates to individual child documents>>
+====
+
== Maintaining Integrity with Updates and Deletes
@@ -223,7 +227,7 @@ Although not recommended, it is also possible to index child documents "anonymou
{ "id": "P11!D61",
"type_s": "DOC",
"name_s": "Warranty Details",
- "content_t": "... lifetime garuntee ..."
+ "content_t": "... lifetime guarantee ..."
}
]
} ]
@@ -304,7 +308,7 @@ $ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=tr
"id":"P11!D61",
"type_s":"DOC",
"name_s":"Warranty Details",
- "content_t":"... lifetime garuntee ...",
+ "content_t":"... lifetime guarantee ...",
"_version_":1673055562829398016}]}]
}}
----
@@ -354,7 +358,7 @@ $ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=tr
<str name="id">P11!D61</str>
<str name="type_s">DOC</str>
<str name="name_s">Warranty Details</str>
- <str name="content_t">... lifetime garuntee ...</str>
+ <str name="content_t">... lifetime guarantee ...</str>
<long name="_version_">1673055562829398016</long></doc></doc>
</result>
</response>
diff --git a/solr/solr-ref-guide/src/searching-nested-documents.adoc b/solr/solr-ref-guide/src/searching-nested-documents.adoc
index 29ea585..057d6d4 100644
--- a/solr/solr-ref-guide/src/searching-nested-documents.adoc
+++ b/solr/solr-ref-guide/src/searching-nested-documents.adoc
@@ -35,7 +35,7 @@ include::indexing-nested-documents.adoc[tag=sample-indexing-deeply-nested-docume
=== Child Doc Transformer
-By default, documents that match a query do not include any of thir nested children in the response. The `[child]` Doc Transformer Can be used enrich query results with the documents' descendants.
+By default, documents that match a query do not include any of their nested children in the response. The `[child]` Doc Transformer Can be used enrich query results with the documents' descendants.
For a detailed explanation of this transformer, and specifics on it's syntax & limitations, please refer to the section <<transforming-result-documents.adoc#child-childdoctransformerfactory, [child] - ChildDocTransformerFactory>>.
@@ -72,7 +72,7 @@ $ curl 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q=descr
"color_s":"RED",
"price_i":42,
"_version_":1672933224035123200,
- "docs":[
+ "manuals":[
{
"id":"P11!D41",
"name_s":"Red Swingline Brochure",
@@ -84,7 +84,7 @@ $ curl 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q=descr
"color_s":"BLACK",
"price_i":3,
"_version_":1672933224035123200}],
- "docs":[
+ "manuals":[
{
"id":"P11!D51",
"name_s":"Quick Reference Guide",
@@ -94,7 +94,7 @@ $ curl 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q=descr
{
"id":"P11!D61",
"name_s":"Warranty Details",
- "content_t":"... lifetime garuntee ...",
+ "content_t":"... lifetime guarantee ...",
"_version_":1672933224035123200}]}]
}}
----
@@ -134,7 +134,7 @@ $ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=tr
{
"id":"P11!D61",
"name_s":"Warranty Details",
- "content_t":"... lifetime garuntee ...",
+ "content_t":"... lifetime guarantee ...",
"_version_":1672933224035123200}]
}}
----
@@ -198,13 +198,13 @@ Here for example is a query where:
* the (sku) documents returned must have a color of "RED"
* the (sku) docments returned must be the descendents of root level (product) documents which have:
-** immediate child "docs" documents which have:
-*** "lifetime garuntee" in their content
-* each return (sku) document also includes any descendent (docs) documents it has
+** immediate child "manuals" documents which have:
+*** "lifetime guarantee" in their content
+* each return (sku) document also includes any descendent (manuals) documents it has
[source,bash]
----
-$ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&fq=color_s:RED&q={!child+of="*:*+-_nest_path_:*"+filters=$parent_fq}&parent_fq={!parent+which="*:*+-_nest_path_:*"}_nest_path_:"/docs"+AND+content_t:"lifetime+garuntee"&fl=*,[child]'
+$ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&fq=color_s:RED&q={!child+of="*:*+-_nest_path_:*"+filters=$parent_fq}&parent_fq={!parent+which="*:*+-_nest_path_:*"}_nest_path_:"/manuals"+AND+content_t:"lifetime+guarantee"&fl=*,[child]'
{
"response":{"numFound":1,"start":0,"maxScore":1.4E-45,"numFoundExact":true,"docs":[
{
@@ -212,7 +212,7 @@ $ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=tr
"color_s":"RED",
"price_i":42,
"_version_":1672933224035123200,
- "docs":[
+ "manuals":[
{
"id":"P11!D41",
"name_s":"Red Swingline Brochure",
diff --git a/solr/solr-ref-guide/src/transforming-result-documents.adoc b/solr/solr-ref-guide/src/transforming-result-documents.adoc
index 95eaf9a..7394097 100644
--- a/solr/solr-ref-guide/src/transforming-result-documents.adoc
+++ b/solr/solr-ref-guide/src/transforming-result-documents.adoc
@@ -168,13 +168,13 @@ Some Examples:
** Matches any documents that are descendents of the current document and have a "nested path" of `/skus` -- but not any children of those `skus`
* childFilter="/skus/color_s:RED"
** Matches any documents that are descendents of the current document; match `color_s:RED`; and have a "nested path" of `/skus` -- but not any children of those `skus`
-* `childFilter="/skus/docs/\*:*"`
-** Matches any documents that are descendents of the current document and have a "nested path" of `/skus/docs` -- but not any children of those `docs`
+* `childFilter="/skus/manuals/\*:*"`
+** Matches any documents that are descendents of the current document and have a "nested path" of `/skus/manuals` -- but not any children of those `manuals`
When paths do not start with a `/` they are treated as "path suffixes":
-* `childFilter="docs/\*:*"`
-** Matches any documents that are descendents of the current document and have a "nested path" that ends with "docs", regardless of how deeply nested they are -- but not any children of those `docs`
+* `childFilter="manuals/\*:*"`
+** Matches any documents that are descendents of the current document and have a "nested path" that ends with "manuals", regardless of how deeply nested they are -- but not any children of those `manuals`
====
diff --git a/solr/solr-ref-guide/src/updating-parts-of-documents.adoc b/solr/solr-ref-guide/src/updating-parts-of-documents.adoc
index ecdc447..ccd09e4 100644
--- a/solr/solr-ref-guide/src/updating-parts-of-documents.adoc
+++ b/solr/solr-ref-guide/src/updating-parts-of-documents.adoc
@@ -156,14 +156,14 @@ As with normal (multiValued) fields, the `set` keyword can be used to replace al
curl -X POST 'http://localhost:8983/solr/gettingstarted/update?commit=true' -H 'Content-Type: application/json' --data-binary '[
{
"id": "P22!S22",
- "docs": { "set": [ { "id": "P22!D77",
- "name_s": "Why Red Pens Are the Best",
- "content_t": "... correcting papers ...",
- },
- { "id": "P22!D88",
- "name_s": "How to get Red ink stains out of fabric",
- "content_t": "... vinegar ...",
- } ] }
+ "manuals": { "set": [ { "id": "P22!D77",
+ "name_s": "Why Red Pens Are the Best",
+ "content_t": "... correcting papers ...",
+ },
+ { "id": "P22!D88",
+ "name_s": "How to get Red ink stains out of fabric",
+ "content_t": "... vinegar ...",
+ } ] }
} ]'
----
@@ -177,10 +177,10 @@ As with normal (multiValued) fields, the `add` keyword can be used to add additi
curl -X POST 'http://localhost:8983/solr/gettingstarted/update?commit=true' -H 'Content-Type: application/json' --data-binary '[
{
"id": "P11!S21",
- "docs": { "add": { "id": "P11!D99",
- "name_s": "Why Red Staplers Are the Best",
- "content_t": "Once upon a time, Mike Judge ...",
- } }
+ "manuals": { "add": { "id": "P11!D99",
+ "name_s": "Why Red Staplers Are the Best",
+ "content_t": "Once upon a time, Mike Judge ...",
+ } }
} ]'
----
@@ -194,7 +194,7 @@ As with normal (multiValued) fields, the `remove` keyword can be used to remove
curl -X POST 'http://localhost:8983/solr/gettingstarted/update?commit=true' -H 'Content-Type: application/json' --data-binary '[
{
"id": "P11!S21",
- "docs": { "remove": { "id": "P11!D41" } }
+ "manuals": { "remove": { "id": "P11!D41" } }
} ]'
----
[lucene-solr] 01/02: SOLR-14383.patch circa 2020-07-30
Posted by ho...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hossman pushed a commit to branch jira/SOLR-14383
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
commit 8423eb9570712489fded77ee98e2f495026a2aac
Author: Chris Hostetter <ho...@apache.org>
AuthorDate: Fri Jul 31 10:37:02 2020 -0700
SOLR-14383.patch circa 2020-07-30
https://issues.apache.org/jira/secure/attachment/13008628/SOLR-14383.patch
---
.../src/indexing-nested-documents.adoc | 424 +++++++++++++++------
solr/solr-ref-guide/src/other-parsers.adoc | 10 +
.../src/searching-nested-documents.adoc | 306 ++++++++-------
.../src/transforming-result-documents.adoc | 43 ++-
.../src/updating-parts-of-documents.adoc | 142 ++++---
5 files changed, 595 insertions(+), 330 deletions(-)
diff --git a/solr/solr-ref-guide/src/indexing-nested-documents.adoc b/solr/solr-ref-guide/src/indexing-nested-documents.adoc
index e699cb2..7b53600 100644
--- a/solr/solr-ref-guide/src/indexing-nested-documents.adoc
+++ b/solr/solr-ref-guide/src/indexing-nested-documents.adoc
@@ -17,137 +17,349 @@
// under the License.
Solr supports indexing nested documents, described here, and ways to <<searching-nested-documents.adoc#searching-nested-documents,search and retrieve>> them very efficiently.
-By way of example, nested documents in Solr can be used to bind a blog post (parent document) with comments (child documents)
--- or products as parent documents and sizes, colors, or other variations as child documents. +
-The parent with all children is referred to as a nested document or "block" and it explains some of the nomenclature of related features.
+By way of examples: nested documents in Solr can be used to bind a blog post (parent document) with comments (child documents) -- or as a way to model major product lines as parent documents, with multiple types of child documents representing individual SKUs (with unique sizes / colors) and supporting documention (either directly nested under the products, or under individual SKUs.
+The "top most" parent with all children is referred to as a "root level" document or "block document" and it explains some of the nomenclature of related features.
At query time, the <<other-parsers.adoc#block-join-query-parsers,Block Join Query Parsers>> can search these relationships,
- and the `<<transforming-result-documents.adoc#child-childdoctransformerfactory,[child]>>` Document Transformer can attach child documents to the result documents.
-In terms of performance, indexing the relationships between documents usually yields much faster queries than an equivalent "query time join",
+ and the `<<transforming-result-documents.adoc#child-childdoctransformerfactory,[child]>>` Document Transformer can attach child (or other "descendent") documents to the result documents.
+In terms of performance, indexing the relationships between documents usually yields much faster queries than an equivalent "<<other-parsers#join-query-parser,query time join>>",
since the relationships are already stored in the index and do not need to be computed.
However, nested documents are less flexible than query time joins as it imposes rules that some applications may not be able to accept.
Nested documents may be indexed via either the XML or JSON data syntax, and is also supported by <<using-solrj.adoc#using-solrj,SolrJ>> with javabin.
+
+[CAUTION]
+====
+.Re-Indexing Considerations
+With the exception of in-place updates, <<#maintaining-integrity-with-updates-and-deletes,blocks of nested documents must be updated/deleted together>>. Modifying or replacing individual child documents requires re-indexing of the entire block (either explicitly/externally, or under the covers inside of Solr). For some applications this may result in a lot of extra indexing overhead and may not be worth the performance gains at query time.
+====
+
+[#example-indexing-syntax]
+== Example Indexing Syntax: Psuedo-Fields
+
+This example shows what it looks like to index two root level "product" documents, each containing two different types of child documents specified in "psuedo-fields": "skus" and "docs". Two of the "sku" type documents have their own nested child "docs" documents...
+
[NOTE]
====
-.Limitation
-With the exception of in-place updates, the whole block must be updated or deleted together, not separately. For some applications this may result in tons of extra indexing and thus may be a deal-breaker.
+Even though the child documents in these examples are provided syntactically as field values syntactically, this is simply a matter of syntax and as such `skus` and `docs` are not actual fields in the documents. Consequently, these field names need not be defined in the schema and probably shouldn't be as it would be confusing. There is no "child document" field type.
+====
+
+//
+// DO NOT MODIFY THESE EXAMPLE DOCS WITH OUT REVIEWING ALL PAGES THAT INCLUDE/REFER BACK TO THESE EXAMPLES
+// INCLUDING THE SEMI-EQUIVILENT ANONYMOUS CHILDREN EXAMPLE AT THE BOTTOM OF THIS PAGE
+//
+[.dynamic-tabs]
+--
+[example.tab-pane#json]
+====
+[.tab-label]*JSON*
+// tag::sample-indexing-deeply-nested-documents[]
+[source,json]
+----
+[{ "id": "P11!prod",
+ "name_s": "Swingline Stapler",
+ "description_t": "The Cadillac of office staplers ...",
+ "skus": [ { "id": "P11!S21",
+ "color_s": "RED",
+ "price_i": 42,
+ "docs": [ { "id": "P11!D41",
+ "name_s": "Red Swingline Brochure",
+ "content_t": "..."
+ } ]
+ },
+ { "id": "P11!S31",
+ "color_s": "BLACK",
+ "price_i": 3
+ } ],
+ "docs": [ { "id": "P11!D51",
+ "name_s": "Quick Reference Guide",
+ "content_t": "How to use your stapler ..."
+ },
+ { "id": "P11!D61",
+ "name_s": "Warranty Details",
+ "content_t": "... lifetime garuntee ..."
+ } ]
+ },
+ { "id": "P22!prod",
+ "name_s": "Mont Blanc Fountain Pen",
+ "description_t": "A Premium Writing Instrument ...",
+ "skus": [ { "id": "P22!S22",
+ "color_s": "RED",
+ "price_i": 89,
+ "docs": [ { "id": "P21!D41",
+ "name_s": "Red Mont Blanc Brochure",
+ "content_t": "..."
+ } ]
+ },
+ { "id": "P22!S32",
+ "color_s": "BLACK",
+ "price_i": 67
+ } ],
+ "docs": [ { "id": "P22!D52",
+ "name_s": "How To Use A Pen",
+ "content_t": "Start by removing the cap ..."
+ } ]
+ } ]
+----
+// end::sample-indexing-deeply-nested-documents[]
+
+[CAUTION]
+=====
+The <<uploading-data-with-index-handlers#json-update-convenience-paths,`/update/json/docs` convenience path>> will automatically flatten complex JSON documents by default -- so to index nested JSON documents make sure to use `/update`.
+=====
====
+[example.tab-pane#xml]
+====
+[.tab-label]*XML*
+[source,xml]
+----
+nocommit: TODO: XML equivilent of JSON above
+----
+====
+
+[example.tab-pane#solrj]
+====
+[.tab-label]*SolrJ*
+[source,java]
+----
+nocommit: TODO: SolrJ equivilent of JSON above
+nocommit: ... do we even have a test proving this works correctly
+nocommit: the SolrInputDocument methods for addChildDocument methods still don't take "field name"
+----
+====
+--
+
+
== Schema Configuration
- * The schema must include an indexed field `\_root_`. Solr automatically populates this with the value of the top/parent ID. +
- `<field name="\_root_" type="string" indexed="true" stored="false" docValues="false" />`
- ** `\_root_` must be set either as stored (stored="true") or doc values (docValues="true") to enable
- <<updating-parts-of-documents#updating-child-documents, atomic updates of nested documents>>. Also, beware of `uniqueBlock(\_root_)` <<json-facet-api#stat-facet-functions,field type limitation>>, if you plan to use one.
- * `\_nest_path_` is populated by Solr automatically with the path of the document in the hierarchy for non-root documents. This field is optional. +
- `<fieldType name="\_nest_path_" class="solr.NestPathField" />
- <field name="\_nest_path_" type="_nest_path_" />`
- * `\_nest_parent_` is populated by Solr automatically to store the ID of each document's parent document (if there is one). This field is optional. +
- `<field name="\_nest_parent_" type="string" indexed="true" stored="true"/>`
- * Nested documents are very much documents in their own right even if certain nested documents hold different information from the parent.
- Therefore:
- ** a field can only be configured one way no matter what sort of document uses it
- ** it may be infeasible to use `required`
- ** even child documents need a unique ID
- * Even though child documents are provided as field values syntactically and with SolrJ, it's a matter of syntax and it isn't an actual field in the schema.
- Consequently, the field need not be defined in the schema and probably shouldn't be as it would be confusing.
- There is no child document field type, at least not yet.
-
-=== Rudimentary Root-only Schemas
-
-These schemas do not contain any other nested related fields apart from `\_root_`.
-Many schemas in existence are this way simply because default configsets are this way, even if the application isn't using nested documents.
-If an application uses nested documents with such a schema, keep in mind that that some related features aren't as effective since there is less information. Mainly the <<searching-nested-documents.adoc#child-doc-transformer,[child]>> transformer returns matching children in a flat list (not nested) and it's attached to the parent using the special field name `\_childDocuments_`.
-
-With such a schema, typically you should have a field that differentiates a root doc from any nested children.
-However this isn't strictly necessary; so long as it's possible to write a query that can select only root documents somehow.
-Such a query is needed for the <<other-parsers.adoc#block-join-query-parsers,block join query parsers>> and <<searching-nested-documents.adoc#child-doc-transformer,[child]>> doc transformer to function.
-
-=== XML Examples
-
-Here are two documents and their child documents.
-It illustrates two styles of adding child documents: the first is associated via a field "comment" (preferred),
-and the second is done in the classic way now referred to as an "anonymous" or "unlabelled" child document.
-This field label relationship is available to the URP chain in Solr but is ultimately discarded unless the special fields are defined.
+Indexing nested documents _requires_ an indexed field named `\_root_`:
[source,xml]
----
-<add>
- <doc>
- <field name="ID">1</field>
- <field name="title">Solr adds block join support</field>
- <field name="content_type">parentDocument</field>
- <field name="content">
- <doc>
- <field name="ID">2</field>
- <field name="comments">SolrCloud supports it too!</field>
- </doc>
- </field>
- </doc>
- <doc>
- <field name="ID">3</field>
- <field name="title">New Lucene and Solr release is out</field>
- <field name="content_type">parentDocument</field>
- <doc>
- <field name="ID">4</field>
- <field name="comments">Lots of new features</field>
- </doc>
- </doc>
-</add>
+<field name="_root_" type="string" indexed="true" />
----
-In this example, we have indexed the parent documents with the field `content_type`, which has the value "parentDocument".
-We could have also used a boolean field, such as `isParent`, with a value of "true", or any other similar approach.
+Solr automatically populates this field in every nested document with the `id` value of the top most parent document in the block.
+
+
+There are several additional schema considerations that should be considered for people who wish to use nested documents:
+
+* Nested child documents are very much documents in their own right even if certain nested documents hold different information from the parent, Therefore:
+** All field names in the schema can only be configured in one -- different types of child documents can not have the same field name configured in different ways.
+** It may be infeasible to use `required` for any field names that aren't reqiured for all types of documents.
+** Even child documents need a _globally_ unique `id`.
+*** When using Solr Cloud it is a _VERY_ good idea to use <<shards-and-indexing-data-in-solrcloud#document-routing,prefix based compositeIds>> with a common prefix for all documents in the block. This makes it much easier to apply <<updating-parts-of-documents#updating-child-documents,atomic updates to individual child documents>>
+* `\_root_` must be configured to either be stored (`stored="true"`) or use doc values (`docValues="true"`) to enable <<updating-parts-of-documents#updating-child-documents,atomic updates of nested documents>>.
+** Also, beware of `uniqueBlock(\_root_)` <<json-facet-api#stat-facet-functions,field type limitation>>, if you plan to use one.
+* `\_nest_path_` is an optional field that (if definied) will be populated by Solr automatically with the ancestor path of each non-root document.
++
+[source,xml]
+----
+<fieldType name="_nest_path_" class="solr.NestPathField" />
+<field name="_nest_path_" type="_nest_path_" />`
+----
+** This field is neccessary if you wish to use <<updating-parts-of-documents#updating-child-documents,atomic updates of nested documents>>
+** This field is neccessary in order for Solr to properly record & reconstruct the nested relationship of documents when using the `<<searching-nested-documents.adoc#child-doc-transformer,[child]>>` doc transformer.
+*** If this field does not exist, the `[child]` transformer will return all descendent child docs as a flattened list -- just as if they had been <<#indexing-anonymous-children,indexed as anonymous children>>.
+** If you do not use `\_nest_path_` it is strongly recomended that every document have some field that differentiates root docs from their nested children -- and differentiates different "types" of child documents. This is not strictly neccessary, so long as it's possible to write a "filter" query that can be used to isolate and select only parent documents for use in the <<other-parsers.adoc#block-join-query-parsers,block join query parsers>> and <<searching-nested-documents.adoc#child [...]
+* `\_nest_parent_` is an optional field that (if defined) will be populated by Solr automatically to store the `id` of each document's _immediate_ parent document (if there is one).
++
+[sourece,xml]
+----
+<field name="_nest_parent_" type="string" indexed="true" stored="true" />
+----
+
+
+== Maintaining Integrity with Updates and Deletes
+
+Blocks of nested documents can be modified simply by adding/replacing the root document with more or fewer child/descendent documents as an application desires. This can either be done explicitly/externaly by an indexing client completely re-indexing the root level document, or internally by Solr when a client uses <<updating-parts-of-documents#updating-child-documents,atomic updates>> to modify child documents. This aspect isn't different than updating any normal document except that [...]
+
+Clients should however be very careful to *never* add a root document that has the same `id` of a child document -- or vice-versa. Solr does not prevent clients from attempting this, but *_it will violate integrity assumptions that Solr expects._*
+
+To delete an entire block of documents, you can simply delete-by-ID using the `id` of the root document. Delete-by-ID will not work with the `id` of a child document, since only root document IDs are considered. (Instead, use <<updating-parts-of-documents#updating-child-documents,atomic updates>> to remove the child document from it's parent)
-=== JSON Examples
+If you use Solr's delete-by-query APIs, you *MUST* be careful to ensure that any deletion query is strutured to ensure no descendent children remain of any documents that are being deleted. *_Doing otherwise will violate integrity assumptions that Solr expects._*
-This example is equivalent to the XML example above.
-Again, the field labelled relationship is preferred.
-The labelled relationship here is one child document but could have been wrapped in array brackets.
-For the anonymous relationship, note the special `\_childDocuments_` key whose contents must be an array of child documents.
+
+
+== Indexing Anonymous Children
+
+Although not recommended, it is also possible to index child documents "anonymously":
+
+[.dynamic-tabs]
+--
+[example.tab-pane#anon_json]
+====
+[.tab-label]*JSON*
[source,json]
----
-[
- {
- "ID": "1",
- "title": "Solr adds block join support",
- "content_type": "parentDocument",
- "comments": [{
- "ID": "2",
- "content": "SolrCloud supports it too!"
- },
- {
- "ID": "3",
- "content": "New filter syntax"
- }
+[{ "id": "P11!prod",
+ "name_s": "Swingline Stapler",
+ "type_s": "PRODUCT",
+ "description_t": "The Cadillac of office staplers ...",
+ "_childDocuments_": [
+ { "id": "P11!S21",
+ "type_s": "SKU",
+ "color_s": "RED",
+ "price_i": 42,
+ "_childDocuments_": [
+ { "id": "P11!D41",
+ "type_s": "DOC",
+ "name_s": "Red Swingline Brochure",
+ "content_t": "..."
+ } ]
+ },
+ { "id": "P11!S31",
+ "type_s": "SKU",
+ "color_s": "BLACK",
+ "price_i": 3
+ },
+ { "id": "P11!D51",
+ "type_s": "DOC",
+ "name_s": "Quick Reference Guide",
+ "content_t": "How to use your stapler ..."
+ },
+ { "id": "P11!D61",
+ "type_s": "DOC",
+ "name_s": "Warranty Details",
+ "content_t": "... lifetime garuntee ..."
+ }
]
- },
- {
- "ID": "4",
- "title": "New Lucene and Solr release is out",
- "content_type": "parentDocument",
- "_childDocuments_": [
- {
- "ID": "5",
- "comments": "Lots of new features"
- }
- ]
- }
-]
+} ]
----
+====
+
+[example.tab-pane#anon_xml]
+====
+[.tab-label]*XML*
+[source,xml]
+----
+nocommit: TODO: XML equivilent of JSON above
+----
+====
+
+[example.tab-pane#anon_solrj]
+====
+[.tab-label]*SolrJ*
+[source,java]
+----
+nocommit: TODO: SolrJ equivilent of JSON above
+----
+====
+
+--
-.Root-Only Mode
-[NOTE]
- In Root-only schemas, these two documents will result in the same docs being indexed (Root-only schemas do not honor nested relationships).
- When queried, child docs will be appended to the _childDocuments_ field/key.
-=== Important: Maintaining Integrity with Updates and Deletes
+This simplified approach was common in older versions of Solr, and can still be used with "Root-Only" schemas that do not contain any other nested related fields apart from `\_root_`. (Many schemas in existence are this way simply because default configsets are this way, even if the application isn't using nested documents.)
-Nested documents (children and all) can simply be replaced by adding a new document with more or fewer documents as an application desires. This aspect isn't different than updating any normal document except that Solr takes care to ensure that all related child documents of the existing version get deleted.
+This approach should *NOT* be used when schemas include a `\_nest_path_` field, as the existence of that field triggers assumptions and changes in behavior in various query time functionality, such as the <<searching-nested-documents.adoc#child-doc-transformer,[child]>>, that will not work when nested documents do not have any intrinsic "nested path" information.
+
+The results of indexing anonymous nested children with a "Root-Only" schema are similar to what happens if you attempt to index "psuedo field" nested documents using a "Root-Only" schema. Notably: since there is no nested path information for the <<searching-nested-documents.adoc#child-doc-transformer,[child]>> transformer to use to reconstruct the structured of a block of documents, it returns all matching children as a flat list, similar in structure to how they were originally indexed:
+
+
+
+[.dynamic-tabs]
+--
+[example.tab-pane#anon_json_out]
+====
+[.tab-label]*JSON*
+[source,bash]
+----
+$ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q=id:P11!prod&fl=*,[child%20parentFilter=%22type_s:PRODUCT%22]'
+{
+ "response":{"numFound":1,"start":0,"maxScore":0.7002023,"numFoundExact":true,"docs":[
+ {
+ "id":"P11!prod",
+ "name_s":"Swingline Stapler",
+ "type_s":"PRODUCT",
+ "description_t":"The Cadillac of office staplers ...",
+ "_version_":1673055562829398016,
+ "_childDocuments_":[
+ {
+ "id":"P11!D41",
+ "type_s":"DOC",
+ "name_s":"Red Swingline Brochure",
+ "content_t":"...",
+ "_version_":1673055562829398016},
+ {
+ "id":"P11!S21",
+ "type_s":"SKU",
+ "color_s":"RED",
+ "price_i":42,
+ "_version_":1673055562829398016},
+ {
+ "id":"P11!S31",
+ "type_s":"SKU",
+ "color_s":"BLACK",
+ "price_i":3,
+ "_version_":1673055562829398016},
+ {
+ "id":"P11!D51",
+ "type_s":"DOC",
+ "name_s":"Quick Reference Guide",
+ "content_t":"How to use your stapler ...",
+ "_version_":1673055562829398016},
+ {
+ "id":"P11!D61",
+ "type_s":"DOC",
+ "name_s":"Warranty Details",
+ "content_t":"... lifetime garuntee ...",
+ "_version_":1673055562829398016}]}]
+ }}
+----
+====
+
+[example.tab-pane#anon_xml_out]
+====
+[.tab-label]*XML*
+[source,bash]
+----
+$ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q=id:P11!prod&fl=*,[child%20parentFilter=%22type_s:PRODUCT%22]&wt=xml'
+<?xml version="1.0" encoding="UTF-8"?>
+<response>
+
+<result name="response" numFound="1" start="0" maxScore="0.7002023" numFoundExact="true">
+ <doc>
+ <str name="id">P11!prod</str>
+ <str name="name_s">Swingline Stapler</str>
+ <str name="type_s">PRODUCT</str>
+ <str name="description_t">The Cadillac of office staplers ...</str>
+ <long name="_version_">1673055562829398016</long>
+ <doc>
+ <str name="id">P11!D41</str>
+ <str name="type_s">DOC</str>
+ <str name="name_s">Red Swingline Brochure</str>
+ <str name="content_t">...</str>
+ <long name="_version_">1673055562829398016</long></doc>
+ <doc>
+ <str name="id">P11!S21</str>
+ <str name="type_s">SKU</str>
+ <str name="color_s">RED</str>
+ <int name="price_i">42</int>
+ <long name="_version_">1673055562829398016</long></doc>
+ <doc>
+ <str name="id">P11!S31</str>
+ <str name="type_s">SKU</str>
+ <str name="color_s">BLACK</str>
+ <int name="price_i">3</int>
+ <long name="_version_">1673055562829398016</long></doc>
+ <doc>
+ <str name="id">P11!D51</str>
+ <str name="type_s">DOC</str>
+ <str name="name_s">Quick Reference Guide</str>
+ <str name="content_t">How to use your stapler ...</str>
+ <long name="_version_">1673055562829398016</long></doc>
+ <doc>
+ <str name="id">P11!D61</str>
+ <str name="type_s">DOC</str>
+ <str name="name_s">Warranty Details</str>
+ <str name="content_t">... lifetime garuntee ...</str>
+ <long name="_version_">1673055562829398016</long></doc></doc>
+</result>
+</response>
+----
+====
-Do *not* add a root document that has the same ID of a child document. _This will violate integrity assumptions that Solr expects._
-To delete a nested document, you can delete it by the ID of the root document.
-If you try to use an ID of a child document, nothing will happen since only root document IDs are considered.
-If you use Solr's delete-by-query APIs, you *have to be careful* to ensure that no children remain of any documents that are being deleted. _Doing otherwise will violate integrity assumptions that Solr expects._
+--
diff --git a/solr/solr-ref-guide/src/other-parsers.adoc b/solr/solr-ref-guide/src/other-parsers.adoc
index 51f97e7..d0bc731 100644
--- a/solr/solr-ref-guide/src/other-parsers.adoc
+++ b/solr/solr-ref-guide/src/other-parsers.adoc
@@ -24,6 +24,16 @@ Many of these parsers are expressed the same way as <<local-parameters-in-querie
== Block Join Query Parsers
+// nocommit: of/which are a PITA to get right in deeply nested docs
+//
+// nocommit: https://issues.apache.org/jira/browse/SOLR-14383?focusedCommentId=17166339&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17166339
+//
+// nocommit: the detailed docs on child/parent parsers need to explain exactly how these should be used
+// nocommit: so people can understand them (and so other sections of the ref-guide can link here for
+// nocommit: actual explanaiton)
+
+
+
There are two query parsers that support block joins. These parsers allow indexing and searching for relational content that has been <<indexing-nested-documents.adoc#indexing-nested-documents, indexed as Nested Documents>>.
The example usage of the query parsers below assumes these two documents and each of their child documents have been indexed:
diff --git a/solr/solr-ref-guide/src/searching-nested-documents.adoc b/solr/solr-ref-guide/src/searching-nested-documents.adoc
index 18e6420..29ea585 100644
--- a/solr/solr-ref-guide/src/searching-nested-documents.adoc
+++ b/solr/solr-ref-guide/src/searching-nested-documents.adoc
@@ -29,174 +29,194 @@ This section does not show case faceting on nested documents. For nested documen
== Query Examples
-For the upcoming examples, assume the following documents have been indexed:
+For the upcoming examples, we'll assume an index containing the same documents covered in <<indexing-nested-documents#example-indexing-syntax,Indexing Nested Documents>>:
-[source,json]
+include::indexing-nested-documents.adoc[tag=sample-indexing-deeply-nested-documents]
+
+=== Child Doc Transformer
+
+By default, documents that match a query do not include any of thir nested children in the response. The `[child]` Doc Transformer Can be used enrich query results with the documents' descendants.
+
+For a detailed explanation of this transformer, and specifics on it's syntax & limitations, please refer to the section <<transforming-result-documents.adoc#child-childdoctransformerfactory, [child] - ChildDocTransformerFactory>>.
+
+A simple query matching all documents with a description that includes "staplers":
+
+[source,bash]
----
-[
- {
- "ID": "1",
- "title": "Cooking Recommendations",
- "tags": ["cooking", "meetup"],
- "posts": [{
- "ID": "2",
- "title": "Cookies",
- "comments": [{
- "ID": "3",
- "content": "Lovely recipe"
- },
- {
- "ID": "4",
- "content": "A-"
- }
- ]
- },
+$ curl 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q=description_t:staplers'
+{
+ "response":{"numFound":1,"start":0,"maxScore":0.30136836,"numFoundExact":true,"docs":[
{
- "ID": "5",
- "title": "Cakes"
- }
- ]
- },
- {
- "ID": "6",
- "title": "For Hire",
- "tags": ["professional", "jobs"],
- "posts": [{
- "ID": "7",
- "title": "Search Engineer",
- "comments": [{
- "ID": "8",
- "content": "I am interested"
- },
- {
- "ID": "9",
- "content": "How large is the team?"
- }
- ]
- },
+ "id":"P11!prod",
+ "name_s":"Swingline Stapler",
+ "description_t":"The Cadillac of office staplers ...",
+ "_version_":1672933224035123200}]
+ }}
+----
+
+The same query with the addition of the `[child]` transformer is shown below. Note that the `numFound` has not changed, we are still matching the same set of documents, but when returning those documents the nested children are also returned as psuedo-fields.
+
+[source,bash]
+----
+$ curl 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q=description_t:staplers&fl=*,[child]'
+{
+ "response":{"numFound":1,"start":0,"maxScore":0.30136836,"numFoundExact":true,"docs":[
{
- "ID": "10",
- "title": "Low level Engineer"
- }
- ]
- }
-]
+ "id":"P11!prod",
+ "name_s":"Swingline Stapler",
+ "description_t":"The Cadillac of office staplers ...",
+ "_version_":1672933224035123200,
+ "skus":[
+ {
+ "id":"P11!S21",
+ "color_s":"RED",
+ "price_i":42,
+ "_version_":1672933224035123200,
+ "docs":[
+ {
+ "id":"P11!D41",
+ "name_s":"Red Swingline Brochure",
+ "content_t":"...",
+ "_version_":1672933224035123200}]},
+
+ {
+ "id":"P11!S31",
+ "color_s":"BLACK",
+ "price_i":3,
+ "_version_":1672933224035123200}],
+ "docs":[
+ {
+ "id":"P11!D51",
+ "name_s":"Quick Reference Guide",
+ "content_t":"How to use your stapler ...",
+ "_version_":1672933224035123200},
+
+ {
+ "id":"P11!D61",
+ "name_s":"Warranty Details",
+ "content_t":"... lifetime garuntee ...",
+ "_version_":1672933224035123200}]}]
+ }}
----
-=== Child Doc Transformer
-Can be used enrich query results with the documents' descendants. +
-For a detailed explanation of this transformer, see the section <<transforming-result-documents.adoc#child-childdoctransformerfactory, [child] - ChildDocTransformerFactory>>.
+=== Child Query Parser
-For example, let us examine this query:
-`q=ID:1,
-fl=ID,[child childFilter=/comments/content:recipe]`. +
-The Child Doc Transformer can be used to enrich matching docs with comments that match a particular filter. +
-In this particular query, the child Filter will only match the first comment of doc(ID:1),
-therefore only that particular comment will be appended to the result. This is a special syntax feature.
+The `{!child}` query parser can be used to search for the _descendent_ documents of parent documents matching a wrapped query. For a detailed explanation of this parser, see the section <<other-parsers.adoc#block-join-children-query-parser, Block Join Children Query Parser>>.
-[source,json]
+Let's consider again the `description_t:staplers` query used above -- if we wrap that query in a `{!child}` query parser then instead of "matching" & returning the product level documents, we instead match all of the _descendent_ child documents of the original query:
+
+[source,bash]
----
- { "response":{"numFound":1,"start":0,"docs":[
- {
- "ID": "1",
- "title": "Cooking Recommendations",
- "tags": ["cooking", "meetup"],
- "posts": [{
- "ID": "2",
- "title": "Cookies",
- "comments": [{
- "ID": "3",
- "content": "Lovely recipe"
- }]
- }]
- }]
- }
- }
+$ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q={!child+of="*:*+-_nest_path_:*"}description_t:staplers'
+{
+ "response":{"numFound":5,"start":0,"maxScore":0.30136836,"numFoundExact":true,"docs":[
+ {
+ "id":"P11!D41",
+ "name_s":"Red Swingline Brochure",
+ "content_t":"...",
+ "_version_":1672933224035123200},
+ {
+ "id":"P11!S21",
+ "color_s":"RED",
+ "price_i":42,
+ "_version_":1672933224035123200},
+ {
+ "id":"P11!S31",
+ "color_s":"BLACK",
+ "price_i":3,
+ "_version_":1672933224035123200},
+ {
+ "id":"P11!D51",
+ "name_s":"Quick Reference Guide",
+ "content_t":"How to use your stapler ...",
+ "_version_":1672933224035123200},
+ {
+ "id":"P11!D61",
+ "name_s":"Warranty Details",
+ "content_t":"... lifetime garuntee ...",
+ "_version_":1672933224035123200}]
+ }}
----
-=== Children Query Parser
+NOTE: The `of` local param is neccessary to tell the `{!child}` parser the set of _all_ "ancestor" documents to consider when looking for matching children. In this example we've used `\*:* -\_nest_path_:*` to indicate we want to consider all documents which don't have a nest path field -- ie: all "root" level document. When dealing with multiple levels of nested documents, it can be very tricky to define a correct `of` param -- see the <<other-parsers.adoc#block-join-children-query-pa [...]
+
+=== Parent Query Parser
-Can be used to retrieve children of a matching document. +
-For a detailed explanation of this parser, see the section <<other-parsers.adoc#block-join-children-query-parser, Block Join Children Query Parser>>.
+The inverse of the `{!child}` query parser is the `{!parent}` query parser, which let's you search for the _ancestor_ documents of some child documents matching a wrapped query. For a detailed explanation of this parser, see the section <<other-parsers.adoc#block-join-parent-query-parser,Block Join Parent Query Parser>>.
-For example, let us examine this query:
-`q={!child of='_nest_path_:/posts}content:"Search Engineer"`. +
-The `'of'` filter returns all posts. This is used to filter out all documents in a particular path of the hierarchy(all parents).
-The second part of the query is a filter for some parents, which we wish to return their children. +
-In this example, all comments of posts which had "Search Engineer in their `content` field will be returned.
+Let's first consider this example of searching for all "sku" type documents that have a color of "RED"...
-[source,json]
+[source,bash]
----
- { "response":{"numFound":2,"start":0,"docs":[
- {
- "ID": "8",
- "content": "I am interested"
- },
- {
- "ID": "9",
- "content": "How large is the team?"
- }
- ]}
- }
+$ curl 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q=color_s:RED'
+{
+ "response":{"numFound":2,"start":0,"maxScore":0.2449984,"numFoundExact":true,"docs":[
+ {
+ "id":"P11!S21",
+ "color_s":"RED",
+ "price_i":42,
+ "_version_":1672933224035123200},
+ {
+ "id":"P22!S22",
+ "color_s":"RED",
+ "price_i":89,
+ "_version_":1672933224436727808}]
+ }}
----
-=== Parents Query Parser
-
-Can be used to retrieve parents of a child document. +
-For a detailed explanation of this parser, see the section <<other-parsers.adoc#block-join-parent-query-parser,Block Join Parent Query Parser>>.
-
-For example, let us examine this query:
-`q={!parent which='-_nest_path_:* \*:*'}title:"Search Engineer"`. +
-The `'which'` filter returns all root documents.
-The second part of this query is a filter to match some child documents.
-This query returns the parent at the root(since all parents filter returns root documents) of each
-matching child document. In this case, all child documents which had `Search Engineer` in their `title` field.
+We can wrap that query in a `{!parent}` query to return the details of all products that have "RED" skus...
-[source,json]
+[source,bash]
----
- { "response":{"numFound":1,"start":0,"docs":[{
- "ID": "6",
- "title": "For Hire",
- "tags": ["professional", "jobs"]
- }
- ]}
- }
+$ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&q={!parent+which="*:*+-_nest_path_:*"}color_s:RED'
+{
+ "response":{"numFound":2,"start":0,"maxScore":1.4E-45,"numFoundExact":true,"docs":[
+ {
+ "id":"P11!prod",
+ "name_s":"Swingline Stapler",
+ "description_t":"The Cadillac of office staplers ...",
+ "_version_":1672933224035123200},
+ {
+ "id":"P22!prod",
+ "name_s":"Mont Blanc Fountain Pen",
+ "description_t":"A Premium Writing Instrument ...",
+ "_version_":1672933224436727808}]
+ }}
----
+
+NOTE: The `which` local param of the `{!parent}` parser serves the same purpse as the `{!child}` parser's `of` param: to define the set of _all_ "ancestor" documents to consider for matching based on their children. In this example we've again used `\*:* -\_nest_path_:*` to indicate we want to consider all documents which don't have a nest path field -- ie: all "root" level document. When dealing with multiple levels of nested documents, it can be very tricky to define a correct `which [...]
+
+
+
=== Combining Block Join Query Parsers with Child Doc Transformer
-The combination of these two features enable seamless creation of powerful queries. +
-For example, querying posts which are under a page tagged as a job, contain the words "Search Engineer".
-The comments for matching posts can also be fetched, all done in a single Solr Query.
+The combination of these two parsers with the `[child] transformer enables seamless creation of very powerful queries.
+
+Here for example is a query where:
-For example, let us examine this query:
-`q=+{!child of='-\_nest_path_:* \*:*'}+tags:"jobs" &fl=*,[child]
-&fq=\_nest_path_:/posts`. +
-This query returns all posts and their comments, which had "Search Engineer" in their title,
-and are indexed under a page tagged with "jobs".
-The comments are appended to the matching posts, since the ChildDocTransformer is specified under the `fl` parameter.
+* the (sku) documents returned must have a color of "RED"
+* the (sku) docments returned must be the descendents of root level (product) documents which have:
+** immediate child "docs" documents which have:
+*** "lifetime garuntee" in their content
+* each return (sku) document also includes any descendent (docs) documents it has
-[source,json]
+[source,bash]
----
- { "response":{"numFound":1,"start":0,"docs":[
- {
- "ID": "7",
- "title": "Search Engineer",
- "comments": [{
- "ID": "8",
- "content": "I am interested"
- },
- {
- "ID": "9",
- "content": "How large is the team?"
- }
- ]
- },
- {
- "ID": "10",
- "title": "Low level Engineer"
- }]
- }
- }
+$ curl --globoff 'http://localhost:8983/solr/gettingstarted/select?omitHeader=true&fq=color_s:RED&q={!child+of="*:*+-_nest_path_:*"+filters=$parent_fq}&parent_fq={!parent+which="*:*+-_nest_path_:*"}_nest_path_:"/docs"+AND+content_t:"lifetime+garuntee"&fl=*,[child]'
+{
+ "response":{"numFound":1,"start":0,"maxScore":1.4E-45,"numFoundExact":true,"docs":[
+ {
+ "id":"P11!S21",
+ "color_s":"RED",
+ "price_i":42,
+ "_version_":1672933224035123200,
+ "docs":[
+ {
+ "id":"P11!D41",
+ "name_s":"Red Swingline Brochure",
+ "content_t":"...",
+ "_version_":1672933224035123200}]}]
+ }}
----
diff --git a/solr/solr-ref-guide/src/transforming-result-documents.adoc b/solr/solr-ref-guide/src/transforming-result-documents.adoc
index 33cb334..95eaf9a 100644
--- a/solr/solr-ref-guide/src/transforming-result-documents.adoc
+++ b/solr/solr-ref-guide/src/transforming-result-documents.adoc
@@ -124,20 +124,24 @@ A default style can be configured by specifying an `args` parameter in your `sol
=== [child] - ChildDocTransformerFactory
-This transformer returns all <<indexing-nested-documents.adoc#indexing-nested-documents,descendant documents>> of each parent document matching your query.
-This is useful when you have indexed nested child documents and want to retrieve the child documents for the relevant parent documents for any type of search query.
+
+This transformer returns all <<indexing-nested-documents.adoc#indexing-nested-documents,descendant documents>> of each parent document matching your query. This is useful when you have indexed nested child documents and want to retrieve the child documents for the relevant parent documents for any type of search query.
+
+Note that this transformer can be used even when the query used to match the result documents is not a <<other-parsers.adoc#block-join-query-parsers,Block Join query>>.
+
[source,plain]
----
-fl=id,[child parentFilter=doc_type:book childFilter=doc_type:chapter limit=100]
+q=book_title:Solr&fl=id,[child parentFilter=doc_type:book childFilter=doc_type:chapter limit=100]
----
-Note that this transformer can be used even though the query itself is not a <<other-parsers.adoc#block-join-query-parsers,Block Join query>>.
+If the documents involved include a `\_nest_path_` field, then it is used to re-create the hierarchical structure of the descendent documents using the original psuedo-field names the documents were indexed with, otherwise the descendent documents are returned as a flat list of <<indexing-nested-documents#indexing-anonymous-children,anonymous children>>.
-When using this transformer, the `parentFilter` parameter must be specified _unless_ the schema declares `\_nest_path_`. It works the same as in all Block Join Queries. Additional optional parameters are:
+`parentFilter`::
+When using a schema that does _not_ include the `\_nest_path_` field, this parameter is mandatory, and serves the same purpose as the `of`/`which` parms in `{!child}`/`{!parent}` query parsers: to identify the set of "all parents" for the purpose of identifying the begining & end of each nested document block. *When a schema _does_ include a `\_nest_path_` field, this parameter is prohibited.*
`childFilter`::
-A query to filter which child documents should be included. This can be particularly useful when you have multiple levels of hierarchical documents. The default is all children. This query supports a special syntax to match nested doc patterns so long as `\_nest_path_` is defined in the schema and the query contains a `/` preceding the first `:`. Example: `childFilter=/comments/content:recipe` Further details of this are experimental.
+A query to filter which child documents should be included. This can be particularly useful when you have multiple levels of hierarchical documents. The default is all children.
`limit`::
The maximum number of child documents to be returned per parent document. The default is `10`.
@@ -148,6 +152,33 @@ The field list which the transformer is to return. The default is the top level
There is a further limitation in which the fields here should be a subset of those specified by the top level `fl` parameter.
+[TIP]
+====
+.Experimental `childFilter` Syntax
+
+When a `\_nest_path_` field is defined, the `childFilter` option supports an experimental syntax to combine a "path syntax" restriction with a more traditional filtering query.
+
+*This syntax is triggered by including a `/` seperated path structure prior to a query that includes a `:` character.*
+
+When the "path" begins with a `/` character, it restricts matches to documents that have that exist "path" of nested psuedo-field documents, starting at the Root document of the block (even if the document being transformed is not a Root level document)
+
+Some Examples:
+
+* `childFilter="/skus/\*:*"`
+** Matches any documents that are descendents of the current document and have a "nested path" of `/skus` -- but not any children of those `skus`
+* childFilter="/skus/color_s:RED"
+** Matches any documents that are descendents of the current document; match `color_s:RED`; and have a "nested path" of `/skus` -- but not any children of those `skus`
+* `childFilter="/skus/docs/\*:*"`
+** Matches any documents that are descendents of the current document and have a "nested path" of `/skus/docs` -- but not any children of those `docs`
+
+When paths do not start with a `/` they are treated as "path suffixes":
+
+* `childFilter="docs/\*:*"`
+** Matches any documents that are descendents of the current document and have a "nested path" that ends with "docs", regardless of how deeply nested they are -- but not any children of those `docs`
+
+====
+
+
=== [shard] - ShardAugmenterFactory
This transformer adds information about what shard each individual document came from in a distributed request.
diff --git a/solr/solr-ref-guide/src/updating-parts-of-documents.adoc b/solr/solr-ref-guide/src/updating-parts-of-documents.adoc
index 23089ba..ecdc447 100644
--- a/solr/solr-ref-guide/src/updating-parts-of-documents.adoc
+++ b/solr/solr-ref-guide/src/updating-parts-of-documents.adoc
@@ -105,105 +105,97 @@ The resulting document in our collection will be:
=== Updating Child Documents
-Solr supports modifying, adding and removing child documents as part of atomic updates. +
-Schema and configuration requirements are detailed in
-<<updating-parts-of-documents#field-storage, Field Storage>> and <<indexing-nested-documents#schema-configuration, Indexing Nested Documents>>. +
-Under the hood, Solr retrieves the whole nested structure, deletes the old documents,
-and reindexes the structure after applying the atomic update. +
-Syntactically, nested/partial updates are very similar to a regular atomic update,
-as demonstrated by the examples below.
-
-[NOTE]
+Solr supports modifying, adding and removing child documents as part of atomic updates. Syntactically, updates changing the children of a document are very similar to a regular atomic updates of simle fields, as demonstrated by the examples below.
+
+Schema and configuration requirements for updating child documents the same <<updating-parts-of-documents#field-storage,Field Storage>> requirements for atomic updates mentioned above, combined with the <<indexing-nested-documents#schema-configuration,schema configuration rules for Indexing Nested Documents>> -- notably:
+* The `\_root_` field must configured with `stored="true"` or `docValues="true"`
+* The `\_nest_path_` field must exist (it is implicitly `docValues="true"`)
+
+Under the hood, When Solr processes atomic updates on nested documents, it retrieves the entire block structure (up to and including the common "Root" document), reindexes the structure after applying the atomic update, and deletes the old documents.
+
+[IMPORTANT]
====
-.\_route_ Parameter
-To ensure each nested update is routed to its respective shard,
-`\_route_` parameter must be set to the root document's ID when the
-update does not have that root document.
+.Routing Updates using child document Ids in SolrCloud
+
+When SolrCloud recieves document updates, the <<shards-and-indexing-data-in-solrcloud#document-routing,document routing>> rules for the collection is used to determine which shard should process the update based on the `id` of the document.
+
+When sending an update that specifies the `id` of a _child document_ this will not work by default: the correct shard to send the document to is based on the `id` of the "Root" document for the block the child document is in, *not* the `id` of the child document being updated.
+
+Solr offers two solutions to address this:
+
+* Clients may specify a <<shards-and-indexing-data-in-solrcloud#document-routing,`\_route_` parameter>>, with the `id` of the Root document as the parameter value, on each update to tell Solr which shard should process the update.
+* Clients can use the (default) `compositeId` router's "prefix routing" feature when indexing all documents to ensure that all child/descendent documents in a Block use the same `id` prefix as the Root level document. This will cause Solr's default routing logic to automatically send child document updates to the correct shard.
+
+All of the examples below use `id` prefixes, so no `\_route_` param will be neccessary for these examples.
====
-If the following document exists in our collection:
+For the upcoming examples, we'll assume an index containing the same documents covered in <<indexing-nested-documents#example-indexing-syntax,Indexing Nested Documents>>:
-[source,json]
-----
-{
- "id":"mydoc",
- "product":"T-Shirt",
- "stock": {
- "id":"mydoc2",
- "color":"red",
- "size": ["L"]
- }
-}
-----
+include::indexing-nested-documents.adoc[tag=sample-indexing-deeply-nested-documents]
-And we apply the following update command:
+==== Modifying Child Document Fields
-[source,json]
+All of the <<#atomic-updates,Atomic Update operations>> mentioned above are supported for "real" fields of Child Documents:
+
+[source,bash]
----
+curl -X POST 'http://localhost:8983/solr/gettingstarted/update?commit=true' -H 'Content-Type: application/json' --data-binary '[
{
- "id":"mydoc",
- "stock": {
- "add":
- {
- "id":"mydoc3",
- "color":"blue",
- "size": ["M"]
- }
- }
-}
+ "id": "P11!S31",
+ "price_i": { "inc": 73 },
+ "color_s": { "set": "GREY" }
+} ]'
----
-The resulting document in our collection will be:
+==== Replacing all child documents
-[source,json]
+As with normal (multiValued) fields, the `set` keyword can be used to replace all child documents in a psuedo-field:
+
+[source,bash]
----
+curl -X POST 'http://localhost:8983/solr/gettingstarted/update?commit=true' -H 'Content-Type: application/json' --data-binary '[
{
- "id":"mydoc",
- "product":"T-Shirt",
- "stock": [{
- "id":"mydoc2",
- "color":"red",
- "size": ["L"]
- },
- {
- "id":"mydoc3",
- "color":"blue",
- "size": ["M"]
- }]
-}
+ "id": "P22!S22",
+ "docs": { "set": [ { "id": "P22!D77",
+ "name_s": "Why Red Pens Are the Best",
+ "content_t": "... correcting papers ...",
+ },
+ { "id": "P22!D88",
+ "name_s": "How to get Red ink stains out of fabric",
+ "content_t": "... vinegar ...",
+ } ] }
+
+} ]'
----
-Documents inside nested structures can also be updated.
-These type of updates require setting the `\_route_` set to the root document's ID
+==== Adding a child document
-If we send this update, setting `\_route_`=mydoc
+As with normal (multiValued) fields, the `add` keyword can be used to add additional child documents to a psuedo-field:
-[source,json]
+[source,bash]
----
+curl -X POST 'http://localhost:8983/solr/gettingstarted/update?commit=true' -H 'Content-Type: application/json' --data-binary '[
{
- "id":"mydoc2",
- "size": {"add": ["S"]}
-}
+ "id": "P11!S21",
+ "docs": { "add": { "id": "P11!D99",
+ "name_s": "Why Red Staplers Are the Best",
+ "content_t": "Once upon a time, Mike Judge ...",
+ } }
+} ]'
----
-The resulting document in our collection will be:
-[source,json]
+==== Removing a child document
+
+As with normal (multiValued) fields, the `remove` keyword can be used to remove a child document (by `id`) from it's psuedo-field:
+
+[source,bash]
----
+curl -X POST 'http://localhost:8983/solr/gettingstarted/update?commit=true' -H 'Content-Type: application/json' --data-binary '[
{
- "id":"mydoc",
- "product":"T-Shirt",
- "stock": [{
- "id":"mydoc2",
- "color":"red",
- "size": ["L", "S"]
- },
- {
- "id":"mydoc3",
- "color":"blue",
- "size": ["M"]
- }]
-}
+ "id": "P11!S21",
+ "docs": { "remove": { "id": "P11!D41" } }
+} ]'
----