You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by ge...@apache.org on 2019/02/07 02:17:46 UTC
[lucene-solr] branch branch_8x updated: SOLR-13042: Miscellaneous
JSON Faceting ref-guide improvements
This is an automated email from the ASF dual-hosted git repository.
gerlowskija pushed a commit to branch branch_8x
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
The following commit(s) were added to refs/heads/branch_8x by this push:
new 199725d SOLR-13042: Miscellaneous JSON Faceting ref-guide improvements
199725d is described below
commit 199725d5d59c684631d78ce1a8811f3668266bf5
Author: Jason Gerlowski <ge...@apache.org>
AuthorDate: Wed Feb 6 19:52:22 2019 -0500
SOLR-13042: Miscellaneous JSON Faceting ref-guide improvements
---
solr/solr-ref-guide/src/blockjoin-faceting.adoc | 2 +-
solr/solr-ref-guide/src/json-facet-api.adoc | 779 ++++++++++-----------
.../src/json-faceting-domain-changes.adoc | 250 +++++++
solr/solr-ref-guide/src/json-query-dsl.adoc | 346 +++++++--
solr/solr-ref-guide/src/json-request-api.adoc | 218 +++---
.../JsonRequestApiHeatmapFacetingTest.java | 108 +++
.../ref_guide_examples/JsonRequestApiTest.java | 563 ++++++++++++++-
7 files changed, 1710 insertions(+), 556 deletions(-)
diff --git a/solr/solr-ref-guide/src/blockjoin-faceting.adoc b/solr/solr-ref-guide/src/blockjoin-faceting.adoc
index fdfd893..18a7408 100644
--- a/solr/solr-ref-guide/src/blockjoin-faceting.adoc
+++ b/solr/solr-ref-guide/src/blockjoin-faceting.adoc
@@ -20,7 +20,7 @@ BlockJoin facets allow you to aggregate children facet counts by their parents.
It is a common requirement that if a parent document has several children documents, all of them need to increment facet value count only once. This functionality is provided by `BlockJoinDocSetFacetComponent`, and `BlockJoinFacetComponent` just an alias for compatibility.
-CAUTION: This functionality is considered deprecated. Users are encouraged to use `uniqueBlock(\_root_)` aggregation under a `terms` facet in the <<json-facet-api.adoc#block-join-counts,JSON Facet API>>.
+CAUTION: This functionality is considered deprecated. Users are encouraged to use `uniqueBlock(\_root_)` aggregation under a `terms` facet in the <<json-faceting-domain-changes.adoc#block-join-domain-changes,JSON Facet API>>.
If this component is used, it must be explicitly enabled for a request handler in `solrconfig.xml`, in the same way as any other <<requesthandlers-and-searchcomponents-in-solrconfig.adoc#requesthandlers-and-searchcomponents-in-solrconfig,search component>>.
This example shows how you could add this search components to `solrconfig.xml` and define it in request handler:
diff --git a/solr/solr-ref-guide/src/json-facet-api.adoc b/solr/solr-ref-guide/src/json-facet-api.adoc
index 87e4487..7655576 100644
--- a/solr/solr-ref-guide/src/json-facet-api.adoc
+++ b/solr/solr-ref-guide/src/json-facet-api.adoc
@@ -2,6 +2,7 @@
:page-tocclass: right
:solr-root-path: ../../
:example-source-dir: {solr-root-path}solrj/src/test/org/apache/solr/client/ref_guide_examples/
+:page-children: json-faceting-domain-changes
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
@@ -21,18 +22,12 @@
== Facet & Analytics Module
-The new Facet & Analytics Module exposed via the JSON Facet API is a rewrite of Solr's previous faceting capabilities, with the following goals:
+The JSON Faceting module exposes similar functionality to Solr's traditional faceting module but with a stronger emphasis on usability. It has several benefits over traditional faceting:
-* First class native JSON API to control faceting and analytics
-** The structured nature of nested sub-facets are more naturally expressed in JSON rather than the flat namespace provided by normal query parameters.
-* First class integrated analytics support
-* Nest any facet type under any other facet type (such as range facet, field facet, query facet)
-* Ability to sort facet buckets by any calculated metric
-* Easier programmatic construction of complex nested facet commands
-* Support a more canonical response format that is easier for clients to parse
-* Support a cleaner way to implement distributed faceting
-* Support better integration with other search features
-* Full integration with the JSON Request API
+* easier programmatic construction of complex or nested facets
+* the nesting and structure offered by JSON makes facets easier to read and understand than the flat namespace of the traditional faceting API.
+* first class support for metrics and analytics
+* more standardized response format makes responses easier for clients to parse and use
== Faceted Search
@@ -43,35 +38,6 @@ There are two main types of facets:
* Facets that partition or categorize data (the domain) into multiple *buckets*
* Facets that calculate data for a given bucket (normally a metric, statistic or analytic function)
-=== Metrics Example
-
-By default, the *domain* for facets starts with all documents that match the base query and any filters. Here's an example that requests various metrics about the root domain:
-
-[source,bash]
-----
-curl http://localhost:8983/solr/techproducts/query -d '
-q=memory&
-fq=inStock:true&
-json.facet={
- "avg_price" : "avg(price)",
- "num_suppliers" : "unique(manu_exact)",
- "median_weight" : "percentile(weight,50)"
-}'
-----
-
-The response to the facet request above will start with documents matching the root domain (docs containing "memory" with inStock:true) then calculate and return the requested metrics:
-
-[...]
-[source,java]
-----
- "facets" : {
- "count" : 4,
- "avg_price" : 109.9950008392334,
- "num_suppliers" : 3,
- "median_weight" : 352.0
- }
-----
-
=== Bucketing Facet Example
Here's an example of a bucketing facet, that partitions documents into bucket based on the `cat` field (short for category), and returns the top 3 buckets:
@@ -83,12 +49,15 @@ Here's an example of a bucketing facet, that partitions documents into bucket ba
[.tab-label]*curl*
[source,bash]
----
-curl http://localhost:8983/solr/techproducts/query -d 'q=*:*&
-json.facet={
- categories : {
- type : terms,
- field : cat, // bucket documents based on the "cat" field
- limit : 3 // retrieve the top 3 buckets ranked by the number of docs in each bucket
+curl http://localhost:8983/solr/techproducts/query -d '
+{
+ "query": "*:*",
+ "facet": {
+ "categories" : {
+ "type": "terms",
+ "field": "cat",
+ "limit": 3
+ }
}
}'
----
@@ -127,88 +96,96 @@ The response below shows us that 32 documents match the default root domain. and
}
----
-=== Making a Facet Request
+=== Stat Facet Example
-In this guide, we will often just present the **facet command block**:
-
-[source,java]
-----
-{
- x: "average(mul(price,popularity))"
-}
-----
-
-To execute a facet command block such as this, you'll need to use the `json.facet` parameter, and provide at least a base query such as `q=\*:*`
-
-[source,bash]
-----
-curl http://localhost:8983/solr/techproducts/query -d 'q=*:*&json.facet=
-{
- x: "avg(mul(price,popularity))"
-}
-'
-----
-
-Another option is to use the JSON Request API to provide the entire request in JSON:
+Stat (also called `aggregation` or `analytic`) facets are useful for displaying information derived from query results, in addition to those results themselves. For example, stat facets can be used to provide context to users on an e-commerce site looking for memory. The example below computes the average price (and other statistics) and would allow a user to gauge whether the memory stick in their cart is a good price.
[.dynamic-tabs]
--
-[example.tab-pane#curljsontermsfacet2]
+[example.tab-pane#curl-json-metrics-facet-1]
====
[.tab-label]*curl*
[source,bash]
----
curl http://localhost:8983/solr/techproducts/query -d '
-{
- query: "*:*", // this is the base query
- filter: [ "inStock:true" ], // a list of filters
- facet: {
- x: "avg(mul(price,popularity))" // and our funky metric of average of price * popularity
- }
-}
-'
+q=memory&
+fq=inStock:true&
+json.facet={
+ "avg_price" : "avg(price)",
+ "num_suppliers" : "unique(manu_exact)",
+ "median_weight" : "percentile(weight,50)"
+}'
----
====
-[example.tab-pane#solrjjsontermsfacet2]
+[example.tab-pane#solrj-json-metrics-facet-1]
====
[.tab-label]*SolrJ*
[source,java,indent=0]
----
-include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-terms-facet2]
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-metrics-facet-1]
----
====
--
-=== JSON Extensions
+The response to the facet request above will start with documents matching the root domain (docs containing "memory" with inStock:true) followed by the requested statistics in a `facets` block:
-The *Noggit* JSON parser that is used by Solr accepts a number of JSON extensions such as,
+[...]
+[source,java]
+----
+ "facets" : {
+ "count" : 4,
+ "avg_price" : 109.9950008392334,
+ "num_suppliers" : 3,
+ "median_weight" : 352.0
+ }
+----
+
+
+== Types of Facets
+There are 4 different types of bucketing facets, which behave in two different ways:
-* bare words can be left unquoted
-* single line comments using either `//` or `#`
-* Multi-line comments using C style /* comments in here */
-* Single quoted strings
-* Allow backslash escaping of any character
-* Allow trailing commas and extra commas. Example: [9,4,3,]
-* Handle nbsp (non-break space, \u00a0) as whitespace.
+* "terms" and "range" facets produce multiple buckets and assign each document in the domain into one (or more) of these buckets
+* "query" and "heatmap" facets always produce a single bucket which all documents in the domain belong to
-== Terms Facet
+Each of these facet-types are covered in detail below.
-The terms facet (or field facet) buckets the domain based on the unique terms / values of a field.
+=== Terms Facet
+A `terms` facet buckets the domain based on the unique values in a field.
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-terms-facet-2]
+====
+[.tab-label]*curl*
[source,bash]
----
-curl http://localhost:8983/solr/techproducts/query -d 'q=*:*&
-json.facet={
- categories:{
- terms: {
- field : cat, // bucket documents based on the "cat" field
- limit : 5 // retrieve the top 5 buckets ranked by the number of docs in each bucket
+curl http://localhost:8983/solr/techproducts/query -d '
+{
+ "query": "*:*",
+ "facet": {
+ categories:{
+ "type": "terms",
+ "field" : "cat",
+ "limit" : 5
}
}
}'
----
+====
+
+[example.tab-pane#solrj-json-terms-facet-2]
+====
+[.tab-label]*SolrJ*
+
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-terms-facet-2]
+----
+====
+--
// TODO: This table has cells that won't work with PDF: https://github.com/ctargett/refguide-asciidoc-poc/issues/13
@@ -220,7 +197,7 @@ json.facet={
|limit |Limits the number of buckets returned. Defaults to 10.
|sort |Specifies how to sort the buckets produced.
-“count” specifies document count, “index” sorts by the index (natural) order of the bucket value. One can also sort by any <<json-facet-api.adoc#aggregation-functions,facet function / statistic>> that occurs in the bucket. The default is “count desc”. This parameter may also be specified in JSON like `sort:{count:desc}`. The sort order may either be “asc” or “desc”
+“count” specifies document count, “index” sorts by the index (natural) order of the bucket value. One can also sort by any <<json-facet-api.adoc#stat-facet-functions,facet function / statistic>> that occurs in the bucket. The default is “count desc”. This parameter may also be specified in JSON like `sort:{count:desc}`. The sort order may either be “asc” or “desc”
|overrequest a|
Number of buckets beyond the `limit` to internally request from shards during a distributed search.
@@ -253,31 +230,76 @@ This parameter indicates the facet algorithm to use:
|prelim_sort |An optional parameter for specifying an approximation of the final `sort` to use during initial collection of top buckets when the <<json-facet-api.adoc#sorting-facets-by-nested-functions,`sort` param is very costly>>.
|===
-== Query Facet
+=== Query Facet
The query facet produces a single bucket of documents that match the domain as well as the specified query.
-An example of the simplest form of the query facet is `"query":"query string"`.
-
-[source,java]
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-query-facet-simple]
+====
+[.tab-label]*curl*
+[source,bash]
----
+curl http://localhost:8983/solr/techproducts/query -d '
{
- high_popularity: {query: "popularity:[8 TO 10]" }
-}
+ "query": "*:*",
+ "facet": {
+ "high_popularity": {
+ "type": "query",
+ "q": "popularity:[8 TO 10]"
+ }
+ }
+}'
----
+====
-An expanded form allows for more parameters and a facet command block to specify sub-facets (either nested facets or metrics):
+[example.tab-pane#solrj-json-query-facet-simple]
+====
+[.tab-label]*SolrJ*
-[source,java]
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-query-facet-simple]
----
+====
+--
+
+Users may also specify sub-facets (either "bucketing" facets or metrics):
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-query-facet-expanded]
+====
+[.tab-label]*curl*
+[source,bash]
+----
+curl http://localhost:8983/solr/techproducts/query -d '
{
- high_popularity : {
- type: query,
- q : "popularity:[8 TO 10]",
- facet : { average_price : "avg(price)" }
+ "query": "*:*",
+ "facet": {
+ "high_popularity": {
+ "type": "query",
+ "q": "popularity:[8 TO 10]",
+ "facet" : {
+ "average_price" : "avg(price)"
+ }
+ }
}
-}
+}'
----
+====
+
+[example.tab-pane#solrj-json-query-facet-expanded]
+====
+[.tab-label]*SolrJ*
+
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-query-facet-expanded]
+----
+====
+--
Example response:
@@ -289,24 +311,45 @@ Example response:
}
----
-== Range Facet
-
-The range facet produces multiple buckets over a date field or numeric field.
+=== Range Facet
-Example:
+The range facet produces multiple buckets over a date or numeric field.
-[source,java]
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-range-facet-simple]
+====
+[.tab-label]*curl*
+[source,bash]
----
+curl http://localhost:8983/solr/techproducts/query -d '
{
- prices : {
- type: range,
- field : price,
- start : 0,
- end : 100,
- gap : 20
+ "query": "*:*",
+ "facet": {
+ "prices": {
+ "type": "range",
+ "field": "price",
+ "start": 0,
+ "end": 100,
+ "gap": 20
+ }
}
-}
+}'
+----
+====
+
+[example.tab-pane#solrj-json-range-facet-simple]
+====
+[.tab-label]*SolrJ*
+
+[source,java,indent=0]
----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-range-facet-simple]
+----
+====
+--
+
+The output from the range facet above would look a bit like:
[source,java]
----
@@ -317,10 +360,10 @@ Example:
"count":5},
{
"val":20.0,
- "count":3},
+ "count":0},
{
"val":40.0,
- "count":2},
+ "count":0},
{
"val":60.0,
"count":1},
@@ -331,9 +374,9 @@ Example:
}
----
-=== Range Facet Parameters
+==== Range Facet Parameters
-To ease migration, the range facet parameter names and semantics largely mirror facet.range query-parameter style faceting. For example "start" here corresponds to "facet.range.start" in a facet.range command.
+Range facet parameter names and semantics largely mirror facet.range query-parameter style faceting. For example "start" here corresponds to "facet.range.start" in a facet.range command.
// TODO: This table has cells that won't work with PDF: https://github.com/ctargett/refguide-asciidoc-poc/issues/13
@@ -366,7 +409,7 @@ By default, the ranges used to compute range faceting between `start` and `end`
|facet |Aggregations, metrics, or nested facets that will be calculated for every returned bucket
|===
-== Heatmap Facet
+=== Heatmap Facet
The `heatmap` facet generates a 2D grid of facet counts for documents having spatial data in each grid cell.
@@ -378,27 +421,45 @@ For example `geom` here corresponds to `facet.heatmap.geom` in a facet.heatmap c
NOTE: Unlike other facets that partition the domain into buckets, `heatmap` facets do not currently support <<Nested Facets>>.
-
-Here's an example query:
-[source,java]
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-heatmap-facet-1]
+====
+[.tab-label]*curl*
+[source,bash]
----
+curl http://localhost:8983/solr/spatialdata/query -d '
{
- hm : {
- type : heatmap,
- field : points_srpt,
- geom : "[-49.492,-180 TO 64.701,73.125]",
- distErrPct : 0.5
+ "query": "*:*",
+ "facet": {
+ "locations": {
+ "type": "heatmap",
+ "field": "location_srpt",
+ "geom": "[\"50 20\" TO \"180 90\"]",
+ "gridLevel": 4
+ }
}
-}
+}'
----
+====
+
+[example.tab-pane#solrj-json-heatmap-facet-1]
+====
+[.tab-label]*SolrJ*
+
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiHeatmapFacetingTest.java[tag=solrj-json-heatmap-facet-1]
+----
+====
+--
And the facet response will look like:
[source,json]
----
{
-"facets":{
- "count":145725,
- "hm":{
+ "facets": {
+ "locations":{
"gridLevel":1,
"columns":6,
"rows":4,
@@ -407,10 +468,12 @@ And the facet response will look like:
"minY":-90.0,
"maxY":90.0,
"counts_ints2D":[[68,1270,459,5359,39456,1713],[123,10472,13620,7777,18376,6239],[88,6,3898,989,1314,255],[0,0,30,1,0,1]]
- }}}
+ }
+ }
+}
----
-== Aggregation Functions
+=== Stat Facet Functions
Unlike all the facets discussed so far, Aggregation functions (also called *facet functions*, *analytic functions*, or *metrics*) do not partition data into buckets. Instead, they calculate something over all the documents in the domain.
@@ -422,40 +485,89 @@ Unlike all the facets discussed so far, Aggregation functions (also called *face
|min |`min(salary)` |minimum value
|max |`max(mul(price,popularity))` |maximum value
|unique |`unique(author)` |number of unique values of the given field. Beyond 100 values it yields not exact estimate
-|uniqueBlock |`uniqueBlock(\_root_)` |same as above with smaller footprint strictly for <<json-facet-api.adoc#block-join-counts,counting the number of Block Join blocks>>. The given field must be unique across blocks, and only singlevalued string fields are supported, docValues are recommended.
+|uniqueBlock |`uniqueBlock(\_root_)` |same as above with smaller footprint strictly for <<json-faceting-domain-changes.adoc#block-join-domain-changes,counting the number of Block Join blocks>>. The given field must be unique across blocks, and only singlevalued string fields are supported, docValues are recommended.
|hll |`hll(author)` |distributed cardinality estimate via hyper-log-log algorithm
|percentile |`percentile(salary,50,75,99,99.9)` |Percentile estimates via t-digest algorithm. When sorting by this metric, the first percentile listed is used as the sort value.
|sumsq |`sumsq(rent)` |sum of squares of field or function
|variance |`variance(rent)` |variance of numeric field or function
|stddev |`stddev(rent)` |standard deviation of field or function
-|relatedness |`relatedness('popularity:[100 TO *]','inStock:true')`|A function for computing a relatedness score of the documents in the domain to a Foreground set, relative to a Background set (both defined as queries). This is primarily for use when building <<Semantic Knowledge Graphs>>.
+|relatedness |`relatedness('popularity:[100 TO *]','inStock:true')`|A function for computing a relatedness score of the documents in the domain to a Foreground set, relative to a Background set (both defined as queries). This is primarily for use when building <<json-facet-api.adoc#relatedness-and-semantic-knowledge-graphs,Semantic Knowledge Graphs>>.
|===
Numeric aggregation functions such as `avg` can be on any numeric field, or on a <<function-queries.adoc#function-queries,nested function>> of multiple numeric fields such as `avg(div(popularity,price))`.
-The most common way of requesting an aggregation function is as a simple containing the expression you wish to compute:
+The most common way of requesting an aggregation function is as a simple String containing the expression you wish to compute:
-[source,javascript]
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-metrics-facet-simple]
+====
+[.tab-label]*curl*
+[source,bash]
----
+curl http://localhost:8983/solr/techproducts/query -d '
{
- "average_roi": "avg(div(popularity,price))"
-}
+ "query": "*:*",
+ "filter": [
+ "price:[1.0 TO *]",
+ "popularity:[0 TO 10]"
+ ],
+ "facet": {
+ "avg_value": "avg(div(popularity,price))"
+ }
+}'
+----
+====
+
+[example.tab-pane#solrj-json-metrics-facet-simple]
+====
+[.tab-label]*SolrJ*
+
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-metrics-facet-simple]
----
+====
+--
An expanded form allows for <<local-parameters-in-queries.adoc#local-parameters-in-queries,Local Parameters>> to be specified. These may be used explicitly by some specialized aggregations such as `<<json-facet-api.adoc#relatedness-options,relatedness()>>`, but can also be used as parameter references to make aggregation expressions more readable, with out needing to use (global) request parameters:
-[source,javascript]
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-metrics-facet-expanded]
+====
+[.tab-label]*curl*
+[source,bash]
----
+curl http://localhost:8983/solr/techproducts/query -d '
{
- "average_roi" : {
- "type": "func",
- "func": "avg(div($numer,$denom))",
- "numer": "mul(popularity,rating)",
- "denom": "mul(price,size)"
+ "query": "*:*",
+ "filter": [
+ "price:[1.0 TO *]",
+ "popularity:[0 TO 10]"
+ ],
+ "facet": {
+ "avg_value" : {
+ "type": "func",
+ "func": "avg(div($numer,$denom))",
+ "numer": "mul(popularity,3.0)",
+ "denom": "price"
+ }
}
-}
+}'
----
+====
+
+[example.tab-pane#solrj-json-metrics-facet-expanded]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-metrics-facet-expanded]
+----
+====
+--
== Nested Facets
@@ -466,91 +578,147 @@ The syntax is identical to top-level facets - just add a `facet` command to the
=== Nested Facet Example
-Let's start off with a simple non-nested terms facet on the genre field:
+Let's start off with a simple non-nested terms facet on the category field `cat`:
-[source,java]
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-unnested-cat-facet]
+====
+[.tab-label]*curl*
+[source,bash]
----
- top_genres:{
- type: terms
- field: genre,
- limit: 5
+curl http://localhost:8983/solr/techproducts/query -d '
+{
+ "query": "*:*",
+ "facet": {
+ "categories": {
+ "type": "terms",
+ "field": "cat",
+ "limit": 3
+ }
}
+}'
----
+====
-Now if we wanted to add a nested facet to find the top 2 authors for each genre bucket:
+[example.tab-pane#solrj-json-unnested-cat-facet]
+====
+[.tab-label]*SolrJ*
-[source,java]
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-simple-terms-facet]
----
- top_genres:{
- type: terms,
- field: genre,
- limit: 5,
- facet:{
- top_authors:{
- type: terms, // nested terms facet on author will be calculated for each parent bucket (genre)
- field: author,
- limit: 2
+====
+--
+
+The response for the facet above will show the top category and the number of documents that falls into each category bucket. Nested facets can be used to gather additional information about each bucket of documents. For example, using the nested facet below, we can find the top categories as well as who the leading manufacturer is in each category:
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-nested-cat-facet]
+====
+[.tab-label]*curl*
+[source,bash]
+----
+curl http://localhost:8983/solr/techproducts/query -d '
+{
+ "query": "*:*",
+ "facet": {
+ "categories": {
+ "type": "terms",
+ "field": "cat",
+ "limit": 3,
+ "facet": {
+ "top_manufacturer": {
+ "type": "terms",
+ "field": "manu_id_s",
+ "limit": 1
+ }
}
}
}
+}'
----
+====
+
+[example.tab-pane#solrj-json-nested-cat-facet]
+====
+[.tab-label]*SolrJ*
+
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-nested-cat-facet]
+----
+====
+--
And the response will look something like:
[source,java]
----
- "facets":{
- "top_genres":{
- "buckets":[
- {
- "val":"Fantasy",
- "count":5432,
- "top_authors":{ // these are the top authors in the "Fantasy" genre
+"facets":{
+ "count":32,
+ "categories":{
+ "buckets":[{
+ "val":"electronics",
+ "count":12,
+ "top_manufacturer":{
"buckets":[{
- "val":"Mercedes Lackey",
- "count":121},
- {
- "val":"Piers Anthony",
- "count":98}
- ]
- }
- },
+ "val":"corsair",
+ "count":3}]}},
{
- "val":"Mystery",
- "count":4322,
- "top_authors":{ // these are the top authors in the "Mystery" genre
+ "val":"currency",
+ "count":4,
+ "top_manufacturer":{
"buckets":[{
- "val":"James Patterson",
- "count":146},
- {
- "val":"Patricia Cornwell",
- "count":132}
- ]
- }
- }
+ "val":"boa",
+ "count":1}]}},
----
-By default "top authors" is defined by simple document count descending, but we could use our aggregation functions to sort by more interesting metrics.
-
=== Sorting Facets By Nested Functions
The default sort for a field or terms facet is by bucket count descending. We can optionally `sort` ascending or descending by any facet function that appears in each bucket.
-[source,java]
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-nested-cat-facet-sorted]
+====
+[.tab-label]*curl*
+[source,bash]
----
+curl http://localhost:8983/solr/techproducts/query -d '
{
- categories:{
- type : terms, // terms facet creates a bucket for each indexed term in the field
- field : cat,
- sort : "x desc", // can also use sort:{x:desc}
- facet : {
- x : "avg(price)", // x = average price for each facet bucket
- y : "max(popularity)" // y = max popularity value in each facet bucket
+ "query": "*:*",
+ "facet": {
+ "categories":{
+ "type" : "terms", // terms facet creates a bucket for each indexed term in the field
+ "field" : "cat",
+ "limit": 3,
+ "sort" : "avg_price desc",
+ "facet" : {
+ "avg_price" : "avg(price)",
+ }
}
}
-}
+}'
----
+====
+
+[example.tab-pane#solrj-json-nested-cat-facet-sorted]
+====
+[.tab-label]*SolrJ*
+
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-nested-cat-facet-sorted]
+----
+====
+--
+
+
In some situations the desired `sort` may be an aggregation function that is very costly to compute for every bucket. A `prelim_sort` option can be used to specify an approximation of the `sort`, for initially ranking the buckets to determine the top candidates (based on the `limit` and `overrequest`). Only after the top candidate buckets have been refined, will the actual `sort` be used.
@@ -576,178 +744,21 @@ In some situations the desired `sort` may be an aggregation function that is ver
== Changing the Domain
-As discussed above, facets compute buckets or statistics based on a "domain" which is typically implicit:
-
- * By default, facets use the set of all documents matching the main query as their domain.
- * Nested "sub-facets" are computed for every bucket of their parent facet, using a domain containing all documents in that bucket.
-
-But users can also override the "domain" of a facet that partitions data, using an explicit `domain` attribute whose value is a JSON Object that can support various options for restricting, expanding, or completely changing the original domain before the buckets are computed for the associated facet.
-
-[NOTE]
-====
-`domain` changes can only be specified on individual facets that do data partitioning -- not statistical/metric facets, or groups of facets.
-
-A `\*:*` query facet with a `domain` change can be used to group multiple sub-facets of any type, for the purpose of applying a common domain change.
-====
-
-
-=== Adding Domain Filters
+As discussed above, facets compute buckets or statistics based on their "domain" of documents.
-The simplest example of a domain change is to specify an additional filter which will be applied to the existing domain. This can be done via the `filter` keyword in the `domain` block of the facet.
+* By default, top-level facets use the set of all documents matching the main query as their domain.
+* Nested "sub-facets" are computed for every bucket of their parent facet, using a domain containing all documents in that bucket.
-Example:
-[source,json]
-----
-{
- categories: {
- type: terms,
- field: cat,
- domain: {filter: "popularity:[5 TO 10]" }
- }
-}
-----
-
-The value of `filter` can be a single query to treat as a filter, or a JSON list of filter queries. Each query can be:
+In addition to this default behavior, domains can be also be widened, narrowed, or changed entirely. The JSON Faceting API supports modifying domains through its `domain` property. This is discussed in more detail <<json-faceting-domain-changes.adoc#json-faceting-domain-changes,here>>
-* a string containing a query in Solr query syntax
-* a reference to a request parameter containing Solr query syntax, of the form: `{param : <request_param_name>}`
+== Special Stat Facet Functions
-When a `filter` option is combined with other `domain` changing options, the filtering is applied _after_ the other domain changes take place.
+Most stat facet functions (`avg`, `sumsq`, etc.) allow users to perform math computations on groups of documents. A few functions are more involved though, and deserve an explanation of their own. These are described in more detail in the sections below.
-=== Filter Exclusions
-
-Domains can also exclude the top-level query or filters via the `excludeTags` keywords in the `domain` block of the facet, expanding the existing domain.
-
-Example:
-[source,bash]
-----
-&q={!tag=top}"running shorts"
-&fq={!tag=COLOR}color:Blue
-&json={
- filter:"{!tag=BRAND}brand:Bosco"
- facet:{
- sizes:{type:terms, field:size},
- colors:{type:terms, field:color, domain:{excludeTags:COLOR} },
- brands:{type:terms, field:brand, domain:{excludeTags:"BRAND,top"} }
- }
-}
-----
-
-The value of `excludeTags` can be a single string tag, array of string tags or comma-separated tags in the single string.
-
-When an `excludeTags` option is combined with other `domain` changing options, it expands the domain _before_ any other domain changes take place.
-
-See also the section on <<faceting.adoc#tagging-and-excluding-filters,multi-select faceting>>.
-
-=== Arbitrary Domain Query
-
-A `query` domain can be specified when you wish to compute a facet against an arbitrary set of documents, regardless of the original domain. The most common use case would be to compute a top level facet against a specific subset of the collection, regardless of the main query. But it can also be useful on nested facets when building <<Semantic Knowledge Graphs>>.
-
-Example:
-[source,json]
-----
-{
- "categories": {
- "type": "terms",
- "field": "cat",
- "domain": {"query": "*:*" }
- }
-}
-----
+=== uniqueBlock() and Block Join Counts
-The value of `query` can be a single query, or a JSON list of queries. Each query can be:
-
-* a string containing a query in Solr query syntax
-* a reference to a request parameter containing Solr query syntax, of the form: `{param: <request_param_name>}`
-
-NOTE: While a `query` domain can be combined with an additional domain `filter`, It is not possible to also use `excludeTags`, because the tags would be meaningless: The `query` domain already completely ignores the top-level query and all previous filters.
-
-=== Block Join Domain Changes
-
-When a collection contains <<uploading-data-with-index-handlers.adoc#nested-child-documents, Block Join child documents>>, the `blockChildren` or `blockParent` domain options can be used transform an existing domain containing one type of document, into a domain containing the documents with the specified relationship (child or parent of) to the documents from the original domain.
-
-Both of these options work similar to the corresponding <<other-parsers.adoc#block-join-query-parsers,Block Join Query Parsers>> by taking in a single String query that exclusively matches all parent documents in the collection. If `blockParent` is used, then the resulting domain will contain all parent documents of the children from the original domain. If If `blockChildren` is used, then the resulting domain will contain all child documents of the parents from the original domain.
-
-Example:
-[source,json,subs="verbatim,callouts"]]
-----
-{
- "colors": { // <1>
- "type": "terms",
- "field": "sku_color", // <2>
- "facet" : {
- "brands" : {
- "type": "terms",
- "field": "product_brand", // <3>
- "domain": {
- "blockParent": "doc_type:product"
- }
- }}}}
-----
-<1> This example assumes we parent documents corresponding to Products, with child documents corresponding to individual SKUs with unique colors, and that our original query was against SKU documents.
-<2> The `colors` facet will be computed against all of the original SKU documents matching our search.
-<3> For each bucket in the `colors` facet, the set of all matching SKU documents will be transformed into the set of corresponding parent Product documents. The resulting `brands` sub-facet will count how many Product documents (that have SKUs with the associated color) exist for each Brand.
-
-=== Join Query Domain Changes
-
-A `join` domain change option can be used to specify arbitrary `from` and `to` fields to use in transforming from the existing domain to a related set of documents.
-
-This works very similar to the <<other-parsers.adoc#join-query-parser,Join Query Parser>>, and has the same limitations when dealing with multi-shard collections.
-
-Example:
-[source,json]
-----
-{
- "colors": {
- "type": "terms",
- "field": "sku_color",
- "facet": {
- "brands": {
- "type": "terms",
- "field": "product_brand",
- "domain" : {
- "join" : {
- "from": "product_id_of_this_sku",
- "to": "id"
- },
- "filter": "doc_type:product"
- }
- }
- }
- }
-}
-
-----
-
-=== Graph Traversal Domain Changes
-
-A `graph` domain change option works similarly to the `join` domain option, but can do traversal multiple hops `from` the existing domain `to` other documents.
-
-This works very similar to the <<other-parsers.adoc#graph-query-parser,Graph Query Parser>>, supporting all of it's optional parameters, and has the same limitations when dealing with multi-shard collections.
-
-Example:
-[source,json]
-----
-{
- "related_brands": {
- "type": "terms",
- "field": "brand",
- "domain": {
- "graph": {
- "from": "related_product_ids",
- "to": "id",
- "maxDepth": 3
- }
- }
- }
-}
-----
-
-== Block Join Counts
-
-When a collection contains <<uploading-data-with-index-handlers.adoc#nested-child-documents, Block Join child documents>>, the `blockChildren` and `blockParent` domain changes mentioned above can be useful when searching for parent documents and you want to compute stats against all of the affected children documents (or vice versa). But in the situation where the _count_ of all the blocks that exist in the current domain is sufficient, a more efficient option is the `uniqueBlock()` agg [...]
-
-=== Block Join Counts Example
+When a collection contains <<uploading-data-with-index-handlers.adoc#nested-child-documents, Block Join child documents>>, the `blockChildren` and `blockParent` <<json-faceting-domain-changes.adoc#block-join-domain-changes, domain changes>> can be useful when searching for parent documents and you want to compute stats against all of the affected children documents (or vice versa).
+But if you only need to know the _count_ of all the blocks that exist in the current domain, a more efficient option is the `uniqueBlock()` aggregate function.
Suppose we have products with multiple SKUs, and we want to count products for each color.
@@ -800,9 +811,9 @@ Aggregation `uniqueBlock(\_root_)` is functionally equivalent to `unique(\_root_
It's recommended to define `limit: -1` for `uniqueBlock` calculation, like in above example,
since default value of `limit` parameter is `10`, while `uniqueBlock` is supposed to be much faster with `-1`.
-== Semantic Knowledge Graphs
+=== relatedness() and Semantic Knowledge Graphs
-The `relatedness(...)` aggregation function allows for sets of documents to be scored relative to Foreground and Background sets of documents, for the purposes of finding ad-hoc relationships that make up a "Semantic Knowledge Graph":
+The `relatedness(...)` stat function allows for sets of documents to be scored relative to Foreground and Background sets of documents, for the purposes of finding ad-hoc relationships that make up a "Semantic Knowledge Graph":
[quote, Grainger et al., 'https://arxiv.org/abs/1609.00464[The Semantic Knowledge Graph]']
____
@@ -816,7 +827,7 @@ Unlike most aggregation functions, the `relatedness(...)` function is aware of w
NOTE: While it's very common to define the Background Set as `\*:*`, or some other super-set of the Foreground Query, it is not strictly required. The `relatedness(...)` function can be used to compare the statistical relatedness of sets of documents to orthogonal foreground/background queries.
[[relatedness-options]]
-=== relatedness() Options
+==== relatedness() Options
When using the extended `type:func` syntax for specifying a `relatedness()` aggregation, an opional `min_popularity` (float) option can be used to specify a lower bound on the `foreground_popularity` and `background_popularity` values, that must be met in order for the `relatedness` score to be valid -- If this `min_popularity` is not met, then the `relatedness` score will be `-Infinity`.
@@ -835,7 +846,7 @@ This can be particularly useful when using a descending sorting on `relatedness(
When sorting on `relatedness(...)` requests can be processed much more quickly by adding a `prelim_sort: "count desc"` option. Increasing the `overrequest` can help improve the accuracy of the top buckets.
====
-=== Semantic Knowledge Graph Example
+==== Semantic Knowledge Graph Example
.Sample Documents
[source,bash,subs="verbatim,callouts"]
@@ -938,15 +949,3 @@ curl -sS -X POST http://localhost:8983/solr/gettingstarted/query -d 'rows=0&q=*:
<5> The state of Colorado (CO) has a _negative_ correlation with the nested Foreground Set -- i.e., "People in Colorado are statistically less likely to be '35+ year old Golfers' then the country as a whole."
<6> The number documents matching `age:[35 TO *]` _and_ `hobbies:golf` _and_ `state:AZ` is 18.75% of the total number of documents in the Background Set
<7> 50% of the documents in the Background Set match `state:AZ`
-
-== References
-
-This documentation was originally adapted largely from the following blog pages:
-
-http://yonik.com/json-facet-api/
-
-http://yonik.com/solr-facet-functions/
-
-http://yonik.com/solr-subfacets/
-
-http://yonik.com/percentiles-for-solr-faceting/
diff --git a/solr/solr-ref-guide/src/json-faceting-domain-changes.adoc b/solr/solr-ref-guide/src/json-faceting-domain-changes.adoc
new file mode 100644
index 0000000..ebf9c96
--- /dev/null
+++ b/solr/solr-ref-guide/src/json-faceting-domain-changes.adoc
@@ -0,0 +1,250 @@
+= JSON Faceting Domain Changes
+:solr-root-path: ../../
+:example-source-dir: {solr-root-path}solrj/src/test/org/apache/solr/client/ref_guide_examples/
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+Facet computation operates on a "domain" of documents. By default, this domain consists of the documents matched by the main query. For sub-facets, the domain consists of all documents placed in their bucket by the parent facet.
+
+Users can also override the "domain" of a facet that partitions data, using an explicit `domain` attribute whose value is a JSON object that can support various options for restricting, expanding, or completely changing the original domain before the buckets are computed for the associated facet.
+
+[NOTE]
+====
+`domain` changes can only be specified on individual facets that do data partitioning -- not statistical/metric facets, or groups of facets.
+
+A `\*:*` query facet with a `domain` change can be used to group multiple sub-facets of any type, for the purpose of applying a common domain change.
+====
+
+
+== Adding Domain Filters
+
+The simplest example of a domain change is to specify an additional filter which will be applied to the existing domain. This can be done via the `filter` keyword in the `domain` block of the facet.
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-facet-filtered-domain]
+====
+[.tab-label]*curl*
+[source,bash]
+----
+curl http://localhost:8983/solr/techproducts/query -d '
+{
+ "query": "*:*",
+ "facet": {
+ "categories": {
+ "type": "terms",
+ "field": "cat",
+ "limit": 3,
+ "domain": {
+ "filter": "popularity:[5 TO 10]"
+ }
+ }
+ }
+}'
+----
+====
+
+[example.tab-pane#solrj-json-facet-filtered-domain]
+====
+[.tab-label]*SolrJ*
+
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-facet-filtered-domain]
+----
+====
+--
+
+The value of `filter` can be a single query to treat as a filter, or a JSON list of filter queries. Each query can be:
+
+* a string containing a query in Solr query syntax
+* a reference to a request parameter containing Solr query syntax, of the form: `{param : <request_param_name>}`
+
+When a `filter` option is combined with other `domain` changing options, the filtering is applied _after_ the other domain changes take place.
+
+== Filter Exclusions
+
+Domains can also be expanded by using the `excludeTags` keyword to discard or ignore particular tagged query filters.
+
+This is used in the example below to show the top two manufacturers matching a search. The search results match the filter `manu_id_s:apple`, but the computed facet discards this filter and operates a domain widened by discarding the `manu_id_s` filter.
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-facet-excludetags-domain]
+====
+[.tab-label]*curl*
+[source,bash]
+----
+curl http://localhost:8983/solr/techproducts/query -d '
+{
+ "query": "cat:electronics",
+ "filter": "{!tag=MANU}manu_id_s:apple",
+ "facet": {
+ "stock": {"type": "terms", "field": "inStock", "limit": 2},
+ "manufacturers": {
+ "type": "terms",
+ "field": "manu_id_s",
+ "limit": 2,
+ "domain": { "excludeTags":"MANU" }
+ }
+ }
+}'
+----
+====
+
+[example.tab-pane#solrj-json-facet-excludetags-domain]
+====
+[.tab-label]*SolrJ*
+
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-facet-excludetags-domain]
+----
+====
+--
+
+The value of `excludeTags` can be a single string tag, array of string tags or comma-separated tags in the single string.
+
+When an `excludeTags` option is combined with other `domain` changing options, it expands the domain _before_ any other domain changes take place.
+
+See also the section on <<faceting.adoc#tagging-and-excluding-filters,multi-select faceting>>.
+
+== Arbitrary Domain Query
+
+A `query` domain can be specified when you wish to compute a facet against an arbitrary set of documents, regardless of the original domain. The most common use case would be to compute a top level facet against a specific subset of the collection, regardless of the main query. But it can also be useful on nested facets when building <<json-facet-api.adoc#relatedness-and-semantic-knowledge-graphs,Semantic Knowledge Graphs>>.
+
+Example:
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-facet-query-domain]
+====
+[.tab-label]*curl*
+[source,bash]
+----
+curl http://localhost:8983/solr/techproducts/query -d '
+{
+ "query": "apple",
+ "facet": {
+ "popular_categories": {
+ "type": "terms",
+ "field": "cat",
+ "domain": { "query": "popularity:[8 TO 10]" },
+ "limit": 3
+ }
+ }
+}'
+----
+====
+
+[example.tab-pane#solrj-json-facet-query-domain]
+====
+[.tab-label]*SolrJ*
+
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-facet-query-domain]
+----
+====
+--
+
+The value of `query` can be a single query, or a JSON list of queries. Each query can be:
+
+* a string containing a query in Solr query syntax
+* a reference to a request parameter containing Solr query syntax, of the form: `{param: <request_param_name>}`
+
+NOTE: While a `query` domain can be combined with an additional domain `filter`, It is not possible to also use `excludeTags`, because the tags would be meaningless: The `query` domain already completely ignores the top-level query and all previous filters.
+
+== Block Join Domain Changes
+
+When a collection contains <<uploading-data-with-index-handlers.adoc#nested-child-documents, Block Join child documents>>, the `blockChildren` or `blockParent` domain options can be used transform an existing domain containing one type of document, into a domain containing the documents with the specified relationship (child or parent of) to the documents from the original domain.
+
+Both of these options work similarly to the corresponding <<other-parsers.adoc#block-join-query-parsers,Block Join Query Parsers>> by taking in a single String query that exclusively matches all parent documents in the collection. If `blockParent` is used, then the resulting domain will contain all parent documents of the children from the original domain. If `blockChildren` is used, then the resulting domain will contain all child documents of the parents from the original domain.
+
+[source,json,subs="verbatim,callouts"]]
+----
+{
+ "colors": { // <1>
+ "type": "terms",
+ "field": "sku_color", // <2>
+ "facet" : {
+ "brands" : {
+ "type": "terms",
+ "field": "product_brand", // <3>
+ "domain": {
+ "blockParent": "doc_type:product"
+ }
+ }}}}
+----
+<1> This example assumes we parent documents corresponding to Products, with child documents corresponding to individual SKUs with unique colors, and that our original query was against SKU documents.
+<2> The `colors` facet will be computed against all of the original SKU documents matching our search.
+<3> For each bucket in the `colors` facet, the set of all matching SKU documents will be transformed into the set of corresponding parent Product documents. The resulting `brands` sub-facet will count how many Product documents (that have SKUs with the associated color) exist for each Brand.
+
+== Join Query Domain Changes
+
+A `join` domain change option can be used to specify arbitrary `from` and `to` fields to use in transforming from the existing domain to a related set of documents.
+
+This works very similar to the <<other-parsers.adoc#join-query-parser,Join Query Parser>>, and has the same limitations when dealing with multi-shard collections.
+
+Example:
+[source,json]
+----
+{
+ "colors": {
+ "type": "terms",
+ "field": "sku_color",
+ "facet": {
+ "brands": {
+ "type": "terms",
+ "field": "product_brand",
+ "domain" : {
+ "join" : {
+ "from": "product_id_of_this_sku",
+ "to": "id"
+ },
+ "filter": "doc_type:product"
+ }
+ }
+ }
+ }
+}
+
+----
+
+== Graph Traversal Domain Changes
+
+A `graph` domain change option works similarly to the `join` domain option, but can do traversal multiple hops `from` the existing domain `to` other documents.
+
+This works very similar to the <<other-parsers.adoc#graph-query-parser,Graph Query Parser>>, supporting all of it's optional parameters, and has the same limitations when dealing with multi-shard collections.
+
+Example:
+[source,json]
+----
+{
+ "related_brands": {
+ "type": "terms",
+ "field": "brand",
+ "domain": {
+ "graph": {
+ "from": "related_product_ids",
+ "to": "id",
+ "maxDepth": 3
+ }
+ }
+ }
+}
+----
diff --git a/solr/solr-ref-guide/src/json-query-dsl.adoc b/solr/solr-ref-guide/src/json-query-dsl.adoc
index 9c9ba64..386f03f 100644
--- a/solr/solr-ref-guide/src/json-query-dsl.adoc
+++ b/solr/solr-ref-guide/src/json-query-dsl.adoc
@@ -1,4 +1,6 @@
= JSON Query DSL
+:solr-root-path: ../../
+:example-source-dir: {solr-root-path}solrj/src/test/org/apache/solr/client/ref_guide_examples/
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
@@ -16,172 +18,362 @@
// specific language governing permissions and limitations
// under the License.
-The JSON Query DSL provides a simple yet powerful query language for the JSON Request API.
+Queries and filters provided in JSON requests can be specified using a rich, powerful query DSL.
-== Structure of JSON Query DSL
-A JSON query can be:
+== Query DSL Structure
+The JSON Request API accepts query values in three different formats:
-* A valid <<the-standard-query-parser.adoc#the-standard-query-parser,query string>> for default `deftype` (the standard query parser in most cases), as in, `title:solr`.
+* A valid <<the-standard-query-parser.adoc#the-standard-query-parser,query string>> that uses the default `deftype` (`lucene`, in most cases). e.g. `title:solr`.
-* A valid <<local-parameters-in-queries.adoc#local-parameters-in-queries,local parameters>> query string, as in, `{!dismax qf=myfield}solr rocks`.
+* A valid <<local-parameters-in-queries.adoc#local-parameters-in-queries,local parameters query string>> that specifies its `deftype` explicitly. e.g. `{!dismax qf=title}solr`.
-* A JSON object with query parser name and arguments.
-The special key `v` in local parameters is replaced by key `query` in JSON object query, as in this example:
+* A valid JSON object with the name of the query parser and any relevant parameters. e.g. `{ "lucene": {"df":"title", "query":"solr"}}`.
+** The top level "query" JSON block generally only has a single property representing the name of the query parser to use. The value for the query parser property is a child block containing any relevant parameters as JSON properties. The whole structure is analogous to a "local-params" query string. The query itself (often represented in local params using the name `v`) is specified with the key `query` instead.
-[source,json]
-{
- "query-parser-name" : {
- "param1": "value1",
- "param2": "value2",
- "query": "a-json-query",
- "another-param": "another-json-query"
- }
-}
+All of these syntaxes can be used to specify queries for either the JSON Request API's `query` or `filter` properties.
+
+=== Query DSL Examples
-== Basic Examples
-The four requests below are equivalent for searching for `solr lucene` in a field named `content`:
+The examples below show how to use each of the syntaxes discussed above to represent a query. Each snippet represents the same basic search: the term `iPod` in a field called `name`:
-. Passing all parameters on URI, with "lucene" as the default query parser.
+. Using the standard query API, with a simple query string
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-ipod-query-basic]
+====
+[.tab-label]*curl*
[source,bash]
-curl -XGET "http://localhost:8983/solr/books/query?q=content:(solr lucene)"
+----
+curl -X GET "http://localhost:8983/solr/techproducts/query?q=name:iPod"
+----
+====
+[example.tab-pane#solrj-ipod-query-basic]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-ipod-query-basic]
+----
+====
+--
-. Using the JSON Query DSL with valid query string for default `deftype`, with "lucene" as default query parser.
+. Using the JSON Request API, with a simple query string
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-ipod-query-dsl-1]
+====
+[.tab-label]*curl*
[source,bash]
-curl -XGET http://localhost:8983/solr/books/query -d '
-{"query": "content:(solr lucene)"}'
+----
+curl -X POST http://localhost:8983/solr/techproducts/query -d '
+{
+ "query" : "name:iPod"
+}'
+----
+====
+[example.tab-pane#solrj-ipod-query-dsl-1]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-ipod-query-dsl-1]
+----
+====
+--
-. Using JSON Query DSL with valid local parameters query defining the "lucene" query parser.
+. Using the JSON Request API, with a local-params string
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-ipod-query-dsl-2]
+====
+[.tab-label]*curl*
[source,bash]
-curl -XGET http://localhost:8983/solr/books/query -d '
-{"query": "{!lucene df=content v='solr lucene'}"}'
-
+----
+curl -X POST http://localhost:8983/solr/techproducts/query -d '
+{
+ "query": "{!lucene df=name v=iPod}"
+}'
+----
+====
+[example.tab-pane#solrj-ipod-query-dsl-2]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-ipod-query-dsl-2]
+----
+====
+--
-. Using JSON Query DSL in verbose way, as a valid JSON object with parser name and arguments.
+. Using the JSON Request API, with a fully expanded JSON object
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-ipod-query-dsl-3]
+====
+[.tab-label]*curl*
[source,bash]
-curl -XGET http://localhost:8983/solr/books/query -d '
-{"query": {"lucene": {
- "df": "content",
- "query": "solr lucene"
- }
- }}'
-
-Note that the JSON query in the examples above is provided under the key `query` of <<json-request-api.adoc#json-request-api,JSON Request API>>.
+----
+curl -X POST http://localhost:8983/solr/techproducts/query -d '
+{
+ "query": {
+ "lucene": {
+ "df": "name",
+ "query": "iPod"
+ }
+ }
+}'
+----
+====
+[example.tab-pane#solrj-ipod-query-dsl-3]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-ipod-query-dsl-3]
+----
+====
+--
== Nested Queries
-Some query parsers accept a query as an argument. JSON Query DSL makes it easier to write and read such complex query.
+Many of Solr's query parsers allow queries to be nested within one another. When these are used, requests using the standard query API quickly become hard to write, read, and understand. These sorts of queries are often much easier to work with in the JSON Request API.
-The three requests below are equivalent for wrapping the above example query (searching for `solr lucene` in field `content`) with a boost query:
+=== Nested Boost Query Example
+As an example, consider the three requests below, which wrap a simple query (the term `iPod` in the field `name`) within a boost query:
-. Passing all parameters on URI.
+. Using the standard query API.
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-ipod-query-boosted-basic]
+====
+[.tab-label]*curl*
[source,bash]
-http://localhost:8983/solr/books/query?q={!boost b=log(popularity) v='{!lucene df=content}(lucene solr)'}
+----
+curl -X GET "http://localhost:8983/solr/techproducts/query?q={!boost b=log(popularity) v=\'{!lucene df=name}iPod\'}"
+----
+====
+[example.tab-pane#solrj-ipod-query-boosted-basic]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-ipod-query-boosted-basic]
+----
+====
+--
-. Converted into JSON Query DSL with use of local parameters.
-As you can see, the special key `v` is replaced by key `query`.
+. Using the JSON Request API, with a mix of fully-expanded and local-params queries.
+As you can see, the special key `v` is replaced with the key `query`.
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-ipod-query-boosted-dsl-1]
+====
+[.tab-label]*curl*
[source,bash]
-curl -XGET http://localhost:8983/solr/books/query -d '
+----
+curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query" : {
"boost": {
- "query": {!lucene df=content}(lucene solr),
+ "query": {!lucene df=name}iPod,
"b": "log(popularity)"
}
}
}'
+----
+====
+[example.tab-pane#solrj-ipod-query-boosted-dsl-1]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-ipod-query-boosted-dsl-1]
+----
+====
+--
-. Using a verbose JSON Query DSL without local parameters.
+. Using the JSON Request API, with all queries fully expanded as JSON
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-ipod-query-boosted-dsl-2]
+====
+[.tab-label]*curl*
[source,bash]
-curl -XGET http://localhost:8983/solr/books/query -d '
+----
+curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query": {
"boost": {
"query": {
"lucene": {
- "df": "content",
- "query": "solr lucene"
+ "df": "name",
+ "query": "iPod"
}
},
"b": "log(popularity)"
}
}
}'
+----
+====
+[example.tab-pane#solrj-ipod-query-boosted-dsl-2]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-ipod-query-boosted-dsl-2]
+----
+====
+--
-== Compound Queries
-With the support of the <<other-parsers.adoc#boolean-query-parser,BoolQParser>>, the JSON Query DSL can create a very powerful nested query.
+=== Nested Boolean Query Example
+Query nesting is commonly seen when combining multiple query clauses together using pseudo-boolean logic with the <<other-parsers.adoc#boolean-query-parser,BoolQParser>>.
-This query searches for books where `content` contains `lucene` or `solr`, `title` contains `solr` and their `ranking` must larger than 3.0:
+The example below shows how the `BoolQParser` can be used to create powerful nested queries. In this example, a user searches for results with `iPod` in the field `name` which are _not_ in the bottom half of the `popularity` rankings.
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-ipod-query-bool]
+====
+[.tab-label]*curl*
[source,bash]
-curl -XGET http://localhost:8983/solr/books/query -d '
+----
+curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query": {
"bool": {
"must": [
- "title:solr",
- {"lucene": {"df: "content", query: "lucene solr"}}
+ {"lucene": {"df": "name", query: "iPod"}}
],
"must_not": [
- {"frange": {"u": "3.0", query: "ranking"}}
+ {"frange": {"l": "0", "u": "5", "query": "popularity"}}
]
}
}
}'
-If lucene is the default query parser query, the above can be rewritten in much less verbose way as in:
+----
+====
+[example.tab-pane#solrj-ipod-query-bool]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-ipod-query-bool]
+----
+====
+--
+
+If lucene is the default query parser, the example above can be simplified to:
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-ipod-query-bool-condensed]
+====
+[.tab-label]*curl*
[source,bash]
-curl -XGET http://localhost:8983/solr/books/query -d '
+----
+curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query": {
"bool": {
"must": [
- "title:solr",
- "content:(lucene solr)"
+ "name:iPod"
],
- "must_not": "{!frange u:3.0}ranking"
+ "must_not": "{!frange l=0 u=5}popularity"
}
}
}'
-== Use JSON Query DSL in JSON Request API
-JSON Query DSL is not only supported with the key `query` but also with the key `filter` of the <<json-request-api.adoc#json-request-api,JSON Request API>>.
+----
+====
+[example.tab-pane#solrj-ipod-query-bool-condensed]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-ipod-query-bool-condensed]
+----
+====
+--
-For example, the above query can be rewritten using filter clause like this:
+== Filter Queries
+The syntaxes discussed above can also be used to specify query filters (under the `filter` key) in addition to the main query itself.
+For example, the above query can be rewritten using a filter clause as:
+
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-ipod-query-bool-filter]
+====
+[.tab-label]*curl*
[source,bash]
-curl -XGET http://localhost:8983/solr/books/query -d '
+----
+curl -X POST http://localhost:8983/solr/techproducts/query -d '
{
"query": {
"bool": {
- "must_not": "{!frange u:3.0}ranking"
+ "must_not": "{!frange l=0 u=5}popularity"
}
},
"filter: [
- "title:solr",
- { "lucene" : {"df: "content", query : "lucene solr" }}
+ "name:iPod"
]
}'
+----
+====
+[example.tab-pane#solrj-ipod-query-bool-filter]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-ipod-query-bool-filter]
+----
+====
+--
+
== Tagging in JSON Query DSL
-Queries and filters might be tagged by inserting JSON object with the single property with name starting with a hash (`#`). Tags can be applied to the single string syntax and to a full JSON object query as well:
+Query and filter clauses can also be individually "tagged". Tags serve as handles for query clauses, allowing them to be referenced from elsewhere in the request. This is most commonly used by the filter-exclusion functionality offered by both <<faceting.adoc#tagging-and-excluding-filters,traditional>> and <<json-faceting-domain-changes.adoc#filter-exclusions,JSON>> faceting.
+
+Queries and filters are tagged by wrapping them in a surrounding JSON object. The name of the tag is specified as a JSON key, with the query string (or object) becoming the value associated with that key. Tag name properties are prefixed with a hash, and may include multiple tags, separated by commas. For example: `{"\#title,tag2,tag3":"title:solr"}`. Note that unlike the rest of the JSON request API which uses lax JSON parsing rules, tags must be surrounded by double-quotes because [...]
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-tagged-query]
+====
+[.tab-label]*curl*
[source,bash]
-curl -XGET http://localhost:8983/solr/books/query -d '
+----
+curl -X POST http://localhost:8983/solr/techproducts/select -d '
{
- "filter: [ // this applies `titletag`
- {"#titletag": "title:solr"},
- { // this applies `tagcontent`
- "#tagcontent":
- {"lucene": {"df:"content", query :"lucene solr"}}
- }
- ]
+ "query": "*:*",
+ "filter": [
+ {
+ "#titleTag": "name:Solr"
+ },
+ {
+ "#inStockTag": "inStock:true"
+ }
+ ]
}'
-There might be many comma separated tags like `{"#title,tag2,tag3":"title:solr"}`. Tags can be used in <<faceting.adoc#tagging-and-excluding-filters,old facets>> and in <<json-facet-api.adoc#filter-exclusions,JSON Facets'>> `excludeTags` as well. Notice that the leading hash requires to use double quotes as in regular JSON.
+----
+====
+[example.tab-pane#solrj-tagged-query]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-tagged-query]
+----
+====
+--
+
+Note that the tags created in the example above have no impact in how the search is executed. Tags will not affect a query unless they are referenced by some other part of the request that uses them.
diff --git a/solr/solr-ref-guide/src/json-request-api.adoc b/solr/solr-ref-guide/src/json-request-api.adoc
index 1f5d46d..28976b6 100644
--- a/solr/solr-ref-guide/src/json-request-api.adoc
+++ b/solr/solr-ref-guide/src/json-request-api.adoc
@@ -19,18 +19,13 @@
// specific language governing permissions and limitations
// under the License.
-The JSON Request API allows a JSON body to be passed for the entire search request.
+Solr supports an alternate request API which accepts requests composed in part or entirely of JSON objects. This alternate API can be preferable in some situations, where its increased readability and flexibility make it easier to use than the entirely query-param driven alternative. There is also some functionality which can only be accessed through this JSON request API, such as much of the analytics capabilities of <<json-facet-api.adoc#json-facet-api,JSON Faceting>>
-The <<json-facet-api.adoc#json-facet-api,JSON Facet API>> is part of the JSON Request API, and allows specification of faceted analytics in JSON.
-
-Here's an example of a search request using query parameters only:
-[source,bash]
-curl "http://localhost:8983/solr/techproducts/query?q=memory&fq=inStock:true"
-
-The same request when passed as JSON in the body:
+== Building JSON Requests
+The core of the JSON Request API is its ability to specify request parameters as JSON in the request body, as shown by the example below:
[.dynamic-tabs]
--
-[example.tab-pane#curlsimplejsonquery]
+[example.tab-pane#curl-simple-json-query]
====
[.tab-label]*curl*
[source,bash]
@@ -42,11 +37,9 @@ curl http://localhost:8983/solr/techproducts/query -d '
}'
----
====
-
-[example.tab-pane#solrjsimplejsonquery]
+[example.tab-pane#solrj-simple-json-query]
====
[.tab-label]*SolrJ*
-
[source,java,indent=0]
----
include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-query-simple]
@@ -54,56 +47,109 @@ include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-query-simple
====
--
-== Passing JSON via Request Parameter
-It may sometimes be more convenient to pass the JSON body as a request parameter rather than in the actual body of the HTTP request. Solr treats a `json` parameter the same as a JSON body.
-
+JSON objects are typically sent in the request body, but they can also be sent as values for `json`-prefixed query parameters. This can be used to override or supplement values specified in the request body. For example the query parameter `json.limit=5` will override any `limit` value provided in the JSON request body. You can also specify the entire JSON body in a single `json` query parameter, as shown in the example below:
[source,bash]
curl http://localhost:8983/solr/techproducts/query -d 'json={"query":"memory"}'
-== Smart Merging of Multiple JSON Parameters
-Multiple `json` parameters in a single request are merged before being interpreted.
-
-* Single-valued elements are overwritten by the last value.
-
-* Multi-valued elements like fields and `filter` are appended.
+=== JSON Parameter Merging
+If multiple `json` parameters are provided in a single request, Solr attempts to merge the parameter values together before processing the request.
-* Parameters of the form `json.<path>=<json_value>` are merged in the appropriate place in the hierarchy. For example a `json.facet` parameter is the same as `facet` within the JSON body.
+The JSON Request API has several properties (`filter`, `fields`, etc) which accept multiple values. During the merging process, all values for these "multivalued" properties are retained. Many properties though (`query`, `limit`, etc.) can have only a single value. When multiple parameter values conflict with one another a single value is chosen based on the following precedence rules:
-* A JSON body, or straight `json` parameters are always parsed first, meaning that other request parameters come after, and overwrite single valued elements.
+* Traditional query parameters (`q`, `rows`, etc.) take first precedence and are used over any other specified values.
+* `json`-prefixed query parameters are considered next.
+* Values specified in the JSON request body have the lowest precedence and are only used if specified nowhere else.
-Smart merging gives the best of both worlds…the structure of JSON with the ability to selectively separate out / decompose parts of the request!
+This layered merging gives the best of both worlds. Requests can be specified using readable, structured JSON. But users also have the flexibility to separate out parts of the request that change often. This can be seen at work in the following example, which combines `json.`-style parameters to override and supplement values found in the main JSON body.
-=== Simple Example
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-query-param-overrides]
+====
+[.tab-label]*curl*
[source,bash]
+----
curl 'http://localhost:8983/solr/techproducts/query?json.limit=5&json.filter="cat:electronics"' -d '
{
query: "memory",
limit: 10,
filter: "inStock:true"
}'
+----
+====
+[example.tab-pane#solrj-json-query-param-overrides]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-query-param-overrides]
+----
+====
+--
-Is equivalent to:
+This is equivalent to:
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-query-param-overrides-equivalent]
+====
+[.tab-label]*curl*
[source,bash]
+----
curl http://localhost:8983/solr/techproducts/query -d '
{
- query: "memory",
- limit: 5, // this single-valued parameter was overwritten.
- filter: ["inStock:true","cat:electronics"] // this multi-valued parameter was appended to.
+ "query": "memory",
+ "limit": 5, // this single-valued parameter was overwritten.
+ "filter": ["inStock:true","cat:electronics"] // this multi-valued parameter was appended to.
}'
+----
+====
+[example.tab-pane#solrj-json-query-param-overrides-equivalent]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-query-param-overrides-equivalent]
+----
+====
+--
-=== Facet Example
-In fact, you don’t even need to start with a JSON body for smart merging to be very useful. Consider the following request composed entirely of request params:
+Similarly, smart merging can be used to create JSON API requests which have no proper request body at all, such as the example below:
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-facet-all-query-params]
+====
+[.tab-label]*curl*
[source,bash]
+----
curl http://localhost:8983/solr/techproducts/query -d 'q=*:*&rows=1&
json.facet.avg_price="avg(price)"&
- json.facet.top_cats={type:terms,field:"cat",limit:5}'
-
-That is equivalent to having the following JSON body or `json` parameter:
+ json.facet.top_cats={type:terms,field:"cat",limit:3}'
+----
+====
+[example.tab-pane#solrj-json-facet-all-query-params]
+====
+[.tab-label]*SolrJ*
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-facet-all-query-params]
+----
+====
+--
-[source,json]
+That is equivalent to the following request:
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-facet-all-query-params-equivalent]
+====
+[.tab-label]*curl*
+[source,bash]
+----
+curl http://localhost:8983/solr/techproducts/query -d '
{
+ "query": "*:*",
+ "limit": 1,
"facet": {
"avg_price": "avg(price)",
"top_cats": {
@@ -113,53 +159,24 @@ That is equivalent to having the following JSON body or `json` parameter:
}
}
}
-
-See the <<json-facet-api.adoc#json-facet-api,JSON Facet API>> for more on faceting and analytics commands in specified in JSON.
-
-
-=== Debugging
-
-If you want to see what your merged/parsed JSON looks like, you can turn on debugging (`debug=timing`), and it will come back under the "json" key along with the other debugging information.
-Note: `debug=true` as well as `debugQuery=true` might have too much performance implication and `debug=query` makes no effect on JSON facet in SolrCloud.
-
-== Passing Parameters via JSON
-We can also pass normal query request parameters in the JSON body within the params block:
-
-[.dynamic-tabs]
---
-[example.tab-pane#curljsonqueryparamsblock]
-====
-[.tab-label]*curl*
-[source,bash]
-----
-curl "http://localhost:8983/solr/techproducts/query?fl=name,price"-d '
-{
- params: {
- q: "memory",
- rows: 1
- }
-}'
+'
----
====
-
-[example.tab-pane#solrjjsonqueryparamsblock]
+[example.tab-pane#solrj-json-facet-all-query-params-equivalent]
====
[.tab-label]*SolrJ*
-
[source,java,indent=0]
----
-include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-query-params-block]
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-facet-all-query-params-equivalent]
----
====
--
-Which is equivalent to:
+See the <<json-facet-api.adoc#json-facet-api,JSON Facet API>> for more on faceting and analytics commands.
-[source,bash]
-curl "http://localhost:8983/solr/techproducts/query?fl=name,price&q=memory&rows=1"
+== Supported Properties and Syntax
-== Parameters Mapping
-Right now only some standard query parameters have JSON equivalents. Unmapped parameters can be passed through request parameters or `params` block as shown above.
+Right now, only some of Solr's traditional query parameters have first class JSON equivalents. Those that do are shown in the table below:
.Standard query parameters to JSON field
|===
@@ -190,29 +207,49 @@ Right now only some standard query parameters have JSON equivalents. Unmapped pa
|`<param_name>`
|===
-== Error Detection
-
-Because we didn’t pollute the root body of the JSON request with the normal Solr request parameters (they are all contained in the params block), we now have the ability to validate requests and return an error for unknown JSON keys.
+Parameters not specified in the table above can still be used in the main body of JSON API requests, but they must be put within a `params` block as shown in the example below.
+[.dynamic-tabs]
+--
+[example.tab-pane#curl-json-query-params-block]
+====
+[.tab-label]*curl*
[source,bash]
-curl http://localhost:8983/solr/techproducts/query -d '
+----
+curl "http://localhost:8983/solr/techproducts/query?fl=name,price"-d '
{
- query : "memory",
- fulter : "inStock:true" // oops, we misspelled "filter"
+ params: {
+ q: "memory",
+ rows: 1
+ }
}'
+----
+====
-And we get an error back containing the error string:
+[example.tab-pane#solrj-json-query-params-block]
+====
+[.tab-label]*SolrJ*
-[source,text]
-"Unknown top-level key in JSON request : fulter"
+[source,java,indent=0]
+----
+include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-query-params-block]
+----
+====
+--
+
+Parameters placed in a `params` block act as if they were added verbatim to the query-parameters of the request. The request above is equivalent to:
+
+[source,bash]
+curl "http://localhost:8983/solr/techproducts/query?fl=name,price&q=memory&rows=1"
-== Parameter Substitution / Macro Expansion
+
+=== Parameter Substitution / Macro Expansion
Of course request templating via parameter substitution works fully with JSON request bodies or parameters as well.
For example:
[.dynamic-tabs]
--
-[example.tab-pane#curljsonquerymacroexpansion]
+[example.tab-pane#curl-json-query-macro-expansion]
====
[.tab-label]*curl*
[source,bash]
@@ -224,7 +261,7 @@ curl "http://localhost:8983/solr/techproducts/query?FIELD=text&TERM=memory" -d '
----
====
-[example.tab-pane#solrjjsonquerymacroexpansion]
+[example.tab-pane#solrj-json-query-macro-expansion]
====
[.tab-label]*SolrJ*
@@ -234,3 +271,20 @@ include::{example-source-dir}JsonRequestApiTest.java[tag=solrj-json-query-macro-
----
====
--
+
+=== JSON Extensions
+
+Solr uses the *Noggit* JSON parser in its request API. Noggit is capable of more relaxed JSON parsing, and allows a number of deviations from the JSON standard:
+
+* bare words can be left unquoted
+* single line comments can be inserted using either `//` or `#`
+* Multi-line ("C style") comments can be inserted using `/\*` and `*/`
+* strings can be single-quoted
+* special characters can be backslash-escaped
+* trailing (extra) commas are silently ignored (e.g. `[9,4,3,]`)
+* nbsp (non-break space, \u00a0) is treated as whitespace.
+
+== Debugging
+
+If you want to see what your merged/parsed JSON looks like, you can turn on debugging (`debug=timing`), and it will come back under the "json" key along with the other debugging information.
+Note that `debug=true` and `debugQuery=true` can often have significant performance implications, and should be reserved for debugging. Also note that `debug=query` has no effect on JSON facets in SolrCloud.
diff --git a/solr/solrj/src/test/org/apache/solr/client/ref_guide_examples/JsonRequestApiHeatmapFacetingTest.java b/solr/solrj/src/test/org/apache/solr/client/ref_guide_examples/JsonRequestApiHeatmapFacetingTest.java
new file mode 100644
index 0000000..4698c7e
--- /dev/null
+++ b/solr/solrj/src/test/org/apache/solr/client/ref_guide_examples/JsonRequestApiHeatmapFacetingTest.java
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.solr.client.ref_guide_examples;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import org.apache.solr.client.solrj.request.CollectionAdminRequest;
+import org.apache.solr.client.solrj.request.json.HeatmapFacetMap;
+import org.apache.solr.client.solrj.request.json.JsonQueryRequest;
+import org.apache.solr.client.solrj.response.QueryResponse;
+import org.apache.solr.client.solrj.response.json.HeatmapJsonFacet;
+import org.apache.solr.client.solrj.response.json.NestableJsonFacet;
+import org.apache.solr.cloud.SolrCloudTestCase;
+import org.apache.solr.common.SolrInputDocument;
+import org.apache.solr.util.ExternalPaths;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+/**
+ * Example SolrJ usage Heatmap facets in the JSON Request API.
+ *
+ * Snippets surrounded by "tag" and "end" comments are extracted and used in the Solr Reference Guide.
+ * <p>
+ * This class is mostly copied from {@link org.apache.solr.client.solrj.request.json.JsonQueryRequestHeatmapFacetingTest}.
+ * The test was duplicated here as the community has previously decided that it's best to keep all buildable ref-guide
+ * snippets together in the same package.
+ */
+public class JsonRequestApiHeatmapFacetingTest extends SolrCloudTestCase {
+ private static final String COLLECTION_NAME = "spatialdata";
+ private static final String CONFIG_NAME = "spatialdata_config";
+ private static final String FIELD = "location_srpt";
+
+ @BeforeClass
+ public static void setupCluster() throws Exception {
+ configureCluster(1)
+ .addConfig(CONFIG_NAME, new File(ExternalPaths.SOURCE_HOME, "solrj/src/test-files/solrj/solr/configsets/spatial/conf").toPath())
+ .configure();
+
+ final List<String> solrUrls = new ArrayList<>();
+ solrUrls.add(cluster.getJettySolrRunner(0).getBaseUrl().toString());
+
+ CollectionAdminRequest.createCollection(COLLECTION_NAME, CONFIG_NAME, 1, 1).process(cluster.getSolrClient());
+
+ indexSpatialData();
+ }
+
+ private static void indexSpatialData() throws Exception {
+ final SolrInputDocument doc1 = new SolrInputDocument("id", "0", FIELD, "ENVELOPE(100, 120, 80, 40)");
+ final SolrInputDocument doc2 = new SolrInputDocument("id", "1", FIELD, "ENVELOPE(-120, -110, 80, 20)");
+ final SolrInputDocument doc3 = new SolrInputDocument("id", "3", FIELD, "POINT(70 60)");
+ final SolrInputDocument doc4 = new SolrInputDocument("id", "4", FIELD, "POINT(91 89)");
+ final List<SolrInputDocument> docs = new ArrayList<>();
+ docs.add(doc1);
+ docs.add(doc2);
+ docs.add(doc3);
+ docs.add(doc4);
+
+ cluster.getSolrClient().add(COLLECTION_NAME, docs);
+ cluster.getSolrClient().commit(COLLECTION_NAME);
+ }
+
+ @Test
+ public void testHeatmapFacet() throws Exception {
+ final List<List<Integer>> expectedHeatmapGrid = Arrays.asList(
+ Arrays.asList(0, 0, 2, 1, 0, 0),
+ Arrays.asList(0, 0, 1, 1, 0, 0),
+ Arrays.asList(0, 1, 1, 1, 0, 0),
+ Arrays.asList(0, 0, 1, 1, 0, 0),
+ Arrays.asList(0, 0, 1, 1, 0, 0),
+ null,
+ null
+ );
+ //tag::solrj-json-heatmap-facet-1[]
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("*:*")
+ .setLimit(0)
+ .withFacet("locations", new HeatmapFacetMap("location_srpt")
+ .setHeatmapFormat(HeatmapFacetMap.HeatmapFormat.INTS2D)
+ .setRegionQuery("[\"50 20\" TO \"180 90\"]")
+ .setGridLevel(4)
+ );
+ //end::solrj-json-heatmap-facet-1[]
+
+ QueryResponse response = request.process(cluster.getSolrClient(), COLLECTION_NAME);
+ final NestableJsonFacet topLevelFacet = response.getJsonFacetingResponse();
+ final HeatmapJsonFacet heatmap = topLevelFacet.getHeatmapFacetByName("locations");
+ final List<List<Integer>> actualHeatmapGrid = heatmap.getCountGrid();
+ assertEquals(expectedHeatmapGrid, actualHeatmapGrid);
+ }
+}
diff --git a/solr/solrj/src/test/org/apache/solr/client/ref_guide_examples/JsonRequestApiTest.java b/solr/solrj/src/test/org/apache/solr/client/ref_guide_examples/JsonRequestApiTest.java
index 83ab801..5a64c3f 100644
--- a/solr/solrj/src/test/org/apache/solr/client/ref_guide_examples/JsonRequestApiTest.java
+++ b/solr/solrj/src/test/org/apache/solr/client/ref_guide_examples/JsonRequestApiTest.java
@@ -20,13 +20,19 @@ package org.apache.solr.client.ref_guide_examples;
import java.io.File;
import java.util.ArrayList;
import java.util.Collection;
+import java.util.HashMap;
import java.util.List;
+import java.util.Map;
import org.apache.solr.client.solrj.SolrClient;
+import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.request.AbstractUpdateRequest;
import org.apache.solr.client.solrj.request.CollectionAdminRequest;
import org.apache.solr.client.solrj.request.ContentStreamUpdateRequest;
+import org.apache.solr.client.solrj.request.json.DomainMap;
import org.apache.solr.client.solrj.request.json.JsonQueryRequest;
+import org.apache.solr.client.solrj.request.json.QueryFacetMap;
+import org.apache.solr.client.solrj.request.json.RangeFacetMap;
import org.apache.solr.client.solrj.request.json.TermsFacetMap;
import org.apache.solr.client.solrj.response.json.BucketJsonFacet;
import org.apache.solr.client.solrj.response.json.NestableJsonFacet;
@@ -79,8 +85,82 @@ public class JsonRequestApiTest extends SolrCloudTestCase {
QueryResponse queryResponse = simpleQuery.process(solrClient, COLLECTION_NAME);
// end::solrj-json-query-simple[]
- assertEquals(0, queryResponse.getStatus());
- assertEquals(expectedResults, queryResponse.getResults().size());
+ assertResponseFoundNumDocs(queryResponse, expectedResults);
+ }
+
+ @Test
+ public void testJsonQueryWithJsonQueryParamOverrides() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+ final int expectedResults = 4;
+
+ // This subtest has its own scope so that it and its twin below can can have identical variable declarations (as they appear as separate snippets in the ref-guide)
+ {
+ // tag::solrj-json-query-param-overrides[]
+ final ModifiableSolrParams overrideParams = new ModifiableSolrParams();
+ final JsonQueryRequest queryWithParamOverrides = new JsonQueryRequest(overrideParams)
+ .setQuery("memory")
+ .setLimit(10)
+ .withFilter("inStock:true");
+ overrideParams.set("json.limit", 5);
+ overrideParams.add("json.filter", "\"cat:electronics\"");
+ QueryResponse queryResponse = queryWithParamOverrides.process(solrClient, COLLECTION_NAME);
+ // end::solrj-json-query-param-overrides[]
+
+ assertResponseFoundNumDocs(queryResponse, expectedResults);
+ }
+
+ // tag::solrj-json-query-param-overrides-equivalent[]
+ final JsonQueryRequest query = new JsonQueryRequest()
+ .setQuery("memory")
+ .setLimit(5)
+ .withFilter("inStock:true")
+ .withFilter("cat:electronics");
+ QueryResponse queryResponse = query.process(solrClient, COLLECTION_NAME);
+ // end::solrj-json-query-param-overrides-equivalent[]
+
+ assertResponseFoundNumDocs(queryResponse, expectedResults);
+ }
+
+ @Test
+ public void testJsonFacetWithAllQueryParams() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ // This subtest has its own scope so that it and its twin below can can have identical variable declarations (as they appear as separate snippets in the ref-guide)
+ {
+ //tag::solrj-json-facet-all-query-params[]
+ final ModifiableSolrParams params = new ModifiableSolrParams();
+ final SolrQuery query = new SolrQuery("*:*");
+ query.setRows(1);
+ query.setParam("json.facet.avg_price", "\"avg(price)\"");
+ query.setParam("json.facet.top_cats", "{type:terms,field:\"cat\",limit:3}");
+ QueryResponse queryResponse = solrClient.query(COLLECTION_NAME, query);
+ //end::solrj-json-facet-all-query-params[]
+
+ NestableJsonFacet topLevelFacet = queryResponse.getJsonFacetingResponse();
+ assertResponseFoundNumDocs(queryResponse, 1);
+ assertHasFacetWithBucketValues(topLevelFacet, "top_cats",
+ new FacetBucket("electronics", 12),
+ new FacetBucket("currency", 4),
+ new FacetBucket("memory", 3));
+ }
+
+ {
+ //tag::solrj-json-facet-all-query-params-equivalent[]
+ final JsonQueryRequest jsonQueryRequest = new JsonQueryRequest()
+ .setQuery("*:*")
+ .setLimit(1)
+ .withStatFacet("avg_price", "avg(price)")
+ .withFacet("top_cats", new TermsFacetMap("cat").setLimit(3));
+ QueryResponse queryResponse = jsonQueryRequest.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-facet-all-query-params-equivalent[]
+
+ NestableJsonFacet topLevelFacet = queryResponse.getJsonFacetingResponse();
+ assertResponseFoundNumDocs(queryResponse, 1);
+ assertHasFacetWithBucketValues(topLevelFacet, "top_cats",
+ new FacetBucket("electronics", 12),
+ new FacetBucket("currency", 4),
+ new FacetBucket("memory", 3));
+ }
}
@Test
@@ -118,8 +198,206 @@ public class JsonRequestApiTest extends SolrCloudTestCase {
QueryResponse queryResponse = simpleQuery.process(solrClient, COLLECTION_NAME);
// end::solrj-json-query-macro-expansion[]
- assertEquals(0, queryResponse.getStatus());
- assertEquals(5, queryResponse.getResults().size());
+ assertResponseFoundNumDocs(queryResponse, 5);
+ }
+
+ @Test
+ public void testJsonQueryDslBasicEquivalents() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ {
+ //tag::solrj-ipod-query-basic[]
+ final SolrQuery query = new SolrQuery("name:iPod");
+ final QueryResponse response = solrClient.query(COLLECTION_NAME, query);
+ //end::solrj-ipod-query-basic[]
+
+ assertResponseFoundNumDocs(response, 3);
+ }
+
+ {
+ //tag::solrj-ipod-query-dsl-1[]
+ final JsonQueryRequest query = new JsonQueryRequest()
+ .setQuery("name:iPod");
+ final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
+ //end::solrj-ipod-query-dsl-1[]
+
+ assertResponseFoundNumDocs(response, 3);
+ }
+
+ {
+ //tag::solrj-ipod-query-dsl-2[]
+ final JsonQueryRequest query = new JsonQueryRequest()
+ .setQuery("{!lucene df=name}iPod");
+ final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
+ //end::solrj-ipod-query-dsl-2[]
+
+ assertResponseFoundNumDocs(response, 3);
+ }
+
+ {
+ //tag::solrj-ipod-query-dsl-3[]
+ final Map<String, Object> queryTopLevel = new HashMap<>();
+ final Map<String, Object> luceneQueryProperties = new HashMap<>();
+ queryTopLevel.put("lucene", luceneQueryProperties);
+ luceneQueryProperties.put("df", "name");
+ luceneQueryProperties.put("query", "iPod");
+ final JsonQueryRequest query = new JsonQueryRequest()
+ .setQuery(queryTopLevel);
+ final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
+ //end::solrj-ipod-query-dsl-3[]
+
+ assertResponseFoundNumDocs(response, 3);
+ }
+ }
+
+ @Test
+ public void testJsonQueryDslBoostEquivalents() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+ QueryResponse[] responses = new QueryResponse[3];
+
+ {
+ //tag::solrj-ipod-query-boosted-basic[]
+ final SolrQuery query = new SolrQuery("{!boost b=log(popularity) v=\'{!lucene df=name}iPod\'}");
+ final QueryResponse response = solrClient.query(COLLECTION_NAME, query);
+ //end::solrj-ipod-query-boosted-basic[]
+
+ responses[0] = response;
+ }
+
+ {
+ //tag::solrj-ipod-query-boosted-dsl-1[]
+ final Map<String, Object> queryTopLevel = new HashMap<>();
+ final Map<String, Object> boostQuery = new HashMap<>();
+ queryTopLevel.put("boost", boostQuery);
+ boostQuery.put("b", "log(popularity)");
+ boostQuery.put("query", "{!lucene df=name}iPod");
+ final JsonQueryRequest query = new JsonQueryRequest()
+ .setQuery(queryTopLevel);
+ final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
+ //end::solrj-ipod-query-boosted-dsl-1[]
+
+ responses[1] = response;
+ }
+
+ {
+ //tag::solrj-ipod-query-boosted-dsl-2[]
+ final Map<String, Object> queryTopLevel = new HashMap<>();
+ final Map<String, Object> boostProperties = new HashMap<>();
+ final Map<String, Object> luceneTopLevel = new HashMap();
+ final Map<String, Object> luceneProperties = new HashMap<>();
+ queryTopLevel.put("boost", boostProperties);
+ boostProperties.put("b", "log(popularity)");
+ boostProperties.put("query", luceneTopLevel);
+ luceneTopLevel.put("lucene", luceneProperties);
+ luceneProperties.put("df", "name");
+ luceneProperties.put("query", "iPod");
+ final JsonQueryRequest query = new JsonQueryRequest()
+ .setQuery(queryTopLevel);
+ final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
+ //end::solrj-ipod-query-boosted-dsl-2[]
+
+ responses[2] = response;
+ }
+
+ for (QueryResponse response : responses) {
+ assertResponseFoundNumDocs(response, 3);
+ assertEquals("MA147LL/A", response.getResults().get(0).get("id"));
+ assertEquals("F8V7067-APL-KIT", response.getResults().get(1).get("id"));
+ assertEquals("IW-02", response.getResults().get(2).get("id"));
+ }
+ }
+
+ @Test
+ public void testJsonBooleanQuery() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+ QueryResponse[] responses = new QueryResponse[3];
+
+ {
+ //tag::solrj-ipod-query-bool[]
+ final Map<String, Object> queryTopLevel = new HashMap<>();
+ final Map<String, Object> boolProperties = new HashMap<>();
+ final List<Object> mustClauses = new ArrayList<>();
+ final List<Object> mustNotClauses = new ArrayList<>();
+ final Map<String, Object> frangeTopLevel = new HashMap<>();
+ final Map<String, Object> frangeProperties = new HashMap<>();
+
+ queryTopLevel.put("bool", boolProperties);
+ boolProperties.put("must", mustClauses);
+ mustClauses.add("name:iPod");
+
+ boolProperties.put("must_not", mustNotClauses);
+ frangeTopLevel.put("frange", frangeProperties);
+ frangeProperties.put("l", 0);
+ frangeProperties.put("u", 5);
+ frangeProperties.put("query", "popularity");
+ mustNotClauses.add(frangeTopLevel);
+
+ final JsonQueryRequest query = new JsonQueryRequest()
+ .setQuery(queryTopLevel);
+ final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
+ //end::solrj-ipod-query-bool[]
+
+ responses[0] = response;
+ }
+
+ {
+ //tag::solrj-ipod-query-bool-condensed[]
+ final Map<String, Object> queryTopLevel = new HashMap<>();
+ final Map<String, Object> boolProperties = new HashMap<>();
+ final List<Object> mustClauses = new ArrayList<>();
+ final List<Object> mustNotClauses = new ArrayList<>();
+ queryTopLevel.put("bool", boolProperties);
+ boolProperties.put("must", "name:iPod");
+ boolProperties.put("must_not", "{!frange l=0 u=5}popularity");
+
+ final JsonQueryRequest query = new JsonQueryRequest()
+ .setQuery(queryTopLevel);
+ final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
+ //end::solrj-ipod-query-bool-condensed[]
+
+ responses[1] = response;
+ }
+
+ {
+ //tag::solrj-ipod-query-bool-filter[]
+ final Map<String, Object> queryTopLevel = new HashMap<>();
+ final Map<String, Object> boolProperties = new HashMap<>();
+ queryTopLevel.put("bool", boolProperties);
+ boolProperties.put("must_not","{!frange l=0 u=5}popularity");
+
+ final JsonQueryRequest query = new JsonQueryRequest()
+ .setQuery(queryTopLevel)
+ .withFilter("name:iPod");
+ final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
+ //end::solrj-ipod-query-bool-filter[]
+
+ responses[2] = response;
+ }
+
+ for (QueryResponse response : responses) {
+ assertResponseFoundNumDocs(response, 1);
+ assertEquals("MA147LL/A", response.getResults().get(0).get("id"));
+ }
+ }
+
+ @Test
+ public void testJsonTaggedQuery() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-tagged-query[]
+ final Map<String, Object> titleTaggedQuery = new HashMap<>();
+ titleTaggedQuery.put("#titleTag", "name:Solr");
+ final Map<String, Object> inStockTaggedQuery = new HashMap<>();
+ inStockTaggedQuery.put("#inStockTag", "inStock:true");
+ final JsonQueryRequest query = new JsonQueryRequest()
+ .setQuery("*:*")
+ .withFilter(titleTaggedQuery)
+ .withFilter(inStockTaggedQuery);
+ final QueryResponse response = query.process(solrClient, COLLECTION_NAME);
+ //end::solrj-tagged-query[]
+
+ assertResponseFoundNumDocs(response, 1);
+ assertEquals("SOLR1000", response.getResults().get(0).get("id"));
}
@Test
@@ -148,13 +426,13 @@ public class JsonRequestApiTest extends SolrCloudTestCase {
public void testTermsFacet2() throws Exception {
SolrClient solrClient = cluster.getSolrClient();
- //tag::solrj-json-terms-facet2[]
+ //tag::solrj-json-terms-facet-2[]
final TermsFacetMap categoryFacet = new TermsFacetMap("cat").setLimit(5);
final JsonQueryRequest request = new JsonQueryRequest()
.setQuery("*:*")
.withFacet("categories", categoryFacet);
QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
- //end::solrj-json-terms-facet2[]
+ //end::solrj-json-terms-facet-2[]
assertEquals(0, queryResponse.getStatus());
assertEquals(32, queryResponse.getResults().getNumFound());
@@ -168,6 +446,274 @@ public class JsonRequestApiTest extends SolrCloudTestCase {
new FacetBucket("graphics card", 2));
}
+ @Test
+ public void testStatFacet1() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-metrics-facet-1[]
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("memory")
+ .withFilter("inStock:true")
+ .withStatFacet("avg_price", "avg(price)")
+ .withStatFacet("num_suppliers", "unique(manu_exact)")
+ .withStatFacet("median_weight", "percentile(weight,50)");
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-metrics-facet-1[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(4, queryResponse.getResults().getNumFound());
+ assertEquals(4, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+ assertEquals(146.66, (double) topLevelFacetingData.getStatFacetValue("avg_price"), 0.5);
+ assertEquals(3, topLevelFacetingData.getStatFacetValue("num_suppliers"));
+ assertEquals(352.0, (double) topLevelFacetingData.getStatFacetValue("median_weight"), 0.5);
+ }
+
+ @Test
+ public void testStatFacetSimple() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-metrics-facet-simple[]
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("*:*")
+ .withFilter("price:[1.0 TO *]")
+ .withFilter("popularity:[0 TO 10]")
+ .withStatFacet("avg_value", "avg(div(popularity,price))");
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-metrics-facet-simple[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(13, queryResponse.getResults().getNumFound());
+ assertEquals(10, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+ assertEquals(0.036, (double) topLevelFacetingData.getStatFacetValue("avg_value"), 0.1);
+ }
+
+ @Test
+ public void testStatFacetExpanded() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-metrics-facet-expanded[]
+ final Map<String, Object> expandedStatFacet = new HashMap<>();
+ expandedStatFacet.put("type", "func");
+ expandedStatFacet.put("func", "avg(div($numer,$denom))");
+ expandedStatFacet.put("numer", "mul(popularity,3.0)");
+ expandedStatFacet.put("denom", "price");
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("*:*")
+ .withFilter("price:[1.0 TO *]")
+ .withFilter("popularity:[0 TO 10]")
+ .withFacet("avg_value", expandedStatFacet);
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-metrics-facet-expanded[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(13, queryResponse.getResults().getNumFound());
+ assertEquals(10, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+ assertEquals(0.108, (double) topLevelFacetingData.getStatFacetValue("avg_value"), 0.1);
+ }
+
+ @Test
+ public void testQueryFacetSimple() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-query-facet-simple[]
+ QueryFacetMap queryFacet = new QueryFacetMap("popularity:[8 TO 10]");
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("*:*")
+ .withFacet("high_popularity", queryFacet);
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-query-facet-simple[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(32, queryResponse.getResults().getNumFound());
+ assertEquals(10, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+ assertEquals(2, topLevelFacetingData.getQueryFacet("high_popularity").getCount());
+ }
+
+ @Test
+ public void testQueryFacetExpanded() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-query-facet-expanded[]
+ QueryFacetMap queryFacet = new QueryFacetMap("popularity:[8 TO 10]")
+ .withStatSubFacet("average_price", "avg(price)");
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("*:*")
+ .withFacet("high_popularity", queryFacet);
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-query-facet-expanded[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(32, queryResponse.getResults().getNumFound());
+ assertEquals(10, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+ assertEquals(2, topLevelFacetingData.getQueryFacet("high_popularity").getCount());
+ assertEquals(199.5, topLevelFacetingData.getQueryFacet("high_popularity").getStatFacetValue("average_price"));
+ }
+
+ @Test
+ public void testRangeFacetSimple() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-range-facet-simple[]
+ RangeFacetMap rangeFacet = new RangeFacetMap("price", 0.0, 100.0, 20.0);
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("*:*")
+ .withFacet("prices", rangeFacet);
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-range-facet-simple[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(32, queryResponse.getResults().getNumFound());
+ assertEquals(10, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+ assertHasFacetWithBucketValues(topLevelFacetingData,"prices",
+ new FacetBucket(0.0f,5),
+ new FacetBucket(20.0f, 0),
+ new FacetBucket(40.0f, 0),
+ new FacetBucket(60.0f, 1),
+ new FacetBucket(80.0f, 1));
+ }
+
+ @Test
+ public void testNestedFacetSimple() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-nested-cat-facet[]
+ final TermsFacetMap topCategoriesFacet = new TermsFacetMap("cat").setLimit(3);
+ final TermsFacetMap topManufacturerFacet = new TermsFacetMap("manu_id_s").setLimit(1);
+ topCategoriesFacet.withSubFacet("top_manufacturers", topManufacturerFacet);
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("*:*")
+ .withFacet("categories", topCategoriesFacet);
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-nested-cat-facet[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(32, queryResponse.getResults().getNumFound());
+ assertEquals(10, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+
+ assertHasFacetWithBucketValues(topLevelFacetingData, "categories",
+ new FacetBucket("electronics", 12),
+ new FacetBucket("currency", 4),
+ new FacetBucket("memory", 3));
+
+ // Check the top manufacturer for each category
+ List<BucketJsonFacet> catBuckets = topLevelFacetingData.getBucketBasedFacets("categories").getBuckets();
+ assertHasFacetWithBucketValues(catBuckets.get(0), "top_manufacturers",
+ new FacetBucket("corsair", 3));
+ assertHasFacetWithBucketValues(catBuckets.get(1), "top_manufacturers",
+ new FacetBucket("boa", 1));
+ assertHasFacetWithBucketValues(catBuckets.get(2), "top_manufacturers",
+ new FacetBucket("corsair", 3));
+ }
+
+ @Test
+ public void testFacetSortedByNestedMetric() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-nested-cat-facet-sorted[]
+ final TermsFacetMap topCategoriesFacet = new TermsFacetMap("cat")
+ .setLimit(3)
+ .withStatSubFacet("avg_price", "avg(price)")
+ .setSort("avg_price desc");
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("*:*")
+ .withFacet("categories", topCategoriesFacet);
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-nested-cat-facet-sorted[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(32, queryResponse.getResults().getNumFound());
+ assertEquals(10, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+ assertHasFacetWithBucketValues(topLevelFacetingData, "categories",
+ new FacetBucket("electronics and computer1", 1),
+ new FacetBucket("graphics card", 2),
+ new FacetBucket("music", 1));
+ }
+
+ @Test
+ public void testFacetFilteredDomain() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-facet-filtered-domain[]
+ final TermsFacetMap categoryFacet = new TermsFacetMap("cat")
+ .setLimit(3)
+ .withDomain(new DomainMap().withFilter("popularity:[5 TO 10]"));
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("*:*")
+ .withFacet("categories", categoryFacet);
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-facet-filtered-domain[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(32, queryResponse.getResults().getNumFound());
+ assertEquals(10, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+ assertHasFacetWithBucketValues(topLevelFacetingData, "categories",
+ new FacetBucket("electronics", 9),
+ new FacetBucket("graphics card", 2),
+ new FacetBucket("hard drive", 2));
+ }
+
+ @Test
+ public void testFacetWidenedExcludeTagsDomain() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-facet-excludetags-domain[]
+ final TermsFacetMap inStockFacet = new TermsFacetMap("inStock").setLimit(2);
+ final TermsFacetMap allManufacturersFacet = new TermsFacetMap("manu_id_s")
+ .setLimit(2)
+ .withDomain(new DomainMap().withTagsToExclude("MANU"));
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("cat:electronics")
+ .withFilter("{!tag=MANU}manu_id_s:apple")
+ .withFacet("stock", inStockFacet)
+ .withFacet("manufacturers", allManufacturersFacet);
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-facet-excludetags-domain[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(1, queryResponse.getResults().getNumFound());
+ assertEquals(1, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+ assertHasFacetWithBucketValues(topLevelFacetingData, "stock",
+ new FacetBucket(true, 1));
+ assertHasFacetWithBucketValues(topLevelFacetingData, "manufacturers",
+ new FacetBucket("corsair", 3),
+ new FacetBucket("belkin", 2));
+ }
+
+ @Test
+ public void testFacetWidenedUsingQueryDomain() throws Exception {
+ SolrClient solrClient = cluster.getSolrClient();
+
+ //tag::solrj-json-facet-query-domain[]
+ final TermsFacetMap inStockFacet = new TermsFacetMap("inStock").setLimit(2);
+ final TermsFacetMap popularCategoriesFacet = new TermsFacetMap("cat")
+ .withDomain(new DomainMap().withQuery("popularity:[8 TO 10]"))
+ .setLimit(3);
+ final JsonQueryRequest request = new JsonQueryRequest()
+ .setQuery("apple")
+ .withFacet("popular_categories", popularCategoriesFacet);
+ QueryResponse queryResponse = request.process(solrClient, COLLECTION_NAME);
+ //end::solrj-json-facet-query-domain[]
+
+ assertEquals(0, queryResponse.getStatus());
+ assertEquals(1, queryResponse.getResults().getNumFound());
+ assertEquals(1, queryResponse.getResults().size());
+ final NestableJsonFacet topLevelFacetingData = queryResponse.getJsonFacetingResponse();
+ assertHasFacetWithBucketValues(topLevelFacetingData, "popular_categories",
+ new FacetBucket("electronics", 1),
+ new FacetBucket("music", 1),
+ new FacetBucket("search", 1));
+ }
+
private class FacetBucket {
private final Object val;
private final int count;
@@ -192,4 +738,9 @@ public class JsonRequestApiTest extends SolrCloudTestCase {
assertEquals(expectedBucket.getCount(), actualBucket.getCount());
}
}
+
+ private void assertResponseFoundNumDocs(QueryResponse response, int expectedNumDocs) {
+ assertEquals(0, response.getStatus());
+ assertEquals(expectedNumDocs, response.getResults().size());
+ }
}