[lucene-solr] branch SOLR-13105-visual updated: SOLR-13105: More transform docs

[jb...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

jbernste pushed a commit to branch SOLR-13105-visual
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git


The following commit(s) were added to refs/heads/SOLR-13105-visual by this push:
     new cc208c8  SOLR-13105: More transform docs
cc208c8 is described below

commit cc208c835305fa103e35a08e1656b38356ef20fc
Author: Joel Bernstein <jb...@apache.org>
AuthorDate: Mon Sep 16 12:41:17 2019 -0400

    SOLR-13105: More transform docs
---
 .../src/images/math-expressions/hashRollup.png     | Bin 0 -> 178158 bytes
 .../src/images/math-expressions/search-matches.png | Bin 0 -> 185235 bytes
 .../src/images/math-expressions/search-page.png    | Bin 0 -> 211086 bytes
 .../src/images/math-expressions/search-resort.png  | Bin 0 -> 170565 bytes
 .../math-expressions/search-resort.textClipping    | Bin 0 -> 215 bytes
 solr/solr-ref-guide/src/transform.adoc             |  48 +++++++++++++++++++--
 6 files changed, 44 insertions(+), 4 deletions(-)

diff --git a/solr/solr-ref-guide/src/images/math-expressions/hashRollup.png b/solr/solr-ref-guide/src/images/math-expressions/hashRollup.png
new file mode 100644
index 0000000..2f5d23c
Binary files /dev/null and b/solr/solr-ref-guide/src/images/math-expressions/hashRollup.png differ
diff --git a/solr/solr-ref-guide/src/images/math-expressions/search-matches.png b/solr/solr-ref-guide/src/images/math-expressions/search-matches.png
new file mode 100644
index 0000000..020b0dd
Binary files /dev/null and b/solr/solr-ref-guide/src/images/math-expressions/search-matches.png differ
diff --git a/solr/solr-ref-guide/src/images/math-expressions/search-page.png b/solr/solr-ref-guide/src/images/math-expressions/search-page.png
new file mode 100644
index 0000000..24a2f8a
Binary files /dev/null and b/solr/solr-ref-guide/src/images/math-expressions/search-page.png differ
diff --git a/solr/solr-ref-guide/src/images/math-expressions/search-resort.png b/solr/solr-ref-guide/src/images/math-expressions/search-resort.png
new file mode 100644
index 0000000..094b28c
Binary files /dev/null and b/solr/solr-ref-guide/src/images/math-expressions/search-resort.png differ
diff --git a/solr/solr-ref-guide/src/images/math-expressions/search-resort.textClipping b/solr/solr-ref-guide/src/images/math-expressions/search-resort.textClipping
new file mode 100644
index 0000000..5b3815ea
Binary files /dev/null and b/solr/solr-ref-guide/src/images/math-expressions/search-resort.textClipping differ
diff --git a/solr/solr-ref-guide/src/transform.adoc b/solr/solr-ref-guide/src/transform.adoc
index e9fb94d..fb0417e 100644
--- a/solr/solr-ref-guide/src/transform.adoc
+++ b/solr/solr-ref-guide/src/transform.adoc
@@ -17,6 +17,9 @@
 // under the License.
 
 
+Streaming Expressions provides a powerful set of functions for transforming result sets. This section
+of the user guide provides an overview of some useful transformations applied to result sets.
+
 == Selecting and Adding Fields
 
 The `select` function wraps another streaming expression and can select a
@@ -30,7 +33,6 @@ of streaming math will be shown later in the userguide.
 
 image::images/math-expressions/select1.png[]
 
-
 == Filtering Tuples
 
 The `having` function can be used to filter tuples in the stream based
@@ -42,6 +44,17 @@ In the example below the `having` function is filtering the output of the
 image::images/math-expressions/having.png[]
 
 
+== Paging
+
+The *record number*, added with the `recNum` function
+can, be filtered on to support paging.
+
+In the example below the `and` function with nested `lt` and `gt` functions are
+used to select records within a specific record number range:
+
+image::images/math-expressions/search-page.png[]
+
+
 == Handling Nulls
 
 The `notNull` and `isNull` function can be used to either replace null values with different values,
@@ -69,12 +82,39 @@ and field that is not in the result set, which always returns false.
 
 image::images/math-expressions/having3.png[]
 
-== Regex Matching
+== Regex Matching and Filtering
 
-The `matches` function can used to perform regex matches on fields ...
+The `matches` function can be used inside of a `having` function
+to test if a field in the record matches a specific
+regular expression. This allows for sophisticated regex matching over search results.
 
-== Paging
+The example below uses the `matches` function to return all records where
+the *complaint_type_s* field ends with *Commercial*.
+
+image::images/math-expressions/search-matches.png[]
 
 == Sorting
 
+The `sort` and `top` function can be used to resort a result set in memory. The `sort` function
+sorts and returns the entire result set based on the sort criteria. The `top` function
+can be used to return the top N values in a result set based on the sort criteria.
+
+image::images/math-expressions/search-resort.png[]
+
 == Rollups
+
+The `rollup` and `hashRollup` functions can be used to perform aggregations over result sets. This
+is different then the `facet`, `facet2D` and `timeseries` aggregation functions which push the aggregations
+into the search engine using.
+
+The `rollup` function performs map-reduce style rollups, which requires the result stream to be sorted by the
+the grouping fields. This allows for aggregations over very high cardinality fields. The `hashRollup` function
+performs rollups keeping all buckets in an in-memory hashmap. This requires enough memory to store all the
+distinct group by fields in memory, but does not require that the underlying stream be sorted.
+
+The example below shows a visualization of the top 5 complaint types
+from a random sample from the `nyc311` complaint database. The `top`
+function is used to select the top 5 complaint types based on
+the `count(*)` field output by the `hashRollup`.
+
+image::images/math-expressions/hashRollup.png[]