You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by jb...@apache.org on 2018/10/28 17:15:29 UTC
[2/3] lucene-solr:branch_7x: SOLR-12913: RefGuide revisions
SOLR-12913: RefGuide revisions
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/4399bcd0
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/4399bcd0
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/4399bcd0
Branch: refs/heads/branch_7x
Commit: 4399bcd0abc30f9a803275b4e8f9a77734b62508
Parents: 3b4c0eb
Author: Joel Bernstein <jb...@apache.org>
Authored: Fri Oct 26 13:31:22 2018 -0400
Committer: Joel Bernstein <jb...@apache.org>
Committed: Sun Oct 28 13:14:55 2018 -0400
----------------------------------------------------------------------
.../src/computational-geometry.adoc | 12 ++---
solr/solr-ref-guide/src/curve-fitting.adoc | 9 ++--
solr/solr-ref-guide/src/dsp.adoc | 48 ++++++++++++++------
solr/solr-ref-guide/src/matrix-math.adoc | 4 +-
solr/solr-ref-guide/src/statistics.adoc | 2 +-
.../src/stream-decorator-reference.adoc | 46 +++++++++----------
solr/solr-ref-guide/src/vector-math.adoc | 4 +-
7 files changed, 75 insertions(+), 50 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/4399bcd0/solr/solr-ref-guide/src/computational-geometry.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/computational-geometry.adoc b/solr/solr-ref-guide/src/computational-geometry.adoc
index 263f6d0..abcdb08 100644
--- a/solr/solr-ref-guide/src/computational-geometry.adoc
+++ b/solr/solr-ref-guide/src/computational-geometry.adoc
@@ -24,14 +24,14 @@ functions.
A convex hull is the smallest convex set of points that encloses a data set. Math expressions has support for computing
the convex hull of a 2D data set. Once a convex hull has been calculated, a set of math expression functions
-can be used to geometrically describe the convex hull.
+can be applied to geometrically describe the convex hull.
The `convexHull` function finds the convex hull of an observation matrix of 2D vectors.
-Each row of the matrix contains a 2D observation.
+Each row of the matrix is a 2D observation.
In the example below a convex hull is calculated for a randomly generated set of 100 2D observations.
-Then the following functions are called on the convex result:
+Then the following functions are called on the convex hull:
-`getBaryCenter`: Returns the 2D point that is the bary center of the convex hull.
@@ -39,7 +39,7 @@ Then the following functions are called on the convex result:
-`getBoundarySize`: Returns the boundary size of the convex hull.
--`getVertices`: Returns 2D points that are the vertices of the convex hull.
+-`getVertices`: Returns a set of 2D points that are the vertices of the convex hull.
[source,text]
@@ -126,11 +126,11 @@ When this expression is sent to the `/stream` handler it responds with:
The `enclosingDisk` function finds the smallest enclosing circle the encloses a 2D data set.
Once an enclosing disk has been calculated, a set of math expression functions
-can be used to geometrically describe the enclosing disk.
+can be applied to geometrically describe the enclosing disk.
In the example below an enclosing disk is calculated for a randomly generated set of 1000 2D observations.
-Then the following functions are called on the enclosing disk result:
+Then the following functions are called on the enclosing disk:
-`getCenter`: Returns the 2D point that is the center of the disk.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/4399bcd0/solr/solr-ref-guide/src/curve-fitting.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/curve-fitting.adoc b/solr/solr-ref-guide/src/curve-fitting.adoc
index dc66eeb..c2086b1 100644
--- a/solr/solr-ref-guide/src/curve-fitting.adoc
+++ b/solr/solr-ref-guide/src/curve-fitting.adoc
@@ -230,10 +230,11 @@ for the x-axis.
The example below shows `harmfit` fitting a single oscillation of a sine wave. `harmfit`
returns the smoothed values at each control point. The return value is also a model which can be used by
-the `predict`, `derivative` and `integrate` function. There are also three helper functions that can be used to
-retrieve the estimated parameters of the fitted model:
+the `predict`, `derivative` and `integrate` functions.
-* `getAmplitude`: Returns the amplitude of sine wave.
+There are also three helper functions that can be used to retrieve the estimated parameters of the fitted model:
+
+* `getAmplitude`: Returns the amplitude of the sine wave.
* `getAngularFrequency`: Returns the angular frequency of the sine wave.
* `getPhase`: Returns the phase of the sine wave.
@@ -241,7 +242,7 @@ retrieve the estimated parameters of the fitted model:
oscillations. This is particularly true if the sine wave has noise. After the curve has been fit it can be
extrapolated to any point in time in the past or future.
-In example below the `harmfit` function fits control points, provided as x and y axes and the
+In the example below the `harmfit` function fits control points, provided as x and y axes, and then the
angular frequency, phase and amplitude are retrieved from the fitted model.
[source,text]
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/4399bcd0/solr/solr-ref-guide/src/dsp.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/dsp.adoc b/solr/solr-ref-guide/src/dsp.adoc
index 04524dc..8f3b24b 100644
--- a/solr/solr-ref-guide/src/dsp.adoc
+++ b/solr/solr-ref-guide/src/dsp.adoc
@@ -21,19 +21,43 @@ Digital Signal Processing (DSP).
== Dot Product
-The `dotProduct` function is used to calculate the dot product of two arrays.
+The `dotProduct` function is used to calculate the dot product of two numeric arrays.
The dot product is a fundamental calculation for the DSP functions discussed in this section. Before diving into
-the more advanced DSP functions, its useful to get a better understanding of how the dot product calculation works.
+the more advanced DSP functions its useful to develop a deeper intuition of the dot product.
-=== Combining Two Arrays
+The dot product operation is performed in two steps:
-The `dotProduct` function can be used to combine two arrays into a single product. A simple example can help
-illustrate this concept.
+1) Element-by-element multiplication of two vectors which produces a vector of products.
+2) Sum the vector of products to produce a scalar result.
+
+This simple bit of math has a number of important applications.
+
+=== Representing Linear Combinations
+
+The `dotProduct` performs the math of a Linear Combination. A Linear Combination has the following form:
+
+[source,text]
+----
+(a1*v1)+(a2*v2)...
+----
+
+In the above example a1 and a2 are random variables that change. v1 and v2 are *constant values*.
+
+When computing the dot product the elements of two vectors are multiplied together and the results are added.
+If the first vector contains random variables and the second vector contains *constant values*
+then the dot product is performing a linear combination.
+
+This scenario comes up again and again in machine learning. For example both linear and logistic regression
+solve for a vector of constant weights. In order to perform a prediction, a dot product is calculated
+between a random observation vector and the constant weight vector. That dot product is a linear combination because
+one of the vectors holds constant weights.
+
+Lets look at simple example of how a linear combination can be used to find the *mean* of a vector of numbers.
In the example below two arrays are set to variables *`a`* and *`b`* and then operated on by the `dotProduct` function.
The output of the `dotProduct` function is set to variable *`c`*.
-Then the `mean` function is then used to compute the mean of the first array which is set to the variable *`d`*.
+The `mean` function is then used to compute the mean of the first array which is set to the variable *`d`*.
Both the dot product and the mean are included in the output.
@@ -527,14 +551,12 @@ When this expression is sent to the `/stream` handler it responds with:
== Oscillate (Sine Wave)
-The `oscillate` function generates a periodic oscillating signal based
-on a parameters. The `oscillate` function can be used to study,
-combine and model sine waves.
+The `oscillate` function generates a periodic oscillating signal which can be used to model and study sine waves.
-The `oscillate` function takes three parameters: amplitude, angular frequency
-and phase and returns a vector contain the y axis points of sine wave.
+The `oscillate` function takes three parameters: *amplitude*, *angular frequency*
+and *phase* and returns a vector containing the y-axis points of a sine wave.
-The y axis points were generated from a sequence 0-127.
+The y-axis points were generated from an x-axis sequence of 0-127.
Below is an example of the `oscillate` function called with an amplitude of
1, and angular frequency of .28 and phase of 1.57.
@@ -551,7 +573,7 @@ image::images/math-expressions/sinewave.png[]
=== Sine Wave Interpolation, Extrapolation
The `oscillate` function returns a function which can be used by the `predict` function to interpolate or extrapolate a sine wave.
-The example below extrapolates the sine wave to a sequence from 0-256.
+The example below extrapolates the sine wave to an x-axis sequence of 0-256.
[source,text]
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/4399bcd0/solr/solr-ref-guide/src/matrix-math.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/matrix-math.adoc b/solr/solr-ref-guide/src/matrix-math.adoc
index 36191c4..b5cce75 100644
--- a/solr/solr-ref-guide/src/matrix-math.adoc
+++ b/solr/solr-ref-guide/src/matrix-math.adoc
@@ -103,7 +103,7 @@ responds with:
}
----
-== Pair sorting vectors
+== Pair Sorting Vectors
The `pairSort` function can be used to sort two vectors based on the values in
the first vector. The sorting operation maintains the pairing between
@@ -115,7 +115,7 @@ the second row in the matrix is the second vector.
The individual vectors can then be accessed using the `rowAt` function.
-The example below performs a pair sort on two vectors and returns the
+The example below performs a pair sort of two vectors and returns the
matrix containing the sorted vectors.
----
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/4399bcd0/solr/solr-ref-guide/src/statistics.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/statistics.adoc b/solr/solr-ref-guide/src/statistics.adoc
index 2d7a548..324a8a6 100644
--- a/solr/solr-ref-guide/src/statistics.adoc
+++ b/solr/solr-ref-guide/src/statistics.adoc
@@ -578,7 +578,7 @@ When this expression is sent to the `/stream` handler it responds with:
== Back Transformations
-Vectors that have been transformed with `log`, `log10`, `sqrt` and `cbrt` functions
+Vectors that have been transformed with the `log`, `log10`, `sqrt` and `cbrt` functions
can be back transformed using the `pow` function.
The example below shows how to back transform data that has been transformed by the
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/4399bcd0/solr/solr-ref-guide/src/stream-decorator-reference.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/stream-decorator-reference.adoc b/solr/solr-ref-guide/src/stream-decorator-reference.adoc
index aa8c55d..d9186c9 100644
--- a/solr/solr-ref-guide/src/stream-decorator-reference.adoc
+++ b/solr/solr-ref-guide/src/stream-decorator-reference.adoc
@@ -735,29 +735,6 @@ leftOuterJoin(
)
----
-[#list_expression]
-== list
-
-The `list` function wraps N Stream Expressions and opens and iterates each stream sequentially.
-This has the effect of concatenating the results of multiple Streaming Expressions.
-
-=== list Parameters
-
-* StreamExpressions ...: N Streaming Expressions
-
-=== list Syntax
-
-[source,text]
-----
-list(tuple(a="hello world"), tuple(a="HELLO WORLD"))
-
-list(search(collection1, q="*:*", fl="id, prod_ss", sort="id asc"),
- search(collection2, q="*:*", fl="id, prod_ss", sort="id asc"))
-
-list(tuple(a=search(collection1, q="*:*", fl="id, prod_ss", sort="id asc")),
- tuple(a=search(collection2, q="*:*", fl="id, prod_ss", sort="id asc")))
-----
-
== hashJoin
The `hashJoin` function wraps two streams, Left and Right, and for every tuple in Left which exists in Right will emit a tuple containing the fields of both tuples. This supports one-to-one, one-to-many, many-to-one, and many-to-many inner join scenarios. The tuples are emitted in the order in which they appear in the Left stream. The order of the streams does not matter. If both tuples contain a field of the same name then the value from the Right stream will be used in the emitted tuple.
@@ -863,6 +840,29 @@ intersect(
)
----
+[#list_expression]
+== list
+
+The `list` function wraps N Stream Expressions and opens and iterates each stream sequentially.
+This has the effect of concatenating the results of multiple Streaming Expressions.
+
+=== list Parameters
+
+* StreamExpressions ...: N Streaming Expressions
+
+=== list Syntax
+
+[source,text]
+----
+list(tuple(a="hello world"), tuple(a="HELLO WORLD"))
+
+list(search(collection1, q="*:*", fl="id, prod_ss", sort="id asc"),
+ search(collection2, q="*:*", fl="id, prod_ss", sort="id asc"))
+
+list(tuple(a=search(collection1, q="*:*", fl="id, prod_ss", sort="id asc")),
+ tuple(a=search(collection2, q="*:*", fl="id, prod_ss", sort="id asc")))
+----
+
== merge
The `merge` function merges two or more streaming expressions and maintains the ordering of the underlying streams. Because the order is maintained, the sorts of the underlying streams must line up with the on parameter provided to the merge function.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/4399bcd0/solr/solr-ref-guide/src/vector-math.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/vector-math.adoc b/solr/solr-ref-guide/src/vector-math.adoc
index d610d4e..6171d77 100644
--- a/solr/solr-ref-guide/src/vector-math.adoc
+++ b/solr/solr-ref-guide/src/vector-math.adoc
@@ -145,7 +145,9 @@ When this expression is sent to the `/stream` handler it responds with:
== Vector Sorting
-An array can be sorted in natural ascending order with `asc` function.
+An array can be sorted in natural ascending order with the `asc` function.
+
+The example below shows the `asc` function sorting an array:
[source,text]
----