You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by ct...@apache.org on 2021/01/07 04:43:31 UTC
[lucene-solr] branch jira/solr-13105-toMerge updated (0896629 ->
e8ec608)
This is an automated email from the ASF dual-hosted git repository.
ctargett pushed a change to branch jira/solr-13105-toMerge
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git.
from 0896629 Merge branch 'visual-guide' into jira/solr-13105-toMerge
new eb62980 Remove old doc left behind in branch merge; fix children list to pass the build
new e8ec608 Fix heading levels; remove TOC page links put in for github viewing
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../solr-ref-guide/src/computational-geometry.adoc | 4 -
solr/solr-ref-guide/src/curve-fitting.adoc | 7 -
solr/solr-ref-guide/src/dsp.adoc | 8 -
solr/solr-ref-guide/src/loading.adoc | 48 ++-
solr/solr-ref-guide/src/logs.adoc | 9 -
solr/solr-ref-guide/src/machine-learning.adoc | 16 +-
solr/solr-ref-guide/src/math-expressions.adoc | 2 +-
solr/solr-ref-guide/src/math-start.adoc | 13 -
solr/solr-ref-guide/src/matrix-math.adoc | 12 +-
solr/solr-ref-guide/src/numerical-analysis.adoc | 8 -
.../src/probability-distributions.adoc | 8 -
solr/solr-ref-guide/src/regression.adoc | 5 -
solr/solr-ref-guide/src/scalar-math.adoc | 7 +-
solr/solr-ref-guide/src/search-sample.adoc | 5 -
solr/solr-ref-guide/src/simulations.adoc | 7 -
solr/solr-ref-guide/src/statistics.adoc | 8 -
solr/solr-ref-guide/src/term-vectors.adoc | 5 -
solr/solr-ref-guide/src/time-series.adoc | 8 -
solr/solr-ref-guide/src/transform.adoc | 10 -
solr/solr-ref-guide/src/variables.adoc | 5 -
solr/solr-ref-guide/src/vector-math.adoc | 9 -
solr/solr-ref-guide/src/vectorization.adoc | 383 ---------------------
22 files changed, 27 insertions(+), 560 deletions(-)
delete mode 100644 solr/solr-ref-guide/src/vectorization.adoc
[lucene-solr] 02/02: Fix heading levels;
remove TOC page links put in for github viewing
Posted by ct...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ctargett pushed a commit to branch jira/solr-13105-toMerge
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
commit e8ec608ff87792e69b8e9ac562984265348a9ffa
Author: Cassandra Targett <ct...@apache.org>
AuthorDate: Wed Jan 6 22:40:05 2021 -0600
Fix heading levels; remove TOC page links put in for github viewing
---
.../solr-ref-guide/src/computational-geometry.adoc | 4 --
solr/solr-ref-guide/src/curve-fitting.adoc | 7 ----
solr/solr-ref-guide/src/dsp.adoc | 8 ----
solr/solr-ref-guide/src/loading.adoc | 48 +++++++++-------------
solr/solr-ref-guide/src/logs.adoc | 9 ----
solr/solr-ref-guide/src/machine-learning.adoc | 16 ++------
solr/solr-ref-guide/src/math-start.adoc | 13 ------
solr/solr-ref-guide/src/matrix-math.adoc | 12 +-----
solr/solr-ref-guide/src/numerical-analysis.adoc | 8 ----
.../src/probability-distributions.adoc | 8 ----
solr/solr-ref-guide/src/regression.adoc | 5 ---
solr/solr-ref-guide/src/scalar-math.adoc | 7 +---
solr/solr-ref-guide/src/search-sample.adoc | 5 ---
solr/solr-ref-guide/src/simulations.adoc | 7 ----
solr/solr-ref-guide/src/statistics.adoc | 8 ----
solr/solr-ref-guide/src/term-vectors.adoc | 5 ---
solr/solr-ref-guide/src/time-series.adoc | 8 ----
solr/solr-ref-guide/src/transform.adoc | 10 -----
solr/solr-ref-guide/src/variables.adoc | 5 ---
solr/solr-ref-guide/src/vector-math.adoc | 9 ----
20 files changed, 26 insertions(+), 176 deletions(-)
diff --git a/solr/solr-ref-guide/src/computational-geometry.adoc b/solr/solr-ref-guide/src/computational-geometry.adoc
index bb3f39d..83139f9 100644
--- a/solr/solr-ref-guide/src/computational-geometry.adoc
+++ b/solr/solr-ref-guide/src/computational-geometry.adoc
@@ -19,10 +19,6 @@
This section of the math expressions user guide covers computational geometry functions.
-<<Convex Hull, Convex Hull>> -
-<<Visualization, Visualization>> -
-<<Enclosing Disk, Enclosing Disk>>
-
== Convex Hull
A convex hull is the smallest convex set of points that encloses a data set. Math expressions has support for computing
diff --git a/solr/solr-ref-guide/src/curve-fitting.adoc b/solr/solr-ref-guide/src/curve-fitting.adoc
index e2cf6ff..c2edcec 100644
--- a/solr/solr-ref-guide/src/curve-fitting.adoc
+++ b/solr/solr-ref-guide/src/curve-fitting.adoc
@@ -18,11 +18,6 @@
These functions support constructing a curve through bivariate non-linear data.
-<<Polynomial Curve Fitting, Polynomial>> -
-<<Gaussian Curve Fitting, Gaussian>> -
-<<Harmonic Curve Fitting, Harmonic>>
-
-
== Polynomial Curve Fitting
The `polyfit` function is a general purpose curve fitter used to model
@@ -140,5 +135,3 @@ used to predict results for the model. This extrapolates the sine wave out to 12
the original model curve had only 19 control points.
image::images/math-expressions/harmfit2.png[]
-
-
diff --git a/solr/solr-ref-guide/src/dsp.adoc b/solr/solr-ref-guide/src/dsp.adoc
index 4a93575..22d59aa 100644
--- a/solr/solr-ref-guide/src/dsp.adoc
+++ b/solr/solr-ref-guide/src/dsp.adoc
@@ -19,14 +19,6 @@
This section of the user guide explores functions that are commonly used in the field of
Digital Signal Processing (DSP).
-<<Convolution, Convolution>> -
-<<Convolutional Smoothing, Smoothing>> -
-<<Cross-Correlation, Cross-Correlation>> -
-<<Find Delay, Find Delay>> -
-<<Oscillate (Sine Wave), Oscillate (Sine Wave)>> -
-<<Autocorrelation, Autocorrelation>> -
-<<Discrete Fourier Transform, Discrete Fourier Transform>>
-
== Convolution
The `conv` function calculates the convolution of two vectors. The convolution is calculated by *reversing*
diff --git a/solr/solr-ref-guide/src/loading.adoc b/solr/solr-ref-guide/src/loading.adoc
index daef99d..dd8005f 100644
--- a/solr/solr-ref-guide/src/loading.adoc
+++ b/solr/solr-ref-guide/src/loading.adoc
@@ -22,13 +22,6 @@ and loading CSV and TSV formatted data. These functions are designed to cut down
time spent on data preparation and allow users to begin data exploration before the data is
loaded into Solr.
-<<Reading Files, Reading>> -
-<<Parsing CSV and TSV Files, Parsing>> -
-<<Visualizing, Visualizing>> -
-<<Selecting Fields and Field Types, Selecting Fields and Types>> -
-<<Loading, Loading >> -
-<<Transforming Data, Transforming>>
-
== Reading Files
The `cat` function can be used to read files under the *userfiles* directory in
@@ -194,13 +187,13 @@ where the output of the `update` function can be spooled to disk.
image::images/math-expressions/update.png[]
-= Transforming Data
+== Transforming Data
Streaming Expressions and Math Expression provide a powerful set of functions
for transforming data. The section below shows some useful transformations that
can be applied while analyzing, visualizing and loading CSV and TSV files.
-== Unique IDs
+=== Unique IDs
Both `parseCSV` and `parseTSV` emit an *id* field if one is not present in the data already.
The *id* field is a concatenation of the file path and the line number. This is a
@@ -222,7 +215,7 @@ Below is an example using the `uuid` function to create a new id.
image::images/math-expressions/selectuuid.png[]
-== Record Numbers
+=== Record Numbers
The `recNum` function can be used inside of a `select` function to add a record number
to each tuple. The record number is useful for tracking location in the result set
@@ -234,7 +227,7 @@ The example below shows the syntax of the `recNum` function:
image::images/math-expressions/recNum.png[]
-== Parsing Dates
+=== Parsing Dates
The `dateTime` function can be used to parse dates into ISO 8601 format
needed for loading into a Solr date time field.
@@ -308,7 +301,7 @@ When this expression is sent to the `/stream` handler it responds with:
}
----
-== String Manipulation
+=== String Manipulation
The `upper`, `lower`, `split`, `valueAt`, `trim` and `concat` functions can be used to manipulate
strings inside of the `select` function.
@@ -368,8 +361,7 @@ When this expression is sent to the `/stream` handler it responds with:
"iris.csv",
"3"
]
- },
- ...
+ }]}}
----
The `valueAt` function can be used to select a specific index from
@@ -377,7 +369,7 @@ a split array.
image::images/math-expressions/valueat.png[]
-== Filtering Results
+=== Filtering Results
The `having` function can be used to filter records. Filtering can be used to systematically
explore specific record sets before indexing or to filter records that are sent for indexing.
@@ -389,28 +381,28 @@ The following boolean functions are supported: `eq`, `gt`, `gteq`, `lt`, `lteq`,
Below are some strategies for using the `having` function to filter records.
-=== Finding a Specific Id or Record Number
+==== Finding a Specific Id or Record Number
The `eq` (equals) function can be used with the `having` expression to filter the result set
to a single record number:
image::images/math-expressions/havingId.png[]
-=== Skipping
+==== Skipping
The `gt` (greater than) function can be used on the `recNum` field to filter the result set to
records with a recNum greater then a specific value:
image::images/math-expressions/skipping.png[]
-=== Paging
+==== Paging
The `and` function with nested `lt` and `gt` functions can be used to select records within a specific
record number range:
image::images/math-expressions/paging.png[]
-=== Striding
+==== Striding
The `eq` and nested `mod` function can be used to stride through the data at specific
record number intervals. This allows for a sample to be taken at different intervals in the data
@@ -418,14 +410,14 @@ in a systematic way.
image::images/math-expressions/striding.png[]
-=== Regex Matching
+==== Regex Matching
The `matches` function can be used to test if a field in the record matches a specific
regular expression. This provides a powerful *grep* like capability over the record set.
image::images/math-expressions/matches.png[]
-== Handling Nulls
+=== Handling Nulls
In most cases nulls do not need to be handled directly unless there is specific logic needed
to handle nulls during the load.
@@ -440,7 +432,7 @@ will simply be left off the record.
In certain scenarios it can be important to directly filter or replace nulls. The sections below cover these
scenarios.
-=== Filtering Nulls
+==== Filtering Nulls
The `having` and `isNull`, `notNull` functions can be combined to filter records that can contain null
values.
@@ -455,7 +447,7 @@ In the example below the `having` function returns all documents because the `is
image::images/math-expressions/havingIsNull.png[]
-=== Replacing Nulls
+==== Replacing Nulls
The `if` function and `isNull`, `notNull` functions can be combined to replace null values inside a `select` function.
@@ -466,7 +458,7 @@ In the second example it replace null *field1* values with the string literal "N
image::images/math-expressions/ifIsNull.png[]
-== Text Analysis
+=== Text Analysis
The `analyze` function can be used from inside a `select` function to analyze
a text field with a Lucene/Solr analyzer. The output of `analyze` is a
@@ -477,14 +469,14 @@ function can be used to expand the list of tokens to a stream of tuples.
There are a number of interesting use cases for the `analyze` function:
-- Previewing the output of different analyzers before indexing.
-- Annotating documents with NLP generated tokens (entity extraction, noun phrases etc...)
+* Previewing the output of different analyzers before indexing.
+* Annotating documents with NLP generated tokens (entity extraction, noun phrases etc...)
before the documents reach the indexing pipeline.
This removes heavy NLP processing from the servers that may also be handling queries. It also allows
more compute resources to be applied to the NLP indexing then is available on the search cluster.
-- Using the `cartesianProduct` function the analyzed tokens can be indexed as individual documents which allows
+* Using the `cartesianProduct` function the analyzed tokens can be indexed as individual documents which allows
analyzed tokens to be searched and analyzed with Solr's aggregation and graph expressions.
-- Also using `cartesianProduct` the analyzed tokens can be aggregated, analyzed and visualized using
+* Also using `cartesianProduct` the analyzed tokens can be aggregated, analyzed and visualized using
Streaming Expressions directly before indexing occurs.
diff --git a/solr/solr-ref-guide/src/logs.adoc b/solr/solr-ref-guide/src/logs.adoc
index d638c48..a049844 100644
--- a/solr/solr-ref-guide/src/logs.adoc
+++ b/solr/solr-ref-guide/src/logs.adoc
@@ -16,19 +16,11 @@
// specific language governing permissions and limitations
// under the License.
-
This section of the user guide provides an introduction to Solr log analytics.
NOTE: This is an appendix of the <<math-expressions.adoc#streaming-Expressions-and-math-expressions,Visual Guide to Streaming Expressions and Math Expressions>>. All the functions described below are convered in detail in the guide.
See the <<math-start.adoc#math-start,Getting Started>> chapter to learn how to get started with visualizations and Apache Zeppelin.
-<<Loading, Loading>> -
-<<Exploring, Exploring>> -
-<<Query Counting, Query Counting>> -
-<<Query Performance, Query Performance>> -
-<<Performance Trouble Shooting, Performance Trouble Shooting>> -
-<<Errors, Errors>>
-
== Loading
The out-of-the-box Solr log format can be loaded into a Solr index using the *postlogs* command line tool
@@ -412,4 +404,3 @@ If the error is followed by a stack trace the stack trace will be present in the
result.
image::images/math-expressions/stack.png[]
-
diff --git a/solr/solr-ref-guide/src/machine-learning.adoc b/solr/solr-ref-guide/src/machine-learning.adoc
index d9e8068..6bf8c63 100644
--- a/solr/solr-ref-guide/src/machine-learning.adoc
+++ b/solr/solr-ref-guide/src/machine-learning.adoc
@@ -20,16 +20,6 @@
This section of the math expressions user guide covers machine learning
functions.
-<<Distance and Distance Measures, Distance>> -
-<<K-Nearest Neighbor (KNN), KNN>> -
-<<K-Nearest Neighbor Regression, KNN Regression>> -
-<<knnSearch, knnSearch>> -
-<<DBSCAN, DBSCAN>> -
-<<K-Means Clustering, K-means>> -
-<<Fuzzy K-Means Clustering, Fuzzy K-means>> -
-<<Feature Scaling, Feature Scaling>>
-
-
== Distance and Distance Matrices
The `distance` function computes the distance for two numeric arrays or a distance matrix for the columns of a matrix.
@@ -430,7 +420,7 @@ Once the clustering has been completed there are a number of useful functions av
for examining and visualizing the clusters and centroids.
-==== Clustered Scatter Plot
+=== Clustered Scatter Plot
In this example we'll again be clustering 2D lat/lon points of rat sightings. But unlike the DBSCAN example, k-means clustering
does not on its own
@@ -461,7 +451,7 @@ insight into the densities of rat sightings throughout the five boroughs of New
example it highlights a cluster of dense sightings in Brooklyn at cluster1
surrounded by less dense but still high activity clusters.
-==== Plotting the Centroids
+=== Plotting the Centroids
The centroids of each cluster can then be plotted on a *map* to visualize the center of the
clusters. In the example below the centroids are extracted from the clusters using the `getCentroids`
@@ -839,4 +829,4 @@ When this expression is sent to the `/stream` handler it responds with:
]
}
}
-----
\ No newline at end of file
+----
diff --git a/solr/solr-ref-guide/src/math-start.adoc b/solr/solr-ref-guide/src/math-start.adoc
index 37968d5..0124923 100644
--- a/solr/solr-ref-guide/src/math-start.adoc
+++ b/solr/solr-ref-guide/src/math-start.adoc
@@ -16,9 +16,6 @@
// specific language governing permissions and limitations
// under the License.
-<<Language, Language>> -
-<<Execution, Execution>> -
-<<Visualization, Visualization>>
== Language
@@ -137,13 +134,3 @@ The `zplot` function has support for plotting *vectors*, *matrices*, *probabilit
There are many examples in the guide which show how to visualize both Streaming Expressions
and Math Expressions.
-
-
-
-
-
-
-
-
-
-
diff --git a/solr/solr-ref-guide/src/matrix-math.adoc b/solr/solr-ref-guide/src/matrix-math.adoc
index a4be9e1..f354228 100644
--- a/solr/solr-ref-guide/src/matrix-math.adoc
+++ b/solr/solr-ref-guide/src/matrix-math.adoc
@@ -21,16 +21,6 @@ as both inputs and outputs of many mathematical functions.
This section of the user guide covers the basics of matrix creation,
manipulation and matrix math.
-<<Matrices, Matrices>> -
-<<Row and Column Labels, Labels>> -
-<<Visualization, Visualization>> -
-<<Accessing Rows and Columns, Rows and Columns>> -
-<<Attributes, Attributes>> -
-<<Dimensions, Dimensions>> -
-<<Transposition, Transposition>> -
-<<Summations, Summations>> -
-<<Scalar Matrix Math, Matrix Math>>
-
== Matrices
A matrix can be created with the `matrix` function.
@@ -466,4 +456,4 @@ responds with:
]
}
}
-----
\ No newline at end of file
+----
diff --git a/solr/solr-ref-guide/src/numerical-analysis.adoc b/solr/solr-ref-guide/src/numerical-analysis.adoc
index e416b7d..41ee44d 100644
--- a/solr/solr-ref-guide/src/numerical-analysis.adoc
+++ b/solr/solr-ref-guide/src/numerical-analysis.adoc
@@ -18,14 +18,6 @@
This section explores the interrelated Math Expressions for Interpolation and Numerical Calculus.
-<<Interpolation, Interpolation>> -
-<<Smoothing Interpolation, Smoothing>> -
-<<Derivatives, Derivatives>> -
-<<Integrals, Integrals>> -
-<<Cumulative Integral Plot, Integral Plot>> -
-<<Bicubic Spline, Bicubic Spline>>
-
-
== Interpolation
Interpolation is used to construct new data points between a set of known control of points.
diff --git a/solr/solr-ref-guide/src/probability-distributions.adoc b/solr/solr-ref-guide/src/probability-distributions.adoc
index 4adf73a..09b757b 100644
--- a/solr/solr-ref-guide/src/probability-distributions.adoc
+++ b/solr/solr-ref-guide/src/probability-distributions.adoc
@@ -19,14 +19,6 @@
This section of the user guide covers the probability distribution
framework included in the math expressions library.
-<<Visualization, Visualization>> -
-<<Continuous Distributions, Continuous Distributions>> -
-<<Discrete Distributions, Discrete Distributions>> -
-<<Sampling, Sampling>> -
-<<Cumulative Probability, Cumulative Probability>> -
-<<Probability, Probability>> -
-<<Multivariate Normal Distribution, Multivariate Normal Distribution>>
-
== Visualization
Probability distributions can be visualized with Zeppelin-Solr using the
diff --git a/solr/solr-ref-guide/src/regression.adoc b/solr/solr-ref-guide/src/regression.adoc
index f0265c8..78b5a40 100644
--- a/solr/solr-ref-guide/src/regression.adoc
+++ b/solr/solr-ref-guide/src/regression.adoc
@@ -18,11 +18,6 @@
The math expressions library supports simple and multivariate linear regression.
-<<Simple Linear Regression, Simple Linear Regression>> -
-<<Regression Plot, Regression Plot>> -
-<<Residual Plot, Residual Plot>> -
-<<Multivariate Linear Regression, Multivariate Linear Regression>>
-
== Simple Linear Regression
The `regress` function is used to build a linear regression model
diff --git a/solr/solr-ref-guide/src/scalar-math.adoc b/solr/solr-ref-guide/src/scalar-math.adoc
index 46b1da0..089936f 100644
--- a/solr/solr-ref-guide/src/scalar-math.adoc
+++ b/solr/solr-ref-guide/src/scalar-math.adoc
@@ -74,7 +74,7 @@ This expression returns the following response:
}
----
-=== Visualization
+== Visualization
In the Zeppelin-Solr interpreter you can simply type in scalar math functions and the
result will be shown in a table format.
@@ -85,8 +85,6 @@ The *Number* visualization can be used to visualize the number with text and ico
image::images/math-expressions/num.png[]
-
-
== Streaming Scalar Math
Scalar math expressions can also be applied to each tuple in a stream
@@ -139,7 +137,7 @@ When this expression is sent to the `/stream` handler it responds with:
}
----
-=== Visualization
+== Visualization
The expression above can be visualized as a table using Zeppelin-Solr.
@@ -158,4 +156,3 @@ The following scalar math functions are available in the math expressions librar
`pow`, `mod`, `ceil`, `floor`, `sin`, `asin`,
`sinh`, `cos`, `acos`, `cosh`, `tan`, `atan`,
`tanh`, `round`, `precision`, `recip`, `sqrt`, `cbrt`
-
diff --git a/solr/solr-ref-guide/src/search-sample.adoc b/solr/solr-ref-guide/src/search-sample.adoc
index 23f0d70..e91983c 100644
--- a/solr/solr-ref-guide/src/search-sample.adoc
+++ b/solr/solr-ref-guide/src/search-sample.adoc
@@ -21,10 +21,6 @@ provides an overview of the key functions for retrieving data for
visualization and statistical analysis: searching, sampling
and aggregation.
-<<Searching, Searching>> - <<Sampling, Sampling>> - <<Aggregation, Aggregation>>
-- <<stats, stats>> - <<facet, facet>> - <<facet2D, facet2D>> - <<timeseries, timeseries>> -
-<<significantTerms, significantTerms>> - <<nodes, nodes>>
-
== Searching
=== Exploring
@@ -323,4 +319,3 @@ image::images/math-expressions/nodestab.png[]
The `nodes` function supports any combination of the following aggregate functions: count(*), sum, avg, min,
max.
-
diff --git a/solr/solr-ref-guide/src/simulations.adoc b/solr/solr-ref-guide/src/simulations.adoc
index bdab0ce..35b98d7 100644
--- a/solr/solr-ref-guide/src/simulations.adoc
+++ b/solr/solr-ref-guide/src/simulations.adoc
@@ -20,13 +20,6 @@ Monte Carlo simulations are commonly used to model the behavior of
stochastic (random) systems. This section of the user guide covers
the basics of performing Monte Carlo simulations with Math Expressions.
-<<Random Time Series, Random Time Series>> -
-<<Autocorrelation, Autocorrelation>> -
-<<Visualizing the Distribution, Visualizing & Fitting the Distribution>> -
-<<Monte Carlo, Monte Carlo>> -
-<<Random Walk, Random Walk>> -
-<<Multivariate Normal Distribution, Multivariate Normal Distribution>>
-
== Random Time Series
The daily movement of stock prices is often described as a "random walk".
diff --git a/solr/solr-ref-guide/src/statistics.adoc b/solr/solr-ref-guide/src/statistics.adoc
index e5f6ada..24787b5 100644
--- a/solr/solr-ref-guide/src/statistics.adoc
+++ b/solr/solr-ref-guide/src/statistics.adoc
@@ -16,17 +16,9 @@
// specific language governing permissions and limitations
// under the License.
-
This section of the user guide covers the core statistical functions
available in math expressions.
-
-<<descriptive-statistics, Descriptive Statistics>> - <<Histograms and Frequency Tables, Histograms>> -
-<<Frequency Tables, Frequency Tables>> - <<Percentiles, Percentiles>> - <<Quantile Plots, Quantile Plots>> -
-<<Correlation and Covariance, Correlation and Covariance>> - <<Statistical Inference Tests, Inference Tests>> -
-<<Transformations, Transformations>> - <<Z-scores, Z-scores>>
-
-
== Descriptive Statistics
The `describe` function returns descriptive statistics for a
diff --git a/solr/solr-ref-guide/src/term-vectors.adoc b/solr/solr-ref-guide/src/term-vectors.adoc
index 3fcf2dd..9736489 100644
--- a/solr/solr-ref-guide/src/term-vectors.adoc
+++ b/solr/solr-ref-guide/src/term-vectors.adoc
@@ -19,11 +19,6 @@
This section of the user guide presents an overview of the text analysis, text analytics
and TF-IDF term vector functions in math expressions.
-<<Text Analysis, Text Analysis>> -
-<<Annotating Documents, Annotating Documents>> -
-<<Text Analytics, Text Analytics>> -
-<<TF-IDF Term Vectors, TF-IDF Term Vectors>>
-
== Text Analysis
The `analyze` function applies a Solr analyzer to a text field and returns the tokens
diff --git a/solr/solr-ref-guide/src/time-series.adoc b/solr/solr-ref-guide/src/time-series.adoc
index bf30d77..2c04899 100644
--- a/solr/solr-ref-guide/src/time-series.adoc
+++ b/solr/solr-ref-guide/src/time-series.adoc
@@ -19,14 +19,6 @@
This section of the user guide provides an overview of some of the time series capabilities available
in Streaming Expressions and Math Expressions.
-<<Time Series Aggregation, Aggregation>> -
-<<Vectorizing the Time Series, Vectorization>> -
-<<Smoothing, Smoothing>> -
-<<Differencing, Differencing>> -
-<<Anomaly Detection, Anomaly Detection>> -
-<<Modeling, Modeling>> -
-<<Forecasting, Forecasting>>
-
== Time Series Aggregation
The `timeseries` function performs fast, distributed time
diff --git a/solr/solr-ref-guide/src/transform.adoc b/solr/solr-ref-guide/src/transform.adoc
index 481799a..11f4371 100644
--- a/solr/solr-ref-guide/src/transform.adoc
+++ b/solr/solr-ref-guide/src/transform.adoc
@@ -20,16 +20,6 @@
Streaming Expressions provides a powerful set of functions for transforming result sets. This section
of the user guide provides an overview of useful transformations applied to result sets.
-
-<<Selecting and Adding Fields, Selecting>> -
-<<Filtering Tuples, Filtering>> -
-<<Paging, Paging>> -
-<<Handling Nulls, Handling Nulls>> -
-<<Regex Matching, Regex Matching>> -
-<<Sorting, Sorting>> -
-<<Rollups, Rollups>>
-
-
== Selecting and Adding Fields
The `select` function wraps another streaming expression can perform the following operations on each tuple
diff --git a/solr/solr-ref-guide/src/variables.adoc b/solr/solr-ref-guide/src/variables.adoc
index 5453933..ee7109c 100644
--- a/solr/solr-ref-guide/src/variables.adoc
+++ b/solr/solr-ref-guide/src/variables.adoc
@@ -20,11 +20,6 @@
This section of the user guide describes how to assign and visualize
variables with math expressions.
-<<The Let Expression, Let>> -
-<<Echoing Variables, Echo>> -
-<<Visualizing Variables, Visualizing>> -
-<<Caching Variables, Caching>>
-
== The Let Expression
The `let` expression sets variables and returns
diff --git a/solr/solr-ref-guide/src/vector-math.adoc b/solr/solr-ref-guide/src/vector-math.adoc
index 2975060..3f3abcd 100644
--- a/solr/solr-ref-guide/src/vector-math.adoc
+++ b/solr/solr-ref-guide/src/vector-math.adoc
@@ -18,15 +18,6 @@
This section covers vector math and vector manipulation functions.
-<<Arrays, Arrays>> -
-<<Visualization, Visualization>> -
-<<Array Operations, Array Operations>> -
-<<Getting Values By Index, valueAt>> -
-<<Sequences, Sequences>> -
-<<Vector Sorting, Sorting>> -
-<<Vector Summarizations and Norms, Summarization>> -
-<<Scalar Vector Math, Vector Math>>
-
== Arrays
Arrays can be created with the `array` function.
[lucene-solr] 01/02: Remove old doc left behind in branch merge;
fix children list to pass the build
Posted by ct...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ctargett pushed a commit to branch jira/solr-13105-toMerge
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
commit eb6298091b0705c53b148ebeb149d2b21ddfa652
Author: Cassandra Targett <ct...@apache.org>
AuthorDate: Wed Jan 6 21:43:46 2021 -0600
Remove old doc left behind in branch merge; fix children list to pass the build
---
solr/solr-ref-guide/src/math-expressions.adoc | 2 +-
solr/solr-ref-guide/src/vectorization.adoc | 383 --------------------------
2 files changed, 1 insertion(+), 384 deletions(-)
diff --git a/solr/solr-ref-guide/src/math-expressions.adoc b/solr/solr-ref-guide/src/math-expressions.adoc
index 3554c90..343696e 100644
--- a/solr/solr-ref-guide/src/math-expressions.adoc
+++ b/solr/solr-ref-guide/src/math-expressions.adoc
@@ -1,5 +1,5 @@
= Streaming Expressions and Math Expressions
-:page-children: visualization, math-start, loading, search-sample, transform, scalar-math, vector-math, variables, matrix-math, term-vectors, statistics, probability-distributions, simulations, time-series, regression, numerical-analysis, curve-fitting, dsp, machine-learning, computational-geometry
+:page-children: visualization, math-start, loading, search-sample, transform, scalar-math, vector-math, variables, matrix-math, term-vectors, statistics, probability-distributions, simulations, time-series, regression, numerical-analysis, curve-fitting, dsp, machine-learning, computational-geometry, logs
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
diff --git a/solr/solr-ref-guide/src/vectorization.adoc b/solr/solr-ref-guide/src/vectorization.adoc
deleted file mode 100644
index 26a6f60..0000000
--- a/solr/solr-ref-guide/src/vectorization.adoc
+++ /dev/null
@@ -1,383 +0,0 @@
-= Streams and Vectorization
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-
-This section of the user guide explores techniques
-for retrieving streams of data from Solr and vectorizing the
-numeric fields.
-
-See the section <<term-vectors.adoc#term-vectors,Text Analysis and Term Vectors>> which describes how to
-vectorize text fields.
-
-== Streams
-
-Streaming Expressions has a wide range of stream sources that can be used to
-retrieve data from SolrCloud collections. Math expressions can be used
-to vectorize and analyze the results sets.
-
-Below are some of the key stream sources:
-
-* *`facet`*: Multi-dimensional aggregations are a powerful tool for generating
-co-occurrence counts for categorical data. The `facet` function uses the JSON facet API
-under the covers to provide fast, distributed, multi-dimension aggregations. With math expressions
-the aggregated results can be pivoted into a co-occurance matrix which can be mined for
-correlations and hidden similarities within the data.
-
-* *`random`*: Random sampling is widely used in statistics, probability and machine learning.
-The `random` function returns a random sample of search results that match a
-query. The random samples can be vectorized and operated on by math expressions and the results
-can be used to describe and make inferences about the entire population.
-
-* *`timeseries`*: The `timeseries`
-expression provides fast distributed time series aggregations, which can be
-vectorized and analyzed with math expressions.
-
-* *`knnSearch`*: K-nearest neighbor is a core machine learning algorithm. The `knnSearch`
-function is a specialized knn algorithm optimized to find the k-nearest neighbors of a document in
-a distributed index. Once the nearest neighbors are retrieved they can be vectorized
-and operated on by machine learning and text mining algorithms.
-
-* *`sql`*: SQL is the primary query language used by data scientists. The `sql` function supports
-data retrieval using a subset of SQL which includes both full text search and
-fast distributed aggregations. The result sets can then be vectorized and operated
-on by math expressions.
-
-* *`jdbc`*: The `jdbc` function allows data from any JDBC compliant data source to be combined with
-streams originating from Solr. Result sets from outside data sources can be vectorized and operated
-on by math expressions in the same manner as result sets originating from Solr.
-
-* *`topic`*: Messaging is an important foundational technology for large scale computing. The `topic`
-function provides publish/subscribe messaging capabilities by treating
-SolrCloud as a distributed message queue. Topics are extremely powerful
-because they allow subscription by query. Topics can be use to support a broad set of
-use cases including bulk text mining operations and AI alerting.
-
-* *`nodes`*: Graph queries are frequently used by recommendation engines and are an important
-machine learning tool. The `nodes` function provides fast, distributed, breadth
-first graph traversal over documents in a SolrCloud collection. The node sets collected
-by the `nodes` function can be operated on by statistical and machine learning expressions to
-gain more insight into the graph.
-
-* *`search`*: Ranked search results are a powerful tool for finding the most relevant
-documents from a large document corpus. The `search` expression
-returns the top N ranked search results that match any
-Solr query, including geo-spatial queries. The smaller set of relevant
-documents can then be explored with statistical, machine learning and
-text mining expressions to gather insights about the data set.
-
-== Assigning Streams to Variables
-
-The output of any streaming expression can be set to a variable.
-Below is a very simple example using the `random` function to fetch
-three random samples from collection1. The random samples are returned
-as tuples which contain name/value pairs.
-
-
-[source,text]
-----
-let(a=random(collection1, q="*:*", rows="3", fl="price_f"))
-----
-
-When this expression is sent to the `/stream` handler it responds with:
-
-[source,json]
-----
-{
- "result-set": {
- "docs": [
- {
- "a": [
- {
- "price_f": 0.7927976
- },
- {
- "price_f": 0.060795486
- },
- {
- "price_f": 0.55128294
- }
- ]
- },
- {
- "EOF": true,
- "RESPONSE_TIME": 11
- }
- ]
- }
-}
-----
-
-== Creating a Vector with the col Function
-
-The `col` function iterates over a list of tuples and copies the values
-from a specific column into an array.
-
-The output of the `col` function is an numeric array that can be set to a
-variable and operated on by math expressions.
-
-Below is an example of the `col` function:
-
-[source,text]
-----
-let(a=random(collection1, q="*:*", rows="3", fl="price_f"),
- b=col(a, price_f))
-----
-
-[source,json]
-----
-{
- "result-set": {
- "docs": [
- {
- "b": [
- 0.42105234,
- 0.85237443,
- 0.7566981
- ]
- },
- {
- "EOF": true,
- "RESPONSE_TIME": 9
- }
- ]
- }
-}
-----
-
-== Applying Math Expressions to the Vector
-
-Once a vector has been created any math expression that operates on vectors
-can be applied. In the example below the `mean` function is applied to
-the vector assigned to variable *`b`*.
-
-[source,text]
-----
-let(a=random(collection1, q="*:*", rows="15000", fl="price_f"),
- b=col(a, price_f),
- c=mean(b))
-----
-
-When this expression is sent to the `/stream` handler it responds with:
-
-[source,json]
-----
-{
- "result-set": {
- "docs": [
- {
- "c": 0.5016035594638814
- },
- {
- "EOF": true,
- "RESPONSE_TIME": 306
- }
- ]
- }
-}
-----
-
-== Creating Matrices
-
-Matrices can be created by vectorizing multiple numeric fields
-and adding them to a matrix. The matrices can then be operated on by
-any math expression that operates on matrices.
-
-[TIP]
-====
-Note that this section deals with the creation of matrices
-from numeric data. The section <<term-vectors.adoc#term-vectors,Text Analysis and Term Vectors>> describes how to build TF-IDF term vector matrices from text fields.
-====
-
-Below is a simple example where four random samples are taken
-from different sub-populations in the data. The `price_f` field of
-each random sample is
-vectorized and the vectors are added as rows to a matrix.
-Then the `sumRows`
-function is applied to the matrix to return a vector containing
-the sum of each row.
-
-[source,text]
-----
-let(a=random(collection1, q="market:A", rows="5000", fl="price_f"),
- b=random(collection1, q="market:B", rows="5000", fl="price_f"),
- c=random(collection1, q="market:C", rows="5000", fl="price_f"),
- d=random(collection1, q="market:D", rows="5000", fl="price_f"),
- e=col(a, price_f),
- f=col(b, price_f),
- g=col(c, price_f),
- h=col(d, price_f),
- i=matrix(e, f, g, h),
- j=sumRows(i))
-----
-
-When this expression is sent to the `/stream` handler it responds with:
-
-[source,json]
-----
-{
- "result-set": {
- "docs": [
- {
- "j": [
- 154390.1293375,
- 167434.89453,
- 159293.258493,
- 149773.42769,
- ]
- },
- {
- "EOF": true,
- "RESPONSE_TIME": 9
- }
- ]
- }
-}
-----
-
-== Facet Co-occurrence Matrices
-
-The `facet` function can be used to quickly perform multi-dimension aggregations of categorical data from
-records stored in a SolrCloud collection. These multi-dimension aggregations can represent co-occurrence
-counts for the values in the dimensions. The `pivot` function can be used to move two dimensional
-aggregations into a co-occurrence matrix. The co-occurrence matrix can then be clustered or analyzed for
-correlations to learn about the hidden connections within the data.
-
-In the example below the `facet` expression is used to generate a two dimensional faceted aggregation.
-The first dimension is the US State that a car was purchased in and the second dimension is the car model.
-This two dimensional facet generates the co-occurrence counts for the number of times a particular car model
-was purchased in a particular state.
-
-
-[source,text]
-----
-facet(collection1, q="*:*", buckets="state, model", bucketSorts="count(*) desc", rows=5, count(*))
-----
-
-When this expression is sent to the `/stream` handler it responds with:
-
-[source,json]
-----
-{
- "result-set": {
- "docs": [
- {
- "state": "NY",
- "model": "camry",
- "count(*)": 13342
- },
- {
- "state": "NJ",
- "model": "accord",
- "count(*)": 13002
- },
- {
- "state": "NY",
- "model": "civic",
- "count(*)": 12901
- },
- {
- "state": "CA",
- "model": "focus",
- "count(*)": 12892
- },
- {
- "state": "TX",
- "model": "f150",
- "count(*)": 12871
- },
- {
- "EOF": true,
- "RESPONSE_TIME": 171
- }
- ]
- }
-}
-----
-
-The `pivot` function can be used to move the facet results into a co-occurrence matrix. In the example below
-The `pivot` function is used to create a matrix where the rows of the matrix are the US States (state) and the
-columns of the matrix are the car models (model). The values in the matrix are the co-occurrence counts (count(*))
- from the facet results. Once the co-occurrence matrix has been created the US States can be clustered
-by car model, or the matrix can be transposed and car models can be clustered by the US States
-where they were bought.
-
-[source,text]
-----
-let(a=facet(collection1, q="*:*", buckets="state, model", bucketSorts="count(*) desc", rows="-1", count(*)),
- b=pivot(a, state, model, count(*)),
- c=kmeans(b, 7))
-----
-
-== Latitude / Longitude Vectors
-
-The `latlonVectors` function wraps a list of tuples and parses a lat/lon location field into
-a matrix of lat/long vectors. Each row in the matrix is a vector that contains the lat/long
-pair for the corresponding tuple in the list. The row labels for the matrix are
-automatically set to the `id` field in the tuples. The lat/lon matrix can then be operated
-on by distance-based machine learning functions using the `haversineMeters` distance measure.
-
-The `latlonVectors` function takes two parameters: a list of tuples and a named parameter called
-`field`, which tells the `latlonVectors` function which field to parse the lat/lon
-vectors from.
-
-Below is an example of the `latlonVectors`.
-
-[source,text]
-----
-let(a=random(collection1, q="*:*", fl="id, loc_p", rows="5"),
- b=latlonVectors(a, field="loc_p"))
-----
-
-When this expression is sent to the `/stream` handler it responds with:
-
-[source,json]
-----
-{
- "result-set": {
- "docs": [
- {
- "b": [
- [
- 42.87183530723629,
- 76.74102353397778
- ],
- [
- 42.91372904094898,
- 76.72874889228416
- ],
- [
- 42.911528804897564,
- 76.70537292977619
- ],
- [
- 42.91143870500213,
- 76.74749913047408
- ],
- [
- 42.904666267479705,
- 76.73933236046092
- ]
- ]
- },
- {
- "EOF": true,
- "RESPONSE_TIME": 21
- }
- ]
- }
-}
-----