You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by jb...@apache.org on 2017/06/07 01:45:45 UTC
[1/8] lucene-solr:branch_6_6: SOLR-10426: Add documentation
Repository: lucene-solr
Updated Branches:
refs/heads/branch_6_6 9b6b3e626 -> da03b1e90
SOLR-10426: Add documentation
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/32822984
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/32822984
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/32822984
Branch: refs/heads/branch_6_6
Commit: 328229842b61bc98a139c022a975e232172d1b87
Parents: 9b6b3e6
Author: Joel Bernstein <jb...@apache.org>
Authored: Thu Jun 1 15:35:10 2017 -0400
Committer: Joel Bernstein <jb...@apache.org>
Committed: Tue Jun 6 21:40:13 2017 -0400
----------------------------------------------------------------------
solr/solr-ref-guide/src/stream-sources.adoc | 25 +++++++++++++++++++++++-
1 file changed, 24 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/32822984/solr/solr-ref-guide/src/stream-sources.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/stream-sources.adoc b/solr/solr-ref-guide/src/stream-sources.adoc
index 7d3a2e2..ad91eb3 100644
--- a/solr/solr-ref-guide/src/stream-sources.adoc
+++ b/solr/solr-ref-guide/src/stream-sources.adoc
@@ -327,7 +327,30 @@ The expression above performs a breadth-first search to find the shortest paths
The search starts from the nodeID "\john@company.com" in the `from_address` field and searches for the nodeID "\jane@company.com" in the `to_address` field. This search is performed iteratively until the `maxDepth` has been reached. Each level in the traversal is implemented as a parallel partitioned nested loop join across the entire collection. The `threads` parameter controls the number of threads performing the join at each level, while the `partitionSize` parameter controls the of number of nodes in each join partition. The `maxDepth` parameter controls the number of levels to traverse. `fq` is a limiting query applied to each level in the traversal.
== shuffle
-//TODO
+
+The `shuffle` expression sorts and exports entire result sets. The `shuffle` expression is similar to the `search` expression except that
+under the covers `shuffle` always uses the /export handler. The `shuffle` expression is designed to be combined with the relational algebra
+decorators that require complete, sorted result sets. Shuffled result sets can be partitioned across worker nodes with the parallel
+stream decorator to perform parallel relational algebra. When used in parallel mode the partitionKeys parameter must be provided.
+
+=== shuffle Parameters
+
+* `collection`: (Mandatory) the collection being searched.
+* `q`: (Mandatory) The query to perform on the Solr index.
+* `fl`: (Mandatory) The list of fields to return.
+* `sort`: (Mandatory) The sort criteria.
+* `zkHost`: Only needs to be defined if the collection being searched is found in a different zkHost than the local stream handler.
+* `partitionKeys`: Comma delimited list of keys to partition the search results by. To be used with the parallel function for parallelizing operations across worker nodes. See the <<stream-decorators.adoc#parallel,parallel>> function for details.
+
+=== shuffle Syntax
+
+[source,text]
+----
+shuffle(collection1,
+ q="*:*",
+ fl="id,a_s,a_i,a_f",
+ sort="a_f asc, a_i asc")
+----
== stats
[3/8] lucene-solr:branch_6_6: SOLR-10504: Add documentation
Posted by jb...@apache.org.
SOLR-10504: Add documentation
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/ab90ab7e
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/ab90ab7e
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/ab90ab7e
Branch: refs/heads/branch_6_6
Commit: ab90ab7e70d931d285f96a2ca90fd660cbe72844
Parents: 30cd665
Author: Joel Bernstein <jb...@apache.org>
Authored: Thu Jun 1 15:59:24 2017 -0400
Committer: Joel Bernstein <jb...@apache.org>
Committed: Tue Jun 6 21:42:11 2017 -0400
----------------------------------------------------------------------
solr/solr-ref-guide/src/stream-sources.adoc | 2 ++
1 file changed, 2 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/ab90ab7e/solr/solr-ref-guide/src/stream-sources.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/stream-sources.adoc b/solr/solr-ref-guide/src/stream-sources.adoc
index fcd006c..e7356ae 100644
--- a/solr/solr-ref-guide/src/stream-sources.adoc
+++ b/solr/solr-ref-guide/src/stream-sources.adoc
@@ -117,6 +117,8 @@ jdbc(
The `echo` function returns a single Tuple echoing its text parameter. `Echo` is the simplest stream source designed to provide text
to a text analyzing stream decorator.
+=== echo Syntax
+
[source,text]
----
echo("Hello world")
[2/8] lucene-solr:branch_6_6: SOLR-10504: Add documentation
Posted by jb...@apache.org.
SOLR-10504: Add documentation
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/30cd6652
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/30cd6652
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/30cd6652
Branch: refs/heads/branch_6_6
Commit: 30cd66522004f5c4e4e4f573b23c8c543471bf43
Parents: 3282298
Author: Joel Bernstein <jb...@apache.org>
Authored: Thu Jun 1 15:56:05 2017 -0400
Committer: Joel Bernstein <jb...@apache.org>
Committed: Tue Jun 6 21:41:18 2017 -0400
----------------------------------------------------------------------
solr/solr-ref-guide/src/stream-sources.adoc | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/30cd6652/solr/solr-ref-guide/src/stream-sources.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/stream-sources.adoc b/solr/solr-ref-guide/src/stream-sources.adoc
index ad91eb3..fcd006c 100644
--- a/solr/solr-ref-guide/src/stream-sources.adoc
+++ b/solr/solr-ref-guide/src/stream-sources.adoc
@@ -113,7 +113,14 @@ jdbc(
----
== echo
-//TODO
+
+The `echo` function returns a single Tuple echoing its text parameter. `Echo` is the simplest stream source designed to provide text
+to a text analyzing stream decorator.
+
+[source,text]
+----
+echo("Hello world")
+----
== facet
[6/8] lucene-solr:branch_6_6: SOLR-10559: Add tuple documentation
Posted by jb...@apache.org.
SOLR-10559: Add tuple documentation
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/c2f2a43a
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/c2f2a43a
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/c2f2a43a
Branch: refs/heads/branch_6_6
Commit: c2f2a43acb9b4a94a1622a4d1feff922353fd09e
Parents: c3b192a
Author: Joel Bernstein <jb...@apache.org>
Authored: Thu Jun 1 17:11:50 2017 -0400
Committer: Joel Bernstein <jb...@apache.org>
Committed: Tue Jun 6 21:43:21 2017 -0400
----------------------------------------------------------------------
solr/solr-ref-guide/src/stream-sources.adoc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/c2f2a43a/solr/solr-ref-guide/src/stream-sources.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/stream-sources.adoc b/solr/solr-ref-guide/src/stream-sources.adoc
index 2bb0805..e8887b4 100644
--- a/solr/solr-ref-guide/src/stream-sources.adoc
+++ b/solr/solr-ref-guide/src/stream-sources.adoc
@@ -485,7 +485,7 @@ topic(checkpointCollection,
The `tuple` function emits a single Tuple with name/value pairs. The values can be set to variables assigned in a `let` expression, literals, Stream Evaluators or
Stream Expressions. In the case of Stream Evaluators the tuple will output the return value from the evaluator.
-This could be a numeric, list or map. If a Stream Expression is set to a value, the `tuple` function will flatten
+This could be a numeric, list or map. If a value is set to a Stream Expression, the `tuple` function will flatten
the tuple stream from the Stream Expression into a list of Tuples.
=== tuple Parameters
[8/8] lucene-solr:branch_6_6: SOLR-10516: Add documention
Posted by jb...@apache.org.
SOLR-10516: Add documention
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/da03b1e9
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/da03b1e9
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/da03b1e9
Branch: refs/heads/branch_6_6
Commit: da03b1e90fd71ebde9770f92fadc6f15b123b3a3
Parents: 90b3879
Author: Joel Bernstein <jb...@apache.org>
Authored: Tue Jun 6 13:07:42 2017 -0400
Committer: Joel Bernstein <jb...@apache.org>
Committed: Tue Jun 6 21:44:59 2017 -0400
----------------------------------------------------------------------
solr/solr-ref-guide/src/stream-decorators.adoc | 23 ++++++++++++++++++---
solr/solr-ref-guide/src/stream-sources.adoc | 5 ++++-
2 files changed, 24 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/da03b1e9/solr/solr-ref-guide/src/stream-decorators.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/stream-decorators.adoc b/solr/solr-ref-guide/src/stream-decorators.adoc
index cbbc9ef..1594833 100644
--- a/solr/solr-ref-guide/src/stream-decorators.adoc
+++ b/solr/solr-ref-guide/src/stream-decorators.adoc
@@ -23,8 +23,6 @@
== cartesianProduct
//TODO
-== cell
-//TODO
== classify
@@ -237,7 +235,26 @@ daemonStream.close();
----
== eval
-//todo
+
+The `eval` function allows for use cases where new streaming expressions are generated on the fly and then evaluated.
+The `eval` function wraps a streaming expression and reads a single tuple from the underlying stream.
+The `eval` function then retrieves a string Streaming Expressions from the `expr_s` field of the tuple.
+The `eval` function then compiles the string Streaming Expression and emits the tuples.
+
+=== eval Parameters
+
+* `StreamExpression`: (Mandatory) The stream which provides the streaming expression to be evaluated.
+
+=== eval Syntax
+
+[source,text]
+----
+eval(expr)
+----
+
+In the example above the `eval` expression reads the first tuple from the underlying expression. It then compiles and
+executes the string Streaming Expression in the epxr_s field.
+
[[StreamingExpressions-executor]]
== executor
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/da03b1e9/solr/solr-ref-guide/src/stream-sources.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/stream-sources.adoc b/solr/solr-ref-guide/src/stream-sources.adoc
index 97a1ca4..973d135 100644
--- a/solr/solr-ref-guide/src/stream-sources.adoc
+++ b/solr/solr-ref-guide/src/stream-sources.adoc
@@ -284,6 +284,7 @@ The `significantTerms` function queries a SolrCloud collection, but instead of r
* `collection`: (Mandatory) The collection that the function is run on.
* `q`: (Mandatory) The query that describes the foreground document set.
+* `field`: (Mandatory) The field to extract the terms from.
* `limit`: (Optional, Default 20) The max number of terms to return.
* `minDocFreq`: (Optional, Defaults to 5 documents) The minimum number of documents the term must appear in on a shard. This is a float value. If greater then 1.0 then it's considered the absolute number of documents. If less then 1.0 it's treated as a percentage of documents.
* `maxDocFreq`: (Optional, Defaults to 30% of documents) The maximum number of documents the term can appear in on a shard. This is a float value. If greater then 1.0 then it's considered the absolute number of documents. If less then 1.0 it's treated as a percentage of documents.
@@ -295,12 +296,14 @@ The `significantTerms` function queries a SolrCloud collection, but instead of r
----
significantTerms(collection1,
q="body:Solr",
+ field="author",
+ limit="50",
minDocFreq="10",
maxDocFreq=".20",
minTermLength="5")
----
-In the example above the `significantTerms` function is querying `collection1` and returning at most 50 significant terms that appear in 10 or more documents but not more then 20% of the corpus.
+In the example above the `significantTerms` function is querying `collection1` and returning at most 50 significant terms from the `authors` field that appear in 10 or more documents but not more then 20% of the corpus.
== shortestPath
[4/8] lucene-solr:branch_6_6: SOLR-10566: Add documentation
Posted by jb...@apache.org.
SOLR-10566: Add documentation
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/204447ff
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/204447ff
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/204447ff
Branch: refs/heads/branch_6_6
Commit: 204447ffda81913d7ae75e42ed87a5c5400d9b6a
Parents: ab90ab7
Author: Joel Bernstein <jb...@apache.org>
Authored: Thu Jun 1 16:26:57 2017 -0400
Committer: Joel Bernstein <jb...@apache.org>
Committed: Tue Jun 6 21:42:36 2017 -0400
----------------------------------------------------------------------
solr/solr-ref-guide/src/stream-sources.adoc | 31 +++++++++++++++++++++++-
1 file changed, 30 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/204447ff/solr/solr-ref-guide/src/stream-sources.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/stream-sources.adoc b/solr/solr-ref-guide/src/stream-sources.adoc
index e7356ae..986a5da 100644
--- a/solr/solr-ref-guide/src/stream-sources.adoc
+++ b/solr/solr-ref-guide/src/stream-sources.adoc
@@ -390,7 +390,36 @@ stats(collection1,
== timeseries
-//TODO
+The `timeseries` function builds a time series aggregation. Under the covers the `timeseries` function uses the
+JSON Facet API as its high performance aggregation engine.
+
+=== timeseries Parameters
+
+* `collection`: (Mandatory) Collection the stats will be aggregated from.
+* `q`: (Mandatory) The query to build the aggregations from.
+* `field`: (Mandatory) The date field for the time series.
+* `start`: (Mandatory) The start of the time series expressed in Solr date or date math syntax.
+* `end`: (Mandatory) The end of the time series expressed in Solr date or date math syntax.
+* `gap`: (Mandatory) The time gap between time series aggregation points expressed in Solr date math syntax.
+* `metrics`: (Mandatory) The metrics to include in the result tuple. Current supported metrics are `sum(col)`, `avg(col)`, `min(col)`, `max(col)` and `count(*)`
+
+=== timeseries Syntax
+
+[source,text]
+----
+timeseries(collection1,
+ q=*:*,
+ field="rec_dt"
+ start="NOW-30DAYS",
+ end="NOW",
+ gap="+1DAY",
+ sum(a_i),
+ max(a_i),
+ max(a_f),
+ avg(a_i),
+ avg(a_f),
+ count(*))
+----
== train
[7/8] lucene-solr:branch_6_6: SOLR-10559: Add tuple documentation
Posted by jb...@apache.org.
SOLR-10559: Add tuple documentation
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/90b38791
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/90b38791
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/90b38791
Branch: refs/heads/branch_6_6
Commit: 90b38791873f943557f747af3f13327fd4ae3245
Parents: c2f2a43
Author: Joel Bernstein <jb...@apache.org>
Authored: Thu Jun 1 17:14:02 2017 -0400
Committer: Joel Bernstein <jb...@apache.org>
Committed: Tue Jun 6 21:43:52 2017 -0400
----------------------------------------------------------------------
solr/solr-ref-guide/src/stream-sources.adoc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/90b38791/solr/solr-ref-guide/src/stream-sources.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/stream-sources.adoc b/solr/solr-ref-guide/src/stream-sources.adoc
index e8887b4..97a1ca4 100644
--- a/solr/solr-ref-guide/src/stream-sources.adoc
+++ b/solr/solr-ref-guide/src/stream-sources.adoc
@@ -497,7 +497,7 @@ the tuple stream from the Stream Expression into a list of Tuples.
[source,text]
----
tuple(a=add(1,1),
- b=search(collection1, q="cat:a", fl="a, b, c", sort"a desc"))
+ b=search(collection1, q="cat:a", fl="a, b, c", sort="a desc"))
----
[5/8] lucene-solr:branch_6_6: SOLR-10559: Add tuple documentation
Posted by jb...@apache.org.
SOLR-10559: Add tuple documentation
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/c3b192ae
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/c3b192ae
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/c3b192ae
Branch: refs/heads/branch_6_6
Commit: c3b192aed2c1b662db064dd37ffc247c01881cc6
Parents: 204447f
Author: Joel Bernstein <jb...@apache.org>
Authored: Thu Jun 1 17:08:52 2017 -0400
Committer: Joel Bernstein <jb...@apache.org>
Committed: Tue Jun 6 21:43:00 2017 -0400
----------------------------------------------------------------------
solr/solr-ref-guide/src/stream-sources.adoc | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/c3b192ae/solr/solr-ref-guide/src/stream-sources.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/stream-sources.adoc b/solr/solr-ref-guide/src/stream-sources.adoc
index 986a5da..2bb0805 100644
--- a/solr/solr-ref-guide/src/stream-sources.adoc
+++ b/solr/solr-ref-guide/src/stream-sources.adoc
@@ -480,3 +480,24 @@ topic(checkpointCollection,
q="topic query",
fl="id, name, country")
----
+
+== tuple
+
+The `tuple` function emits a single Tuple with name/value pairs. The values can be set to variables assigned in a `let` expression, literals, Stream Evaluators or
+Stream Expressions. In the case of Stream Evaluators the tuple will output the return value from the evaluator.
+This could be a numeric, list or map. If a Stream Expression is set to a value, the `tuple` function will flatten
+the tuple stream from the Stream Expression into a list of Tuples.
+
+=== tuple Parameters
+
+* name=value pairs
+
+=== tuple Syntax
+
+[source,text]
+----
+tuple(a=add(1,1),
+ b=search(collection1, q="cat:a", fl="a, b, c", sort"a desc"))
+----
+
+