You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datasketches.apache.org by le...@apache.org on 2020/02/24 06:44:22 UTC
[incubator-datasketches-website] branch Update created (now fa0c283)
This is an automated email from the ASF dual-hosted git repository.
leerho pushed a change to branch Update
in repository https://gitbox.apache.org/repos/asf/incubator-datasketches-website.git.
at fa0c283 Restructuring of ToC
This branch includes the following new commits:
new de23090 Updated top Nav + lots of little edits.
new e6271b6 update community page
new 851557a Fix link
new 61510e3 Add Feature Matrix
new 07979a3 Update ToC
new fa0c283 Restructuring of ToC
The 6 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[incubator-datasketches-website] 03/06: Fix link
Posted by le...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
leerho pushed a commit to branch Update
in repository https://gitbox.apache.org/repos/asf/incubator-datasketches-website.git
commit 851557a357b14de0b57cb92a0870af68070a596f
Author: Lee Rhodes <le...@users.noreply.github.com>
AuthorDate: Sun Feb 23 14:48:21 2020 -0800
Fix link
---
docs/Quantiles/QuantilesStreamAStudy.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/Quantiles/QuantilesStreamAStudy.md b/docs/Quantiles/QuantilesStreamAStudy.md
index c4ab5aa..e1a43f4 100644
--- a/docs/Quantiles/QuantilesStreamAStudy.md
+++ b/docs/Quantiles/QuantilesStreamAStudy.md
@@ -33,7 +33,7 @@ Please get familiar with the [Definitions]({{site.docs_dir}}/Quantiles/Definitio
## The Paper
The implementation of this Quantiles Sketch was originally inspired by
-[Mergable Summaries]https://dl.acm.org/citation.cfm?id=2213562), PODS, May, 2012 paper by Agarwal, Cormode, Huang, Phillips, Wei, and Yi.
+[Mergable Summaries](https://dl.acm.org/citation.cfm?id=2213562), PODS, May, 2012 paper by Agarwal, Cormode, Huang, Phillips, Wei, and Yi.
## The Input Data
The data file used for this evaluation, *streamA.txt*, is real data extracted from one of our back-end servers. It represents one hour of web-site time-spent data measured in milliseconds. The data in this file has a smooth and well-behaved value distribution with a wide dynamic range. It is a text file and consists of consecutive strings of numeric values separated by a line-feeds. Its size is about 2GB.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[incubator-datasketches-website] 05/06: Update ToC
Posted by le...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
leerho pushed a commit to branch Update
in repository https://gitbox.apache.org/repos/asf/incubator-datasketches-website.git
commit 07979a343884452b16eba75e3ae9fd3100780dba
Author: Lee Rhodes <le...@users.noreply.github.com>
AuthorDate: Sun Feb 23 21:55:11 2020 -0800
Update ToC
---
docs/Architecture/KeyFeatures.md | 27 ++---
docs/CommandLine/CommandLine-old.md | 198 ------------------------------------
docs/CommandLine/CommandLine.md | 23 -----
src/main/resources/docgen/toc.json | 19 +---
4 files changed, 19 insertions(+), 248 deletions(-)
diff --git a/docs/Architecture/KeyFeatures.md b/docs/Architecture/KeyFeatures.md
index 3fe4d3f..3f52935 100644
--- a/docs/Architecture/KeyFeatures.md
+++ b/docs/Architecture/KeyFeatures.md
@@ -19,11 +19,13 @@ layout: doc_page
specific language governing permissions and limitations
under the License.
-->
-<h2>Key Features</h2>
+## Key Features
-<h3>Common Sketch Properties</h3>
+### [Sketch Features Matrix]({{site.docs_dir}}/Architecture/FeatureMatrix)
- * Please refer to the [Sketch Criteria]({{site.docs_dir}}/Architecture/SketchCriteria.html) for all sketches in the library.
+### Common Sketch Properties
+
+ * Please refer to the [Sketch Criteria]({{site.docs_dir}}/Architecture/SketchCriteria.html) for the criteria for sketches to be included in the library.
* Query results are <b>approximate</b> but within well defined error bounds that are user
configurable by trading off sketch size with accuracy.
* Designed for <a href="{{site.docs_dir}}/LargeScale.html">Large-scale</a> computing environments
@@ -39,7 +41,7 @@ and are heavily used within Yahoo / Verizon-Media.
* Comprehensive <b>unit tests</b> and testing tools are provided.
* Extensive documentation with the systems developer in mind.
-<h3>Built-In, General Purpose Functions</h3>
+### Built-In, General Purpose Functions
* General purpose <a href="{{site.docs_dir}}/Memory/MemoryPackage.html">Memory Package</a> for managing data off the Java Heap.
This enables systems designers the ability to manage their own large data heaps with
@@ -48,7 +50,7 @@ its garbage collection.
* General purpose implementaion of Austin Appleby's 128-bit MurmurHash3 algorithm,
with a number of useful extensions.
-<h3>Robust, High Quality Implementations.</h3>
+### Robust, High Quality Implementations.
* Extensive test code leveraging <a href="https://testng.org">TestNG</a>.
* Speed and accuracy performance characterization testing code
@@ -63,17 +65,16 @@ its garbage collection.
<a href="https://www.oracle.com/technetwork/java/index.html">Java JDK8</a> standards.
* Suitable for production environments.
-<h3>Opportunities to Extend</h3>
+### Opportunities to Extend
* There is ample opportunity for interested parties to contribute additional algorithms in this exciting area.
+## Key Algorithms
-<h2>Key Algorithms</h2>
-
-<h3>Count Distinct / Count Unique</h3>
+### Count Distinct / Count Unique
-<h4>Solves Computational Challenges Associated with Unique Identifiers</h4>
+#### Solves Computational Challenges Associated with Unique Identifiers
* <b>Estimating cardinality</b> of a stream with many duplicates
* Performing <a href="{{site.docs_dir}}/Theta/ThetaSketchSetOps.html">set operations</a> (e.g., Union, Intersection,
@@ -84,17 +85,17 @@ its garbage collection.
for operation on the java heap or off-heap.
* <a href="{{site.docs_dir}}/HLL/HLL.html">The Hyper-Log Log algorithms</a> when sketch size is of utmost concern.
-<h3>Quantiles</h3>
+### Quantiles
* Get normal or inverse PDFs or CDFs of the distributions of any numeric value from your raw data in a
single pass.
* Well defined error bounds on the result.
-<h3>Frequent Items</h3>
+### Frequent Items
* Get the most frequent items from a stream of items.
-<h3>Tuple Sketch</h3>
+### Tuple Sketch
* Associative sketches that are useful for performing approximate join operations and
extracting other kinds of behavior associated with unique identifiers.
diff --git a/docs/CommandLine/CommandLine-old.md b/docs/CommandLine/CommandLine-old.md
deleted file mode 100644
index e693f2e..0000000
--- a/docs/CommandLine/CommandLine-old.md
+++ /dev/null
@@ -1,198 +0,0 @@
----
-layout: doc_page
----
-<!--
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
--->
-<b>Note that these instructions work on unix-based systems including macs. Windows systems will
-need something similar.</b>
-
-<h2>Creating the command-line <i>sketch</i></h2>
-
-<h3>Clone and install <i>sketches-core</i> and <i>sketches-misc</i></h3>
- * Clone sketches-core and sketches-misc repositories into separate directories on your system.
- * Run <i>mvn install -DskipTests -Dgpg.skip</i> on both directories. This will download JAR
- files into your local .m2/repository. Take note of the version numbers for the
- sketches-core-X.Y.Z.jar and the sketches-misc-X.Y.Z.jar
-
-<h3>Create <i>sketch</i> executable</h3>
-
-Place the following in an empty text file called "sketch" and update the version numbers and the
-path to your local <i>.m2/repository</i> directory:
-
- #!/bin/bash
- # Update version numbers and the path to your local .m2/repository as necessary
-
- COREVER="0.5.2"
- MISCVER="0.1.0"
- M2PATH="/path/to/.m2/repository"
-
- COREPATH="$M2PATH/com/yahoo/datasketches/sketches-core/$COREVER/sketches-core-$COREVER.jar"
- MISCPATH="$M2PATH/com/yahoo/datasketches/sketches-misc/$MISCVER/sketches-misc-$MISCVER.jar"
- CLSPATH="$COREPATH:$MISCPATH"
-
- java -cp $CLSPATH org.apache.datasketches.cmd.CommandLine $@
-
-Move this <i>sketch</i> file to a local system directory accessible from anywhere in your system,
-and make it executable.
-
- cp sketch /usr/local/bin/sketch
- chmod +x /usr/local/bin/sketch
-
-Test your executable. You should see something like the following:
-
- sketch
-
- NAME
- sketch - sketch Uniques, Quantiles, Histograms, or Frequent Items.
- SYNOPSIS
- sketch (this help)
- sketch TYPE help
- sketch TYPE [SIZE] [FILE]
- DESCRIPTION
- Write a sketch(TYPE, SIZE) of FILE to standard output.
- TYPE is required.
- If SIZE is omitted, internal defaults are used.
- If FILE is omitted, Standard In is assumed.
- TYPE DESCRIPTION
- sketch uniq : Sketch the unique string items of a stream.
- sketch rank : Sketch the rank-value distribution of a numeric value stream.
- sketch hist : Sketch the linear-axis value-frequency distribution of numeric value stream.
- sketch loghist : Sketch the log-axis value-frequency distribution of numeric value stream.
- sketch freq : Sketch the Heavy Hitters of a string item stream.
-
- UNIQ SYNOPSIS
- sketch uniq help
- sketch uniq [SIZE] [FILE]
-
- RANK SYNOPSIS
- sketch rank help
- sketch rank [SIZE] [FILE]
-
- HIST SYNOPSIS
- sketch hist help
- sketch hist [SIZE] [FILE]
-
- LOGHIST SYNOPSIS
- sketch loghist help
- sketch loghist [SIZE] [FILE]
-
- FREQ SYNOPSIS
- sketch freq help
- sketch freq [SIZE] [FILE]
-
-You can create a test data file, with duplicate values, like this:
-
- $ python -c "exec(\"import random\\nfor _ in range(10000000): print random.randint(1,10000000)\")" > manyNumbers.txt
-
-Now you can do either something like this:
-
- $ cat manyNumbers.txt | sketch uniq
- or
- $ cat manyNumbers.txt | sketch uniq 16000
-
-or like this:
-
- $ sketch uniq manyNumbers.txt
- or
- $ sketch uniq 16000 manyNumbers.txt
-
-Providing the size allows you to tune the accuracy.
-
-Be sure to compare the speed of the above to the conventional method:
-
- $ cat manyNumbers.txt | sort | uniq | wc -l
-
-<h2>Creating the command-line <i>demo</i></h2>
-
-If you haven't already, clone and install <i>sketches-core</i> and <i>sketches-misc</i> as in the
-previous example.
-
-Create the <i>demo</i> executable with the same content as the <i>sketch</i> executable except
-for the last line:
-
- java -cp $CLSPATH org.apache.datasketches.demo.ExactVsSketchDemo $@
-
-Move this <i>demo</i> file to a local system directory accessible from anywhere in your system,
-and make it executable.
-
- cp demo /usr/local/bin/demo
- chmod +x /usr/local/bin/demo
-
-When run, the output should look something like this:
-
- demo
-
- # COMPUTE DISTINCT COUNT EXACTLY:
- ## BUILD FILE:
- Time Min:Sec.mSec = 0:17.569
- Total Values: 100,000,000
- Build Rate: 175 nSec/Value
- Exact Uniques: 50,002,776
- File Size Bytes: 1,693,331,301
-
- ## SORT & REMOVE DUPLICATES
- Unix cmd: sort -u -o tmp/sorted.txt tmp/test.txt
- Time Min:Sec.mSec = 1:49.571
-
- ## LINE COUNT
- Unix cmd: wc -l tmp/sorted.txt
- Time Min:Sec.mSec = 0:00.900
- Output from wc command:
- 50002776 tmp/sorted.txt
-
- Total Exact Time Min:Sec.mSec = 2:08.040
-
-
- # COMPUTE DISTINCT COUNT USING SKETCHES
- ## USING THETA SKETCH
- Time Min:Sec.mSec = 0:00.614
- Total Values: 100,000,000
- Build Rate: 6 nSec/Value
- Exact Uniques: 50,002,776
- ## SKETCH STATS
- Sketch Estimate of Uniques: 50,098,990
- Sketch Actual Relative Error: 0.192%
- Sketch 95%ile Error Bounds : +/- 1.563%
- Max Sketch Size Bytes: 262,144
- Speedup Factor 208.5
-
- ## USING HLL SKETCH
- Time Min:Sec.mSec = 0:02.212
- Total Values: 100,000,000
- Build Rate: 22 nSec/Value
- Exact Uniques: 50,002,776
- ## SKETCH STATS
- Sketch Estimate of Uniques: 49,784,556
- Sketch Actual Relative Error: -0.436%
- Sketch 95%ile Error Bounds : +/- 1.306%
- Max Sketch Size Bytes: 8,192
- Speedup Factor 57.9
-
-The first part builds a file, separately timed, of 100M numbers with roughly 50% duplicates.
-The second part sorts and removes duplicates using the unix <i>sort -u</i> command and may take
-several minutes to run, so be patient. The third part does a line count using the unix <i>wc -l</i>
-command.
-
-After that, two different sketch trials are run, one with a <i>Theta Sketch</i> and the
-other with a compact implementation of Flajolet's <i>HLL Sketch</i>. Sketches do not require a
-pre-built file. They run in true streaming mode with the random values generated on the fly.
-
-Check out the statistics!
-
-Enjoy!
diff --git a/docs/CommandLine/CommandLine.md b/docs/CommandLine/CommandLine.md
deleted file mode 100644
index 46f2a64..0000000
--- a/docs/CommandLine/CommandLine.md
+++ /dev/null
@@ -1,23 +0,0 @@
----
-layout: doc_page
----
-<!--
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
--->
-<b>Note: The command-line utility is obsolete and has been removed. A completely new CL utility is in development and should be released soon.</b>
-
diff --git a/src/main/resources/docgen/toc.json b/src/main/resources/docgen/toc.json
index 1c99ccf..6b0659e 100644
--- a/src/main/resources/docgen/toc.json
+++ b/src/main/resources/docgen/toc.json
@@ -8,10 +8,13 @@
{"class":"Doc", "desc" : "Sketch Elements", "dir" : "", "file": "SketchElements" },
{"class":"Doc", "desc" : "Large Scale Computing", "dir" : "", "file": "LargeScale" },
+ {"class":"Doc", "desc" : "Overview Slide Deck", "dir" : "", "file": "DataSketches_deck", "pdf":"true" },
+ {"class":"Doc", "desc" : "Who Uses", "dir" : "", "file": "WhoUses" },
- { "class":"Dropdown", "desc" : "Architecture", "array":
+ { "class":"Dropdown", "desc" : "Architecture & Design", "array":
[
- {"class":"Doc", "desc" : "Key Features", "dir" : "Architecture", "file": "KeyFeatures" },
+ {"class":"Doc", "desc" : "Key Features", "dir" : "Architecture", "file": "KeyFeatures" },
+ {"class":"Doc", "desc" : "Sketch Feature Matrix", "dir" : "Architecture", "file": "FeatureMatrix" },
{"class":"Doc", "desc" : "Components", "dir" : "Architecture", "file": "Components" },
{"class":"Doc", "desc" : "Sketches by Component", "dir" : "Architecture", "file": "SketchesByComponent" },
{"class":"Doc", "desc" : "Sketch Criteria", "dir" : "Architecture", "file": "SketchCriteria" },
@@ -19,7 +22,6 @@
{"class":"Doc", "desc" : "Notes on Concurrency", "dir" : "Architecture", "file": "Concurrency" },
]
},
- {"class":"Doc", "desc" : "Overview Slide Deck", "dir" : "", "file": "DataSketches_deck", "pdf":"true" },
]
},
{ "class":"Dropdown", "desc" : "Community", "array":
@@ -230,17 +232,6 @@
]
},
- { "class":"Dropdown", "desc" : "Command Line", "array":
- [
- {"class":"Doc", "desc" : "Creating Command Line Executables", "dir" : "CommandLine", "file": "CommandLine" },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Who Uses", "array":
- [
- {"class":"Doc", "desc" : "Who Uses", "dir" : "", "file": "WhoUses" },
- ]
- },
]
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[incubator-datasketches-website] 02/06: update community page
Posted by le...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
leerho pushed a commit to branch Update
in repository https://gitbox.apache.org/repos/asf/incubator-datasketches-website.git
commit e6271b67a10d3dc1efa81c58d535d59a54f4d76b
Author: Lee Rhodes <le...@users.noreply.github.com>
AuthorDate: Sun Feb 23 14:37:31 2020 -0800
update community page
---
docs/Community/index.md | 3 ---
1 file changed, 3 deletions(-)
diff --git a/docs/Community/index.md b/docs/Community/index.md
index 063f5db..c02542d 100644
--- a/docs/Community/index.md
+++ b/docs/Community/index.md
@@ -20,9 +20,6 @@ There are many ways that are available for our community to communicate with eac
<br/><br/>
* **[GitHub Components]({{site.docs_dir}}/Architecture/Components.html):** Our library is made up of components that are partitioned into GitHub repositories. If you have a specific issue or bug report that impacts only one of these components please open an issue on the respective component. If you are a developer and wish to submit a PR, please choose the appropriate repository.
-## Downloads
-* [Downloads]({{site.docs_dir}}/Community/Downloads.html)
-
## Contributing
We are always open to contributions from our community. Contributions can be of many forms: documentation, testing, science as well as bug fixes, code enhancements, code reviews, feature suggestions, usability feedback, etc. Contributions usually take the form of a Pull Request (PR), but if you wish to contribute and not sure how, please contact us on our [dev]({{page.dev}}) list.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[incubator-datasketches-website] 04/06: Add Feature Matrix
Posted by le...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
leerho pushed a commit to branch Update
in repository https://gitbox.apache.org/repos/asf/incubator-datasketches-website.git
commit 61510e333d7fc9230bac22f2c6f8f38fb05e45cb
Author: Lee Rhodes <le...@users.noreply.github.com>
AuthorDate: Sun Feb 23 21:54:51 2020 -0800
Add Feature Matrix
---
docs/Architecture/FeatureMatrix.md | 76 ++++++++++++++++++++++++++++++++++++++
docs/Community/Research.md | 66 ++++++++++++++-------------------
docs/MajorSketchFamilies.md | 3 +-
3 files changed, 105 insertions(+), 40 deletions(-)
diff --git a/docs/Architecture/FeatureMatrix.md b/docs/Architecture/FeatureMatrix.md
new file mode 100644
index 0000000..81f5681
--- /dev/null
+++ b/docs/Architecture/FeatureMatrix.md
@@ -0,0 +1,76 @@
+---
+layout: doc_page
+---
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+# Sketch Feature Matrix
+
+Use the following table to compare the capabilities of the different sketch families.
+
+<div>
+<table>
+<tr style="font-weight:bold"><td colspan="2"></td><td colspan="3">Languages</td><td colspan="4">Set Operations</td><td colspan="4">System Integrations</td><td colspan="5">Misc.</td></tr>
+
+<tr style="font-weight:bold"><td>Type</td><td>Sketch</td><td>Java</td><td>C++</td><td>Python</td><td>Union</td><td>Inter-section</td><td>Difference</td><td>Jaccard</td><td>Hive</td><td>Pig</td><td>Druid<sup>1</sup></td><td>Spark<sup>2</sup></td><td>Con-current</td><td>Compact</td><td>Java Generics</td><td>Off Java Heap</td><td>Error Bounds</td></tr>
+
+<tr style="font-weight:bold"><td colspan="18">Major Sketches</td></tr>
+<tr><td>Cardinality/FM85</td><td>CpcSketch</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td>Y</td><td>Y</td><td></td><td></td><td></td><td>Y</td><td></td><td></td><td>Y</td></tr>
+<tr><td>Cardinality/FM85</td><td>HllSketch</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td>Y</td></tr>
+<tr><td>Cardinality/Theta</td><td>Sketch</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td></td><td>Y</td><td>Y</td></tr>
+<tr><td>Cardinality/Tuple</td><td>Sketch</td><td>Y</td><td></td><td></td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td><td>Y</td><td>Y</td><td>Y</td></tr>
+<tr><td>Quantiles/Cormode</td><td>DoublesSketch</td><td>Y</td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td>Y</td><td></td><td>Y</td><td>Y</td></tr>
+<tr><td>Quantiles/Cormode</td><td>ItemsSketch</td><td>Y</td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td>Y</td></tr>
+<tr><td>Quantiles/KLL</td><td>FloatsSketch</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td></tr>
+<tr><td>Frequencies</td><td>LongsSketch</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td></tr>
+<tr><td>Frequencies</td><td>ItemsSketch</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td>Y</td></tr>
+<tr><td>Sampling</td><td>ReservoirLongsSketch</td><td>Y</td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td></tr>
+<tr><td>Sampling</td><td>ReserviorItemsSketch</td><td>Y</td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td>Y</td></tr>
+<tr><td>Sampling</td><td>VarOptItemsSketch</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td>Y</td></tr>
+
+<tr style="font-weight:bold"><td colspan="18">Specialty Sketches</td></tr>
+
+<tr><td>Cardinality/FM85</td><td>UniqueCountMap</td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td></tr>
+<tr><td>Cardinality/Tuple</td><td>FdtSketch</td><td>Y</td><td></td><td></td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td></tr>
+<tr><td>Cardinality/Tuple</td><td>ArrayOfDoublesSketch</td><td>Y</td><td></td><td></td><td>Y</td><td>Y</td><td>Y</td><td></td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td>Y</td><td></td><td>Y</td><td>Y</td></tr>
+<tr><td>Cardinality/Tuple</td><td>DoubleSketch</td><td>Y</td><td></td><td></td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td></tr>
+<tr><td>Cardinality/Tuple</td><td>IntegerSketch</td><td>Y</td><td></td><td></td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td></tr>
+<tr><td>Cardinality/Tuple</td><td>ArrayOfStringsSketch</td><td>Y</td><td></td><td></td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td></tr>
+<tr><td>Cardinality/Tuple</td><td>EngagementTest<sup>3</sup></td><td>Y</td><td></td><td></td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td></tr>
+</table>
+</div>
+
+<sup>1</sup> Integrated into Druid<br>
+<sup>2</sup> Example Code on website<br>
+<sup>3</sup> Example Code in test/.../tuple/aninteger
+
+
+## Definitions
+
+### Type (a.k.a. Sketch Family)
+
+See [Research]({{site.docs_dir}}/Community/Research.html) for references in [...]
+
+* **Cardinality/FM85** Derivations of [FM85]. They include the popular HyperLogLog (HLL) Sketch as well as the Compressed Probabilistic Counting (CPC) Sketch, which has a completely different theoretical derivation and is superior to the HLL sketch in terms of accuracy per byte of storage.
+* **Cardinality/Theata** Derivations of [BJKST02].
+* **Cardinality/Tuple** An Extension of the Theta family that adds attributes to each hash-key.
+* **Quantiles/Cormode** Based on [AC+13]
+* **Quantiles/KLL** Based on [KLL16].
+* **Frequencies** Based on [ABL+17].
+* **Sampling** Two families, The simple reservoir sketch is based on Knuth, algorithm R. The VarOpt sketch is based on [CDKLT09].
+
diff --git a/docs/Community/Research.md b/docs/Community/Research.md
index c3ec0a0..2ed2782 100644
--- a/docs/Community/Research.md
+++ b/docs/Community/Research.md
@@ -52,7 +52,7 @@ The library has been adapted throughout industry and government. For example, at
Beyond its utility in deployed systems, the process of developing of developing the Data Sketches library has led to interesting research. This has involved important contributions to both the theory of streaming algorithms as well as addressing issues that are crucial in real-world stream engines but are often ignored in the academic literature. These issues include mergeability, and dealing with *weighted* stream updates (i.e., data streams where each piece of data comes with an associ [...]
-In particular, work on the Data Sketches library has led to novel algorithms achieving state of the art practical performance for identifying frequent items in data streams [ABL+17], and mergeable summaries for unique count queries [DLRT16]. On the theoretical side, work on the library has led to the resolution of the space complexity of streaming approximation algorithms for quantile queries, which was a longstanding open question [KLL16], as well as for the problem of identifying frequ [...]
+In particular, work on the Data Sketches library has led to novel algorithms achieving state of the art practical performance for identifying frequent items in data streams [ABL+17], and mergeable summaries for unique count queries [DLRT16] including intersections and differences as an extension of [BJKST02]. On the theoretical side, work on the library has led to the resolution of the space complexity of streaming approximation algorithms for quantile queries, which was a longstanding o [...]
The cutting edge of scientific inquiry is to build more powerful algorithms and, at the same time, to devise new theorems and proofs that certify that these algorithms work well enough to draw robust conclusions. This library is dedicated to that quest. By working in the open source community we are confident that there are major opportunities to incorporate algorithms for new and richer types of queries into the library, as well as to improve the efficiency of the algorithms that are a [...]
@@ -60,7 +60,7 @@ The cutting edge of scientific inquiry is to build more powerful algorithms and,
In a recent pre-print that grew out of work on the Data Sketches library [Lan17], Kevin Lang describes several streaming algorithms for estimating the number of distinct elements in a data stream. The algorithms have a better space/accuracy tradeoff than the previous state of the art algorithm, Hyperloglog (HLL) [FFGM07], which has been considered the gold standard in practical performance for this problem for nearly a decade. Specifically, for a given accuracy level, Lang’s algorithms u [...]
-Regarding runtime, Lang’s pre-print shows that some of his algorithms have comparable speed to straightforward implementations of HLL, but are somewhat slower than heavily optimized HLL implementations. Significant research remains to optimize the speed of the new algorithms, determine which variant algorithm performs best on real data and is most suitable for a production environment, and finally to produce a production-quality implementation of this algorithm.
+Regarding runtime, Lang’s pre-print shows that some of his algorithms have comparable speed to straightforward implementations of HLL, but are somewhat slower for serialization than heavily optimized HLL implementations. Significant research remains to optimize the speed of the new algorithms, determine which variant algorithm performs best on real data and is most suitable for a production environment, and finally to produce a production-quality implementation of this algorithm.
## Algorithms For Anomaly Detection
@@ -98,59 +98,47 @@ This solution suffices in some applications, but for other applications the chun
## References
-[ABL+17]
-Daniel Anderson, Pryce Bevan, Kevin J. Lang, Edo Liberty, Lee Rhodes, and Justin Thaler. A high-performance algorithm for identifying frequent items in data streams. In *ACM IMC 2017 (To Appear)*, 2017. [Preliminary paper](https://arxiv.org/abs/1705.07001).
+**[ABL+17]** Daniel Anderson, Pryce Bevan, Kevin J. Lang, Edo Liberty, Lee Rhodes, and Justin Thaler. A high-performance algorithm for identifying frequent items in data streams. In *ACM IMC 2017 (To Appear)*, 2017. [Preliminary paper](https://arxiv.org/abs/1705.07001).
-[AC+13]
-Pankaj K. Agarwal, Graham Cormode, Zengfeng Huang, Jeff M. Phillips, Zhewei Wei, Ke Yi. Mergeable summaries. *ACM Trans. Database Syst.* 38(4): 26:1-26:28, 2013
+**[AC+13]** Pankaj K. Agarwal, Graham Cormode, Zengfeng Huang, Jeff M. Phillips, Zhewei Wei, Ke Yi. Mergeable summaries. In *ACM Trans. Database Syst.* 38(4): 26:1-26:28, 2013
-[AM04]
-Arvind Arasu and Gurmeet Singh Manku. Approximate counts and quantiles over sliding windows. In *ACM PODS Proceedings '04*, pages 286–296, 2004.
+**[AM04]** Arvind Arasu and Gurmeet Singh Manku. Approximate counts and quantiles over sliding windows. In *ACM PODS Proceedings '04*, pages 286–296, 2004.
-[CCM10]
-Amit Chakrabarti, Graham Cormode, and Andrew McGregor. A near-optimal algorithm for estimating the entropy of a stream. *ACM Trans. Algorithms*, 6(3):51:1–51:21, 2010.
+**[BJKST02]** Bar-Yossef Z., Jayram T.S., Kumar R., Sivakumar D., Trevisan L. Counting Distinct Elements in a Data Stream. In *Rolim J.D.P., Vadhan S. (eds) Randomization and Approximation Techniques in Computer Science*. RANDOM 2002.
-[CEM+15]
-Michael B. Cohen, Sam Elder, Cameron Musco, Christopher Musco, and Madalina Persu. Dimensionality reduction for k-means clustering and low rank approximation. In *ACM STOC Proceedings '15*, pages 163–172, 2015.
+**[CCM10]** Amit Chakrabarti, Graham Cormode, and Andrew McGregor. A near-optimal algorithm for estimating the entropy of a stream. In *ACM Trans. Algorithms*, 6(3):51:1–51:21, 2010.
-[DLRT16]
-Anirban Dasgupta, Kevin J. Lang, Lee Rhodes, and Justin Thaler. A framework for estimating stream expression cardinalities. In *EDBT/ICDT Proceedings '16 *, pages 6:1–6:17, 2016. [Talk Slides](https://github.com/apache/incubator-datasketches-website/blob/master/docs/pdf/icdt-talk.pdf).
+**[CDKLT09]** Edith Cohen, Nick Duffield, Haim Kaplan, Carsten Lund, Mikkel Thorup. Stream sampling for variance-optimal estimation of subset sums. In **Proceedings of the 20th ACM-SIAM Symposium on Discrete Algorithms** SODA 2009.
-[FFGM07]
-Philippe Flajolet, E ́ric Fusy, Olivier Gandouet, and Fre ́de ́ric Meunier. Hyperloglog: the analysis of a near-optimal cardinality estimation algorithm. In *DMTCS Conference on Analysis of Algorithms*, pages 137–156, 2007.
+**[CEM+15]** Michael B. Cohen, Sam Elder, Cameron Musco, Christopher Musco, and Madalina Persu. Dimensionality reduction for k-means clustering and low rank approximation. In *ACM STOC Proceedings '15*, pages 163–172, 2015.
-[GDD+03]
-Lukasz Golab, David DeHaan, Erik D. Demaine, Alejandro Lopez-Ortiz, and J. Ian Munro. Identifying frequent items in sliding windows over on-line packet streams. In *ACM IMC Proceedings '03*, pages 173–178, 2003.
+**[DLRT16]** Anirban Dasgupta, Kevin J. Lang, Lee Rhodes, and Justin Thaler. A framework for estimating stream expression cardinalities. In *EDBT/ICDT Proceedings '16 *, pages 6:1–6:17, 2016. [Paper](https://arxiv.org/pdf/1510.01455.pdf). [Talk Slides](https://github.com/apache/incubator-datasketches-website/blob/master/docs/pdf/icdt-talk.pdf).
-[GLP16] Mina Ghashami, Edo Liberty, Jeff M. Phillips. Efficient Frequent Directions Algorithm for Sparse Matrices. In *ACM SIGKDD Proceedings '16*, pages 845-854, 2016
+**[FFGM07]** Philippe Flajolet, E ́ric Fusy, Olivier Gandouet, and Fre ́de ́ric Meunier. Hyperloglog: the analysis of a near-optimal cardinality estimation algorithm. In *DMTCS Conference on Analysis of Algorithms*, pages 137–156, 2007.
-[GT02]
-Phillip B. Gibbons and Srikanta Tirthapura. Distributed streams algorithms for sliding windows. In *ACM SPAA Proceedings '02*, pages 63–72, New York, NY, USA, 2002.
+**[FM85]** P. Flajolet and G. N. Martin. Probabilistic counting algorithms for data base applications. In *Journal of Computer and System Sciences*, 31:182–209, 1985.
-[KLL16]
-Zohar S. Karnin, Kevin J. Lang, and Edo Liberty. Optimal quantile approximation in streams. In *IEEE FOCS Proceedings '16*, pages 71–78, 2016.
+**[GDD+03]** Lukasz Golab, David DeHaan, Erik D. Demaine, Alejandro Lopez-Ortiz, and J. Ian Munro. Identifying frequent items in sliding windows over on-line packet streams. In *ACM IMC Proceedings '03*, pages 173–178, 2003.
-[Lan17]
-Kevin J Lang. Back to the future: an even more nearly optimal cardinality estimation algorithm. *arXiv preprint* <https://arxiv.org/abs/1708.06839>, 2017.
+**[GLP16]** Mina Ghashami, Edo Liberty, Jeff M. Phillips. Efficient Frequent Directions Algorithm for Sparse Matrices. In *ACM SIGKDD Proceedings '16*, pages 845-854, 2016
-[Lib13]
-Edo Liberty. Simple and deterministic matrix sketching. In *ACM KDD Proceedings '13*, pages 581– 588, 2013.
+**[GT02]** Phillip B. Gibbons and Srikanta Tirthapura. Distributed streams algorithms for sliding windows. In *ACM SPAA Proceedings '02*, pages 63–72, New York, NY, USA, 2002.
-[LMTU16]
-Edo Liberty, Michael Mitzenmacher, Justin Thaler, and Jonathan Ullman. Space lower bounds for itemset frequency sketches. In *ACM PODS Proceedings '16*, pages 441–454, 2016.
+**[KLL16]** Zohar S. Karnin, Kevin J. Lang, and Edo Liberty. Optimal quantile approximation in streams. In *IEEE FOCS Proceedings '16*, pages 71–78, 2016.
-[McG14]
-Andrew McGregor. Graph stream algorithms: a survey. *ACM SIGMOD Record*, 43(1):9–20, 2014.
+**[Lan17]** Kevin J Lang. Back to the future: an even more nearly optimal cardinality estimation algorithm. In *arXiv preprint* <https://arxiv.org/abs/1708.06839>, 2017.
-[MST12]
-Michael Mitzenmacher, Thomas Steinke, and Justin Thaler. Hierarchical heavy hitters with the space saving algorithm. In *SIAM ALENEX Proceedings '12*, pages 160–174, 2012.
+**[Lib13]** Edo Liberty. Simple and deterministic matrix sketching. In *ACM KDD Proceedings '13*, pages 581– 588, 2013.
-[RLS+ 15]
-Lee Rhodes, Kevin Lang, Alexander Saydakov, Justin Thaler, Edo Liberty, and Jon Malkin. DataSketches: A Java software library for streaming data algorithms. Apache License, Version 2.0, 2015. <https://datasketches.apahce.org>.
+**[LMTU16]** Edo Liberty, Michael Mitzenmacher, Justin Thaler, and Jonathan Ullman. Space lower bounds for itemset frequency sketches. In *ACM PODS Proceedings '16*, pages 441–454, 2016.
-[Tha07]
-Justin Thaler. REU project website: A near-optimal algorithm for computing the entropy of a stream, 2007. <https://people.cs.georgetown.edu/jthaler>.
+**[McG14]** Andrew McGregor. Graph stream algorithms: a survey. In *ACM SIGMOD Record*, 43(1):9–20, 2014.
-[VSGB05]
-Shobha Venkataraman, Dawn Xiaodong Song, Phillip B. Gibbons, and Avrim Blum. New streaming algorithms for fast detection of superspreaders. In *Internet Society NDSS Proceedings*, 2005.
+**[MST12]** Michael Mitzenmacher, Thomas Steinke, and Justin Thaler. Hierarchical heavy hitters with the space saving algorithm. In *SIAM ALENEX Proceedings '12*, pages 160–174, 2012.
+
+**[RLS+15]** Lee Rhodes, Kevin Lang, Alexander Saydakov, Justin Thaler, Edo Liberty, and Jon Malkin. **Apache DataSketches**: A Java software library for streaming data algorithms. Apache License, Version 2.0, 2015. <https://datasketches.apache.org>.
+
+**[Tha07]** Justin Thaler. REU project website: A near-optimal algorithm for computing the entropy of a stream, 2007. <https://people.cs.georgetown.edu/jthaler>.
+
+**[VSGB05]** Shobha Venkataraman, Dawn Xiaodong Song, Phillip B. Gibbons, and Avrim Blum. New streaming algorithms for fast detection of superspreaders. In *Internet Society NDSS Proceedings*, 2005.
diff --git a/docs/MajorSketchFamilies.md b/docs/MajorSketchFamilies.md
index 32a3da2..53c005a 100644
--- a/docs/MajorSketchFamilies.md
+++ b/docs/MajorSketchFamilies.md
@@ -59,7 +59,8 @@ Use the following table to compare the capabilities of the different sketch fami
<sup>2</sup> Example Code on website<br>
<sup>3</sup> Example Code in test/.../tuple/aninteger
-----
+
+---
# High-Level Descriptions
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[incubator-datasketches-website] 01/06: Updated top Nav + lots of
little edits.
Posted by le...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
leerho pushed a commit to branch Update
in repository https://gitbox.apache.org/repos/asf/incubator-datasketches-website.git
commit de2309012e609b1f885b5af2d475ea19887ffd95
Author: Lee Rhodes <le...@users.noreply.github.com>
AuthorDate: Sun Feb 23 14:37:11 2020 -0800
Updated top Nav + lots of little edits.
Added bigger logos to TopNav, lots of text wording cleanup. Github nav
now points to explanation page.
---
_includes/page_footer.html | 4 +-
_includes/page_header.html | 19 ++-
_includes/site_head.html | 5 +-
css/header.css | 24 ++--
docs/Architecture/Components.md | 6 +-
docs/Architecture/KeyFeatures.md | 4 +-
docs/Architecture/OrderSensitivity.md | 31 ++---
docs/Architecture/SketchesByComponent.md | 14 +--
docs/Community/Downloads.md | 2 +-
docs/HLL/Hll_vs_CS_Hllpp.md | 2 +-
docs/MajorSketchFamilies.md | 40 +++---
docs/Quantiles/DruidApproxHistogramStudy.md | 2 +-
docs/Quantiles/MomentsSketchStudy.md | 2 +-
docs/SketchElements.md | 3 -
docs/SketchOrigins.md | 4 -
docs/TheChallenge.md | 2 +-
docs/Tuple/TupleEngagementExample.md | 2 +-
img/datasketches-HorizontalColor-1.svg | 52 ++++++++
img/datasketches-HorizontalWhite.svg | 31 +++++
img/datasketches-ManColor-3.svg | 27 ++++
img/datasketches-ManWhite.svg | 12 ++
...calWhite.svg => datasketches-VerticalWhite.svg} | 0
img/feather.svg | 138 +++++++++++++++++++++
index.md | 17 +--
24 files changed, 352 insertions(+), 91 deletions(-)
diff --git a/_includes/page_footer.html b/_includes/page_footer.html
index cbedf23..10290eb 100644
--- a/_includes/page_footer.html
+++ b/_includes/page_footer.html
@@ -1,4 +1,4 @@
-<!-- Start page_footer.html include -->
+<!-- Start _include/page_footer.html -->
<footer class="ds-footer">
<div class="container">
<div class="text-center">
@@ -15,4 +15,4 @@
</div>
</div>
</footer>
-<!-- End page_footer.html include -->
+<!-- End _include/page_footer.html -->
diff --git a/_includes/page_header.html b/_includes/page_header.html
index 6795f5d..ad92d80 100644
--- a/_includes/page_header.html
+++ b/_includes/page_header.html
@@ -1,4 +1,4 @@
-<!-- Start page_header.html include -->
+<!-- Start _include/page_header.html -->
<div class="navbar navbar-inverse navbar-static-top ds-nav">
<div class="container">
<div class="navbar-header">
@@ -8,9 +8,8 @@
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
- <!-- navbar-brand: https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css -->
- <!-- /css/main.css:57:.logo -->
- <a class="navbar-brand logo" href="/">DataSketches</a>
+ <a href="/" style="padding-top: 0px; padding-bottom: 0px;">
+ <span class="ds-small-h-logo"></span></a>
</div>
<div class="navbar-collapse collapse">
<ul class="nav navbar-nav navbar-right">
@@ -23,7 +22,7 @@
<span class="fa fa-download"></span> DOWNLOAD</a>
</li>
<li>
- <a href="https://github.com/apache?utf8=%E2%9C%93&q=datasketches&type=&language=">
+ <a href="/docs/Architecture/Components.html">
<span class="fa fa-github"></span> GITHUB</a>
</li>
<li>
@@ -31,12 +30,12 @@
<span class="fa fa-paper-plane"></span> RESEARCH</a>
</li>
<li>
- <a href="https://lists.apache.org/list.html?users@datasketches.apache.org">
- <span class="fa fa-comment"></span> FORUM</a>
+ <a href="/docs/Community" style="padding-top: 0; padding-bottom: 0;">
+ <img class="ds-small-man" src="/img/datasketches-ManWhite.svg"/>COMMUNITY</a>
</li>
<ul class="nav navbar-nav navbar-right ds-nav">
- <li class="dropdown ds-nav">
- <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false"><img class="apache-logo" src="https://www.apache.org/foundation/press/kit/feather.svg"/>Apache <span class="caret"></span></a>
+ <li class="dropdown ds-nav" >
+ <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false" style="padding-top: 0; padding-bottom: 0;"><img class="apache-logo" src="/img/feather.svg"/>Apache <span class="caret"></span></a>
<ul class="dropdown-menu ds-nav">
<li><a href="https://www.apache.org/" target="_blank">Foundation</a></li>
<li><a href="https://www.apache.org/events/current-event" target="_blank">Events</a></li>
@@ -51,4 +50,4 @@
</div>
</div>
</div>
-<!-- End page_header.html include -->
+<!-- End _include/page_header.html -->
diff --git a/_includes/site_head.html b/_includes/site_head.html
index 46d25fb..2e41c9b 100644
--- a/_includes/site_head.html
+++ b/_includes/site_head.html
@@ -1,4 +1,4 @@
-<!-- Start site_head.html include-->
+<!-- Start _include/site_head.html -->
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="">
@@ -21,9 +21,8 @@
<link rel="stylesheet" href="/css/docs.css">
-
<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML-full">
</script>
<script src="https://code.jquery.com/jquery.min.js"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/js/bootstrap.min.js"></script>
-<!-- End site_head.html include-->
+<!-- End _include/site_head.html -->
diff --git a/css/header.css b/css/header.css
index b9c59dd..d6bb837 100644
--- a/css/header.css
+++ b/css/header.css
@@ -59,18 +59,23 @@
background-size: 300px 300px;
background-repeat: no-repeat;
background-position: center;
- background-image: url('/img/datasketches_VerticalWhite.svg');
+ background-image: url('/img/datasketches-VerticalWhite.svg');
}
-.y-bootlogo {
- display: inline-block;
+.ds-small-man {
+ width: 35.99px;
+ height: 50px;
+}
+
+.ds-small-h-logo {
+ display: block;
margin: 0 auto;
- width: 100px;
- height: 35px;
- background-size: 100px 35px;
+ width: 117.51px;
+ height: 50px;
+ background-size: 117.51px 50px;
background-repeat: no-repeat;
- background-position: 0px 6px;
- background-image: url('/img/Yahoo_white_small.png');
+ background-position: center;
+ background-image: url('/img/datasketches-HorizontalWhite.svg');
}
.ds-booticon {
@@ -176,9 +181,10 @@ a:hover, a:focus {
}
.apache-logo {
- height: 20px;
+ height: 50px;
margin-right: 5px;
}
+
.dropdown:hover .dropdown-menu {
display: block;
margin-top: 0; // remove the gap so it doesn't close
diff --git a/docs/Architecture/Components.md b/docs/Architecture/Components.md
index bb6cdbf..8cc585c 100644
--- a/docs/Architecture/Components.md
+++ b/docs/Architecture/Components.md
@@ -12,10 +12,10 @@ If you like what you see give us a **Star** on one of these two sites!
* **[Java](https://github.com/apache/incubator-datasketches-java)** (Versioned, Apache Released) This is the original and the most comprehensive collection of sketch algorithms. It has a dependence on the Memory component and the Java Adaptors have a dependence on this component.
-* **[C++/Python](https://github.com/apache/incubator-datasketches-cpp)** (Versioned, Apache Released) This is newer and provides most of the major algorithms available in Java. Our C++ adaptors have a dependence on this component. The Pybind adaptors for Python are included here for all the C++ sketches.
+* **[C++/Python](https://github.com/apache/incubator-datasketches-cpp)** (Versioned, Apache Released) This is newer and provides most of the major algorithms available in Java. Our C++ adaptors have a dependence on this component. The Pybind adaptors for Python are included for all the C++ sketches.
## Adapters
-Apapters integrate the core components into the aggregation APIs of specific data processing systems. Some of these adapters are available as part of the library, other adapters may be directly integrated into the specific data processing system.
+Apapters integrate the core components into the aggregation APIs of specific data processing systems. Some of these adapters are available as part of the library, other adapters are directly integrated into the target data processing application.
### Java Adaptors
* **[Apache Hive](https://github.com/apache/incubator-datasketches-hive)** (Versioned, Apache Released)
@@ -34,7 +34,7 @@ them available to the PostgreSQL database users. PostgreSQL users should downloa
## Other Components
* **[Memory](https://github.com/apache/incubator-datasketches-memory):** (Versioned, Apache Released) This is a low-level library that enables fast access to off-heap memory for Java.
* **[Characterization](https://github.com/apache/incubator-datasketches-characterization):** This is a collection of Java and C++ code that we use for long-running studies of accuracy and speed performance over many different parameters. Feel free to run these tests to reproduce many of the graphs and charts you see on our website.
-* **[Vector (Experimental)](https://github.com/apache/incubator-datasketches-vector):** This component implements the [Frequent Directions Algorithm](https://datasketches.apache.org/docs/Community/Research.html) [GLP16]. It is still experimental in that the theoretical work has not yet supplied a suitable measure of error for production work. It can be used as is, but it will not go through a formal Apache Release until we can find a way to provide better error properties. It has a dep [...]
+* **[Vector (Experimental)](https://github.com/apache/incubator-datasketches-vector):** This component implements the [Frequent Directions Algorithm](/docs/Community/Research.html) [GLP16]. It is still experimental in that the theoretical work has not yet supplied a suitable measure of error for production work. It can be used as is, but it will not go through a formal Apache Release until we can find a way to provide better error properties. It has a dependence on the Memory component.
* **[Website](https://github.com/apache/incubator-datasketches-website):** This repository is the home of our website and is constantly being updated with new material.
diff --git a/docs/Architecture/KeyFeatures.md b/docs/Architecture/KeyFeatures.md
index 00a0c97..3fe4d3f 100644
--- a/docs/Architecture/KeyFeatures.md
+++ b/docs/Architecture/KeyFeatures.md
@@ -23,7 +23,7 @@ layout: doc_page
<h3>Common Sketch Properties</h3>
- * [Sketch Criteria]({{site.docs_dir}}/Architecture/SketchCriteria.html) for all sketches in the library.
+ * Please refer to the [Sketch Criteria]({{site.docs_dir}}/Architecture/SketchCriteria.html) for all sketches in the library.
* Query results are <b>approximate</b> but within well defined error bounds that are user
configurable by trading off sketch size with accuracy.
* Designed for <a href="{{site.docs_dir}}/LargeScale.html">Large-scale</a> computing environments
@@ -33,7 +33,7 @@ layout: doc_page
<a href="https://hive.apache.org/">Hive</a>,
<a href="https://druid.io">Druid</a>,
<a href="https://spark.apache.org">Spark</a>),
-and are heavily used within Yahoo.
+and are heavily used within Yahoo / Verizon-Media.
* <b>Maven deployable</b> and registered with
<a href="https://search.maven.org/#search|ga|1|DataSketches">The Central Repository</a>.
* Comprehensive <b>unit tests</b> and testing tools are provided.
diff --git a/docs/Architecture/OrderSensitivity.md b/docs/Architecture/OrderSensitivity.md
index 689cca5..3f0c3e4 100644
--- a/docs/Architecture/OrderSensitivity.md
+++ b/docs/Architecture/OrderSensitivity.md
@@ -20,44 +20,47 @@ layout: doc_page
under the License.
-->
## Sketching and Order Sensitivity
+Definitions:
-Sketching by its nature is a stochastic process and in general we cannot guarantee _order insensitivity_.
-All of our sketches (frequency, quantiles, Theta and HLL) should be assumed to be _order sensitive_.
-The only "guarantee" that we offer is that the "true value" (computed using brute-force techniques) should be within our approximate error bounds with the specified confidence.
+* **Absolute Order Insensitivity** Any permutation of the order of a given input stream produces the exact same result.
+* **Bounded Order Insensitivity** Any permutation of the order of a given input stream produces a result that is still within the defined error bounds of the sketch and confidence.
-Having said that there are a few exceptions to this "assume order sensitivity" guideline.
+Sketching by its nature is a stochastic process and in general we cannot guarantee _absolute order insensitivity_. However, some of our sketches, with the correct configuration, can meet this definition, but in general, we do not recommend users depending on this strict definition of order insensitivity.
-### Theta Sketches
+Nonetheless, all of our sketches do qualify as being _bounded order sensitive_.
+In other words, the "true value" (computed using brute-force techniques) should be within our approximate error bounds with the specified confidence.
-Only the QuickSelect Sketch (the default) can be order insensitive and ONLY if the final sketch is "trimmed" back to a maximum of _K_ values before an estimate is retrieved. For example:
+
+### Example: Theta Sketches
+
+Only the internal QuickSelect Sketch (the default) can be order insensitive and _iff_ the final sketch is "trimmed" back to a maximum of _K_ values before an estimate is retrieved. For example:
```java
UpdateSketch sk = Sketches.updateSketchBuilder().build();
-for (...) { /* load sketch with > 2K values */ }
-double est = sk.getEstimate(); //this may be order sensitive (but not if sketch is in exact mode)
+for (...) { /* load sketch with > 2 * K values */ }
+double est = sk.getEstimate(); //this may be order sensitive
UpdateSketch sk2 = sk.rebuild(); //trims retained entries back to K
double est2 = sk2.getEstimate(); //this will be order insensitive
```
-If you want a Compact Sketch to be order insensitive, you must _rebuild()_, first than do _compact()_.
+If you want a Compact Sketch to be order insensitive, you must first _rebuild()_, than do _compact()_.
When doing Unions with Theta Sketches, the getResult(...) automatically trims the result back to _K_.
-The impact of the rebuild() is that the error will not be as good as the un-trimmed sketch, but you will get your desired order insensitivity. [For example](https://datasketches.apache.org/docs/Theta/ThetaAccuracyPlots.html).
+The impact of the rebuild() is that the error will not be as good as the un-trimmed sketch, but you will get your desired order insensitivity. [For example](/docs/Theta/ThetaAccuracyPlots.html).
### HLL Sketches
-If you use the _getCompositeEstimate()_, the result should be order insensitive, but is less accurate than the _getEstimate()_, which uses the HIP estimator. Unfortunately, the HIP estimator does not "survive" the union process so the error of the HLL sketches that have gone through a union process generally must fall back on the composite estimator. (This is tracked internally and is rather complex as there are some special cases where the HIP estimator can still be used.)
+HLL sketches used stand-alone, are _bounded order insensitive_. After any merge / union operation the sketch qualifies as
+_absolute order insensitive_, but is less accurate.
-### The HIP Estimator
-The HIP estimator is inherently order sensitive, but provides significantly improved error properties over the composite approach, so it is too important to ignore.
### System Testing and Sketches
There are two primary ways that a "reference" standard is often obtained to use when system testing with sketches:
* Brute Force computation of the correct result. The recommended approach.
-* Assuming some prior test run produced the correct result. I do not recommend this, but many system teams do this anyway. Even if the sketches are working correctly, this can result in double-sided error, so be careful!
+* Assuming some prior test run produced the correct result. This is not recommended, but many system teams do this anyway. Even if the sketches are working correctly, this can result in double-sided error, so be careful!
Given a Brute Force reference, the proper way to establish correctness of the result of a test is to use the upper and lower bounds (or equivalent, depending on the sketch) provided by the sketch. Suppose you use 2-sigma confidence bounds. Then out of 1000 _statistically independent_ trials (runs), ~50 of the results will be outside the 2-sigma bounds.
diff --git a/docs/Architecture/SketchesByComponent.md b/docs/Architecture/SketchesByComponent.md
index b1bc5c8..afa367d 100644
--- a/docs/Architecture/SketchesByComponent.md
+++ b/docs/Architecture/SketchesByComponent.md
@@ -19,8 +19,8 @@ layout: doc_page
specific language governing permissions and limitations
under the License.
-->
-# Sketches by Component
-
+# Sketches by [Component Repository](https://github.com/apache?utf8=%E2%9C%93&q=datasketches)
+
The DataSketches Library is organized into the following repository groups:
## Java
@@ -29,7 +29,7 @@ The DataSketches Library is organized into the following repository groups:
This repository has the core-java sketching classes, which are leveraged by some of the other repositories.
This repository has no external dependencies outside of the DataSketches/memory repository, Java and TestNG for unit tests.
This code is versioned and the latest release can be obtained from
-<a href="https://www.apache.org/dyn/closer.cgi?path=/incubator/datasketches/java">incubator-datasketches-java<a/>.
+<a href="https://www.apache.org/dyn/closer.cgi?path=/incubator/datasketches/java">incubator-datasketches-java</a>.
<b>High-level Repositories Structure</b>
@@ -53,7 +53,7 @@ org.apache.datasketches.tuple.Strings | A Tuple sketch with a Summary of an arra
### incubator-datasketches-memory
This code is versioned and the latest release can be obtained from
-<a href="https://www.apache.org/dyn/closer.cgi?path=/incubator/datasketches/memory">incubator-datasketches-memory<a/>.
+<a href="https://www.apache.org/dyn/closer.cgi?path=/incubator/datasketches/memory">incubator-datasketches-memory</a>.
Memory Packages | Package Description
-------------------------------|---------------------
@@ -65,7 +65,7 @@ This repository contains Hive UDFs and UDAFs for use within Hadoop grid enviornm
This code has dependencies on sketches-core as well as Hadoop and Hive.
Users of this code are advised to use Maven to bring in all the required dependencies.
This code is versioned and the latest release can be obtained from
-<a href="https://www.apache.org/dyn/closer.cgi?path=/incubator/datasketches/hive">incubator-datasketches-hive<a/>.
+<a href="https://www.apache.org/dyn/closer.cgi?path=/incubator/datasketches/hive">incubator-datasketches-hive</a>.
Sketches-hive Packages | Package Description
-------------------------------------|---------------------
@@ -82,7 +82,7 @@ This repository contains Pig User Defined Functions (UDF) for use within Hadoop
This code has dependencies on sketches-core as well as Hadoop and Pig.
Users of this code are advised to use Maven to bring in all the required dependencies.
This code is versioned and the latest release can be obtained from
-<a href="https://www.apache.org/dyn/closer.cgi?path=/incubator/datasketches/pig">incubator-datasketches-pig<a/>.
+<a href="https://www.apache.org/dyn/closer.cgi?path=/incubator/datasketches/pig">incubator-datasketches-pig</a>.
Sketches-pig Packages | Package Description
-----------------------------------|---------------------
@@ -124,7 +124,7 @@ org.apache.datasketches.characterization.uniquecount | Base Profiles for Unique
### incubator-datasketches-vector
-This component implements the [Frequent Directions Algorithm](https://datasketches.apache.org/docs/Community/Research.html) [GLP16]. It is still experimental in that the theoretical work has not yet supplied a suitable measure of error for production work. It can be used as is, but it will not go through a formal Apache Release until we can find a way to provide better error properties. It has a dependence on the Memory component.
+This component implements the [Frequent Directions Algorithm](/docs/Community/Research.html) [GLP16]. It is still experimental in that the theoretical work has not yet supplied a suitable measure of error for production work. It can be used as is, but it will not go through a formal Apache Release until we can find a way to provide better error properties. It has a dependence on the Memory component.
## C++ and Python
diff --git a/docs/Community/Downloads.md b/docs/Community/Downloads.md
index 25bedd7..6f13f7c 100644
--- a/docs/Community/Downloads.md
+++ b/docs/Community/Downloads.md
@@ -22,7 +22,7 @@ layout: doc_page
## Downloads
### Download Zip Files
-Choose the most recent release version from
+Choose the most recent release version from one of these mirrors:
[incubator-datasketches-xxx](https://www.apache.org/dyn/closer.cgi?path=/incubator/datasketches).
### Download Java Jar Files
diff --git a/docs/HLL/Hll_vs_CS_Hllpp.md b/docs/HLL/Hll_vs_CS_Hllpp.md
index 8418288..110a335 100644
--- a/docs/HLL/Hll_vs_CS_Hllpp.md
+++ b/docs/HLL/Hll_vs_CS_Hllpp.md
@@ -226,7 +226,7 @@ Depending on the chosen configuration, the *HllSketch* can be from one to almost
****
* [1] [DataSketches HllSketch GitHub](https://github.com/apache/incubator-datasketches-java/tree/master/src/main/java/org/apache/datasketches/hll)
-* [2] [DataSketches HllSketch JavaDocs](https://datasketches.apache.org/api/java/snapshot/apidocs/index.html)
+* [2] [DataSketches HllSketch JavaDocs](/api/java/snapshot/apidocs/index.html)
* [3] [HyperLogLogPlus GitHub](https://github.com/addthis/stream-lib/blob/master/src/main/java/com/clearspring/analytics/stream/cardinality/HyperLogLogPlus.java)
* [4] [Google: HyperLogLog in Practice: Algorithmic Engineering of a State of The Art Cardinality Estimation Algorithm](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/40671.pdf)
* [5] The Root-Mean-Square of the Relative Error (RMS-RE) is sensitive to bias of the mean if there is any. However, if the bias is zero RMS-RE will produce the same results as the theoretical Relative Standard Error (RSE) of the stochastic process.
diff --git a/docs/MajorSketchFamilies.md b/docs/MajorSketchFamilies.md
index 2943044..32a3da2 100644
--- a/docs/MajorSketchFamilies.md
+++ b/docs/MajorSketchFamilies.md
@@ -21,11 +21,13 @@ layout: doc_page
-->
# Sketch Capability Matrix
+Use the following table to compare the capabilities of the different sketch families.
+
<div>
<table>
<tr style="font-weight:bold"><td colspan="2"></td><td colspan="3">Languages</td><td colspan="4">Set Operations</td><td colspan="4">System Integrations</td><td colspan="5">Misc.</td></tr>
-<tr style="font-weight:bold"><td>Type</td><td>Sketch</td><td>Java</td><td>C++</td><td>Python</td><td>Union</td><td>Inter-section</td><td>Difference</td><td>Jaccard</td><td>Hive</td><td>Pig</td><td>Druid<sup>1</sup></td><td>Spark<sup>2</sup></td><td>Con-current</td><td>Compact</td><td>Java Generics</td><td>Off-Heap</td><td>Error Bounds</td></tr>
+<tr style="font-weight:bold"><td>Type</td><td>Sketch</td><td>Java</td><td>C++</td><td>Python</td><td>Union</td><td>Inter-section</td><td>Difference</td><td>Jaccard</td><td>Hive</td><td>Pig</td><td>Druid<sup>1</sup></td><td>Spark<sup>2</sup></td><td>Con-current</td><td>Compact</td><td>Java Generics</td><td>Off Java Heap</td><td>Error Bounds</td></tr>
<tr style="font-weight:bold"><td colspan="18">Major Sketches</td></tr>
<tr><td>Cardinality/FM85</td><td>CpcSketch</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td>Y</td><td>Y</td><td></td><td></td><td></td><td>Y</td><td></td><td></td><td>Y</td></tr>
@@ -39,7 +41,7 @@ layout: doc_page
<tr><td>Frequencies</td><td>ItemsSketch</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td>Y</td></tr>
<tr><td>Sampling</td><td>ReservoirLongsSketch</td><td>Y</td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>Y</td></tr>
<tr><td>Sampling</td><td>ReserviorItemsSketch</td><td>Y</td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td>Y</td></tr>
-<tr><td>Sampling</td><td>VarOptItemsSketch</td><td>Y</td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td>Y</td></tr>
+<tr><td>Sampling</td><td>VarOptItemsSketch</td><td>Y</td><td>Y</td><td>Y</td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td></td><td></td><td></td><td>Y</td><td></td><td>Y</td></tr>
<tr style="font-weight:bold"><td colspan="18">Specialty Sketches</td></tr>
@@ -54,7 +56,7 @@ layout: doc_page
</div>
<sup>1</sup> Integrated into Druid<br>
-<sup>2</sup> Example Code on website
+<sup>2</sup> Example Code on website<br>
<sup>3</sup> Example Code in test/.../tuple/aninteger
----
@@ -65,7 +67,7 @@ layout: doc_page
## Cardinality Sketches
### CPC Sketch: Estimating Stream Cardinalities more efficiently than the famous HLL sketch!
-This sketch was developed by the late Keven Lang, our chief scientist at the time, is an amazing *tour de force* of scientific design and engineering and has substantially better accuracy / per stored size than the famous HLL sketch. The theory and demonstration of its performance is detailed in Lang's paper [Back to the Future: an Even More Nearly Optimal Cardinality Estimation Algorithm](https://arxiv.org/abs/1708.06839). This sketch is available in Java, C++ and Python.
+This sketch was developed by the late Keven J. Lang, our chief scientist at the time. It is an amazing *tour de force* of scientific design and engineering and has substantially better accuracy / per stored size than the famous HLL sketch. The theory and demonstration of its performance is detailed in Lang's paper [Back to the Future: an Even More Nearly Optimal Cardinality Estimation Algorithm](https://arxiv.org/abs/1708.06839).
### [Theta Sketches]({{site.docs_dir}}/Theta/ThetaSketchFramework.html): Estimating Stream Expression Cardinalities
Internet content, search and media companies like Yahoo, Google, Facebook, etc., collect many tens of billions of event records from the many millions of users to their web sites each day. These events can be classified by many different dimensions, such as the page visited and user location and profile information. Each event also contains some unique identifiers associated with the user, specific device (cell phone, tablet, or computer) and the web browser used.
@@ -77,12 +79,12 @@ These same unique identifiers will appear on every page that the user visits. I
Computing cardinalities with massive data requires lots of computer resources and time.
However, if an approximate answer to these problems is acceptable, [Theta Sketches]({{site.docs_dir}}/Theta/ThetaSketchFramework.html) can provide reasonable estimates, in a single pass, orders of magnitude faster, even fast enough for analysis in near-real time.
-The [theta/Sketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/theta/Sketch.java) can operate both on-heap and off-heap, has powerful Union, Intersection, AnotB and Jaccard operators, has a high-performance concurrent form for multi-threaded environments, has both immutable compact, and updatable representations, and is quite fast. It is available in Java, C++ and Python. Because of its flexibility, it is one of the most popular [...]
+The [theta/Sketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/theta/Sketch.java) can operate both on-heap and off-heap, has powerful Union, Intersection, AnotB and Jaccard operators, has a high-performance concurrent form for multi-threaded environments, has both immutable compact, and updatable representations, and is quite fast. Because of its flexibility, it is one of the most popular sketches in our library.
### [Tuple Sketches]({{site.docs_dir}}/Tuple/TupleOverview.html): Extending Theta Sketches to Perform Associative Analysis
-It is often not enough to perform stream expressions on sets of unique identifiers, it is very valuable to be able to associate additive data with these identifiers, such as impression counts, clicks or timestamps. Tuple Sketches are a natural extension of the Theta sketch and have Java Genric forms, that enable the user do define the sketch with arbitrary "summary" data. The [tuple/Sketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datas [...]
+It is often not enough to perform stream expressions on sets of unique identifiers, it is very valuable to be able to associate additive data with these identifiers, such as impression counts, clicks or timestamps. Tuple Sketches are a natural extension of the Theta sketch and have Java Genric forms that enable the user to define the sketch with arbitrary "summary" data. The [tuple/Sketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datask [...]
-The Tuple sketch is effectively infinitely extendable and there are several common variants of the Tuple Sketch, which also serve as examples on how to extend the base classes, that are also in the library, including:
+The Tuple sketch is effectively infinitely extendable and there are several common variants of the Tuple Sketch, which also serve as examples on how to extend the base classes that are also in the library, including:
- [tuple/adouble/DoubleSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/tuple/adouble/DoubleSketch.java) with a single column of *double* values as the *summary*.
- [tuple/aninteger/IntegerSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/tuple/aninteger/IntegerSketch.java) with a single column of *int* values as the *summary*.
@@ -91,12 +93,12 @@ The Tuple sketch is effectively infinitely extendable and there are several comm
### [HyperLogLog Sketches]({{site.docs_dir}}/HLL/HLL.html): Estimating Stream Cardinalities
-The HyperLogLog (HLL) is a cardinality sketch similar to the above Theta sketches except they are anywhere from 2 to 16 times smaller in size. The HLL sketches can be Unioned, but set intersection and difference operations are not provided intrinsically, because the resulting error would be quite poor. If your application only requires cardinality estimation and Unioning and space is at a premium, the HLL sketch provided could be your best choice.
+The HyperLogLog (HLL) is a cardinality sketch similar to the above Theta sketches except they are anywhere from 2 to 16 times smaller in size. The HLL sketches can be merged via the Union operator, but set intersection and difference operations are not provided intrinsically, because the resulting error would be quite poor. If your application only requires cardinality estimation and merging and space is at a premium, the HLL or CPC sketches would be your best choice.
-The [hll/HllSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/hll/HllSketch.java) can operate both on-heap and off-heap, provides the Union operators, and has both immutable compact, and updatable representations. It is available in Java, C++ and Python.
+The [hll/HllSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/hll/HllSketch.java) can operate both on-heap and off-heap, provides the Union operators, and has both immutable compact and updatable representations.
### [HyperLogLog Map Sketch]({{site.docs_dir}}/HLL/HllMap.html): Estimating Stream Cardinalities of Key-Value Pairs
-This is a specially designed sketch that addresses the problem of individually tracking value cardinalities of Key-Value (K,V) pairs in real-time, where the number of keys can be very large, such as IP addresses, or Geo identifiers, etc. Assigning individual sketches to each key would create unnecessary overhead. This sketch streamlines the process with much better space management. This [hllmap/UniqueCountMap](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/j [...]
+This is a specially designed sketch that addresses the problem of individually tracking value cardinalities of Key-Value (K,V) pairs in real-time, where the number of keys can be very large, such as IP addresses, or Geo identifiers, etc. Assigning individual sketches to each key would create unnecessary overhead. This sketch streamlines the process with much better space management. This [hllmap/UniqueCountMap](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/j [...]
## Quantiles Sketches
@@ -105,32 +107,31 @@ There are many situations where is valuable to understand the distribution of va
<img class="doc-img-full" src="{{site.docs_img_dir}}/quantiles/TimeSpentHistogram.png" alt="TimeSpentHistogram" />
-There are two different families of quantiles sketches, the original [quantiles/DoublesSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/quantiles/DoublesSketch.java), which can be operated either on-heap or off-heap, and is also available in a Java Generic form for arbitrary comparable objects. It is only available in Java.
+There are two different families of quantiles sketches, the original [quantiles/DoublesSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/quantiles/DoublesSketch.java), which can be operated either on-heap or off-heap, and is also available in a Java Generic form for arbitrary comparable objects.
-Later we developed the [kll/KllFloatsSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/kll/KllFloatsSketch.java) (Named after its authors), which is also a quantiles sketch, that achieves near optimal small size for a given accuracy. It is only available on-heap. It is available in Java, C++ and Python.
+Later we developed the [kll/KllFloatsSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/kll/KllFloatsSketch.java) (Named after its authors), which is also a quantiles sketch, that achieves near optimal small size for a given accuracy.
## Frequent Items / Heavy Hitters Sketches
### [Frequent Items Sketches]({{site.docs_dir}}/Frequency/FrequentItemsOverview.html): Finding the Heavy Hitter Objects from a Stream
-It is very useful to be able to scan a stream of objects, such as song titles, and be able to quickly identify those items that occur most frequently. The term <i>Heavy Hitter</i> is defined to be an item that occurs more frequently than some fractional share of the overall count of items
-in the stream including duplicates. Suppose you have a stream of 1M song titles, but in that stream there are only 100K song titles that are unique. If any single title consumes more than 10% of the stream elements it is a Heavy Hitter, and the 10% is a threshold parameter we call epsilon.
+It is very useful to be able to scan a stream of objects, such as song titles, and be able to quickly identify those titles that occur most frequently. The term <i>Heavy Hitter</i> is defined to be an item that occurs more frequently than its fair share of occurrences. This "fair share" is simply the total count of all occurrences of all items divided by the number of distinct items. Suppose you have a stream of 1M song titles, but in that stream there are only 100K song titles that ar [...]
-The accuracy of a Frequent Items Sketch is proportional to the configured size of the sketch, the larger the sketch, the smaller is the epsilon threshold that can detect Heavy Hitters. This sketch is available in two forms, as the [frequencies/LongsSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/frequencies/LongsSketch.java) used for processing a stream of tuples {*long*, weight}, and the [frequencies/ItemsSketch](https://gi [...]
+The accuracy of a Frequent Items Sketch is proportional to the configured size of the sketch, the larger the sketch, the smaller is the epsilon threshold that can detect Heavy Hitters. This sketch is available in two forms, as the [frequencies/LongsSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/frequencies/LongsSketch.java) used for processing a stream of tuples {*long*, weight}, and the [frequencies/ItemsSketch](https://gi [...]
### [Frequent Distinct Tuples Sketch]({{site.docs_dir}}/Frequency/FrequentDistinctTuplesSketch.html): Finding the Heavy Hitter tuples from a Stream.
Suppose our data is a stream of pairs {IP address, User ID} and we want to identify the IP addresses that have the most distinct User IDs. Or conversely, we would like to identify the User IDs that have the most distinct IP addresses. This is a common challenge in the analysis of big data and the FDT sketch helps solve this problem using probabilistic techniques.
More generally, given a multiset of tuples with *N* dimensions *{d1,d2, d3, …, dN}*, and a primary subset of dimensions *M < N*, our task is to identify the combinations of *M* subset dimensions that have the most frequent number of distinct combinations of the *N - M* non-primary dimensions.
-The [fdt/FdtSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/fdt/FdtSketch.java) is currently only available in Java, but because it is an extension of the Tuple Sketch family, it inherits many of the same properties: it can operate both on-heap and off-heap, includes both Union and Intersection operators, has both immutable compact, and updatable representations.
+The [fdt/FdtSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/fdt/FdtSketch.java) is currently only available in Java, but because it is an extension of the Tuple Sketch family, it inherits many of the same properties: it can operate both on-heap and off-heap, it includes both Union and Intersection operators, and it has both immutable compact and updatable representations.
### Frequent Directions: Distributed, mergeable Singular Value Decomposition
-Part of a new separate sketches-vector package, Frequent Directions is in many ways a generalization of the Frequent Items sketch to handle vector data. This sketch computes an approximate singular value decomposition (SVD) of a matrix, providing a projection matrix that can be used for dimensionality reduction. SVD is a key technique in many recommender systems, providing shopping suggestions based on a customer's past purchases compared with other similar customers. This sketch is stil [...]
+Part of a new separate datasketches-vector component, Frequent Directions is in many ways a generalization of the Frequent Items sketch in order to handle vector data. This sketch computes an approximate singular value decomposition (SVD) of a matrix, providing a projection matrix that can be used for dimensionality reduction. SVD is a key technique in many recommender systems, such as providing shopping suggestions based on a customer's past purchases compared with other similar custome [...]
## Sampling Sketches
-### [Sampling Sketches]({{site.docs_dir}}/Sampling/ReservoirSampling.html): Uniform Sampling of a Stream into a fixed size space
-This family of sketches implements an enhanced version of the famous Reservoir sampling algorithm and extends it with the capabilities that large-scale distributed systems really need: mergability (even with different sized sketches), uses Java Generics so that the base classes can be trivially extended for any input type (even polymorphic types), and an extensible means of performing serialization and deserialization.
+### [Sampling Sketches]({{site.docs_dir}}/Sampling/ReservoirSampling.html): Uniform and Weighted Sampling of a Stream into a fixed size space
+This family of sketches implements an enhanced version of the famous Reservoir sampling algorithm and extends it with the capabilities that large-scale distributed systems really need: mergability (even with different sized sketches). The Java implementaion uses Java Generics so that the base classes can be trivially extended for any input type (even polymorphic types), and also enables an extensible means of performing serialization and deserialization.
The [sampling/ReservoirLongsSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/sampling/ReservoirLongsSketch.java) accepts a stream of *long* values as identifiers with a weight of one, and produces a result Reservoir of a pre-determined size that represents a uniform random sample of the stream.
@@ -138,6 +139,5 @@ The [sampling/ReservoirItemsSketch](https://github.com/apache/incubator-datasket
The [sampling/VarOptItemsSketch](https://github.com/apache/incubator-datasketches-java/blob/master/src/main/java/org/apache/datasketches/sampling/VarOptItemsSketch.java) extends the Reservoir family to weighted sampling, additionally providing subset sum estimates from the sample with provably optimal variance.
-This family is currently only available in Java.
diff --git a/docs/Quantiles/DruidApproxHistogramStudy.md b/docs/Quantiles/DruidApproxHistogramStudy.md
index 5506c51..e0b95f9 100644
--- a/docs/Quantiles/DruidApproxHistogramStudy.md
+++ b/docs/Quantiles/DruidApproxHistogramStudy.md
@@ -25,7 +25,7 @@ The goal of this article is to compare the accuracy performance of the Druid bui
Please get familiar with the [Definitions]({{site.docs_dir}}/Quantiles/Definitions.html) for quantiles.
-Compare this study with the DataSketches [Quantiles StreamA Study](https://datasketches.apache.org/docs/Quantiles/QuantilesStreamAStudy.html) with the same input data.
+Compare this study with the DataSketches [Quantiles StreamA Study](/docs/Quantiles/QuantilesStreamAStudy.html) with the same input data.
## Versions
diff --git a/docs/Quantiles/MomentsSketchStudy.md b/docs/Quantiles/MomentsSketchStudy.md
index 4ffb504..96d636b 100644
--- a/docs/Quantiles/MomentsSketchStudy.md
+++ b/docs/Quantiles/MomentsSketchStudy.md
@@ -25,7 +25,7 @@ The goal of this article is to compare the accuracy performance of the Moments S
Please get familiar with the [Definitions]({{site.docs_dir}}/Quantiles/Definitions.html) for quantiles.
-Compare this study with the DataSketches [Quantiles StreamA Study](https://datasketches.apache.org/docs/Quantiles/QuantilesStreamAStudy.html) with the same input data.
+Compare this study with the DataSketches [Quantiles StreamA Study](/docs/Quantiles/QuantilesStreamAStudy.html) with the same input data.
## Versions
diff --git a/docs/SketchElements.md b/docs/SketchElements.md
index 2bdec1c..e6388e1 100644
--- a/docs/SketchElements.md
+++ b/docs/SketchElements.md
@@ -60,6 +60,3 @@ The sketch only needs to see each item in the stream once.
be merged without losing accuracy.
* Approximate. As an example, for unique count sketches the relative error bounds
are a function of the configured size of the sketch.
-
-With this background, let's examine some of the
-<a href="{{site.docs_dir}}/Architecture/KeyFeatures.html">Key Features</a> of the DataSketches library.
diff --git a/docs/SketchOrigins.md b/docs/SketchOrigins.md
index f58f4a6..760d43f 100644
--- a/docs/SketchOrigins.md
+++ b/docs/SketchOrigins.md
@@ -48,10 +48,6 @@ Synopses for Massive Data: Samples, Histograms, Wavelets, Sketches</a> by
<a href="https://www2.warwick.ac.uk/fac/sci/dcs/people/graham_cormode/">Graham Cormode</a>, et al,
is an excellent review of this field.
-At this point it is useful to describe the
-<a href="/docs/SketchElements.html">sketch elements</a> of a common sub-class of sketching
-algorithms used for solving the
-<a href="https://en.wikipedia.org/wiki/Count-distinct_problem">count-distinct</a> problem.
________
<sup>1</sup><small>Also known as "Approximate Query Processing", see
diff --git a/docs/TheChallenge.md b/docs/TheChallenge.md
index 7ed2e17..93f313a 100644
--- a/docs/TheChallenge.md
+++ b/docs/TheChallenge.md
@@ -116,7 +116,7 @@ Processing the continuous real-time stream from the edge web servers is possible
### Big Win #7: Resource Utilization and Cost
-It has been our experience at Yahoo, that a good implementation of these large analysis systems using sketches reduces the overall cost of the system considerably. It is difficult to quote exact numbers as your mileage will vary as it is system and data dependent.
+It has been our experience at Yahoo/VM, that a good implementation of these large analysis systems using sketches reduces the overall cost of the system considerably. It is difficult to quote exact numbers as your mileage will vary as it is system and data dependent.
diff --git a/docs/Tuple/TupleEngagementExample.md b/docs/Tuple/TupleEngagementExample.md
index bc0df9e..8e56f5d 100644
--- a/docs/Tuple/TupleEngagementExample.md
+++ b/docs/Tuple/TupleEngagementExample.md
@@ -65,7 +65,7 @@ Once we have our 30 day sketches, we merge all 30 sketches together into one fin
To help us code our example we will leverage the [IntegerSketch Package](https://github.com/apache/incubator-datasketches-java/tree/master/src/main/java/org/apache/datasketches/tuple/aninteger) from the library. This package consists of 5 classes, the _IntegerSketch_ and 4 helper classes, all of which extend generic classes of the parent _tuple_ package. Normally, the user/developer would develop these 5 classes to solve a particular analysis problem. These 5 classes can serve as an exa [...]
-Please refer to the [Tuple Overview](https://datasketches.apache.org/docs/Tuple/TupleOverview.html) section on this website for a quick review of how the Tuple Sketch works.
+Please refer to the [Tuple Overview](/docs/Tuple/TupleOverview.html) section on this website for a quick review of how the Tuple Sketch works.
### IntegerSketch class
```java
diff --git a/img/datasketches-HorizontalColor-1.svg b/img/datasketches-HorizontalColor-1.svg
new file mode 100755
index 0000000..66fc1dd
--- /dev/null
+++ b/img/datasketches-HorizontalColor-1.svg
@@ -0,0 +1,52 @@
+<svg id="Layer_1" data-name="Layer 1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="839" height="357" viewBox="0 0 839 357">
+ <defs>
+ <linearGradient id="linear-gradient" x1="13.66" y1="106.04" x2="210.12" y2="269.11" gradientUnits="userSpaceOnUse">
+ <stop offset="0" stop-color="#c70000"/>
+ <stop offset="0" stop-color="#ffa017"/>
+ <stop offset="1" stop-color="#c70000"/>
+ </linearGradient>
+ <linearGradient id="linear-gradient-2" x1="42.07" y1="71.81" x2="238.53" y2="234.89" gradientUnits="userSpaceOnUse">
+ <stop offset="0" stop-color="#c70000"/>
+ <stop offset="0" stop-color="#ffa017"/>
+ <stop offset="0.71" stop-color="#c90501"/>
+ <stop offset="1" stop-color="#4d0000"/>
+ </linearGradient>
+ <linearGradient id="linear-gradient-3" x1="388.26" y1="238.77" x2="659.2" y2="152.74" xlink:href="#linear-gradient-2"/>
+ <linearGradient id="linear-gradient-4" x1="393.86" y1="256.4" x2="664.8" y2="170.37" xlink:href="#linear-gradient-2"/>
+ <linearGradient id="linear-gradient-5" x1="401.37" y1="280.05" x2="672.31" y2="194.02" xlink:href="#linear-gradient-2"/>
+ <linearGradient id="linear-gradient-6" x1="403.2" y1="285.82" x2="674.14" y2="199.79" xlink:href="#linear-gradient-2"/>
+ <linearGradient id="linear-gradient-7" x1="409.51" y1="305.69" x2="680.45" y2="219.66" xlink:href="#linear-gradient-2"/>
+ <linearGradient id="linear-gradient-8" x1="411.46" y1="311.83" x2="682.4" y2="225.8" xlink:href="#linear-gradient-2"/>
+ <linearGradient id="linear-gradient-9" x1="419.57" y1="337.36" x2="690.51" y2="251.33" xlink:href="#linear-gradient-2"/>
+ <linearGradient id="linear-gradient-10" x1="424.42" y1="352.63" x2="695.36" y2="266.6" xlink:href="#linear-gradient-2"/>
+ </defs>
+ <title>ApacheDataSketch_ColorHorizontal</title>
+ <g>
+ <g>
+ <circle cx="93.7" cy="172.4" r="24.7" fill="url(#linear-gradient)"/>
+ <path d="M199.1,81.5c16.7-22.5,45-60.4,52.8-69.6,0-.2-18.2,5.4-18.2,5.4s-29.3,41.4-48.5,74.6-34.8,60.5-73,138.1c-10.4,21-22.2,16.3-38.3,1.8S19.2,180.7,19.2,156c0-29.4,21.9-36.5,37.6-39.6,8.8-1.8,12.1,1.2,12.1,3.5s-9,5.2-16.3,3.2c-2.1-.6,2,5.1,11.3,5.1,5.9,0,17.6-2.8,17.6-8.6,0-3.5-3.3-7.8-16-7.8-9.2,0-26.2,5-35,10.3s-19.2,14.1-19.2,27.2c0,40,79.6,103.5,88.5,109.1,0-.2-13.4,38.3-13.4,59.4,0,15.3,2.4,26.2,2.9,27.3s27.7-20.6,51.6-27.8,46.8-14.1,92.8-13.9c0,0-47.4-7.3-85-23.9-40.8-18.1 [...]
+ </g>
+ <g>
+ <path d="M205.6,185.1h26.7c21.6,0,32.3,14.7,32.3,36.4s-11.3,35.9-33,35.9h-26Zm25.6,63.6c16.4,0,23.1-10.6,23.1-27.2s-6-27.8-22.5-27.8H215.6v55Z" fill="#4d0000"/>
+ <path d="M272.4,243.6c0-12.2,8.9-14.8,22.4-16.5,7.7-.8,12.6-2.1,12.6-7.4s-3.9-9.2-11.1-9.2-12.3,3.9-12.6,10.1H275c.4-8.8,7.1-16.7,21.2-16.7,4.9,0,9.3.9,12.6,2.8s7,7.2,7,14.2v24.4c0,4.2,1,6.4,5.5,5.5h.4v6.3a14.4,14.4,0,0,1-4.8.8c-5.6,0-8.3-2.7-9.3-8.2h-.1c-3.3,4.6-8.6,8.7-17.5,8.7S272.4,253.6,272.4,243.6Zm35.1-5.5v-8.5c-2.6,1.6-7.1,2.9-11.4,3.7-8.6,1.4-14.2,3-14.2,10.2s5.1,8.3,10.4,8.3C302.7,251.8,307.5,244.2,307.5,238.1Z" fill="#4d0000"/>
+ <path d="M324.9,205.2h8.2V188.8H342v16.4h10.8v7H342v32.9c0,3.8,2.1,4.9,5.4,4.9a19.4,19.4,0,0,0,4.6-.7h.3v7.6a31.7,31.7,0,0,1-7.3.9c-7-.1-11.9-2.9-11.9-10.6v-35h-8.2Z" fill="#4d0000"/>
+ <path d="M360.1,243.6c0-12.2,9-14.8,22.5-16.5,7.7-.8,12.6-2.1,12.6-7.4s-4-9.2-11.2-9.2-12.2,3.9-12.5,10.1h-8.8c.4-8.8,7.1-16.7,21.2-16.7,5,0,9.4.9,12.7,2.8s7,7.2,7,14.2v24.4c0,4.2.9,6.4,5.5,5.5h.4v6.3a15.1,15.1,0,0,1-4.9.8c-5.5,0-8.2-2.7-9.2-8.2h-.1c-3.3,4.6-8.6,8.7-17.5,8.7S360.1,253.6,360.1,243.6Zm35.2-5.5v-8.5c-2.6,1.6-7.2,2.9-11.5,3.7-8.5,1.4-14.1,3-14.1,10.2s5.1,8.3,10.4,8.3C390.5,251.8,395.3,244.2,395.3,238.1Z" fill="#4d0000"/>
+ <path d="M413.7,234h9.8c1.2,10.8,8.4,16.9,21.5,16.9,9.6,0,16.6-4.3,16.6-13.1s-8.4-11.6-21-14.4-23.9-6.5-23.9-20.4c0-12.1,10.2-19.3,25.1-19.3,17.8,0,25.7,8.9,27.5,21.3h-9.6c-1.3-8.8-7.6-13.5-17.8-13.5s-15.8,3.9-15.8,11.1,7.9,10.1,18.8,12.3c14,2.8,26.7,6.4,26.7,22.6,0,13.7-11.3,21.3-26.9,21.3C424.5,258.8,414.5,247.8,413.7,234Z" fill="url(#linear-gradient-3)"/>
+ <path d="M498.6,229.9l-7.4,7v20.5h-9V185.1h9v34.3c0,1.8-.2,5.5-.3,8l22.6-22.2h11.3l-19.9,18.7,22.2,33.5H516.5Z" fill="url(#linear-gradient-4)"/>
+ <path d="M529.1,231.3c0-15.1,10.1-27.4,25-27.4,7.8,0,13.8,3,17.9,8.1s6.4,12.6,6.4,21.5h-40c.7,10.2,6.2,17.9,16.3,17.9,7.2,0,12.2-3.4,13.8-8.9h8.9c-2.4,9.7-10.7,16.3-22.8,16.3C538.6,258.8,529.1,246.6,529.1,231.3Zm9.5-4.3h30.2c-.5-10.1-5.6-16.1-14.7-16.1S539.7,217.7,538.6,227Z" fill="url(#linear-gradient-5)"/>
+ <path d="M583.9,205.2H592V188.8h9v16.4h10.8v7H601v32.9c0,3.8,2,4.9,5.3,4.9a18.6,18.6,0,0,0,4.6-.7h.3v7.6a31,31,0,0,1-7.3.9c-7-.1-11.9-2.9-11.9-10.6v-35h-8.1Z" fill="url(#linear-gradient-6)"/>
+ <path d="M617.9,231.3c0-15.1,9.8-27.4,25.5-27.4,12.5,0,20.7,7.1,22.5,18.3h-9c-1.3-6.8-6-11-13.5-11-10.7,0-16.3,8.9-16.3,20.1s5.6,20.1,16.3,20.1c8.1,0,13-4.7,13.7-12.5H666c-1.2,12.5-9.4,19.9-22.6,19.9C627.7,258.8,617.9,246.6,617.9,231.3Z" fill="url(#linear-gradient-7)"/>
+ <path d="M683.9,212.8h.3c3.9-5.6,9.1-8.9,16.7-8.9,10.9,0,17.6,6.1,17.6,17v36.5h-9V222c0-6.7-4.3-10.5-11.4-10.5s-13.9,5.7-13.9,14.1v31.8h-9.1V185.1h9.1V204C684.2,206.6,684,210.2,683.9,212.8Z" fill="url(#linear-gradient-8)"/>
+ <path d="M727.8,231.3c0-15.1,10-27.4,25-27.4,7.8,0,13.8,3,17.9,8.1s6.4,12.6,6.4,21.5h-40c.6,10.2,6.2,17.9,16.3,17.9,7.2,0,12.1-3.4,13.8-8.9H776c-2.3,9.7-10.7,16.3-22.7,16.3C737.2,258.8,727.8,246.6,727.8,231.3Zm9.4-4.3h30.3c-.5-10.1-5.7-16.1-14.7-16.1S738.4,217.7,737.2,227Z" fill="url(#linear-gradient-9)"/>
+ <path d="M783.4,241.5h9.2c1,7.6,6.9,10.1,14.3,10.1s11.8-3.6,11.8-8.2-4.4-7.2-13.6-9.1-19.4-4.2-19.4-15.7c0-8.9,6.9-14.6,19.1-14.6s19.6,6.1,20.8,15.2h-9c-.8-5.4-4.5-8.4-12-8.4s-10.1,3.3-10.1,7.3c0,5.5,5.1,6.5,13.9,8.3,10.4,2,19.3,4.3,19.3,16.5,0,10.6-8.7,16-21,16C792.4,258.9,784.2,252.1,783.4,241.5Z" fill="url(#linear-gradient-10)"/>
+ </g>
+ <g>
+ <path d="M248.5,135.7h6.9L268,170.4h-6.9l-2.7-8.3H245.2l-2.8,8.3h-6.5Zm-1.7,21.7h10l-2.7-8.1c-1.3-4.1-2.2-8.3-2.2-8.3h0s-1.2,5-2.3,8.3Z" fill="#4d0000"/>
+ <path d="M271.4,145.5h5.8v3.7h.1a8.6,8.6,0,0,1,7.8-4.4c6.5,0,10.7,5.2,10.7,13.2s-4.2,13.2-10.9,13.2a8.7,8.7,0,0,1-7.6-4h-.1a47.2,47.2,0,0,1,.2,4.9v6.6h-6Zm18.4,12.6c0-4.9-2-8.5-6.1-8.5s-6.6,3.8-6.6,8.5,2.1,8.4,6.7,8.4S289.8,163,289.8,158.1Z" fill="#4d0000"/>
+ <path d="M315.3,167.2h-.1a9.1,9.1,0,0,1-7.9,3.8c-4.8,0-8.5-2.5-8.5-7.4s4.3-7,10.3-7.7c4.1-.5,6.1-1.2,6.1-3.4s-1.5-3.6-4.5-3.6-4.9,1.6-5.1,4H300c.2-4.2,3.6-8.1,10.7-8.1,2.8,0,5,.4,6.7,1.5s3.5,3.7,3.5,7v13.2c0,2,.4,3.1.9,3.5v.4h-5.7A6.6,6.6,0,0,1,315.3,167.2Zm.1-5.9v-3.7a15.7,15.7,0,0,1-4.9,1.7c-3.6.7-5.7,1.4-5.7,4.2s2,3.4,4.3,3.4C313.1,166.9,315.4,164.1,315.4,161.3Z" fill="#4d0000"/>
+ <path d="M325.1,158c0-7.4,4.9-13.2,12.6-13.2s10.3,3.6,11.2,9h-5.8a5.2,5.2,0,0,0-5.4-4.5c-4.3,0-6.6,3.6-6.6,8.7s2.3,8.5,6.6,8.5,5.2-1.9,5.6-5.2H349c-.3,5.8-4.6,9.8-11.3,9.8S325.1,165.3,325.1,158Z" fill="#4d0000"/>
+ <path d="M358.7,149.1h.2a8.6,8.6,0,0,1,7.7-4.3c5.3,0,8.6,3.3,8.6,8.7v16.9h-6V154.5c0-3-1.7-4.8-4.7-4.8s-5.7,2.5-5.7,6.1v14.6h-5.9V135.7h5.9V145C358.8,145.7,358.8,147.6,358.7,149.1Z" fill="#4d0000"/>
+ <path d="M379.1,158c0-7.4,5-13.2,12.4-13.2a11.5,11.5,0,0,1,8.6,3.5c2.4,2.6,3.6,6.5,3.5,11.1H385c.4,4.3,2.7,7.2,6.9,7.2s4.7-1.2,5.5-3.3h5.8c-1.3,4.5-5.3,7.8-11.4,7.8C383.9,171.1,379.1,165.3,379.1,158Zm6-2.3h12.4c-.2-4.1-2.4-6.6-6-6.6S385.6,151.8,385.1,155.7Z" fill="#4d0000"/>
+ </g>
+ </g>
+</svg>
diff --git a/img/datasketches-HorizontalWhite.svg b/img/datasketches-HorizontalWhite.svg
new file mode 100755
index 0000000..69a0e4a
--- /dev/null
+++ b/img/datasketches-HorizontalWhite.svg
@@ -0,0 +1,31 @@
+<svg id="Layer_1" data-name="Layer 1" xmlns="http://www.w3.org/2000/svg" width="839" height="357" viewBox="0 0 839 357">
+ <title>ApacheDataSketch_HorizontalWhite</title>
+ <g>
+ <g>
+ <circle cx="93.7" cy="172.4" r="24.7" fill="#fff"/>
+ <path d="M199.1,81.5c16.7-22.5,45-60.4,52.8-69.6,0-.2-18.2,5.4-18.2,5.4s-29.3,41.4-48.5,74.6-34.8,60.5-73,138.1c-10.4,21-22.2,16.3-38.3,1.8S19.2,180.7,19.2,156c0-29.4,21.9-36.5,37.6-39.6,8.8-1.8,12.1,1.2,12.1,3.5s-9,5.2-16.3,3.2c-2.1-.6,2,5.1,11.3,5.1,5.9,0,17.6-2.8,17.6-8.6,0-3.5-3.3-7.8-16-7.8-9.2,0-26.2,5-35,10.3s-19.2,14.1-19.2,27.2c0,40,79.6,103.5,88.5,109.1,0-.2-13.4,38.3-13.4,59.4,0,15.3,2.4,26.2,2.9,27.3s27.7-20.6,51.6-27.8,46.8-14.1,92.8-13.9c0,0-47.4-7.3-85-23.9-40.8-18.1 [...]
+ </g>
+ <g>
+ <path d="M205.6,185.1h26.7c21.6,0,32.3,14.7,32.3,36.4s-11.3,35.9-33,35.9h-26Zm25.6,63.6c16.4,0,23.1-10.6,23.1-27.2s-6-27.8-22.5-27.8H215.6v55Z" fill="#fff"/>
+ <path d="M272.4,243.6c0-12.2,8.9-14.8,22.4-16.5,7.7-.8,12.6-2.1,12.6-7.4s-3.9-9.2-11.1-9.2-12.3,3.9-12.6,10.1H275c.4-8.8,7.1-16.7,21.2-16.7,4.9,0,9.3.9,12.6,2.8s7,7.2,7,14.2v24.4c0,4.2,1,6.4,5.5,5.5h.4v6.3a14.4,14.4,0,0,1-4.8.8c-5.6,0-8.3-2.7-9.3-8.2h-.1c-3.3,4.6-8.6,8.7-17.5,8.7S272.4,253.6,272.4,243.6Zm35.1-5.5v-8.5c-2.6,1.6-7.1,2.9-11.4,3.7-8.6,1.4-14.2,3-14.2,10.2s5.1,8.3,10.4,8.3C302.7,251.8,307.5,244.2,307.5,238.1Z" fill="#fff"/>
+ <path d="M324.9,205.2h8.2V188.8H342v16.4h10.8v7H342v32.9c0,3.8,2.1,4.9,5.4,4.9a19.4,19.4,0,0,0,4.6-.7h.3v7.6a31.7,31.7,0,0,1-7.3.9c-7-.1-11.9-2.9-11.9-10.6v-35h-8.2Z" fill="#fff"/>
+ <path d="M360.1,243.6c0-12.2,9-14.8,22.5-16.5,7.7-.8,12.6-2.1,12.6-7.4s-4-9.2-11.2-9.2-12.2,3.9-12.5,10.1h-8.8c.4-8.8,7.1-16.7,21.2-16.7,5,0,9.4.9,12.7,2.8s7,7.2,7,14.2v24.4c0,4.2.9,6.4,5.5,5.5h.4v6.3a15.1,15.1,0,0,1-4.9.8c-5.5,0-8.2-2.7-9.2-8.2h-.1c-3.3,4.6-8.6,8.7-17.5,8.7S360.1,253.6,360.1,243.6Zm35.2-5.5v-8.5c-2.6,1.6-7.2,2.9-11.5,3.7-8.5,1.4-14.1,3-14.1,10.2s5.1,8.3,10.4,8.3C390.5,251.8,395.3,244.2,395.3,238.1Z" fill="#fff"/>
+ <path d="M413.7,234h9.8c1.2,10.8,8.4,16.9,21.5,16.9,9.6,0,16.6-4.3,16.6-13.1s-8.4-11.6-21-14.4-23.9-6.5-23.9-20.4c0-12.1,10.2-19.3,25.1-19.3,17.8,0,25.7,8.9,27.5,21.3h-9.6c-1.3-8.8-7.6-13.5-17.8-13.5s-15.8,3.9-15.8,11.1,7.9,10.1,18.8,12.3c14,2.8,26.7,6.4,26.7,22.6,0,13.7-11.3,21.3-26.9,21.3C424.5,258.8,414.5,247.8,413.7,234Z" fill="#fff"/>
+ <path d="M498.6,229.9l-7.4,7v20.5h-9V185.1h9v34.3c0,1.8-.2,5.5-.3,8l22.6-22.2h11.3l-19.9,18.7,22.2,33.5H516.5Z" fill="#fff"/>
+ <path d="M529.1,231.3c0-15.1,10.1-27.4,25-27.4,7.8,0,13.8,3,17.9,8.1s6.4,12.6,6.4,21.5h-40c.7,10.2,6.2,17.9,16.3,17.9,7.2,0,12.2-3.4,13.8-8.9h8.9c-2.4,9.7-10.7,16.3-22.8,16.3C538.6,258.8,529.1,246.6,529.1,231.3Zm9.5-4.3h30.2c-.5-10.1-5.6-16.1-14.7-16.1S539.7,217.7,538.6,227Z" fill="#fff"/>
+ <path d="M583.9,205.2H592V188.8h9v16.4h10.8v7H601v32.9c0,3.8,2,4.9,5.3,4.9a18.6,18.6,0,0,0,4.6-.7h.3v7.6a31,31,0,0,1-7.3.9c-7-.1-11.9-2.9-11.9-10.6v-35h-8.1Z" fill="#fff"/>
+ <path d="M617.9,231.3c0-15.1,9.8-27.4,25.5-27.4,12.5,0,20.7,7.1,22.5,18.3h-9c-1.3-6.8-6-11-13.5-11-10.7,0-16.3,8.9-16.3,20.1s5.6,20.1,16.3,20.1c8.1,0,13-4.7,13.7-12.5H666c-1.2,12.5-9.4,19.9-22.6,19.9C627.7,258.8,617.9,246.6,617.9,231.3Z" fill="#fff"/>
+ <path d="M683.9,212.8h.3c3.9-5.6,9.1-8.9,16.7-8.9,10.9,0,17.6,6.1,17.6,17v36.5h-9V222c0-6.7-4.3-10.5-11.4-10.5s-13.9,5.7-13.9,14.1v31.8h-9.1V185.1h9.1V204C684.2,206.6,684,210.2,683.9,212.8Z" fill="#fff"/>
+ <path d="M727.8,231.3c0-15.1,10-27.4,25-27.4,7.8,0,13.8,3,17.9,8.1s6.4,12.6,6.4,21.5h-40c.6,10.2,6.2,17.9,16.3,17.9,7.2,0,12.1-3.4,13.8-8.9H776c-2.3,9.7-10.7,16.3-22.7,16.3C737.2,258.8,727.8,246.6,727.8,231.3Zm9.4-4.3h30.3c-.5-10.1-5.7-16.1-14.7-16.1S738.4,217.7,737.2,227Z" fill="#fff"/>
+ <path d="M783.4,241.5h9.2c1,7.6,6.9,10.1,14.3,10.1s11.8-3.6,11.8-8.2-4.4-7.2-13.6-9.1-19.4-4.2-19.4-15.7c0-8.9,6.9-14.6,19.1-14.6s19.6,6.1,20.8,15.2h-9c-.8-5.4-4.5-8.4-12-8.4s-10.1,3.3-10.1,7.3c0,5.5,5.1,6.5,13.9,8.3,10.4,2,19.3,4.3,19.3,16.5,0,10.6-8.7,16-21,16C792.4,258.9,784.2,252.1,783.4,241.5Z" fill="#fff"/>
+ </g>
+ <g>
+ <path d="M248.5,135.7h6.9L268,170.4h-6.9l-2.7-8.3H245.2l-2.8,8.3h-6.5Zm-1.7,21.7h10l-2.7-8.1c-1.3-4.1-2.2-8.3-2.2-8.3h0s-1.2,5-2.3,8.3Z" fill="#fff"/>
+ <path d="M271.4,145.5h5.8v3.7h.1a8.6,8.6,0,0,1,7.8-4.4c6.5,0,10.7,5.2,10.7,13.2s-4.2,13.2-10.9,13.2a8.7,8.7,0,0,1-7.6-4h-.1a47.2,47.2,0,0,1,.2,4.9v6.6h-6Zm18.4,12.6c0-4.9-2-8.5-6.1-8.5s-6.6,3.8-6.6,8.5,2.1,8.4,6.7,8.4S289.8,163,289.8,158.1Z" fill="#fff"/>
+ <path d="M315.3,167.2h-.1a9.1,9.1,0,0,1-7.9,3.8c-4.8,0-8.5-2.5-8.5-7.4s4.3-7,10.3-7.7c4.1-.5,6.1-1.2,6.1-3.4s-1.5-3.6-4.5-3.6-4.9,1.6-5.1,4H300c.2-4.2,3.6-8.1,10.7-8.1,2.8,0,5,.4,6.7,1.5s3.5,3.7,3.5,7v13.2c0,2,.4,3.1.9,3.5v.4h-5.7A6.6,6.6,0,0,1,315.3,167.2Zm.1-5.9v-3.7a15.7,15.7,0,0,1-4.9,1.7c-3.6.7-5.7,1.4-5.7,4.2s2,3.4,4.3,3.4C313.1,166.9,315.4,164.1,315.4,161.3Z" fill="#fff"/>
+ <path d="M325.1,158c0-7.4,4.9-13.2,12.6-13.2s10.3,3.6,11.2,9h-5.8a5.2,5.2,0,0,0-5.4-4.5c-4.3,0-6.6,3.6-6.6,8.7s2.3,8.5,6.6,8.5,5.2-1.9,5.6-5.2H349c-.3,5.8-4.6,9.8-11.3,9.8S325.1,165.3,325.1,158Z" fill="#fff"/>
+ <path d="M358.7,149.1h.2a8.6,8.6,0,0,1,7.7-4.3c5.3,0,8.6,3.3,8.6,8.7v16.9h-6V154.5c0-3-1.7-4.8-4.7-4.8s-5.7,2.5-5.7,6.1v14.6h-5.9V135.7h5.9V145C358.8,145.7,358.8,147.6,358.7,149.1Z" fill="#fff"/>
+ <path d="M379.1,158c0-7.4,5-13.2,12.4-13.2a11.5,11.5,0,0,1,8.6,3.5c2.4,2.6,3.6,6.5,3.5,11.1H385c.4,4.3,2.7,7.2,6.9,7.2s4.7-1.2,5.5-3.3h5.8c-1.3,4.5-5.3,7.8-11.4,7.8C383.9,171.1,379.1,165.3,379.1,158Zm6-2.3h12.4c-.2-4.1-2.4-6.6-6-6.6S385.6,151.8,385.1,155.7Z" fill="#fff"/>
+ </g>
+ </g>
+</svg>
diff --git a/img/datasketches-ManColor-3.svg b/img/datasketches-ManColor-3.svg
new file mode 100644
index 0000000..3ff3621
--- /dev/null
+++ b/img/datasketches-ManColor-3.svg
@@ -0,0 +1,27 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg width="239px" height="332px" viewBox="0 0 239 332" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
+ <!-- Generator: Sketch 52.2 (67145) - http://www.bohemiancoding.com/sketch -->
+ <title>ManColor</title>
+ <desc>Created with Sketch.</desc>
+ <defs>
+ <linearGradient x1="-111.918367%" y1="-84.4285714%" x2="285.693878%" y2="245.612245%" id="linearGradient-1">
+ <stop stop-color="#C62026" offset="0%"></stop>
+ <stop stop-color="#F9A01F" offset="0%"></stop>
+ <stop stop-color="#C62026" offset="100%"></stop>
+ </linearGradient>
+ <linearGradient x1="12.7996647%" y1="17.9758561%" x2="94.450964%" y2="66.9148346%" id="linearGradient-2">
+ <stop stop-color="#C62026" offset="0%"></stop>
+ <stop stop-color="#F9A01F" offset="0%"></stop>
+ <stop stop-color="#C82027" offset="71%"></stop>
+ <stop stop-color="#481211" offset="100%"></stop>
+ </linearGradient>
+ </defs>
+ <g id="Page-1" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
+ <g id="ApacheDataSketch_VerticalColor">
+ <g id="Group">
+ <circle id="Oval" fill="url(#linearGradient-1)" cx="81.7" cy="159.9" r="24.5"></circle>
+ <path d="M186.3,69.7 C202.8,47.3 230.9,9.8 238.6,0.7 C238.6,0.4 220.6,6 220.6,6 C220.6,6 191.5,47 172.5,80 C153.5,113 138,140 100.1,217 C89.8,237.8 78,233.2 62.1,218.7 C46.2,204.2 7.9,168.1 7.9,143.5 C7.9,114.4 29.6,107.4 45.1,104.3 C53.9,102.5 57.2,105.5 57.2,107.8 C57.2,110.1 48.2,113 41,110.9 C38.9,110.3 43,116 52.2,116 C58,116 69.6,113.3 69.6,107.4 C69.6,104 66.4,99.7 53.8,99.7 C44.6,99.7 27.8,104.7 19.1,109.9 C10.4,115.1 0,124 0,136.9 C0,176.6 78.9,239.6 87.8,245.1 C [...]
+ </g>
+ </g>
+ </g>
+</svg>
\ No newline at end of file
diff --git a/img/datasketches-ManWhite.svg b/img/datasketches-ManWhite.svg
new file mode 100644
index 0000000..a0c9bed
--- /dev/null
+++ b/img/datasketches-ManWhite.svg
@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg width="239px" height="332px" viewBox="0 0 239 332" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
+ <!-- Generator: Sketch 52.2 (67145) - http://www.bohemiancoding.com/sketch -->
+ <title>Slice 1</title>
+ <desc>Created with Sketch.</desc>
+ <g id="Page-1" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
+ <g id="datasketches-ManWhite" fill="#FFFFFF">
+ <circle id="Oval" cx="81.7" cy="159.9" r="24.5"></circle>
+ <path d="M186.3,69.7 C202.8,47.3 230.9,9.8 238.6,0.7 C238.6,0.4 220.6,6 220.6,6 C220.6,6 191.5,47 172.5,80 C153.5,113 138,140 100.1,217 C89.8,237.8 78,233.2 62.1,218.7 C46.2,204.2 7.9,168.1 7.9,143.5 C7.9,114.4 29.6,107.4 45.1,104.3 C53.9,102.5 57.2,105.5 57.2,107.8 C57.2,110.1 48.2,113 41,110.9 C38.9,110.3 43,116 52.2,116 C58,116 69.6,113.3 69.6,107.4 C69.6,104 66.4,99.7 53.8,99.7 C44.6,99.7 27.8,104.7 19.1,109.9 C10.4,115.1 0,124 0,136.9 C0,176.6 78.9,239.6 87.8,245.1 C87.8 [...]
+ </g>
+ </g>
+</svg>
\ No newline at end of file
diff --git a/img/datasketches_VerticalWhite.svg b/img/datasketches-VerticalWhite.svg
similarity index 100%
rename from img/datasketches_VerticalWhite.svg
rename to img/datasketches-VerticalWhite.svg
diff --git a/img/feather.svg b/img/feather.svg
new file mode 100644
index 0000000..2c4e274
--- /dev/null
+++ b/img/feather.svg
@@ -0,0 +1,138 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Generator: Adobe Illustrator 19.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg version="1.1" id="Feather" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+ viewBox="0 0 2392.5 4226.6" enable-background="new 0 0 2392.5 4226.6" xml:space="preserve">
+<linearGradient id="SVGID_1_" gradientUnits="userSpaceOnUse" x1="-5167.0962" y1="697.5549" x2="-4570.1162" y2="1395.619" gradientTransform="matrix(0.4226 -0.9063 0.9063 0.4226 3144.8108 -4619.2983)">
+ <stop offset="0" style="stop-color:#F69923"/>
+ <stop offset="0.3123" style="stop-color:#F79A23"/>
+ <stop offset="0.8383" style="stop-color:#E97826"/>
+</linearGradient>
+<path fill="url(#SVGID_1_)" d="M1798.9,20.1C1732.6,59.2,1622.5,170,1491,330.5l120.8,228c84.8-121.3,170.9-230.4,257.8-323.6
+ c6.7-7.4,10.2-10.9,10.2-10.9c-3.4,3.6-6.8,7.3-10.2,10.9c-28.1,31-113.4,130.5-242.1,328.1c123.9-6.2,314.3-31.5,469.6-58.1
+ c46.2-258.8-45.3-377.3-45.3-377.3S1935.5-60.6,1798.9,20.1z"/>
+<path fill="none" d="M1594.4,1320.7c0.9-0.2,1.8-0.3,2.7-0.5l-17.4,1.9c-1.1,0.5-2,1-3.1,1.4
+ C1582.6,1322.6,1588.5,1321.6,1594.4,1320.7z"/>
+<path fill="none" d="M1471.1,1729.1c-9.9,2.2-20,3.9-30.2,5.4C1451.1,1733,1461.2,1731.2,1471.1,1729.1z"/>
+<path fill="none" d="M633.1,2645.2c1.3-3.4,2.6-6.8,3.8-10.2c26.6-70.2,52.9-138.4,79-204.9c29.3-74.6,58.2-146.8,86.8-216.8
+ c30.1-73.8,59.8-145.1,89.1-214c30.7-72.3,61-141.9,90.7-208.9c24.2-54.5,48-107.3,71.5-158.4c7.8-17,15.6-33.9,23.4-50.6
+ c15.4-33.1,30.7-65.6,45.7-97.3c13.9-29.3,27.7-57.9,41.4-86c4.5-9.4,9.1-18.6,13.6-27.9c0.7-1.5,1.5-3,2.2-4.5l-14.8,1.6
+ l-11.8-23.2c-1.1,2.3-2.3,4.5-3.5,6.8c-21.2,42.1-42.2,84.6-63,127.5c-12,24.8-24,49.7-35.9,74.7c-33,69.3-65.5,139.2-97.4,209.6
+ c-32.3,71.1-63.9,142.6-94.9,214.2c-30.5,70.3-60.3,140.7-89.6,210.9c-29.2,70.1-57.7,140-85.6,209.4
+ c-29.1,72.5-57.4,144.3-84.8,215.3c-6.2,16-12.4,32-18.5,48c-22,57.3-43.4,113.8-64.3,169.6l18.6,36.7l16.6-1.8
+ c0.6-1.7,1.2-3.4,1.8-5C580.1,2786.5,606.7,2714.9,633.1,2645.2z"/>
+<path fill="none" d="M1433.2,1735.7L1433.2,1735.7c0.1,0,0.1-0.1,0.2-0.1C1433.4,1735.6,1433.3,1735.6,1433.2,1735.7z"/>
+<path fill="#BE202E" d="M1393.2,1934.8c-15.4,2.8-31.3,5.5-47.6,8.3c-0.1,0-0.2,0.1-0.3,0.1c8.2-1.2,16.3-2.4,24.3-3.8
+ C1377.6,1938,1385.4,1936.5,1393.2,1934.8z"/>
+<path opacity="0.35" fill="#BE202E" d="M1393.2,1934.8c-15.4,2.8-31.3,5.5-47.6,8.3c-0.1,0-0.2,0.1-0.3,0.1
+ c8.2-1.2,16.3-2.4,24.3-3.8C1377.6,1938,1385.4,1936.5,1393.2,1934.8z"/>
+<path fill="#BE202E" d="M1433.6,1735.5c0,0-0.1,0-0.1,0.1c-0.1,0-0.1,0.1-0.2,0.1c2.6-0.3,5.1-0.8,7.6-1.1
+ c10.3-1.5,20.4-3.3,30.2-5.4C1458.8,1731.2,1446.3,1733.4,1433.6,1735.5L1433.6,1735.5L1433.6,1735.5z"/>
+<path opacity="0.35" fill="#BE202E" d="M1433.6,1735.5c0,0-0.1,0-0.1,0.1c-0.1,0-0.1,0.1-0.2,0.1c2.6-0.3,5.1-0.8,7.6-1.1
+ c10.3-1.5,20.4-3.3,30.2-5.4C1458.8,1731.2,1446.3,1733.4,1433.6,1735.5L1433.6,1735.5L1433.6,1735.5z"/>
+<linearGradient id="SVGID_2_" gradientUnits="userSpaceOnUse" x1="-9585.3418" y1="620.5048" x2="-5326.209" y2="620.5048" gradientTransform="matrix(0.4226 -0.9063 0.9063 0.4226 3144.8108 -4619.2983)">
+ <stop offset="0.3233" style="stop-color:#9E2064"/>
+ <stop offset="0.6302" style="stop-color:#C92037"/>
+ <stop offset="0.7514" style="stop-color:#CD2335"/>
+ <stop offset="1" style="stop-color:#E97826"/>
+</linearGradient>
+<path fill="url(#SVGID_2_)" d="M1255.7,1147.6c36.7-68.6,73.9-135.7,111.5-201c39-67.8,78.5-133.6,118.4-197c2.3-3.7,4.7-7.5,7-11.3
+ c39.4-62.4,79.2-122.4,119.3-179.8l-120.8-228c-9.1,11.1-18.2,22.4-27.5,33.9c-34.8,43.4-71,90.1-108.1,139.6
+ c-41.8,55.8-84.8,115.4-128.5,177.9c-40.3,57.8-81.2,118.3-122.1,180.9c-34.8,53.3-69.8,108.2-104.5,164.5c-1.3,2.1-2.6,4.2-3.9,6.3
+ l157.2,310.5C1187.3,1277.6,1221.3,1212,1255.7,1147.6z"/>
+<linearGradient id="SVGID_3_" gradientUnits="userSpaceOnUse" x1="-9071.207" y1="1047.6898" x2="-6533.1782" y2="1047.6898" gradientTransform="matrix(0.4226 -0.9063 0.9063 0.4226 3144.8108 -4619.2983)">
+ <stop offset="0" style="stop-color:#282662"/>
+ <stop offset="9.548390e-02" style="stop-color:#662E8D"/>
+ <stop offset="0.7882" style="stop-color:#9F2064"/>
+ <stop offset="0.9487" style="stop-color:#CD2032"/>
+</linearGradient>
+<path fill="url(#SVGID_3_)" d="M539.7,2897.1c-20.8,57.2-41.7,115.4-62.7,174.9c-0.3,0.9-0.6,1.7-0.9,2.6c-3,8.4-5.9,16.8-8.9,25.2
+ c-14.1,40.1-26.4,76.2-54.5,158.3c46.3,21.1,83.5,76.7,118.7,139.8c-3.7-65.3-30.8-126.7-82.1-174.2
+ c228.3,10.3,425-47.4,526.7-214.3c9.1-14.9,17.4-30.5,24.9-47.2c-46.2,58.6-103.5,83.5-211.4,77.4c-0.2,0.1-0.5,0.2-0.7,0.3
+ c0.2-0.1,0.5-0.2,0.7-0.3c158.8-71.1,238.5-139.3,308.9-252.4c16.7-26.8,32.9-56.1,49.5-88.6C1009,2841.2,848.1,2881.8,678.6,2851
+ l-127.1,13.9C547.5,2875.6,543.6,2886.3,539.7,2897.1z"/>
+<linearGradient id="SVGID_4_" gradientUnits="userSpaceOnUse" x1="-9346.126" y1="580.817" x2="-5086.9941" y2="580.817" gradientTransform="matrix(0.4226 -0.9063 0.9063 0.4226 3144.8108 -4619.2983)">
+ <stop offset="0.3233" style="stop-color:#9E2064"/>
+ <stop offset="0.6302" style="stop-color:#C92037"/>
+ <stop offset="0.7514" style="stop-color:#CD2335"/>
+ <stop offset="1" style="stop-color:#E97826"/>
+</linearGradient>
+<path fill="url(#SVGID_4_)" d="M599,2612.4c27.5-71,55.8-142.8,84.8-215.3c27.8-69.4,56.4-139.2,85.6-209.4
+ c29.2-70.2,59.1-140.5,89.6-210.9c31-71.6,62.7-143.1,94.9-214.2c31.9-70.3,64.4-140.3,97.4-209.6c11.9-25,23.9-49.9,35.9-74.7
+ c20.8-42.9,41.8-85.4,63-127.5c1.1-2.3,2.3-4.5,3.5-6.8l-157.2-310.5c-2.6,4.2-5.1,8.4-7.7,12.6c-36.6,59.8-73.1,121-108.9,183.5
+ c-36.2,63.1-71.7,127.4-106.4,192.6c-29.3,55-57.9,110.5-85.7,166.5c-5.6,11.4-11.1,22.6-16.6,33.9
+ c-34.3,70.5-65.2,138.6-93.2,204.1c-31.7,74.2-59.6,145.1-84,212.3c-16.1,44.2-30.7,86.9-44.1,127.9c-11,35-21.5,70.1-31.4,105
+ c-23.5,82.3-43.7,164.4-60.3,246.2L516.2,2830c20.9-55.8,42.3-112.3,64.3-169.6C586.6,2644.5,592.8,2628.4,599,2612.4z"/>
+<linearGradient id="SVGID_5_" gradientUnits="userSpaceOnUse" x1="-9035.5029" y1="638.4408" x2="-6797.2012" y2="638.4408" gradientTransform="matrix(0.4226 -0.9063 0.9063 0.4226 3144.8108 -4619.2983)">
+ <stop offset="0" style="stop-color:#282662"/>
+ <stop offset="9.548390e-02" style="stop-color:#662E8D"/>
+ <stop offset="0.7882" style="stop-color:#9F2064"/>
+ <stop offset="0.9487" style="stop-color:#CD2032"/>
+</linearGradient>
+<path fill="url(#SVGID_5_)" d="M356.1,2529.2c-19.8,99.8-33.9,199.2-41,298c-0.2,3.5-0.6,6.9-0.8,10.4
+ c-49.3-79-181.3-156.1-181-155.4c94.5,137,166.2,273,176.9,406.5c-50.6,10.4-119.9-4.6-200-34.1c83.5,76.7,146.2,97.9,170.6,103.6
+ c-76.7,4.8-156.6,57.5-237.1,118.2c117.7-48,212.8-67,280.9-51.6C216.6,3530.6,108.3,3868.2,0,4226.6c33.2-9.8,53-32.1,64.1-62.3
+ c19.3-64.9,147.4-490.7,348.1-1050.4c5.7-15.9,11.5-31.9,17.3-48c1.6-4.5,3.3-9,4.9-13.4c21.2-58.7,43.2-118.6,65.9-179.7
+ c5.2-13.9,10.4-27.8,15.6-41.8c0.1-0.3,0.2-0.6,0.3-0.8l-157.8-311.8C357.7,2521.9,356.8,2525.5,356.1,2529.2z"/>
+<linearGradient id="SVGID_6_" gradientUnits="userSpaceOnUse" x1="-9346.126" y1="1021.6218" x2="-5086.9941" y2="1021.6218" gradientTransform="matrix(0.4226 -0.9063 0.9063 0.4226 3144.8108 -4619.2983)">
+ <stop offset="0.3233" style="stop-color:#9E2064"/>
+ <stop offset="0.6302" style="stop-color:#C92037"/>
+ <stop offset="0.7514" style="stop-color:#CD2335"/>
+ <stop offset="1" style="stop-color:#E97826"/>
+</linearGradient>
+<path fill="url(#SVGID_6_)" d="M1178.1,1370.3c-4.5,9.2-9,18.5-13.6,27.9c-13.6,28.1-27.4,56.7-41.4,86
+ c-15.1,31.7-30.3,64.1-45.7,97.3c-7.8,16.7-15.5,33.5-23.4,50.6c-23.5,51.1-47.3,103.9-71.5,158.4c-29.7,67-60,136.6-90.7,208.9
+ c-29.3,68.9-59,140.2-89.1,214c-28.6,70-57.5,142.3-86.8,216.8c-26.1,66.5-52.4,134.7-79,204.9c-1.3,3.4-2.6,6.8-3.8,10.2
+ c-26.4,69.7-53,141.3-79.8,214.7c-0.6,1.7-1.2,3.4-1.8,5l127.1-13.9c-2.5-0.5-5.1-0.8-7.6-1.3c152-18.9,354-132.5,484.6-272.7
+ c60.2-64.6,114.8-140.8,165.3-230c37.6-66.4,72.9-140,106.5-221.5c29.4-71.2,57.6-148.3,84.8-231.9c-34.9,18.4-74.9,31.9-119,41.3
+ c-7.7,1.6-15.6,3.2-23.6,4.6c-8,1.4-16.1,2.7-24.3,3.8l0,0l0,0c0.1,0,0.2-0.1,0.3-0.1c141.7-54.5,231.1-159.8,296.1-288.7
+ c-37.3,25.4-97.9,58.7-170.5,74.7c-9.9,2.2-20,3.9-30.2,5.4c-2.6,0.4-5.1,0.8-7.6,1.1l0,0l0,0c0.1,0,0.1-0.1,0.2-0.1
+ c0,0,0.1,0,0.1-0.1l0,0c49.2-20.6,90.7-43.6,126.7-70.8c7.7-5.8,15.2-11.8,22.4-18.1c11-9.5,21.4-19.5,31.4-30
+ c6.4-6.7,12.6-13.6,18.6-20.8c14.1-16.8,27.3-34.9,39.7-54.6c3.8-6,7.5-12.1,11.2-18.4c4.7-9.1,9.2-18,13.6-26.8
+ c19.8-39.8,35.6-75.3,48.2-106.5c6.3-15.6,11.8-30,16.5-43.4c1.9-5.3,3.7-10.5,5.4-15.5c5-15,9.1-28.3,12.3-40
+ c4.8-17.5,7.7-31.4,9.3-41.5l0,0l0,0c-4.8,3.8-10.3,7.6-16.5,11.3c-42.8,25.6-116.2,48.8-175.4,59.7l116.7-12.8l-116.7,12.8
+ c-0.9,0.2-1.8,0.3-2.7,0.5c-5.9,1-11.9,1.9-17.9,2.9c1.1-0.5,2-1,3.1-1.4l-399.3,43.8C1179.6,1367.4,1178.9,1368.8,1178.1,1370.3z"
+ />
+<linearGradient id="SVGID_7_" gradientUnits="userSpaceOnUse" x1="-9610.334" y1="999.733" x2="-5351.2017" y2="999.733" gradientTransform="matrix(0.4226 -0.9063 0.9063 0.4226 3144.8108 -4619.2983)">
+ <stop offset="0.3233" style="stop-color:#9E2064"/>
+ <stop offset="0.6302" style="stop-color:#C92037"/>
+ <stop offset="0.7514" style="stop-color:#CD2335"/>
+ <stop offset="1" style="stop-color:#E97826"/>
+</linearGradient>
+<path fill="url(#SVGID_7_)" d="M1627.6,563.1c-35.5,54.5-74.3,116.4-116,186.5c-2.2,3.6-4.4,7.4-6.6,11.1
+ c-36,60.7-74.3,127.3-114.5,200.3c-34.8,63-71,130.6-108.6,203.3c-32.8,63.3-66.7,130.5-101.5,201.6l399.3-43.8
+ c116.3-53.5,168.3-101.9,218.8-171.9c13.4-19.3,26.9-39.5,40.3-60.4c41-64,81.2-134.5,117.2-204.6c34.7-67.7,65.3-134.8,88.8-195.3
+ c14.9-38.5,26.9-74.3,35.2-105.7c7.3-27.7,13-54,17.4-79.1C1941.9,531.6,1751.5,557,1627.6,563.1z"/>
+<path fill="#BE202E" d="M1369.6,1939.4c-8,1.4-16.1,2.7-24.3,3.8l0,0C1353.5,1942.1,1361.6,1940.8,1369.6,1939.4z"/>
+<path opacity="0.35" fill="#BE202E" d="M1369.6,1939.4c-8,1.4-16.1,2.7-24.3,3.8l0,0C1353.5,1942.1,1361.6,1940.8,1369.6,1939.4z"/>
+<linearGradient id="SVGID_8_" gradientUnits="userSpaceOnUse" x1="-9346.126" y1="1152.7261" x2="-5086.9941" y2="1152.7261" gradientTransform="matrix(0.4226 -0.9063 0.9063 0.4226 3144.8108 -4619.2983)">
+ <stop offset="0.3233" style="stop-color:#9E2064"/>
+ <stop offset="0.6302" style="stop-color:#C92037"/>
+ <stop offset="0.7514" style="stop-color:#CD2335"/>
+ <stop offset="1" style="stop-color:#E97826"/>
+</linearGradient>
+<path fill="url(#SVGID_8_)" d="M1369.6,1939.4c-8,1.4-16.1,2.7-24.3,3.8l0,0C1353.5,1942.1,1361.6,1940.8,1369.6,1939.4z"/>
+<path fill="#BE202E" d="M1433.2,1735.7c2.6-0.3,5.1-0.8,7.6-1.1C1438.3,1734.9,1435.8,1735.3,1433.2,1735.7L1433.2,1735.7z"/>
+<path opacity="0.35" fill="#BE202E" d="M1433.2,1735.7c2.6-0.3,5.1-0.8,7.6-1.1C1438.3,1734.9,1435.8,1735.3,1433.2,1735.7
+ L1433.2,1735.7z"/>
+<linearGradient id="SVGID_9_" gradientUnits="userSpaceOnUse" x1="-9346.126" y1="1137.7247" x2="-5086.9941" y2="1137.7247" gradientTransform="matrix(0.4226 -0.9063 0.9063 0.4226 3144.8108 -4619.2983)">
+ <stop offset="0.3233" style="stop-color:#9E2064"/>
+ <stop offset="0.6302" style="stop-color:#C92037"/>
+ <stop offset="0.7514" style="stop-color:#CD2335"/>
+ <stop offset="1" style="stop-color:#E97826"/>
+</linearGradient>
+<path fill="url(#SVGID_9_)" d="M1433.2,1735.7c2.6-0.3,5.1-0.8,7.6-1.1C1438.3,1734.9,1435.8,1735.3,1433.2,1735.7L1433.2,1735.7z"
+ />
+<path fill="#BE202E" d="M1433.5,1735.6c0,0,0.1,0,0.1-0.1l0,0l0,0l0,0C1433.6,1735.5,1433.5,1735.5,1433.5,1735.6z"/>
+<path opacity="0.35" fill="#BE202E" d="M1433.5,1735.6c0,0,0.1,0,0.1-0.1l0,0l0,0l0,0C1433.6,1735.5,1433.5,1735.5,1433.5,1735.6z"
+ />
+<linearGradient id="SVGID_10_" gradientUnits="userSpaceOnUse" x1="-6953.4072" y1="1134.7161" x2="-6011.9995" y2="1134.7161" gradientTransform="matrix(0.4226 -0.9063 0.9063 0.4226 3144.8108 -4619.2983)">
+ <stop offset="0.3233" style="stop-color:#9E2064"/>
+ <stop offset="0.6302" style="stop-color:#C92037"/>
+ <stop offset="0.7514" style="stop-color:#CD2335"/>
+ <stop offset="1" style="stop-color:#E97826"/>
+</linearGradient>
+<path fill="url(#SVGID_10_)" d="M1433.5,1735.6c0,0,0.1,0,0.1-0.1l0,0l0,0l0,0C1433.6,1735.5,1433.5,1735.5,1433.5,1735.6z"/>
+<path fill="#6D6E71" d="M2218.7,387v11.7h27.4v77.4h12.7v-77.4h27.6V387H2218.7z M2377,387l-30.6,62.2l-30.8-62.2h-15.3v89.1h11.7
+ v-70.7l30.3,61.3h8.1l30.3-61.3v70.7h11.7V387H2377z"/>
+</svg>
diff --git a/index.md b/index.md
index 5d5574f..c1c6b37 100644
--- a/index.md
+++ b/index.md
@@ -4,6 +4,7 @@ title: DataSketches
layout: html_page
id: home
---
+<!-- Start /index.md -->
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
@@ -23,7 +24,6 @@ id: home
under the License.
-->
-<!-- Start index.md -->
<link rel="stylesheet" type="text/css" href="css/index.css">
<link rel="stylesheet" type="text/css" href="css/header.css">
@@ -37,13 +37,15 @@ id: home
<a href="https://en.wikipedia.org/wiki/Stochastic" style="color: #EDE379"><i>stochastic</i></a>
<a href="https://en.wikipedia.org/wiki/Streaming_algorithm" style="color: #EDE379"><i>streaming algorithms</i></a></p>
<!--<p class="lead" style="font-size: 16px; line-height: 1.0; margin-bottom: 15px"><i>"Excellence in theoretically informed algorithm engineering" -- Graham Cormode</i></p> -->
+ <!--
<p>
<a class="btn btn-lg btn-outline-inverse" href="overview.html"><span class="fa fa-info-circle"></span> Overview</a>
<a class="btn btn-lg btn-outline-inverse" href="/docs/Community/Downloads.html"><span class="fa fa-download"></span> Download</a>
<a class="btn btn-lg btn-outline-inverse" href="https://github.com/apache?utf8=%E2%9C%93&q=datasketches"><span class="fa fa-github"></span> GitHub</a>
<a class="btn btn-lg btn-outline-inverse" href="/docs/Community/Research.html"><span class="fa fa-paper-plane"></span> Research</a>
- <a class="btn btn-lg btn-outline-inverse" href="https://lists.apache.org/list.html?users@datasketches.apache.org"><span class="fa fa-comment"></span> Contact Us</a>
+ <a class="btn btn-lg btn-outline-inverse" href="/docs/Community/index.html" style="padding-top: 5px; padding-bottom: 0px; padding-left: 11.64px; padding-right: 12px;"><img class="ds-small-man" src="/img/datasketches-ManWhite.svg"/>Community</a>
</p>
+ -->
</div>
</div>
</div>
@@ -57,7 +59,7 @@ id: home
<p>If approximate results are acceptable, there is a class of specialized algorithms, called streaming algorithms, or <a href="/docs/SketchOrigins.html">sketches</a> that can produce results orders-of magnitude faster and with mathematically proven error bounds. For interactive queries there may not be other viable alternatives, and in the case of real-time analysis, sketches are the only known solution.</p>
-<p>For any system that needs to extract useful information from big data these sketches are a required toolkit that should be tightly integrated into their analysis capabilities. This technology has helped Yahoo successfully reduce data processing times from days to hours or minutes on a number of its internal platforms.</p>
+<p>For any system that needs to extract useful information from big data these sketches are a required toolkit that should be tightly integrated into their analysis capabilities. This technology has helped Yahoo (Verizon Media) successfully reduce data processing times from days to hours or minutes on a number of its internal platforms.</p>
<p>This site is dedicated to providing key sketch algorithms of production quality. Contributions are welcome from those in the big data community interested in further development of this science and art.</p>
</div>
@@ -82,8 +84,9 @@ id: home
</a>
<p class="text-justify">This library has been specifically designed for big data systems.
Included are adaptors for Hadoop Pig and Hive, which also can be used as examples for other systems,
- and many other capabilities typically required in big data analysis systems.
- For example, a Memory package for managing large off-heap memory data structures.</p>
+ and many other capabilities typically required in big data analysis systems, such as compatible
+ binary representations across languages (Java, C++, Python) and platforms.
+ </p>
</div>
<div class="col-md-4">
@@ -100,6 +103,4 @@ id: home
</div>
</div>
</div>
-
-
-<!-- End index.md -->
+<!-- End /index.md -->
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org
[incubator-datasketches-website] 06/06: Restructuring of ToC
Posted by le...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
leerho pushed a commit to branch Update
in repository https://gitbox.apache.org/repos/asf/incubator-datasketches-website.git
commit fa0c283a3718a556e8744d234b5162113b47881e
Author: Lee Rhodes <le...@users.noreply.github.com>
AuthorDate: Sun Feb 23 22:42:56 2020 -0800
Restructuring of ToC
---
_includes/toc.html | 408 ++++++++++-----------
.../apache/datasketches/docgen/TocGenerator.java | 3 +-
src/main/resources/docgen/toc.json | 326 ++++++++--------
3 files changed, 367 insertions(+), 370 deletions(-)
diff --git a/_includes/toc.html b/_includes/toc.html
index f7f10a2..939a5e6 100644
--- a/_includes/toc.html
+++ b/_includes/toc.html
@@ -11,252 +11,246 @@
<li><a href="{{site.docs_dir}}/SketchOrigins.html">Sketch Origins</a></li>
<li><a href="{{site.docs_dir}}/SketchElements.html">Sketch Elements</a></li>
<li><a href="{{site.docs_dir}}/LargeScale.html">Large Scale Computing</a></li>
+ <li><a href="{{site.docs_pdf_dir}}/DataSketches_deck.pdf">Overview Slide Deck</a></li>
+ <li><a href="{{site.docs_dir}}/WhoUses.html">Who Uses</a></li>
- <p id="architecture">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_architecture">Architecture</a>
+ <p id="architecture-and-design">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_architecture_and_design">Architecture And Design</a>
</p>
- <div class="collapse" id="collapse_architecture">
+ <div class="collapse" id="collapse_architecture_and_design">
<li><a href="{{site.docs_dir}}/Architecture/KeyFeatures.html">Key Features</a></li>
+ <li><a href="{{site.docs_dir}}/Architecture/FeatureMatrix.html">Sketch Feature Matrix</a></li>
<li><a href="{{site.docs_dir}}/Architecture/Components.html">Components</a></li>
<li><a href="{{site.docs_dir}}/Architecture/SketchesByComponent.html">Sketches by Component</a></li>
<li><a href="{{site.docs_dir}}/Architecture/SketchCriteria.html">Sketch Criteria</a></li>
<li><a href="{{site.docs_dir}}/Architecture/OrderSensitivity.html">Notes on Order Sensitivity</a></li>
<li><a href="{{site.docs_dir}}/Architecture/Concurrency.html">Notes on Concurrency</a></li>
</div>
- <li><a href="{{site.docs_pdf_dir}}/DataSketches_deck.pdf">Overview Slide Deck</a></li>
- </div>
-
- <p id="community">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_community">Community</a>
- </p>
- <div class="collapse" id="collapse_community">
- <li><a href="{{site.docs_dir}}/Community/index.html">Community</a></li>
</div>
- <p id="research">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_research">Research</a>
+ <p id="sketch-families">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_sketch_families">Sketch Families</a>
</p>
- <div class="collapse" id="collapse_research">
- <li><a href="{{site.docs_dir}}/Community/Research.html">Research</a></li>
- </div>
-
- <p id="distinct-counting">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_distinct_counting">Distinct Counting</a>
- </p>
- <div class="collapse" id="collapse_distinct_counting">
- <li><a href="{{site.docs_dir}}/DistinctCountCapabilitiesMatrix.html">Capabilities Matrix</a></li>
+ <div class="collapse" id="collapse_sketch_families">
- <p id="hyperloglog-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_hyperloglog_sketches">HyperLogLog Sketches</a>
+ <p id="distinct-counting">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_distinct_counting">Distinct Counting</a>
</p>
- <div class="collapse" id="collapse_hyperloglog_sketches">
- <li><a href="{{site.docs_dir}}/HLL/HLL.html">HLL Sketch</a></li>
- <li><a href="{{site.docs_dir}}/HLL/HllMap.html">HLL Map Sketch</a></li>
+ <div class="collapse" id="collapse_distinct_counting">
+ <li><a href="{{site.docs_dir}}/DistinctCountCapabilitiesMatrix.html">Capabilities Matrix</a></li>
- <p id="hll-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_hll_examples">HLL Examples</a>
+ <p id="hyperloglog-sketches">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_hyperloglog_sketches">HyperLogLog Sketches</a>
</p>
- <div class="collapse" id="collapse_hll_examples">
- <li><a href="{{site.docs_dir}}/HLL/HllJavaExample.html">HLL Sketch Java Example</a></li>
- <li><a href="{{site.docs_dir}}/HLL/HllPigUDFs.html">HLL Sketch Pig UDFs</a></li>
- <li><a href="{{site.docs_dir}}/HLL/HllHiveUDFs.html">HLL Sketch Hive UDFs</a></li>
- <li><a href="{{site.docs_dir}}/DruidIntegration.html">Using Sketches in Druid</a></li>
+ <div class="collapse" id="collapse_hyperloglog_sketches">
+ <li><a href="{{site.docs_dir}}/HLL/HLL.html">HLL Sketch</a></li>
+ <li><a href="{{site.docs_dir}}/HLL/HllMap.html">HLL Map Sketch</a></li>
+
+ <p id="hll-examples">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_hll_examples">HLL Examples</a>
+ </p>
+ <div class="collapse" id="collapse_hll_examples">
+ <li><a href="{{site.docs_dir}}/HLL/HllJavaExample.html">HLL Sketch Java Example</a></li>
+ <li><a href="{{site.docs_dir}}/HLL/HllPigUDFs.html">HLL Sketch Pig UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/HLL/HllHiveUDFs.html">HLL Sketch Hive UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/DruidIntegration.html">Using Sketches in Druid</a></li>
+ </div>
+
+ <p id="hll-studies">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_hll_studies">HLL Studies</a>
+ </p>
+ <div class="collapse" id="collapse_hll_studies">
+ <li><a href="{{site.docs_dir}}/HLL/Hll_vs_G_Hllpp.html">HLL vs Google HLL++</a></li>
+ <li><a href="{{site.docs_dir}}/HLL/Hll_vs_CS_Hllpp.html">HLL vs Clearspring HLL++</a></li>
+ <li><a href="{{site.docs_dir}}/HLL/HllSketchVsDruidHyperLogLogCollector.html">HLL Sketch vs Druid HyperLogLogCollector</a></li>
+ </div>
</div>
- <p id="hll-studies">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_hll_studies">HLL Studies</a>
+ <p id="theta-sketches">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_sketches">Theta Sketches</a>
</p>
- <div class="collapse" id="collapse_hll_studies">
- <li><a href="{{site.docs_dir}}/HLL/Hll_vs_G_Hllpp.html">HLL vs Google HLL++</a></li>
- <li><a href="{{site.docs_dir}}/HLL/Hll_vs_CS_Hllpp.html">HLL vs Clearspring HLL++</a></li>
- <li><a href="{{site.docs_dir}}/HLL/HllSketchVsDruidHyperLogLogCollector.html">HLL Sketch vs Druid HyperLogLogCollector</a></li>
+ <div class="collapse" id="collapse_theta_sketches">
+ <li><a href="{{site.docs_dir}}/Theta/ThetaSketchFramework.html">Theta Sketch Framework</a></li>
+
+ <p id="theta-examples">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_examples">Theta Examples</a>
+ </p>
+ <div class="collapse" id="collapse_theta_examples">
+ <li><a href="{{site.docs_dir}}/Theta/ConcurrentThetaSketch.html">Concurrent Theta Sketch</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaJavaExample.html">Theta Sketch Java Example</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaSparkExample.html">Theta Sketch Spark Example</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaPigUDFs.html">Theta Sketch Pig UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaHiveUDFs.html">Theta Sketch Hive UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/DruidIntegration.html">Using Sketches in Druid</a></li>
+ </div>
+
+ <p id="kmv-tutorial">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_kmv_tutorial">KMV Tutorial</a>
+ </p>
+ <div class="collapse" id="collapse_kmv_tutorial">
+ <li><a href="{{site.docs_dir}}/Theta/InverseEstimate.html">The Inverse Estimate</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/KMVempty.html">Empty Sketch</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/KMVfirstEst.html">First Estimator</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/KMVbetterEst.html">Better Estimator</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/KMVrejection.html">Rejection Rules</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/KMVupdateVkth.html">Update V(kth) Rule</a></li>
+ </div>
+
+ <p id="set-operations-and-p-sampling">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_set_operations_and_p-sampling">Set Operations and P-sampling</a>
+ </p>
+ <div class="collapse" id="collapse_set_operations_and_p-sampling">
+ <li><a href="{{site.docs_dir}}/Theta/ThetaSketchSetOps.html">Set Operations</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaPSampling.html"><i>p</i>-Sampling</a></li>
+ </div>
+
+ <p id="accuracy">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_accuracy">Accuracy</a>
+ </p>
+ <div class="collapse" id="collapse_accuracy">
+ <li><a href="{{site.docs_dir}}/Theta/ThetaAccuracy.html">Basic Accuracy</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaAccuracyPlots.html">Accuracy Plots</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaErrorTable.html">Relative Error Table</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaSketchSetOpsAccuracy.html">SetOp Accuracy</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/AccuracyOfDifferentKUnions.html">Unions With Different k</a></li>
+ </div>
+
+ <p id="size">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_size">Size</a>
+ </p>
+ <div class="collapse" id="collapse_size">
+ <li><a href="{{site.docs_dir}}/Theta/ThetaSize.html">Theta Sketch Size</a></li>
+ </div>
+
+ <p id="speed">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_speed">Speed</a>
+ </p>
+ <div class="collapse" id="collapse_speed">
+ <li><a href="{{site.docs_dir}}/Theta/ThetaUpdateSpeed.html">Update Speed</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaMergeSpeed.html">Merge Speed</a></li>
+ </div>
+
+ <p id="theta-sketch-theory">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_sketch_theory">Theta Sketch Theory</a>
+ </p>
+ <div class="collapse" id="collapse_theta_sketch_theory">
+ <li><a href="{{site.docs_pdf_dir}}/ThetaSketchFramework.pdf">Theta Sketch Framework (PDF)</a></li>
+ <li><a href="{{site.docs_pdf_dir}}/SketchEquations.pdf">Sketch Equations (PDF)</a></li>
+ <li><a href="{{site.docs_pdf_dir}}/DataSketches.pdf">DataSketches (PDF)</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaConfidenceIntervals.html">Confidence Intervals Notes</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaMergingAlgorithm.html">Merging Algorithm Notes</a></li>
+ <li><a href="{{site.docs_dir}}/Theta/ThetaReferences.html">Theta References</a></li>
+ </div>
</div>
- </div>
-
- <p id="theta-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_sketches">Theta Sketches</a>
- </p>
- <div class="collapse" id="collapse_theta_sketches">
- <li><a href="{{site.docs_dir}}/Theta/ThetaSketchFramework.html">Theta Sketch Framework</a></li>
- <p id="theta-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_examples">Theta Examples</a>
+ <p id="tuple-sketches">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_tuple_sketches">Tuple Sketches</a>
</p>
- <div class="collapse" id="collapse_theta_examples">
- <li><a href="{{site.docs_dir}}/Theta/ConcurrentThetaSketch.html">Concurrent Theta Sketch</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaJavaExample.html">Theta Sketch Java Example</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaSparkExample.html">Theta Sketch Spark Example</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaPigUDFs.html">Theta Sketch Pig UDFs</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaHiveUDFs.html">Theta Sketch Hive UDFs</a></li>
- <li><a href="{{site.docs_dir}}/DruidIntegration.html">Using Sketches in Druid</a></li>
+ <div class="collapse" id="collapse_tuple_sketches">
+ <li><a href="{{site.docs_dir}}/Tuple/TupleOverview.html">Tuple Overview</a></li>
+
+ <p id="tuple-examples">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_tuple_examples">Tuple Examples</a>
+ </p>
+ <div class="collapse" id="collapse_tuple_examples">
+ <li><a href="{{site.docs_dir}}/Tuple/TupleJavaExample.html">Tuple Java Example</a></li>
+ <li><a href="{{site.docs_dir}}/Tuple/TupleEngagementExample.html">Tuple Engagement Example</a></li>
+ <li><a href="{{site.docs_dir}}/Frequency/FrequentDistinctTuplesSketch.html">Frequent Distinct Tuples Sketch</a></li>
+ <li><a href="{{site.docs_dir}}/Tuple/TuplePigUDFs.html">Tuple Pig UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/Tuple/TupleHiveUDFs.html">Tuple Hive UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/DruidIntegration.html">Using Sketches in Druid</a></li>
+ </div>
</div>
+ </div>
- <p id="kmv-tutorial">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_kmv_tutorial">KMV Tutorial</a>
- </p>
- <div class="collapse" id="collapse_kmv_tutorial">
- <li><a href="{{site.docs_dir}}/Theta/InverseEstimate.html">The Inverse Estimate</a></li>
- <li><a href="{{site.docs_dir}}/Theta/KMVempty.html">Empty Sketch</a></li>
- <li><a href="{{site.docs_dir}}/Theta/KMVfirstEst.html">First Estimator</a></li>
- <li><a href="{{site.docs_dir}}/Theta/KMVbetterEst.html">Better Estimator</a></li>
- <li><a href="{{site.docs_dir}}/Theta/KMVrejection.html">Rejection Rules</a></li>
- <li><a href="{{site.docs_dir}}/Theta/KMVupdateVkth.html">Update V(kth) Rule</a></li>
- </div>
+ <p id="most-frequent">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_most_frequent">Most Frequent</a>
+ </p>
+ <div class="collapse" id="collapse_most_frequent">
+ <li><a href="{{site.docs_dir}}/Frequency/FrequencySketchesOverview.html">Frequency Sketches Overview</a></li>
- <p id="set-operations-and-p-sampling">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_set_operations_and_p-sampling">Set Operations and P-sampling</a>
+ <p id="frequent-item-sketches">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_frequent_item_sketches">Frequent Item Sketches</a>
</p>
- <div class="collapse" id="collapse_set_operations_and_p-sampling">
- <li><a href="{{site.docs_dir}}/Theta/ThetaSketchSetOps.html">Set Operations</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaPSampling.html"><i>p</i>-Sampling</a></li>
+ <div class="collapse" id="collapse_frequent_item_sketches">
+ <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsOverview.html">Frequent Items Overview</a></li>
+ <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsErrorTable.html">Frequent Items Error Table</a></li>
+ <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsReferences.html">Frequent Items References</a></li>
+
+ <p id="most-frequent-examples">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_most_frequent_examples">Most Frequent Examples</a>
+ </p>
+ <div class="collapse" id="collapse_most_frequent_examples">
+ <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsJavaExample.html">Frequent Items Java Example</a></li>
+ <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsPigUDFs.html">Frequent Items Pig UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsHiveUDFs.html">Frequent Items Hive UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/DruidIntegration.html">Using Sketches in Druid</a></li>
+ </div>
</div>
- <p id="accuracy">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_accuracy">Accuracy</a>
+ <p id="frequent-distinct-sketches">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_frequent_distinct_sketches">Frequent Distinct Sketches</a>
</p>
- <div class="collapse" id="collapse_accuracy">
- <li><a href="{{site.docs_dir}}/Theta/ThetaAccuracy.html">Basic Accuracy</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaAccuracyPlots.html">Accuracy Plots</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaErrorTable.html">Relative Error Table</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaSketchSetOpsAccuracy.html">SetOp Accuracy</a></li>
- <li><a href="{{site.docs_dir}}/Theta/AccuracyOfDifferentKUnions.html">Unions With Different k</a></li>
+ <div class="collapse" id="collapse_frequent_distinct_sketches">
+ <li><a href="{{site.docs_dir}}/Frequency/FrequentDistinctTuplesSketch.html">Frequent Distinct Tuples Sketch</a></li>
</div>
+ </div>
- <p id="size">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_size">Size</a>
- </p>
- <div class="collapse" id="collapse_size">
- <li><a href="{{site.docs_dir}}/Theta/ThetaSize.html">Theta Sketch Size</a></li>
- </div>
+ <p id="quantiles-and-histograms">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_and_histograms">Quantiles And Histograms</a>
+ </p>
+ <div class="collapse" id="collapse_quantiles_and_histograms">
+ <li><a href="{{site.docs_dir}}/Quantiles/QuantilesOverview.html">Quantiles Overview</a></li>
+ <li><a href="{{site.docs_dir}}/Quantiles/QuantilesAccuracy.html">Quantiles Accuracy and Size</a></li>
- <p id="speed">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_speed">Speed</a>
+ <p id="quantiles-examples">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_examples">Quantiles Examples</a>
</p>
- <div class="collapse" id="collapse_speed">
- <li><a href="{{site.docs_dir}}/Theta/ThetaUpdateSpeed.html">Update Speed</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaMergeSpeed.html">Merge Speed</a></li>
+ <div class="collapse" id="collapse_quantiles_examples">
+ <li><a href="{{site.docs_dir}}/Quantiles/QuantilesJavaExample.html">Quantiles Sketch Java Example</a></li>
+ <li><a href="{{site.docs_dir}}/Quantiles/QuantilesPigUDFs.html">Quantiles Sketch Pig UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/Quantiles/QuantilesHiveUDFs.html">Quantiles Sketch Hive UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/DruidIntegration.html">Using Sketches in Druid</a></li>
</div>
- <p id="theta-sketch-theory">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_theta_sketch_theory">Theta Sketch Theory</a>
+ <p id="quantiles-studies">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_studies">Quantiles Studies</a>
</p>
- <div class="collapse" id="collapse_theta_sketch_theory">
- <li><a href="{{site.docs_pdf_dir}}/ThetaSketchFramework.pdf">Theta Sketch Framework (PDF)</a></li>
- <li><a href="{{site.docs_pdf_dir}}/SketchEquations.pdf">Sketch Equations (PDF)</a></li>
- <li><a href="{{site.docs_pdf_dir}}/DataSketches.pdf">DataSketches (PDF)</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaConfidenceIntervals.html">Confidence Intervals Notes</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaMergingAlgorithm.html">Merging Algorithm Notes</a></li>
- <li><a href="{{site.docs_dir}}/Theta/ThetaReferences.html">Theta References</a></li>
+ <div class="collapse" id="collapse_quantiles_studies">
+ <li><a href="{{site.docs_dir}}/Quantiles/KLLSketch.html">New KLL sketch and comparison with DoublesSketch</a></li>
+ <li><a href="{{site.docs_dir}}/Quantiles/KllSketchVsTDigest.html">KLL sketch vs t-digest</a></li>
+ <li><a href="{{site.docs_dir}}/Quantiles/DruidApproxHistogramStudy.html">Druid Approximate Histogram</a></li>
+ <li><a href="{{site.docs_dir}}/Quantiles/MomentsSketchStudy.html">Moments Sketch Study</a></li>
+ <li><a href="{{site.docs_dir}}/Quantiles/QuantilesStreamAStudy.html">Quantiles StreamA Study</a></li>
</div>
- </div>
-
- <p id="tuple-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_tuple_sketches">Tuple Sketches</a>
- </p>
- <div class="collapse" id="collapse_tuple_sketches">
- <li><a href="{{site.docs_dir}}/Tuple/TupleOverview.html">Tuple Overview</a></li>
- <p id="tuple-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_tuple_examples">Tuple Examples</a>
+ <p id="quantiles-sketch-theory">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_sketch_theory">Quantiles Sketch Theory</a>
</p>
- <div class="collapse" id="collapse_tuple_examples">
- <li><a href="{{site.docs_dir}}/Tuple/TupleJavaExample.html">Tuple Java Example</a></li>
- <li><a href="{{site.docs_dir}}/Tuple/TupleEngagementExample.html">Tuple Engagement Example</a></li>
- <li><a href="{{site.docs_dir}}/Frequency/FrequentDistinctTuplesSketch.html">Frequent Distinct Tuples Sketch</a></li>
- <li><a href="{{site.docs_dir}}/Tuple/TuplePigUDFs.html">Tuple Pig UDFs</a></li>
- <li><a href="{{site.docs_dir}}/Tuple/TupleHiveUDFs.html">Tuple Hive UDFs</a></li>
- <li><a href="{{site.docs_dir}}/DruidIntegration.html">Using Sketches in Druid</a></li>
+ <div class="collapse" id="collapse_quantiles_sketch_theory">
+ <li><a href="{{site.docs_pdf_dir}}/Quantiles_KLL.pdf">Optimal Quantile Approximation in Streams</a></li>
+ <li><a href="{{site.docs_dir}}/Quantiles/QuantilesReferences.html">Quantiles References</a></li>
</div>
</div>
- </div>
- <p id="most-frequent">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_most_frequent">Most Frequent</a>
- </p>
- <div class="collapse" id="collapse_most_frequent">
- <li><a href="{{site.docs_dir}}/Frequency/FrequencySketchesOverview.html">Frequency Sketches Overview</a></li>
-
- <p id="frequent-item-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_frequent_item_sketches">Frequent Item Sketches</a>
+ <p id="sampling">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_sampling">Sampling</a>
</p>
- <div class="collapse" id="collapse_frequent_item_sketches">
- <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsOverview.html">Frequent Items Overview</a></li>
- <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsErrorTable.html">Frequent Items Error Table</a></li>
- <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsReferences.html">Frequent Items References</a></li>
+ <div class="collapse" id="collapse_sampling">
+ <li><a href="{{site.docs_dir}}/Sampling/ReservoirSampling.html">Reservoir Sampling</a></li>
+ <li><a href="{{site.docs_dir}}/Sampling/ReservoirSamplingPerformance.html">Reservoir Sampling Performance</a></li>
+ <li><a href="{{site.docs_dir}}/Sampling/VarOptSampling.html">VarOpt Sampling</a></li>
- <p id="most-frequent-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_most_frequent_examples">Most Frequent Examples</a>
+ <p id="sampling-examples">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_sampling_examples">Sampling Examples</a>
</p>
- <div class="collapse" id="collapse_most_frequent_examples">
- <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsJavaExample.html">Frequent Items Java Example</a></li>
- <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsPigUDFs.html">Frequent Items Pig UDFs</a></li>
- <li><a href="{{site.docs_dir}}/Frequency/FrequentItemsHiveUDFs.html">Frequent Items Hive UDFs</a></li>
- <li><a href="{{site.docs_dir}}/DruidIntegration.html">Using Sketches in Druid</a></li>
+ <div class="collapse" id="collapse_sampling_examples">
+ <li><a href="{{site.docs_dir}}/Sampling/ReservoirSamplingJava.html">Reservoir Sampling Java Example</a></li>
+ <li><a href="{{site.docs_dir}}/Sampling/ReservoirSamplingPigUDFs.html">Reservoir Sampling Pig UDFs</a></li>
+ <li><a href="{{site.docs_dir}}/Sampling/VarOptSamplingJava.html">VarOpt Sampling Java Example</a></li>
+ <li><a href="{{site.docs_dir}}/Sampling/VarOptPigUDFs.html">VarOpt Sampling Pig UDFs</a></li>
</div>
</div>
-
- <p id="frequent-distinct-sketches">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_frequent_distinct_sketches">Frequent Distinct Sketches</a>
- </p>
- <div class="collapse" id="collapse_frequent_distinct_sketches">
- <li><a href="{{site.docs_dir}}/Frequency/FrequentDistinctTuplesSketch.html">Frequent Distinct Tuples Sketch</a></li>
- </div>
- </div>
-
- <p id="quantiles-and-histograms">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_and_histograms">Quantiles And Histograms</a>
- </p>
- <div class="collapse" id="collapse_quantiles_and_histograms">
- <li><a href="{{site.docs_dir}}/Quantiles/QuantilesOverview.html">Quantiles Overview</a></li>
- <li><a href="{{site.docs_dir}}/Quantiles/QuantilesAccuracy.html">Quantiles Accuracy and Size</a></li>
-
- <p id="quantiles-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_examples">Quantiles Examples</a>
- </p>
- <div class="collapse" id="collapse_quantiles_examples">
- <li><a href="{{site.docs_dir}}/Quantiles/QuantilesJavaExample.html">Quantiles Sketch Java Example</a></li>
- <li><a href="{{site.docs_dir}}/Quantiles/QuantilesPigUDFs.html">Quantiles Sketch Pig UDFs</a></li>
- <li><a href="{{site.docs_dir}}/Quantiles/QuantilesHiveUDFs.html">Quantiles Sketch Hive UDFs</a></li>
- <li><a href="{{site.docs_dir}}/DruidIntegration.html">Using Sketches in Druid</a></li>
- </div>
-
- <p id="quantiles-studies">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_studies">Quantiles Studies</a>
- </p>
- <div class="collapse" id="collapse_quantiles_studies">
- <li><a href="{{site.docs_dir}}/Quantiles/KLLSketch.html">New KLL sketch and comparison with DoublesSketch</a></li>
- <li><a href="{{site.docs_dir}}/Quantiles/KllSketchVsTDigest.html">KLL sketch vs t-digest</a></li>
- <li><a href="{{site.docs_dir}}/Quantiles/DruidApproxHistogramStudy.html">Druid Approximate Histogram</a></li>
- <li><a href="{{site.docs_dir}}/Quantiles/MomentsSketchStudy.html">Moments Sketch Study</a></li>
- <li><a href="{{site.docs_dir}}/Quantiles/QuantilesStreamAStudy.html">Quantiles StreamA Study</a></li>
- </div>
-
- <p id="quantiles-sketch-theory">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_quantiles_sketch_theory">Quantiles Sketch Theory</a>
- </p>
- <div class="collapse" id="collapse_quantiles_sketch_theory">
- <li><a href="{{site.docs_pdf_dir}}/Quantiles_KLL.pdf">Optimal Quantile Approximation in Streams</a></li>
- <li><a href="{{site.docs_dir}}/Quantiles/QuantilesReferences.html">Quantiles References</a></li>
- </div>
- </div>
-
- <p id="sampling">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_sampling">Sampling</a>
- </p>
- <div class="collapse" id="collapse_sampling">
- <li><a href="{{site.docs_dir}}/Sampling/ReservoirSampling.html">Reservoir Sampling</a></li>
- <li><a href="{{site.docs_dir}}/Sampling/ReservoirSamplingPerformance.html">Reservoir Sampling Performance</a></li>
- <li><a href="{{site.docs_dir}}/Sampling/VarOptSampling.html">VarOpt Sampling</a></li>
-
- <p id="sampling-examples">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_sampling_examples">Sampling Examples</a>
- </p>
- <div class="collapse" id="collapse_sampling_examples">
- <li><a href="{{site.docs_dir}}/Sampling/ReservoirSamplingJava.html">Reservoir Sampling Java Example</a></li>
- <li><a href="{{site.docs_dir}}/Sampling/ReservoirSamplingPigUDFs.html">Reservoir Sampling Pig UDFs</a></li>
- <li><a href="{{site.docs_dir}}/Sampling/VarOptSamplingJava.html">VarOpt Sampling Java Example</a></li>
- <li><a href="{{site.docs_dir}}/Sampling/VarOptPigUDFs.html">VarOpt Sampling Pig UDFs</a></li>
- </div>
</div>
<p id="memory-package">
@@ -267,18 +261,18 @@
<li><a href="{{site.docs_dir}}/Memory/MemoryPerformance.html">Memory Package Performance</a></li>
</div>
- <p id="command-line">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_command_line">Command Line</a>
+ <p id="community">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_community">Community</a>
</p>
- <div class="collapse" id="collapse_command_line">
- <li><a href="{{site.docs_dir}}/CommandLine/CommandLine.html">Creating Command Line Executables</a></li>
+ <div class="collapse" id="collapse_community">
+ <li><a href="{{site.docs_dir}}/Community/index.html">Community</a></li>
</div>
- <p id="who-uses">
- <a data-toggle="collapse" class="menu collapsed" href="#collapse_who_uses">Who Uses</a>
+ <p id="research">
+ <a data-toggle="collapse" class="menu collapsed" href="#collapse_research">Research</a>
</p>
- <div class="collapse" id="collapse_who_uses">
- <li><a href="{{site.docs_dir}}/WhoUses.html">Who Uses</a></li>
+ <div class="collapse" id="collapse_research">
+ <li><a href="{{site.docs_dir}}/Community/Research.html">Research</a></li>
</div>
</div>
<script>
diff --git a/src/main/java/org/apache/datasketches/docgen/TocGenerator.java b/src/main/java/org/apache/datasketches/docgen/TocGenerator.java
index 2baf995..78cd4f5 100644
--- a/src/main/java/org/apache/datasketches/docgen/TocGenerator.java
+++ b/src/main/java/org/apache/datasketches/docgen/TocGenerator.java
@@ -45,7 +45,8 @@ public class TocGenerator {
/**
* To create the Table of Contents for the website:
* <ol>
- * <li>Edit the JSON source file (referenced below) for the structure you want.</li>
+ * <li>Edit the JSON source file (referenced below) for the structure you want.
+ * Be careful not to use any HTML reserved symbols! </li>
* <li>Execute this test. The result will be placed in the proper location as part of the web
* source.</li>
* <li>Stage the changes and push the web site source to origin.</li>
diff --git a/src/main/resources/docgen/toc.json b/src/main/resources/docgen/toc.json
index 6b0659e..c04d905 100644
--- a/src/main/resources/docgen/toc.json
+++ b/src/main/resources/docgen/toc.json
@@ -6,12 +6,11 @@
{"class":"Doc", "desc" : "The Major Sketch Families", "dir" : "", "file": "MajorSketchFamilies" },
{"class":"Doc", "desc" : "Sketch Origins", "dir" : "", "file": "SketchOrigins" },
{"class":"Doc", "desc" : "Sketch Elements", "dir" : "", "file": "SketchElements" },
-
{"class":"Doc", "desc" : "Large Scale Computing", "dir" : "", "file": "LargeScale" },
{"class":"Doc", "desc" : "Overview Slide Deck", "dir" : "", "file": "DataSketches_deck", "pdf":"true" },
{"class":"Doc", "desc" : "Who Uses", "dir" : "", "file": "WhoUses" },
- { "class":"Dropdown", "desc" : "Architecture & Design", "array":
+ { "class":"Dropdown", "desc" : "Architecture And Design", "array":
[
{"class":"Doc", "desc" : "Key Features", "dir" : "Architecture", "file": "KeyFeatures" },
{"class":"Doc", "desc" : "Sketch Feature Matrix", "dir" : "Architecture", "file": "FeatureMatrix" },
@@ -24,214 +23,217 @@
},
]
},
- { "class":"Dropdown", "desc" : "Community", "array":
- [
- {"class":"Doc", "desc" : "Community", "dir" : "Community", "file": "index" }
- ]
- },
- { "class":"Dropdown", "desc" : "Research", "array":
- [
- {"class":"Doc", "desc" : "Research", "dir" : "Community", "file": "Research" }
- ]
- },
- { "class":"Dropdown", "desc" : "Distinct Counting", "array":
+ { "class":"Dropdown", "desc" : "Sketch Families", "array":
[
- {"class":"Doc", "desc" : "Capabilities Matrix", "dir" : "", "file": "DistinctCountCapabilitiesMatrix" },
-
- { "class":"Dropdown", "desc" : "HyperLogLog Sketches", "array":
+ { "class":"Dropdown", "desc" : "Distinct Counting", "array":
[
- {"class":"Doc", "desc" : "HLL Sketch", "dir" : "HLL", "file": "HLL" },
- {"class":"Doc", "desc" : "HLL Map Sketch", "dir" : "HLL", "file": "HllMap" },
- { "class":"Dropdown", "desc" : "HLL Examples", "array":
- [
- {"class":"Doc", "desc" : "HLL Sketch Java Example", "dir" : "HLL", "file": "HllJavaExample" },
- {"class":"Doc", "desc" : "HLL Sketch Pig UDFs", "dir" : "HLL", "file": "HllPigUDFs" },
- {"class":"Doc", "desc" : "HLL Sketch Hive UDFs", "dir" : "HLL", "file": "HllHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- { "class":"Dropdown", "desc" : "HLL Studies", "array":
+ {"class":"Doc", "desc" : "Capabilities Matrix", "dir" : "", "file": "DistinctCountCapabilitiesMatrix" },
+
+ { "class":"Dropdown", "desc" : "HyperLogLog Sketches", "array":
[
- {"class":"Doc", "desc" : "HLL vs Google HLL++", "dir" : "HLL", "file": "Hll_vs_G_Hllpp" },
- {"class":"Doc", "desc" : "HLL vs Clearspring HLL++", "dir" : "HLL", "file": "Hll_vs_CS_Hllpp" },
- {"class":"Doc", "desc" : "HLL Sketch vs Druid HyperLogLogCollector", "dir" : "HLL", "file": "HllSketchVsDruidHyperLogLogCollector" },
+ {"class":"Doc", "desc" : "HLL Sketch", "dir" : "HLL", "file": "HLL" },
+ {"class":"Doc", "desc" : "HLL Map Sketch", "dir" : "HLL", "file": "HllMap" },
+ { "class":"Dropdown", "desc" : "HLL Examples", "array":
+ [
+ {"class":"Doc", "desc" : "HLL Sketch Java Example", "dir" : "HLL", "file": "HllJavaExample" },
+ {"class":"Doc", "desc" : "HLL Sketch Pig UDFs", "dir" : "HLL", "file": "HllPigUDFs" },
+ {"class":"Doc", "desc" : "HLL Sketch Hive UDFs", "dir" : "HLL", "file": "HllHiveUDFs" },
+ {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
+ ]
+ },
+ { "class":"Dropdown", "desc" : "HLL Studies", "array":
+ [
+ {"class":"Doc", "desc" : "HLL vs Google HLL++", "dir" : "HLL", "file": "Hll_vs_G_Hllpp" },
+ {"class":"Doc", "desc" : "HLL vs Clearspring HLL++", "dir" : "HLL", "file": "Hll_vs_CS_Hllpp" },
+ {"class":"Doc", "desc" : "HLL Sketch vs Druid HyperLogLogCollector", "dir" : "HLL", "file": "HllSketchVsDruidHyperLogLogCollector" },
+ ]
+ },
]
},
- ]
- },
-
- { "class":"Dropdown", "desc" : "Theta Sketches", "array":
- [
- { "class":"Doc", "desc" : "Theta Sketch Framework", "dir" : "Theta", "file": "ThetaSketchFramework" },
-
- { "class":"Dropdown", "desc" : "Theta Examples", "array":
+
+ { "class":"Dropdown", "desc" : "Theta Sketches", "array":
[
- {"class":"Doc", "desc" : "Concurrent Theta Sketch", "dir" : "Theta", "file": "ConcurrentThetaSketch" },
- {"class":"Doc", "desc" : "Theta Sketch Java Example", "dir" : "Theta", "file": "ThetaJavaExample" },
- {"class":"Doc", "desc" : "Theta Sketch Spark Example", "dir" : "Theta", "file": "ThetaSparkExample" },
- {"class":"Doc", "desc" : "Theta Sketch Pig UDFs", "dir" : "Theta", "file": "ThetaPigUDFs" },
- {"class":"Doc", "desc" : "Theta Sketch Hive UDFs", "dir" : "Theta", "file": "ThetaHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
+ { "class":"Doc", "desc" : "Theta Sketch Framework", "dir" : "Theta", "file": "ThetaSketchFramework" },
+
+ { "class":"Dropdown", "desc" : "Theta Examples", "array":
+ [
+ {"class":"Doc", "desc" : "Concurrent Theta Sketch", "dir" : "Theta", "file": "ConcurrentThetaSketch" },
+ {"class":"Doc", "desc" : "Theta Sketch Java Example", "dir" : "Theta", "file": "ThetaJavaExample" },
+ {"class":"Doc", "desc" : "Theta Sketch Spark Example", "dir" : "Theta", "file": "ThetaSparkExample" },
+ {"class":"Doc", "desc" : "Theta Sketch Pig UDFs", "dir" : "Theta", "file": "ThetaPigUDFs" },
+ {"class":"Doc", "desc" : "Theta Sketch Hive UDFs", "dir" : "Theta", "file": "ThetaHiveUDFs" },
+ {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
+ ]
+ },
+
+ { "class":"Dropdown", "desc" : "KMV Tutorial", "array":
+ [
+ {"class":"Doc", "desc" : "The Inverse Estimate", "dir" : "Theta", "file": "InverseEstimate" },
+ {"class":"Doc", "desc" : "Empty Sketch", "dir" : "Theta", "file": "KMVempty" },
+ {"class":"Doc", "desc" : "First Estimator", "dir" : "Theta", "file": "KMVfirstEst" },
+ {"class":"Doc", "desc" : "Better Estimator", "dir" : "Theta", "file": "KMVbetterEst" },
+ {"class":"Doc", "desc" : "Rejection Rules", "dir" : "Theta", "file": "KMVrejection" },
+ {"class":"Doc", "desc" : "Update V(kth) Rule", "dir" : "Theta", "file": "KMVupdateVkth" },
+ ]
+ },
+
+ { "class":"Dropdown", "desc" : "Set Operations and P-sampling", "array":
+ [
+ {"class":"Doc", "desc" : "Set Operations", "dir" : "Theta", "file": "ThetaSketchSetOps" },
+ {"class":"Doc", "desc" : "<i>p</i>-Sampling", "dir" : "Theta", "file": "ThetaPSampling" },
+ ]
+ },
+
+ { "class":"Dropdown", "desc" : "Accuracy", "array":
+ [
+ {"class":"Doc", "desc" : "Basic Accuracy", "dir" : "Theta", "file": "ThetaAccuracy" },
+ {"class":"Doc", "desc" : "Accuracy Plots", "dir" : "Theta", "file": "ThetaAccuracyPlots" },
+ {"class":"Doc", "desc" : "Relative Error Table", "dir" : "Theta", "file": "ThetaErrorTable" },
+ {"class":"Doc", "desc" : "SetOp Accuracy", "dir" : "Theta", "file": "ThetaSketchSetOpsAccuracy" },
+ {"class":"Doc", "desc" : "Unions With Different k", "dir" : "Theta", "file": "AccuracyOfDifferentKUnions" },
+ ]
+ },
+
+ { "class":"Dropdown", "desc" : "Size", "array":
+ [
+ {"class":"Doc", "desc" : "Theta Sketch Size", "dir" : "Theta", "file": "ThetaSize" },
+ ]
+ },
+
+ { "class":"Dropdown", "desc" : "Speed", "array":
+ [
+ {"class":"Doc", "desc" : "Update Speed", "dir" : "Theta", "file": "ThetaUpdateSpeed" },
+ {"class":"Doc", "desc" : "Merge Speed", "dir" : "Theta", "file": "ThetaMergeSpeed" },
+ ]
+ },
+
+ { "class":"Dropdown", "desc" : "Theta Sketch Theory", "array":
+ [
+ {"class":"Doc", "desc" : "Theta Sketch Framework (PDF)", "dir" : "", "file": "ThetaSketchFramework", "pdf":"true" },
+ {"class":"Doc", "desc" : "Sketch Equations (PDF)", "dir" : "", "file": "SketchEquations", "pdf":"true" },
+ {"class":"Doc", "desc" : "DataSketches (PDF)", "dir" : "", "file": "DataSketches", "pdf":"true" },
+ {"class":"Doc", "desc" : "Confidence Intervals Notes", "dir" : "Theta", "file": "ThetaConfidenceIntervals" },
+ {"class":"Doc", "desc" : "Merging Algorithm Notes", "dir" : "Theta", "file": "ThetaMergingAlgorithm" },
+ {"class":"Doc", "desc" : "Theta References", "dir" : "Theta", "file": "ThetaReferences" },
+ ]
+ }
]
},
-
- { "class":"Dropdown", "desc" : "KMV Tutorial", "array":
+
+ { "class":"Dropdown", "desc" : "Tuple Sketches", "array":
[
- {"class":"Doc", "desc" : "The Inverse Estimate", "dir" : "Theta", "file": "InverseEstimate" },
- {"class":"Doc", "desc" : "Empty Sketch", "dir" : "Theta", "file": "KMVempty" },
- {"class":"Doc", "desc" : "First Estimator", "dir" : "Theta", "file": "KMVfirstEst" },
- {"class":"Doc", "desc" : "Better Estimator", "dir" : "Theta", "file": "KMVbetterEst" },
- {"class":"Doc", "desc" : "Rejection Rules", "dir" : "Theta", "file": "KMVrejection" },
- {"class":"Doc", "desc" : "Update V(kth) Rule", "dir" : "Theta", "file": "KMVupdateVkth" },
+ {"class":"Doc", "desc" : "Tuple Overview", "dir" : "Tuple", "file": "TupleOverview" },
+ { "class":"Dropdown", "desc" : "Tuple Examples", "array":
+ [
+ {"class":"Doc", "desc" : "Tuple Java Example", "dir" : "Tuple", "file": "TupleJavaExample" },
+ {"class":"Doc", "desc" : "Tuple Engagement Example", "dir" : "Tuple", "file": "TupleEngagementExample" },
+ {"class":"Doc", "desc" : "Frequent Distinct Tuples Sketch", "dir" : "Frequency", "file": "FrequentDistinctTuplesSketch" },
+ {"class":"Doc", "desc" : "Tuple Pig UDFs", "dir" : "Tuple", "file": "TuplePigUDFs" },
+ {"class":"Doc", "desc" : "Tuple Hive UDFs", "dir" : "Tuple", "file": "TupleHiveUDFs" },
+ {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
+ ]
+ },
]
},
+ ]
+ },
- { "class":"Dropdown", "desc" : "Set Operations and P-sampling", "array":
+ { "class":"Dropdown", "desc" : "Most Frequent", "array":
+ [
+ {"class":"Doc", "desc" : "Frequency Sketches Overview", "dir" : "Frequency", "file": "FrequencySketchesOverview" },
+
+ { "class":"Dropdown", "desc" : "Frequent Item Sketches", "array":
[
- {"class":"Doc", "desc" : "Set Operations", "dir" : "Theta", "file": "ThetaSketchSetOps" },
- {"class":"Doc", "desc" : "<i>p</i>-Sampling", "dir" : "Theta", "file": "ThetaPSampling" },
+ {"class":"Doc", "desc" : "Frequent Items Overview", "dir" : "Frequency", "file": "FrequentItemsOverview" },
+ {"class":"Doc", "desc" : "Frequent Items Error Table", "dir" : "Frequency", "file": "FrequentItemsErrorTable" },
+ {"class":"Doc", "desc" : "Frequent Items References", "dir" : "Frequency", "file": "FrequentItemsReferences" },
+
+ { "class":"Dropdown", "desc" : "Most Frequent Examples", "array":
+ [
+ {"class":"Doc", "desc" : "Frequent Items Java Example", "dir" : "Frequency", "file": "FrequentItemsJavaExample" },
+ {"class":"Doc", "desc" : "Frequent Items Pig UDFs", "dir" : "Frequency", "file": "FrequentItemsPigUDFs" },
+ {"class":"Doc", "desc" : "Frequent Items Hive UDFs", "dir" : "Frequency", "file": "FrequentItemsHiveUDFs" },
+ {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
+ ]
+ },
]
},
-
- { "class":"Dropdown", "desc" : "Accuracy", "array":
+
+ { "class":"Dropdown", "desc" : "Frequent Distinct Sketches", "array":
[
- {"class":"Doc", "desc" : "Basic Accuracy", "dir" : "Theta", "file": "ThetaAccuracy" },
- {"class":"Doc", "desc" : "Accuracy Plots", "dir" : "Theta", "file": "ThetaAccuracyPlots" },
- {"class":"Doc", "desc" : "Relative Error Table", "dir" : "Theta", "file": "ThetaErrorTable" },
- {"class":"Doc", "desc" : "SetOp Accuracy", "dir" : "Theta", "file": "ThetaSketchSetOpsAccuracy" },
- {"class":"Doc", "desc" : "Unions With Different k", "dir" : "Theta", "file": "AccuracyOfDifferentKUnions" },
+ {"class":"Doc", "desc" : "Frequent Distinct Tuples Sketch", "dir" : "Frequency", "file": "FrequentDistinctTuplesSketch" },
]
- },
+ }
+ ]
+ },
- { "class":"Dropdown", "desc" : "Size", "array":
+ { "class":"Dropdown", "desc" : "Quantiles And Histograms", "array":
+ [
+ {"class":"Doc", "desc" : "Quantiles Overview", "dir" : "Quantiles", "file": "QuantilesOverview" },
+ {"class":"Doc", "desc" : "Quantiles Accuracy and Size", "dir" : "Quantiles", "file": "QuantilesAccuracy" },
+
+ { "class":"Dropdown", "desc" : "Quantiles Examples", "array":
[
- {"class":"Doc", "desc" : "Theta Sketch Size", "dir" : "Theta", "file": "ThetaSize" },
+ {"class":"Doc", "desc" : "Quantiles Sketch Java Example", "dir" : "Quantiles", "file": "QuantilesJavaExample" },
+ {"class":"Doc", "desc" : "Quantiles Sketch Pig UDFs", "dir" : "Quantiles", "file": "QuantilesPigUDFs" },
+ {"class":"Doc", "desc" : "Quantiles Sketch Hive UDFs", "dir" : "Quantiles", "file": "QuantilesHiveUDFs" },
+ {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
]
},
-
- { "class":"Dropdown", "desc" : "Speed", "array":
+ { "class":"Dropdown", "desc" : "Quantiles Studies", "array":
[
- {"class":"Doc", "desc" : "Update Speed", "dir" : "Theta", "file": "ThetaUpdateSpeed" },
- {"class":"Doc", "desc" : "Merge Speed", "dir" : "Theta", "file": "ThetaMergeSpeed" },
+ {"class":"Doc", "desc" : "New KLL sketch and comparison with DoublesSketch", "dir" : "Quantiles", "file": "KLLSketch" },
+ {"class":"Doc", "desc" : "KLL sketch vs t-digest", "dir" : "Quantiles", "file": "KllSketchVsTDigest" },
+ {"class":"Doc", "desc" : "Druid Approximate Histogram", "dir" : "Quantiles", "file": "DruidApproxHistogramStudy" },
+ {"class":"Doc", "desc" : "Moments Sketch Study", "dir" : "Quantiles", "file": "MomentsSketchStudy" },
+ {"class":"Doc", "desc" : "Quantiles StreamA Study", "dir" : "Quantiles", "file": "QuantilesStreamAStudy" },
]
},
-
- { "class":"Dropdown", "desc" : "Theta Sketch Theory", "array":
+ { "class":"Dropdown", "desc" : "Quantiles Sketch Theory", "array":
[
- {"class":"Doc", "desc" : "Theta Sketch Framework (PDF)", "dir" : "", "file": "ThetaSketchFramework", "pdf":"true" },
- {"class":"Doc", "desc" : "Sketch Equations (PDF)", "dir" : "", "file": "SketchEquations", "pdf":"true" },
- {"class":"Doc", "desc" : "DataSketches (PDF)", "dir" : "", "file": "DataSketches", "pdf":"true" },
- {"class":"Doc", "desc" : "Confidence Intervals Notes", "dir" : "Theta", "file": "ThetaConfidenceIntervals" },
- {"class":"Doc", "desc" : "Merging Algorithm Notes", "dir" : "Theta", "file": "ThetaMergingAlgorithm" },
- {"class":"Doc", "desc" : "Theta References", "dir" : "Theta", "file": "ThetaReferences" },
+ {"class":"Doc", "desc" : "Optimal Quantile Approximation in Streams", "dir" : "", "file": "Quantiles_KLL", "pdf":"true" },
+ {"class":"Doc", "desc" : "Quantiles References", "dir" : "Quantiles", "file": "QuantilesReferences" },
]
}
]
},
- { "class":"Dropdown", "desc" : "Tuple Sketches", "array":
+ { "class":"Dropdown", "desc" : "Sampling", "array":
[
- {"class":"Doc", "desc" : "Tuple Overview", "dir" : "Tuple", "file": "TupleOverview" },
- { "class":"Dropdown", "desc" : "Tuple Examples", "array":
+ {"class":"Doc", "desc" : "Reservoir Sampling", "dir" : "Sampling", "file": "ReservoirSampling" },
+ {"class":"Doc", "desc" : "Reservoir Sampling Performance", "dir" : "Sampling", "file": "ReservoirSamplingPerformance" },
+ {"class":"Doc", "desc" : "VarOpt Sampling", "dir" : "Sampling", "file": "VarOptSampling" },
+
+ { "class":"Dropdown", "desc" : "Sampling Examples", "array":
[
- {"class":"Doc", "desc" : "Tuple Java Example", "dir" : "Tuple", "file": "TupleJavaExample" },
- {"class":"Doc", "desc" : "Tuple Engagement Example", "dir" : "Tuple", "file": "TupleEngagementExample" },
- {"class":"Doc", "desc" : "Frequent Distinct Tuples Sketch", "dir" : "Frequency", "file": "FrequentDistinctTuplesSketch" },
- {"class":"Doc", "desc" : "Tuple Pig UDFs", "dir" : "Tuple", "file": "TuplePigUDFs" },
- {"class":"Doc", "desc" : "Tuple Hive UDFs", "dir" : "Tuple", "file": "TupleHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
+ {"class":"Doc", "desc" : "Reservoir Sampling Java Example", "dir" : "Sampling", "file": "ReservoirSamplingJava" },
+ {"class":"Doc", "desc" : "Reservoir Sampling Pig UDFs", "dir" : "Sampling", "file": "ReservoirSamplingPigUDFs" },
+ {"class":"Doc", "desc" : "VarOpt Sampling Java Example", "dir" : "Sampling", "file": "VarOptSamplingJava" },
+ {"class":"Doc", "desc" : "VarOpt Sampling Pig UDFs", "dir" : "Sampling", "file": "VarOptPigUDFs" },
]
},
]
},
- ]
+ ]
},
-
- { "class":"Dropdown", "desc" : "Most Frequent", "array":
- [
- {"class":"Doc", "desc" : "Frequency Sketches Overview", "dir" : "Frequency", "file": "FrequencySketchesOverview" },
-
- { "class":"Dropdown", "desc" : "Frequent Item Sketches", "array":
- [
- {"class":"Doc", "desc" : "Frequent Items Overview", "dir" : "Frequency", "file": "FrequentItemsOverview" },
- {"class":"Doc", "desc" : "Frequent Items Error Table", "dir" : "Frequency", "file": "FrequentItemsErrorTable" },
- {"class":"Doc", "desc" : "Frequent Items References", "dir" : "Frequency", "file": "FrequentItemsReferences" },
-
- { "class":"Dropdown", "desc" : "Most Frequent Examples", "array":
- [
- {"class":"Doc", "desc" : "Frequent Items Java Example", "dir" : "Frequency", "file": "FrequentItemsJavaExample" },
- {"class":"Doc", "desc" : "Frequent Items Pig UDFs", "dir" : "Frequency", "file": "FrequentItemsPigUDFs" },
- {"class":"Doc", "desc" : "Frequent Items Hive UDFs", "dir" : "Frequency", "file": "FrequentItemsHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- ]
- },
-
- { "class":"Dropdown", "desc" : "Frequent Distinct Sketches", "array":
- [
- {"class":"Doc", "desc" : "Frequent Distinct Tuples Sketch", "dir" : "Frequency", "file": "FrequentDistinctTuplesSketch" },
- ]
- }
- ]
- },
-
- { "class":"Dropdown", "desc" : "Quantiles And Histograms", "array":
+ { "class":"Dropdown", "desc" : "Memory Package", "array":
[
- {"class":"Doc", "desc" : "Quantiles Overview", "dir" : "Quantiles", "file": "QuantilesOverview" },
- {"class":"Doc", "desc" : "Quantiles Accuracy and Size", "dir" : "Quantiles", "file": "QuantilesAccuracy" },
-
- { "class":"Dropdown", "desc" : "Quantiles Examples", "array":
- [
- {"class":"Doc", "desc" : "Quantiles Sketch Java Example", "dir" : "Quantiles", "file": "QuantilesJavaExample" },
- {"class":"Doc", "desc" : "Quantiles Sketch Pig UDFs", "dir" : "Quantiles", "file": "QuantilesPigUDFs" },
- {"class":"Doc", "desc" : "Quantiles Sketch Hive UDFs", "dir" : "Quantiles", "file": "QuantilesHiveUDFs" },
- {"class":"Doc", "desc" : "Using Sketches in Druid", "dir" : "", "file": "DruidIntegration" },
- ]
- },
- { "class":"Dropdown", "desc" : "Quantiles Studies", "array":
- [
- {"class":"Doc", "desc" : "New KLL sketch and comparison with DoublesSketch", "dir" : "Quantiles", "file": "KLLSketch" },
- {"class":"Doc", "desc" : "KLL sketch vs t-digest", "dir" : "Quantiles", "file": "KllSketchVsTDigest" },
- {"class":"Doc", "desc" : "Druid Approximate Histogram", "dir" : "Quantiles", "file": "DruidApproxHistogramStudy" },
- {"class":"Doc", "desc" : "Moments Sketch Study", "dir" : "Quantiles", "file": "MomentsSketchStudy" },
- {"class":"Doc", "desc" : "Quantiles StreamA Study", "dir" : "Quantiles", "file": "QuantilesStreamAStudy" },
- ]
- },
- { "class":"Dropdown", "desc" : "Quantiles Sketch Theory", "array":
- [
- {"class":"Doc", "desc" : "Optimal Quantile Approximation in Streams", "dir" : "", "file": "Quantiles_KLL", "pdf":"true" },
- {"class":"Doc", "desc" : "Quantiles References", "dir" : "Quantiles", "file": "QuantilesReferences" },
- ]
- }
+ {"class":"Doc", "desc" : "Memory Package", "dir" : "Memory", "file": "MemoryPackage" },
+ {"class":"Doc", "desc" : "Memory Package Performance", "dir" : "Memory", "file": "MemoryPerformance" },
]
},
- { "class":"Dropdown", "desc" : "Sampling", "array":
+ { "class":"Dropdown", "desc" : "Community", "array":
[
- {"class":"Doc", "desc" : "Reservoir Sampling", "dir" : "Sampling", "file": "ReservoirSampling" },
- {"class":"Doc", "desc" : "Reservoir Sampling Performance", "dir" : "Sampling", "file": "ReservoirSamplingPerformance" },
- {"class":"Doc", "desc" : "VarOpt Sampling", "dir" : "Sampling", "file": "VarOptSampling" },
-
- { "class":"Dropdown", "desc" : "Sampling Examples", "array":
- [
- {"class":"Doc", "desc" : "Reservoir Sampling Java Example", "dir" : "Sampling", "file": "ReservoirSamplingJava" },
- {"class":"Doc", "desc" : "Reservoir Sampling Pig UDFs", "dir" : "Sampling", "file": "ReservoirSamplingPigUDFs" },
- {"class":"Doc", "desc" : "VarOpt Sampling Java Example", "dir" : "Sampling", "file": "VarOptSamplingJava" },
- {"class":"Doc", "desc" : "VarOpt Sampling Pig UDFs", "dir" : "Sampling", "file": "VarOptPigUDFs" },
- ]
- },
+ {"class":"Doc", "desc" : "Community", "dir" : "Community", "file": "index" },
]
},
-
- { "class":"Dropdown", "desc" : "Memory Package", "array":
+ { "class":"Dropdown", "desc" : "Research", "array":
[
- {"class":"Doc", "desc" : "Memory Package", "dir" : "Memory", "file": "MemoryPackage" },
- {"class":"Doc", "desc" : "Memory Package Performance", "dir" : "Memory", "file": "MemoryPerformance" },
+ {"class":"Doc", "desc" : "Research", "dir" : "Community", "file": "Research" },
]
},
-
]
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org