You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datasketches.apache.org by gi...@apache.org on 2024/01/16 18:16:26 UTC
(datasketches-python) branch gh-pages updated: deploy: 253de6ab557f5905ff94a162866e7c8760d65ff7
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/datasketches-python.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new 562325a deploy: 253de6ab557f5905ff94a162866e7c8760d65ff7
562325a is described below
commit 562325a43acfa5aa9ef6c18bf04d4c889e653363
Author: jmalkin <jm...@users.noreply.github.com>
AuthorDate: Tue Jan 16 18:16:21 2024 +0000
deploy: 253de6ab557f5905ff94a162866e7c8760d65ff7
---
docs/main/_sources/count_min_sketch.rst.txt | 1 -
docs/main/_sources/cpc.rst.txt | 12 +-
docs/main/_sources/density_sketch.rst.txt | 19 +-
docs/main/_sources/ebpps.rst.txt | 33 ++
docs/main/_sources/frequent_items.rst.txt | 69 +++-
docs/main/_sources/hyper_log_log.rst.txt | 39 ++-
docs/main/_sources/index.rst.txt | 44 ++-
docs/main/_sources/jaccard.rst.txt | 23 ++
docs/main/_sources/kernel.rst.txt | 21 ++
docs/main/_sources/kll.rst.txt | 55 ++-
docs/main/_sources/ks_test.rst.txt | 14 +
docs/main/_sources/quantiles_depr.rst.txt | 59 +++-
docs/main/_sources/req.rst.txt | 47 ++-
docs/main/_sources/serde.rst.txt | 34 ++
docs/main/_sources/theta.rst.txt | 55 ++-
docs/main/_sources/tuple.rst.txt | 64 +++-
docs/main/_sources/tuple_policy.rst.txt | 25 ++
docs/main/_sources/varopt.rst.txt | 42 +++
docs/main/count_min_sketch.html | 11 +
docs/main/cpc.html | 54 +++
docs/main/density_sketch.html | 45 ++-
docs/main/ebpps.html | 246 +++++++++++++
docs/main/frequent_items.html | 219 +++++++++++-
docs/main/genindex.html | 511 ++++++++++++++++++++++++----
docs/main/hyper_log_log.html | 173 +++++++++-
docs/main/index.html | 35 ++
docs/main/jaccard.html | 217 ++++++++++++
docs/main/kernel.html | 191 +++++++++++
docs/main/kll.html | 175 +++++++---
docs/main/ks_test.html | 213 ++++++++++++
docs/main/objects.inv | Bin 3290 -> 3873 bytes
docs/main/quantiles_depr.html | 181 +++++++---
docs/main/req.html | 127 +++++--
docs/main/search.html | 11 +
docs/main/searchindex.js | 2 +-
docs/main/serde.html | 258 ++++++++++++++
docs/main/theta.html | 224 ++++++++++++
docs/main/tuple.html | 269 +++++++++++++--
docs/main/tuple_policy.html | 235 +++++++++++++
docs/main/varopt.html | 308 +++++++++++++++++
40 files changed, 4049 insertions(+), 312 deletions(-)
diff --git a/docs/main/_sources/count_min_sketch.rst.txt b/docs/main/_sources/count_min_sketch.rst.txt
index 2f66c8d..b53b02e 100644
--- a/docs/main/_sources/count_min_sketch.rst.txt
+++ b/docs/main/_sources/count_min_sketch.rst.txt
@@ -17,7 +17,6 @@ heavy hitters.
:members:
:undoc-members:
:exclude-members: deserialize, suggest_num_buckets, suggest_num_hashes
- :member-order: groupwise
.. rubric:: Static Methods:
diff --git a/docs/main/_sources/cpc.rst.txt b/docs/main/_sources/cpc.rst.txt
index 2959b02..6a28d38 100644
--- a/docs/main/_sources/cpc.rst.txt
+++ b/docs/main/_sources/cpc.rst.txt
@@ -13,8 +13,7 @@ For additional security this sketch can be configured with a user-specified hash
.. autoclass:: _datasketches.cpc_sketch
:members:
:undoc-members:
- :exclude-members: deserialize,
- :member-order: groupwise
+ :exclude-members: deserialize
.. rubric:: Static Methods:
@@ -22,3 +21,12 @@ For additional security this sketch can be configured with a user-specified hash
.. rubric:: Non-static Methods:
+ .. automethod:: __init__
+
+
+.. autoclass:: _datasketches.cpc_union
+ :members:
+ :undoc-members:
+ :exclude-members: deserialize
+
+ .. automethod:: __init__
diff --git a/docs/main/_sources/density_sketch.rst.txt b/docs/main/_sources/density_sketch.rst.txt
index 8326ebf..07d8318 100644
--- a/docs/main/_sources/density_sketch.rst.txt
+++ b/docs/main/_sources/density_sketch.rst.txt
@@ -1,5 +1,8 @@
Density Sketch
--------------
+
+.. currentmodule:: datasketches
+
Builds a coreset from the given set of input points.
Provides density estimate at a given point.
@@ -9,9 +12,17 @@ https://proceedings.mlr.press/v99/karnin19a/karnin19a.pdf
Inspired by the following implementation: https://github.com/edoliberty/streaming-quantiles/blob/f688c8161a25582457b0a09deb4630a81406293b/gde.py
-.. autoclass:: datasketches.density_sketch
+Requires the use of a :class:`KernelFunction` to compute the distance between two vectors.
+
+.. autoclass:: density_sketch
:members:
:undoc-members:
-
-.. autoclass:: datasketches.GaussianKernel
- :members:
\ No newline at end of file
+ :exclude-members: deserialize
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
diff --git a/docs/main/_sources/ebpps.rst.txt b/docs/main/_sources/ebpps.rst.txt
new file mode 100644
index 0000000..b225320
--- /dev/null
+++ b/docs/main/_sources/ebpps.rst.txt
@@ -0,0 +1,33 @@
+Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling
+---------------------------------------------------------------------
+
+.. currentmodule:: datasketches
+
+An EBPPS sketch produces a randome sample of data from a stream of items, ensuring that the probability
+of including an item is always exactly equal to the item's size. The size of an item is defined as its
+weight relative to the total weight of all items seen so far by the sketch. In contrast to VarOpt sampling,
+this sketch may return fewer than `k` items in order to keep the probability of including an item strictly
+proportional to its size.
+
+This sketch is based on: B. Hentschel, P. J. Haas, Y. Tian
+"Exact PPS Sampling with Bounded Sample Size",
+Information Processing Letters, 2023.
+
+EBPPS sampling is related to reservoir sampling, but handles unequal item weights.
+Feeding the sketch items with a uniform weight value will produce a sample equivalent to reservoir sampling.
+
+.. note::
+ Serializing and deserializing this sketch requires the use of a :class:`PyObjectSerDe`.
+
+.. autoclass:: ebpps_sketch
+ :members:
+ :undoc-members:
+ :exclude-members: deserialize
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
diff --git a/docs/main/_sources/frequent_items.rst.txt b/docs/main/_sources/frequent_items.rst.txt
index bb580e5..76d81f5 100644
--- a/docs/main/_sources/frequent_items.rst.txt
+++ b/docs/main/_sources/frequent_items.rst.txt
@@ -1,8 +1,10 @@
Frequent Items
--------------
-This sketch is useful for tracking approximate frequencies of items of type `<T>` with optional associated counts `(<T> item, int count)`
-that are members of a multiset of such items.
+.. currentmodule:: datasketches
+
+This sketch is useful for tracking approximate frequencies of items (``object`` or ``string``) with optional associated
+integer counts that are members of a multiset of such items.
The true frequency of an item is defined to be the sum of associated counts.
This implementation provides the following capabilities:
@@ -16,38 +18,38 @@ This implementation provides the following capabilities:
**Space Usage**
-The sketch is initialized with a maximum map size, `maxMapSize`, that specifies the maximum physical length of the internal hash map of the form `(<T> item, int count)`.
-The maximum map size is always a power of 2, defined through the variables `lg_max_map_size`.
+The sketch is initialized with a maximum map size, ``maxMapSize``, that specifies the maximum physical length of the internal hash map of the form ``(object item, int count)``.
+The maximum map size is always a power of 2, defined through the variables ``lg_max_map_size``.
The hash map starts at a very small size (8 entries) and grows as needed up to the specified maximum map size.
-Excluding external space required for the item objects, the internal memory space usage of this sketch is `18 * mapSize bytes` (assuming 8 bytes for each reference),
+Excluding external space required for the item objects, the internal memory space usage of this sketch is ``18 * mapSize`` bytes (assuming 8 bytes for each reference),
plus a small constant number of additional bytes.
-The internal memory space usage of this sketch will never exceed `18 * maxMapSize` bytes, plus a small constant number of additional bytes.
+The internal memory space usage of this sketch will never exceed ``18 * maxMapSize`` bytes, plus a small constant number of additional bytes.
**Maximum Capacity of the Sketch**
-The `LOAD_FACTOR` for the hash map is internally set at :math:`75\%`, which means at any time the map capacity of `(item, count)` pairs is `mapCap = 0.75 * mapSize`.
-The maximum capacity of `(item, count)`` pairs of the sketch is `maxMapCap = 0.75 * maxMapSize`.
+The ``LOAD_FACTOR`` for the hash map is internally set at :math:`75\%`, which means at any time the map capacity of ``(item, count)`` pairs is ``mapCap = 0.75 * mapSize``.
+The maximum capacity of ``(item, count)`` pairs of the sketch is ``maxMapCap = 0.75 * maxMapSize``.
-**Updating the sketch with `(item, count)` pairs**
+**Updating the sketch with ``(item, count)`` pairs**
-If the item is found in the hash map, the mapped count field (the "counter") is incremented by the incoming count; otherwise, a new counter `"(item, count) pair"` is created.
+If the item is found in the hash map, the mapped count field (the "counter") is incremented by the incoming count; otherwise, a new counter ``(item, count)`` pair is created.
If the number of tracked counters reaches the maximum capacity of the hash map, the sketch decrements all of the counters (by an approximately computed median)
and removes any non-positive counters.
**Accuracy**
-If fewer than `0.75 * maxMapSize` different items are inserted into the sketch, the estimated frequencies returned by the sketch will be exact.
+If fewer than ``0.75 * maxMapSize`` different items are inserted into the sketch, the estimated frequencies returned by the sketch will be exact.
The logic of the frequent items sketch is such that the stored counts and true counts are never too different.
More specifically, for any item, the sketch can return an estimate of the true frequency of item, along with upper and lower bounds on the frequency (that hold deterministically).
For this implementation and for a specific active item, it is guaranteed that the true frequency will be between the Upper Bound (UB) and the Lower Bound (LB) computed for that item.
-Specifically, `(UB- LB) ≤ W * epsilon`, where :math:`W` denotes the sum of all item counts, and :math:`epsilon = 3.5/M`, where :math:`epsilon = M` is the maxMapSize.
+Specifically, ``(UB- LB) ≤ W * epsilon``, where :math:`W` denotes the sum of all item counts, and :math:`epsilon = 3.5/M`, where :math:`epsilon = M` is the maxMapSize.
This is a worst-case guarantee that applies to arbitrary inputs.
-For inputs typically seen in practice, `(UB-LB)` is usually much smaller.
+For inputs typically seen in practice, ``(UB-LB)`` is usually much smaller.
**Background**
@@ -63,12 +65,45 @@ Variants of it were discovered and rediscovered and redesigned several times ove
For speed, we do employ some randomization that introduces a small probability that our proof of the worst-case bound might not apply to a given run.
However, we have ensured that this probability is extremely small.
For example, if the stream causes one table purge (rebuild), our proof of the worst-case bound applies with a probability of at least `1 - 1E-14`.
-If the stream causes `1E9` purges, our proof applies with a probability of at least `1 - 1E-5`.
+If the stream causes ``1E9`` purges, our proof applies with a probability of at least ``1 - 1E-5``.
+
+There are two flavors of Frequent Items Sketches, one with generic items (objects) and another specific to strings.
+The string version is a legacy name from before the library supported generic objects and is retained
+only for backwards compatibility.
+
+.. note::
+ The :class:`frequent_items_sketch` uses an input object's ``__hash__`` and ``__eq__`` methods.
+
+.. note::
+ Serializing and deserializing the :class:`frequent_items_sketch` requires the use of a :class:`PyObjectSerDe`.
+
+.. autoclass:: frequent_items_error_type
+
+ .. autoattribute:: NO_FALSE_POSITIVES
+ :annotation: : Returns only true positives but may miss some heavy hitters.
+
+ .. autoattribute:: NO_FALSE_NEGATIVES
+ :annotation: : Does not miss any heavy hitters but may return false positives.
+
+
+.. autoclass:: frequent_items_sketch
+ :members:
+ :undoc-members:
+ :exclude-members: deserialize, get_epsilon_for_lg_size, get_apriori_error
+ :member-order: groupwise
-Parameter: <T> The type of item to be tracked by this sketch
+ .. rubric:: Static Methods:
+ .. automethod:: deserialize
+ .. automethod:: get_epsilon_for_lg_size
+ .. automethod:: get_apriori_error
-.. autoclass:: _datasketches.frequent_items_sketch
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
+
+
+.. autoclass:: frequent_strings_sketch
:members:
:undoc-members:
:exclude-members: deserialize, get_epsilon_for_lg_size, get_apriori_error
@@ -81,3 +116,5 @@ Parameter: <T> The type of item to be tracked by this sketch
.. automethod:: get_apriori_error
.. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
diff --git a/docs/main/_sources/hyper_log_log.rst.txt b/docs/main/_sources/hyper_log_log.rst.txt
index ec79d93..287209b 100644
--- a/docs/main/_sources/hyper_log_log.rst.txt
+++ b/docs/main/_sources/hyper_log_log.rst.txt
@@ -7,22 +7,29 @@ If the ONLY use case for sketching is counting uniques and merging, the HLL sket
This implementation offers three different types of HLL sketch, each with different trade-offs with accuracy, space and performance.
These types are specified with the target_hll_type parameter.
-In terms of accuracy, all three types, for the same lg_config_k, have the same error distribution as a function of n, the number of unique values fed to the sketch.
-The configuration parameter `lg_config_k` is the log-base-2 of `K`, where `K` is the number of buckets or slots for the sketch.
+In terms of accuracy, all three types, for the same lg_config_k, have the same error distribution as a function of ``n``, the number of unique values fed to the sketch.
+The configuration parameter ``lg_config_k`` is the log-base-2 of ``k``, where ``k`` is the number of buckets or slots for the sketch.
-During warmup, when the sketch has only received a small number of unique items (up to about 10% of `K`), this implementation leverages a new class of estimator algorithms with significantly better accuracy.
+During warmup, when the sketch has only received a small number of unique items (up to about 10% of ``k``), this implementation leverages a new class of estimator algorithms with significantly better accuracy.
+
+
+.. autoclass:: _datasketches.tgt_hll_type
+
+ .. autoattribute:: HLL_4
+ :annotation: : 4 bits per entry
+
+ .. autoattribute:: HLL_6
+ :annotation: : 6 bits per entry
+
+ .. autoattribute:: HLL_8
+ :annotation: : 8 bits per entry
-This sketch also offers the capability of operating off-heap.
-Given a WritableMemory object created by the user, the sketch will perform all of its updates and internal phase transitions in that object, which can actually reside either on-heap or off-heap based on how it is configured.
-In large systems that must update and merge many millions of sketches, having the sketch operate off-heap avoids the serialization and deserialization costs of moving sketches to and from off-heap memory-mapped files, for example, and eliminates big garbage collection delays.
.. autoclass:: _datasketches.hll_sketch
:members:
:undoc-members:
:exclude-members: deserialize, get_max_updatable_serialization_bytes, get_rel_err
- :member-order: groupwise
-
.. rubric:: Static Methods:
.. automethod:: deserialize
@@ -30,3 +37,19 @@ In large systems that must update and merge many millions of sketches, having th
.. automethod:: get_rel_err
.. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
+
+.. autoclass:: _datasketches.hll_union
+ :members:
+ :undoc-members:
+ :exclude-members: get_rel_err
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: get_rel_err
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
+
\ No newline at end of file
diff --git a/docs/main/_sources/index.rst.txt b/docs/main/_sources/index.rst.txt
index b6998c2..c65b781 100644
--- a/docs/main/_sources/index.rst.txt
+++ b/docs/main/_sources/index.rst.txt
@@ -55,7 +55,37 @@ Quantile Estimation
kll
req
quantiles_depr
-
+
+
+Random Sampling
+###############
+
+.. toctree::
+ :maxdepth: 1
+
+ varopt
+ ebpps
+
+Helper Classes
+##############
+
+.. toctree::
+ :maxdepth: 1
+
+ serde
+ kernel
+ jaccard
+ tuple_policy
+ ks_test
+
+
+These classes are required for certain sketches or specific functionality withing sketches.
+Some of them are abstract base classes, but in those cases there is at least one reference example of a
+concrete class.
+
+
+
+
.. note::
This project is under active development.
@@ -67,15 +97,3 @@ Indices and tables
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
-
-
-.. .. automodule:: datasketches
-.. :members:
-
-.. .. automodule:: _datasketches
-.. :members:
-
-..
-..
-
-.. distinct_count
diff --git a/docs/main/_sources/jaccard.rst.txt b/docs/main/_sources/jaccard.rst.txt
new file mode 100644
index 0000000..d3b9bfa
--- /dev/null
+++ b/docs/main/_sources/jaccard.rst.txt
@@ -0,0 +1,23 @@
+Jaccard Similarity
+##################
+
+.. currentmodule:: datasketches
+
+These objects provide measures related to the `Jaccard similarity <https://en.wikipedia.org/wiki/Jaccard_similarity>`_
+of :class:`theta_sketch` and :class:`tuple_sketch` objects.
+
+Note that there are separate classes to be used for theta and tuple sketches.
+
+.. autoclass:: theta_jaccard_similarity
+
+ .. automethod:: jaccard
+ .. automethod:: exactly_equal
+ .. automethod:: similarity_test
+ .. automethod:: dissimilarity_test
+
+.. autoclass:: tuple_jaccard_similarity
+
+ .. automethod:: jaccard
+ .. automethod:: exactly_equal
+ .. automethod:: similarity_test
+ .. automethod:: dissimilarity_test
diff --git a/docs/main/_sources/kernel.rst.txt b/docs/main/_sources/kernel.rst.txt
new file mode 100644
index 0000000..bdc2c72
--- /dev/null
+++ b/docs/main/_sources/kernel.rst.txt
@@ -0,0 +1,21 @@
+Kernel Function
+###############
+
+.. currentmodule:: datasketches
+
+A `kernel function <https://en.wikipedia.org/wiki/Positive-definite_kernel>`_ is a specific type of
+mathematical funciton that is particularly useful in certain machine learning and pattern recognition
+contexts. The :class:`density_sketch` performs approximate
+`kernel density estimation <https://en.wikipedia.org/wiki/Kernel_density_estimation>`_ which, unsurprisingly,
+relies on the use of such a kernel function.
+
+The library provides an abstract base class :class:`KernelFunction` and an example implementation of a
+Gaussian (also known as a Radial Basis Function) kernel. Custom classes must override the base class
+and provide a floating point value as a score indicating the similarity of two input vectors.
+
+.. autoclass:: KernelFunction
+
+ .. automethod:: __call__
+
+.. autoclass:: GaussianKernel
+ :show-inheritance:
\ No newline at end of file
diff --git a/docs/main/_sources/kll.rst.txt b/docs/main/_sources/kll.rst.txt
index 1be3dfa..0e54b44 100644
--- a/docs/main/_sources/kll.rst.txt
+++ b/docs/main/_sources/kll.rst.txt
@@ -1,5 +1,8 @@
KLL Sketch
----------
+
+.. currentmodule:: datasketches
+
Implementation of a very compact quantiles sketch with lazy compaction scheme
and nearly optimal accuracy per retained item.
See `Optimal Quantile Approximation in Streams`.
@@ -103,22 +106,66 @@ Additionally, the interval may be quite large for certain distributions.
- Let `q_hi = estimated quantile of rank (r + eps)`.
- Then `q_lo ≤ q ≤ q_hi`, with 99% confidence.
+.. note::
+ For the :class:`kll_items_sketch`, objects must be comparable with ``__lt__``.
+.. note::
+ Serializing and deserializing a :class:`kll_items_sketch` requires the use of a :class:`PyObjectSerDe`.
-.. autoclass:: _datasketches.kll_ints_sketch
+.. autoclass:: kll_ints_sketch
:members:
:undoc-members:
+ :exclude-members: deserialize, get_normalized_rank_error
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+ .. automethod:: get_normalized_rank_error
+
+ .. rubric:: Non-static Methods:
-.. autoclass:: _datasketches.kll_floats_sketch
+ .. automethod:: __init__
+
+.. autoclass:: kll_floats_sketch
:members:
:undoc-members:
+ :exclude-members: deserialize, get_normalized_rank_error
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+ .. automethod:: get_normalized_rank_error
+
+ .. rubric:: Non-static Methods:
-.. autoclass:: _datasketches.kll_doubles_sketch
+ .. automethod:: __init__
+
+.. autoclass:: kll_doubles_sketch
:members:
:undoc-members:
+ :exclude-members: deserialize, get_normalized_rank_error
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+ .. automethod:: get_normalized_rank_error
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
-.. autoclass:: _datasketches.kll_items_sketch
+.. autoclass:: kll_items_sketch
:members:
:undoc-members:
+ :exclude-members: deserialize, get_normalized_rank_error
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+ .. automethod:: get_normalized_rank_error
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
diff --git a/docs/main/_sources/ks_test.rst.txt b/docs/main/_sources/ks_test.rst.txt
new file mode 100644
index 0000000..c9bc19b
--- /dev/null
+++ b/docs/main/_sources/ks_test.rst.txt
@@ -0,0 +1,14 @@
+Kolmogorov-Smirnov Test
+#######################
+
+.. currentmodule:: datasketches
+
+A `Kolmogorov-Smirnov Test <https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test>`_` is
+a test of equality of two distributions to determine if they are likely to have come from the same
+underlying distribution.
+The DataSketches library provides a modified form of the test that takes into account the error
+in each underlying sketch in the analysis.
+
+Currently, the test assumes both input sketches are of the same family and data type.
+
+.. autofunction:: ks_test
diff --git a/docs/main/_sources/quantiles_depr.rst.txt b/docs/main/_sources/quantiles_depr.rst.txt
index a2d12ea..18dd5df 100644
--- a/docs/main/_sources/quantiles_depr.rst.txt
+++ b/docs/main/_sources/quantiles_depr.rst.txt
@@ -1,6 +1,11 @@
Quantiles Sketch (Deprecated)
-----------------------------
+
+.. currentmodule:: datasketches
+
This is a deprecated quantiles sketch that is included for cross-language compatibility.
+Most new projects will favor the KLL sketch over this one, or the REQ sketch for higher accuracy
+at the very edge of a distribution.
This is a stochastic streaming sketch that enables near-real time analysis of the
approximate distribution from a very large stream in a single pass.
@@ -38,18 +43,64 @@ For example, the median item returned from `get_quantile(0.5)` will be between t
from the hypothetically sorted array of input items at normalized ranks of 0.483 and 0.517, with
a confidence of about 99%.
-.. autoclass:: _datasketches.quantiles_ints_sketch
+.. note::
+ For the :class:`quantiles_items_sketch`, objects must be comparable with ``__lt__``.
+
+.. note::
+ Serializing and deserializing a :class:`quantiles_items_sketch` requires the use of a :class:`PyObjectSerDe`.
+
+.. autoclass:: quantiles_ints_sketch
:members:
:undoc-members:
+ :exclude-members: deserialize, get_normalized_rank_error
+
+ .. rubric:: Static Methods:
-.. autoclass:: _datasketches.quantiles_floats_sketch
+ .. automethod:: deserialize
+ .. automethod:: get_normalized_rank_error
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
+
+.. autoclass:: quantiles_floats_sketch
:members:
:undoc-members:
+ :exclude-members: deserialize, get_normalized_rank_error
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+ .. automethod:: get_normalized_rank_error
-.. autoclass:: _datasketches.quantiles_doubles_sketch
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
+
+.. autoclass:: quantiles_doubles_sketch
:members:
:undoc-members:
+ :exclude-members: deserialize, get_normalized_rank_error
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+ .. automethod:: get_normalized_rank_error
+
+ .. rubric:: Non-static Methods:
-.. autoclass:: _datasketches.quantiles_items_sketch
+ .. automethod:: __init__
+
+.. autoclass:: quantiles_items_sketch
:members:
:undoc-members:
+ :exclude-members: deserialize, get_normalized_rank_error
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+ .. automethod:: get_normalized_rank_error
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
diff --git a/docs/main/_sources/req.rst.txt b/docs/main/_sources/req.rst.txt
index 1df4d08..3f63e01 100644
--- a/docs/main/_sources/req.rst.txt
+++ b/docs/main/_sources/req.rst.txt
@@ -1,5 +1,8 @@
Relative Error Quantiles (REQ) Sketch
-------------------------------------
+
+.. currentmodule:: datasketches
+
This is an implementation based on the `paper <https://arxiv.org/abs/2004.01668>`_ "Relative Error Streaming Quantiles" by Graham Cormode, Zohar Karnin, Edo Liberty, Justin Thaler, Pavel Veselý, and loosely derived from a Python prototype written by Pavel Veselý.
This implementation differs from the algorithm described in the paper in the following:
@@ -27,14 +30,50 @@ This implementation allows the user to use both the `INCLUSIVE` criterion and th
This implementation provides extensive debug visibility into the operation of the sketch with two levels of detail output.
This is not only useful for debugging, but is a powerful tool to help users understand how the sketch works.
-.. autoclass:: _datasketches.req_ints_sketch
+.. note::
+ For the :class:`req_items_sketch`, objects must be comparable with ``__lt__``.
+
+.. note::
+ Serializing and deserializing a :class:`req_items_sketch` requires the use of a :class:`PyObjectSerDe`.
+
+.. autoclass:: req_ints_sketch
:members:
:undoc-members:
+ :exclude-members: deserialize, get_RSE
+
+ .. rubric:: Static Methods:
-.. autoclass:: _datasketches.req_floats_sketch
+ .. automethod:: deserialize
+ .. automethod:: get_RSE
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
+
+.. autoclass:: req_floats_sketch
:members:
:undoc-members:
+ :exclude-members: deserialize, get_RSE
+
+ .. rubric:: Static Methods:
-.. autoclass:: _datasketches.req_items_sketch
+ .. automethod:: deserialize
+ .. automethod:: get_RSE
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
+
+.. autoclass:: req_items_sketch
:members:
- :undoc-members:
\ No newline at end of file
+ :undoc-members:
+ :exclude-members: deserialize, get_RSE
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+ .. automethod:: get_RSE
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
diff --git a/docs/main/_sources/serde.rst.txt b/docs/main/_sources/serde.rst.txt
new file mode 100644
index 0000000..ccac1d8
--- /dev/null
+++ b/docs/main/_sources/serde.rst.txt
@@ -0,0 +1,34 @@
+Serialize/Deserialize (SerDe)
+#############################
+
+.. currentmodule:: datasketches
+
+A SerDe is a class used to serialize items sketches to a :class:`bytes` object in binary.
+Several example SerDes are provided as references.
+
+The use of binary-compatible SerDes in different languages is critical for cross-language compatibility.
+
+Each implementation must extend the :class:`PyObjectSerDe` class and override all three of its methods.
+
+.. autoclass:: PyObjectSerDe
+
+ .. automethod:: get_size
+ .. automethod:: to_bytes
+ .. automethod:: from_bytes
+
+
+The provided SerDes are:
+
+.. autoclass:: PyStringsSerDe
+ :show-inheritance:
+
+.. autoclass:: PyIntsSerDe
+ :show-inheritance:
+
+.. autoclass:: PyLongsSerDe
+ :show-inheritance:
+
+.. autoclass:: PyFloatsSerDe
+ :show-inheritance:
+
+.. autoclass:: PyDoublesSerDe
diff --git a/docs/main/_sources/theta.rst.txt b/docs/main/_sources/theta.rst.txt
index 4593f37..94951f9 100644
--- a/docs/main/_sources/theta.rst.txt
+++ b/docs/main/_sources/theta.rst.txt
@@ -1,5 +1,10 @@
Theta Sketch
------------
+
+.. currentmodule:: datasketches
+
+Theta sketches are used for distinct counting.
+
The theta package contains the basic sketch classes that are members of the `Theta Sketch Framework <https://datasketches.apache.org/docs/Theta/ThetaSketchFramework.html>`_.
There is a separate Tuple package for many of the sketches that are derived from the same algorithms defined in the Theta Sketch Framework paper.
@@ -9,6 +14,52 @@ Theta sketch is a practical variant of the K-Minimum Values sketch which avoids
hash values on every insertion to the sketch.
It has better error properties than the HyperLogLog sketch for set operations beyond the simple union.
-.. autoclass:: _datasketches.theta_sketch
+Set operations (union, intersection, A-not-B) are performed through the use of dedicated objects.
+
+Several `Jaccard similarity <https://en.wikipedia.org/wiki/Jaccard_similarity>`_
+measures can be computed between theta sketches with the :class:`theta_jaccard_similarity` class.
+
+.. autoclass:: theta_sketch
+ :members:
+ :undoc-members:
+
+.. autoclass:: update_theta_sketch
:members:
- :undoc-members:
\ No newline at end of file
+ :undoc-members:
+
+ .. automethod:: __init__
+
+
+.. autoclass:: compact_theta_sketch
+ :members:
+ :undoc-members:
+ :exclude-members: deserialize
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
+
+
+.. autoclass:: theta_union
+ :members:
+ :undoc-members:
+
+ .. automethod:: __init__
+
+
+.. autoclass:: theta_intersection
+ :members:
+ :undoc-members:
+
+ .. automethod:: __init__
+
+
+.. autoclass:: theta_a_not_b
+ :members:
+ :undoc-members:
+
+ .. automethod:: __init__
diff --git a/docs/main/_sources/tuple.rst.txt b/docs/main/_sources/tuple.rst.txt
index 4bca37f..e4a369f 100644
--- a/docs/main/_sources/tuple.rst.txt
+++ b/docs/main/_sources/tuple.rst.txt
@@ -1,18 +1,64 @@
Tuple Sketch
------------
-Tuple sketches are an extension of Theta sketch that allow the keeping of arbitrary summaries associated with each retained key
-(for example, a count for every key).
+.. currentmodule:: datasketches
-.. autoclass:: datasketches.tuple_sketch
+Tuple sketches are an extension of Theta sketches, meaning they provide estimate of distinct counts, that
+allow the keeping of arbitrary summaries associated with each retained key
+(for example, a count for every key). The use of a :class:`tuple_sketch` requires a :class:`TuplePolicy` which
+defines how summaries are created, updated, merged, or intersected. The library provides a few basic
+examples of :class:`TuplePolicy` implementations, but the right custom summary and policy can allow very
+complicated analysis to be performed quite easily.
+
+Set operations (union, intersection, A-not-B) are performed through the use of dedicated objects.
+
+Several `Jaccard similarity <https://en.wikipedia.org/wiki/Jaccard_similarity>`_
+measures can be computed between theta sketches with the :class:`tuple_jaccard_similarity` class.
+
+.. note::
+ Serializing and deserializing this sketch requires the use of a :class:`PyObjectSerDe`.
+
+.. autoclass:: tuple_sketch
+ :members:
+ :undoc-members:
+
+.. autoclass:: update_tuple_sketch
+ :members:
+ :undoc-members:
+
+ .. automethod:: __init__
+
+
+.. autoclass:: compact_tuple_sketch
+ :members:
+ :undoc-members:
+ :exclude-members: deserialize
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
+
+
+.. autoclass:: tuple_union
:members:
:undoc-members:
-.. autoclass:: datasketches.AccumulatorPolicy
- :members:
+ .. automethod:: __init__
+
-.. autoclass:: datasketches.MaxIntPolicy
- :members:
+.. autoclass:: tuple_intersection
+ :members:
+ :undoc-members:
+
+ .. automethod:: __init__
+
+
+.. autoclass:: tuple_a_not_b
+ :members:
+ :undoc-members:
-.. autoclass:: datasketches.MinIntPolicy
- :members:
\ No newline at end of file
+ .. automethod:: __init__
diff --git a/docs/main/_sources/tuple_policy.rst.txt b/docs/main/_sources/tuple_policy.rst.txt
new file mode 100644
index 0000000..3f5781f
--- /dev/null
+++ b/docs/main/_sources/tuple_policy.rst.txt
@@ -0,0 +1,25 @@
+Tuple Policy
+############
+
+.. currentmodule:: datasketches
+
+A Tuple Policy is needed when using a :class:`tuple_sketch` in order to specify how the
+summary values should be created, updated, combined, or intersected. A summary can consist of
+any python object.
+
+Each implementation must extend the abstract base class :class:`TuplePolicy`.
+
+.. autoclass:: TuplePolicy
+
+ .. automethod:: create_summary
+ .. automethod:: update_summary
+ .. automethod:: __call__
+
+.. autoclass:: AccumulatorPolicy
+ :show-inheritance:
+
+.. autoclass:: MinIntPolicy
+ :show-inheritance:
+
+.. autoclass:: MaxIntPolicy
+ :show-inheritance:
diff --git a/docs/main/_sources/varopt.rst.txt b/docs/main/_sources/varopt.rst.txt
new file mode 100644
index 0000000..d837835
--- /dev/null
+++ b/docs/main/_sources/varopt.rst.txt
@@ -0,0 +1,42 @@
+Variance Optimal Sampling (VarOpt)
+----------------------------------
+
+.. currentmodule:: datasketches
+
+A VarOpt sketch samples data from a stream of items. The sketch is desinged for optimal (minimum)
+variance when querying the sketch to estimate subset sums of items matching a provided predicate.
+The sketch will produce a sample of size `k` (or smaller if fewer items have been presented), with
+the probability of including an item roughly corresponding it the item's weight relative to the total
+weight of all items presented to the sketch.
+
+VarOpt sampling is related to reservoir sampling, with improved error bounds for subset sum estimation.
+Feeding the sketch items with a uniform weight value will produce a sample equivalent to reservoir sampling.
+
+.. note::
+ Serializing and deserializing this sketch requires the use of a :class:`PyObjectSerDe`.
+
+.. autoclass:: var_opt_sketch
+ :members:
+ :undoc-members:
+ :exclude-members: deserialize
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
+
+.. autoclass:: var_opt_union
+ :members:
+ :undoc-members:
+ :exclude-members: deserialize
+
+ .. rubric:: Static Methods:
+
+ .. automethod:: deserialize
+
+ .. rubric:: Non-static Methods:
+
+ .. automethod:: __init__
diff --git a/docs/main/count_min_sketch.html b/docs/main/count_min_sketch.html
index e9a1e55..1b59b00 100644
--- a/docs/main/count_min_sketch.html
+++ b/docs/main/count_min_sketch.html
@@ -86,6 +86,17 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
diff --git a/docs/main/cpc.html b/docs/main/cpc.html
index 9b93333..9d29680 100644
--- a/docs/main/cpc.html
+++ b/docs/main/cpc.html
@@ -49,6 +49,7 @@
<li class="toctree-l1 current"><a class="current reference internal" href="#">Compressed Probabilistic Counting (CPC)</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.cpc_sketch"><code class="docutils literal notranslate"><span class="pre">cpc_sketch</span></code></a><ul>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.cpc_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">cpc_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.cpc_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">cpc_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.cpc_sketch.get_estimate"><code class="docutils literal notranslate"><span class="pre">cpc_sketch.get_estimate</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.cpc_sketch.get_lower_bound"><code class="docutils literal notranslate"><span class="pre">cpc_sketch.get_lower_bound</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.cpc_sketch.get_upper_bound"><code class="docutils literal notranslate"><span class="pre">cpc_sketch.get_upper_bound</span></code></a></li>
@@ -59,6 +60,12 @@
<li class="toctree-l3"><a class="reference internal" href="#datasketches.cpc_sketch.update"><code class="docutils literal notranslate"><span class="pre">cpc_sketch.update</span></code></a></li>
</ul>
</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.cpc_union"><code class="docutils literal notranslate"><span class="pre">cpc_union</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.cpc_union.__init__"><code class="docutils literal notranslate"><span class="pre">cpc_union.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.cpc_union.get_result"><code class="docutils literal notranslate"><span class="pre">cpc_union.get_result</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.cpc_union.update"><code class="docutils literal notranslate"><span class="pre">cpc_union.update</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="theta.html">Theta Sketch</a></li>
@@ -77,6 +84,17 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -124,6 +142,20 @@ For additional security this sketch can be configured with a user-specified hash
</dd></dl>
<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.cpc_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lg_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class= [...]
+<dd><p>Creates a new CPC sketch</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>lg_k</strong> (<em>int</em><em>, </em><em>optional</em>) – base 2 logarithm of the number of bins in the sketch</p></li>
+<li><p><strong>seed</strong> (<em>int</em><em>, </em><em>optional</em>) – seed value for the hash function</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.cpc_sketch.get_estimate">
<span class="sig-name descname"><span class="pre">get_estimate</span></span><a class="headerlink" href="#datasketches.cpc_sketch.get_estimate" title="Link to this definition"></a></dt>
@@ -186,6 +218,28 @@ For additional security this sketch can be configured with a user-specified hash
</dd></dl>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.cpc_union">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">cpc_union</span></span><a class="headerlink" href="#datasketches.cpc_union" title="Link to this definition"></a></dt>
+<dd><dl class="py method">
+<dt class="sig sig-object py" id="datasketches.cpc_union.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lg_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">seed</span></span><span class="p"><span class= [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.cpc_union.get_result">
+<span class="sig-name descname"><span class="pre">get_result</span></span><a class="headerlink" href="#datasketches.cpc_union.get_result" title="Link to this definition"></a></dt>
+<dd><p>Returns a CPC sketch with the result of the union</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.cpc_union.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.cpc_union.update" title="Link to this definition"></a></dt>
+<dd><p>Updates the union with the provided CPC sketch</p>
+</dd></dl>
+
+</dd></dl>
+
</section>
diff --git a/docs/main/density_sketch.html b/docs/main/density_sketch.html
index 698d60d..fe30c7e 100644
--- a/docs/main/density_sketch.html
+++ b/docs/main/density_sketch.html
@@ -53,7 +53,8 @@
<ul class="current">
<li class="toctree-l1 current"><a class="current reference internal" href="#">Density Sketch</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.density_sketch"><code class="docutils literal notranslate"><span class="pre">density_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.density_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">density_sketch.deserialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.density_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">density_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.density_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">density_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.density_sketch.dim"><code class="docutils literal notranslate"><span class="pre">density_sketch.dim</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.density_sketch.get_estimate"><code class="docutils literal notranslate"><span class="pre">density_sketch.get_estimate</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.density_sketch.is_empty"><code class="docutils literal notranslate"><span class="pre">density_sketch.is_empty</span></code></a></li>
@@ -67,7 +68,6 @@
<li class="toctree-l3"><a class="reference internal" href="#datasketches.density_sketch.update"><code class="docutils literal notranslate"><span class="pre">density_sketch.update</span></code></a></li>
</ul>
</li>
-<li class="toctree-l2"><a class="reference internal" href="#datasketches.GaussianKernel"><code class="docutils literal notranslate"><span class="pre">GaussianKernel</span></code></a></li>
</ul>
</li>
</ul>
@@ -81,6 +81,17 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -115,13 +126,32 @@ Provides density estimate at a given point.</p>
“Discrepancy, Coresets, and Sketches in Machine Learning”
<a class="reference external" href="https://proceedings.mlr.press/v99/karnin19a/karnin19a.pdf">https://proceedings.mlr.press/v99/karnin19a/karnin19a.pdf</a></p>
<p>Inspired by the following implementation: <a class="reference external" href="https://github.com/edoliberty/streaming-quantiles/blob/f688c8161a25582457b0a09deb4630a81406293b/gde.py">https://github.com/edoliberty/streaming-quantiles/blob/f688c8161a25582457b0a09deb4630a81406293b/gde.py</a></p>
+<p>Requires the use of a <a class="reference internal" href="kernel.html#datasketches.KernelFunction" title="datasketches.KernelFunction"><code class="xref py py-class docutils literal notranslate"><span class="pre">KernelFunction</span></code></a> to compute the distance between two vectors.</p>
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.density_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">density_sketch</span></span><a class="headerlink" href="#datasketches.density_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.density_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.density_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">kernel</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="re [...]
+<dd><p>Reads a bytes object and returns the corresponding density_sketch</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.density_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">dim</span></span><span class="p"><span class="pre [...]
+<dd><p>Creates a new density sketch</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>k</strong> (<em>int</em>) – controls the size and error of the sketch</p></li>
+<li><p><strong>dim</strong> (<em>int</em>) – dimension of the input data</p></li>
+<li><p><strong>kernel</strong> (<a class="reference internal" href="kernel.html#datasketches.KernelFunction" title="datasketches.KernelFunction"><em>KernelFunction</em></a>) – instance of a kernel</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
<dl class="py property">
<dt class="sig sig-object py" id="datasketches.density_sketch.dim">
@@ -191,11 +221,6 @@ Provides density estimate at a given point.</p>
</dd></dl>
-<dl class="py class">
-<dt class="sig sig-object py" id="datasketches.GaussianKernel">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">GaussianKernel</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bandwidth</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span> [...]
-<dd></dd></dl>
-
</section>
diff --git a/docs/main/ebpps.html b/docs/main/ebpps.html
new file mode 100644
index 0000000..70532cf
--- /dev/null
+++ b/docs/main/ebpps.html
@@ -0,0 +1,246 @@
+<!DOCTYPE html>
+<html class="writer-html5" lang="en" data-content_root="./">
+<head>
+ <meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
+
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <title>Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling — datasketches 0.1 documentation</title>
+ <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
+ <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+
+
+ <!--[if lt IE 9]>
+ <script src="_static/js/html5shiv.min.js"></script>
+ <![endif]-->
+
+ <script src="_static/jquery.js?v=5d32c60e"></script>
+ <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
+ <script src="_static/documentation_options.js?v=2709fde1"></script>
+ <script src="_static/doctools.js?v=888ff710"></script>
+ <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+ <script src="_static/js/theme.js"></script>
+ <link rel="index" title="Index" href="genindex.html" />
+ <link rel="search" title="Search" href="search.html" />
+ <link rel="next" title="Serialize/Deserialize (SerDe)" href="serde.html" />
+ <link rel="prev" title="Variance Optimal Sampling (VarOpt)" href="varopt.html" />
+</head>
+
+<body class="wy-body-for-nav">
+ <div class="wy-grid-for-nav">
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-scroll">
+ <div class="wy-side-nav-search" >
+
+
+
+ <a href="index.html" class="icon icon-home">
+ datasketches
+ </a>
+<div role="search">
+ <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
+ <ul>
+<li class="toctree-l1"><a class="reference internal" href="hyper_log_log.html">HyperLogLog (HLL)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="cpc.html">Compressed Probabilistic Counting (CPC)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="theta.html">Theta Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple.html">Tuple Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="density_sketch.html">Density Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="count_min_sketch.html">CountMin Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="frequent_items.html">Frequent Items</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1 current"><a class="current reference internal" href="#">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.ebpps_sketch"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.c"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.c</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.get_serialized_size_bytes"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.get_serialized_size_bytes</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.is_empty"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.is_empty</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.k"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.k</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.merge"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.merge</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.n"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.n</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.serialize"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.serialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.to_string"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.to_string</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.ebpps_sketch.update"><code class="docutils literal notranslate"><span class="pre">ebpps_sketch.update</span></code></a></li>
+</ul>
+</li>
+</ul>
+</li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
+</ul>
+
+ </div>
+ </div>
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">datasketches</a>
+ </nav>
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="Page navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
+ <li class="breadcrumb-item active">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</li>
+ <li class="wy-breadcrumbs-aside">
+ <a href="_sources/ebpps.rst.txt" rel="nofollow"> View page source</a>
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+ <div itemprop="articleBody">
+
+ <section id="exact-and-bounded-probabilitiy-proportional-to-size-ebpps-sampling">
+<h1>Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling<a class="headerlink" href="#exact-and-bounded-probabilitiy-proportional-to-size-ebpps-sampling" title="Link to this heading"></a></h1>
+<p>An EBPPS sketch produces a randome sample of data from a stream of items, ensuring that the probability
+of including an item is always exactly equal to the item’s size. The size of an item is defined as its
+weight relative to the total weight of all items seen so far by the sketch. In contrast to VarOpt sampling,
+this sketch may return fewer than <cite>k</cite> items in order to keep the probability of including an item strictly
+proportional to its size.</p>
+<p>This sketch is based on: B. Hentschel, P. J. Haas, Y. Tian
+“Exact PPS Sampling with Bounded Sample Size”,
+Information Processing Letters, 2023.</p>
+<p>EBPPS sampling is related to reservoir sampling, but handles unequal item weights.
+Feeding the sketch items with a uniform weight value will produce a sample equivalent to reservoir sampling.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Serializing and deserializing this sketch requires the use of a <a class="reference internal" href="serde.html#datasketches.PyObjectSerDe" title="datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code></a>.</p>
+</div>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">ebpps_sketch</span></span><a class="headerlink" href="#datasketches.ebpps_sketch" title="Link to this definition"></a></dt>
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.deserialize">
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">serde</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="ref [...]
+<dd><p>Reads a bytes object and returns the corresponding ebpps_sketch</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <sp [...]
+<dd><p>Creates a new EBPPS sketch instance</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>k</strong> (<em>int</em>) – Maximum number of samples in the sketch</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.c">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">c</span></span><a class="headerlink" href="#datasketches.ebpps_sketch.c" title="Link to this definition"></a></dt>
+<dd><p>The expected number of samples returned upon a call to get_result() or the creation of an iterator. The number is a floating point value, where the fractional portion represents the probability of including a “partial item” from the sample. The value C should be no larger than the sketch’s configured value of k, although numerical precision limitations mean it may exceed k by double precision floating point error margins in certain cases.</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.get_serialized_size_bytes">
+<span class="sig-name descname"><span class="pre">get_serialized_size_bytes</span></span><a class="headerlink" href="#datasketches.ebpps_sketch.get_serialized_size_bytes" title="Link to this definition"></a></dt>
+<dd><p>Computes the size in bytes needed to serialize the current sketch</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.is_empty">
+<span class="sig-name descname"><span class="pre">is_empty</span></span><a class="headerlink" href="#datasketches.ebpps_sketch.is_empty" title="Link to this definition"></a></dt>
+<dd><p>Returns True if the sketch is empty, otherwise False</p>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.k">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">k</span></span><a class="headerlink" href="#datasketches.ebpps_sketch.k" title="Link to this definition"></a></dt>
+<dd><p>The sketch’s maximum configured sample size</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.merge">
+<span class="sig-name descname"><span class="pre">merge</span></span><a class="headerlink" href="#datasketches.ebpps_sketch.merge" title="Link to this definition"></a></dt>
+<dd><p>Merges the sketch with the given sketch</p>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.n">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">n</span></span><a class="headerlink" href="#datasketches.ebpps_sketch.n" title="Link to this definition"></a></dt>
+<dd><p>The total stream length</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.serialize">
+<span class="sig-name descname"><span class="pre">serialize</span></span><a class="headerlink" href="#datasketches.ebpps_sketch.serialize" title="Link to this definition"></a></dt>
+<dd><p>Serializes the sketch into a bytes object</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.to_string">
+<span class="sig-name descname"><span class="pre">to_string</span></span><a class="headerlink" href="#datasketches.ebpps_sketch.to_string" title="Link to this definition"></a></dt>
+<dd><p>Produces a string summary of the sketch and optionally prints the items</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.ebpps_sketch.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.ebpps_sketch.update" title="Link to this definition"></a></dt>
+<dd><p>Updates the sketch with the given value and weight</p>
+</dd></dl>
+
+</dd></dl>
+
+</section>
+
+
+ </div>
+ </div>
+ <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
+ <a href="varopt.html" class="btn btn-neutral float-left" title="Variance Optimal Sampling (VarOpt)" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+ <a href="serde.html" class="btn btn-neutral float-right" title="Serialize/Deserialize (SerDe)" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
+ </div>
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>© Copyright 2023.</p>
+ </div>
+
+ Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
+ <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
+ provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+
+</footer>
+ </div>
+ </div>
+ </section>
+ </div>
+ <script>
+ jQuery(function () {
+ SphinxRtdTheme.Navigation.enable(true);
+ });
+ </script>
+
+</body>
+</html>
\ No newline at end of file
diff --git a/docs/main/frequent_items.html b/docs/main/frequent_items.html
index 1758583..75be0b8 100644
--- a/docs/main/frequent_items.html
+++ b/docs/main/frequent_items.html
@@ -59,10 +59,16 @@
</ul>
<ul class="current">
<li class="toctree-l1 current"><a class="current reference internal" href="#">Frequent Items</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.frequent_items_error_type"><code class="docutils literal notranslate"><span class="pre">frequent_items_error_type</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_items_error_type.NO_FALSE_POSITIVES"><code class="docutils literal notranslate"><span class="pre">frequent_items_error_type.NO_FALSE_POSITIVES</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_items_error_type.NO_FALSE_NEGATIVES"><code class="docutils literal notranslate"><span class="pre">frequent_items_error_type.NO_FALSE_NEGATIVES</span></code></a></li>
+</ul>
+</li>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.frequent_items_sketch"><code class="docutils literal notranslate"><span class="pre">frequent_items_sketch</span></code></a><ul>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_items_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">frequent_items_sketch.deserialize()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_items_sketch.get_epsilon_for_lg_size"><code class="docutils literal notranslate"><span class="pre">frequent_items_sketch.get_epsilon_for_lg_size()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_items_sketch.get_apriori_error"><code class="docutils literal notranslate"><span class="pre">frequent_items_sketch.get_apriori_error()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_items_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">frequent_items_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_items_sketch.epsilon"><code class="docutils literal notranslate"><span class="pre">frequent_items_sketch.epsilon</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_items_sketch.get_estimate"><code class="docutils literal notranslate"><span class="pre">frequent_items_sketch.get_estimate</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_items_sketch.get_frequent_items"><code class="docutils literal notranslate"><span class="pre">frequent_items_sketch.get_frequent_items</span></code></a></li>
@@ -78,6 +84,26 @@
<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_items_sketch.update"><code class="docutils literal notranslate"><span class="pre">frequent_items_sketch.update</span></code></a></li>
</ul>
</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.frequent_strings_sketch"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.get_epsilon_for_lg_size"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.get_epsilon_for_lg_size()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.get_apriori_error"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.get_apriori_error()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.epsilon"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.epsilon</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.get_estimate"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.get_estimate</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.get_frequent_items"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.get_frequent_items</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.get_lower_bound"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.get_lower_bound</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.get_serialized_size_bytes"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.get_serialized_size_bytes</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.get_upper_bound"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.get_upper_bound</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.is_empty"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.is_empty</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.merge"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.merge</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.num_active_items"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.num_active_items</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.serialize"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.serialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.to_string"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.to_string</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.total_weight"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.total_weight</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.frequent_strings_sketch.update"><code class="docutils literal notranslate"><span class="pre">frequent_strings_sketch.update</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
</ul>
@@ -85,6 +111,17 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -113,8 +150,8 @@
<section id="frequent-items">
<h1>Frequent Items<a class="headerlink" href="#frequent-items" title="Link to this heading"></a></h1>
-<p>This sketch is useful for tracking approximate frequencies of items of type <cite><T></cite> with optional associated counts <cite>(<T> item, int count)</cite>
-that are members of a multiset of such items.
+<p>This sketch is useful for tracking approximate frequencies of items (<code class="docutils literal notranslate"><span class="pre">object</span></code> or <code class="docutils literal notranslate"><span class="pre">string</span></code>) with optional associated
+integer counts that are members of a multiset of such items.
The true frequency of an item is defined to be the sum of associated counts.</p>
<p>This implementation provides the following capabilities:</p>
<ul class="simple">
@@ -126,27 +163,27 @@ The true frequency of an item is defined to be the sum of associated counts.</p>
<li><p><em>Serialize/Deserialize</em> to/from a byte array.</p></li>
</ul>
<p><strong>Space Usage</strong></p>
-<p>The sketch is initialized with a maximum map size, <cite>maxMapSize</cite>, that specifies the maximum physical length of the internal hash map of the form <cite>(<T> item, int count)</cite>.
-The maximum map size is always a power of 2, defined through the variables <cite>lg_max_map_size</cite>.</p>
+<p>The sketch is initialized with a maximum map size, <code class="docutils literal notranslate"><span class="pre">maxMapSize</span></code>, that specifies the maximum physical length of the internal hash map of the form <code class="docutils literal notranslate"><span class="pre">(object</span> <span class="pre">item,</span> <span class="pre">int</span> <span class="pre">count)</span></code>.
+The maximum map size is always a power of 2, defined through the variables <code class="docutils literal notranslate"><span class="pre">lg_max_map_size</span></code>.</p>
<p>The hash map starts at a very small size (8 entries) and grows as needed up to the specified maximum map size.</p>
-<p>Excluding external space required for the item objects, the internal memory space usage of this sketch is <cite>18 * mapSize bytes</cite> (assuming 8 bytes for each reference),
+<p>Excluding external space required for the item objects, the internal memory space usage of this sketch is <code class="docutils literal notranslate"><span class="pre">18</span> <span class="pre">*</span> <span class="pre">mapSize</span></code> bytes (assuming 8 bytes for each reference),
plus a small constant number of additional bytes.
-The internal memory space usage of this sketch will never exceed <cite>18 * maxMapSize</cite> bytes, plus a small constant number of additional bytes.</p>
+The internal memory space usage of this sketch will never exceed <code class="docutils literal notranslate"><span class="pre">18</span> <span class="pre">*</span> <span class="pre">maxMapSize</span></code> bytes, plus a small constant number of additional bytes.</p>
<p><strong>Maximum Capacity of the Sketch</strong></p>
-<p>The <cite>LOAD_FACTOR</cite> for the hash map is internally set at <span class="math notranslate nohighlight">\(75\%\)</span>, which means at any time the map capacity of <cite>(item, count)</cite> pairs is <cite>mapCap = 0.75 * mapSize</cite>.
-The maximum capacity of <cite>(item, count)`</cite> pairs of the sketch is <cite>maxMapCap = 0.75 * maxMapSize</cite>.</p>
-<p><strong>Updating the sketch with `(item, count)` pairs</strong></p>
-<p>If the item is found in the hash map, the mapped count field (the “counter”) is incremented by the incoming count; otherwise, a new counter <cite>“(item, count) pair”</cite> is created.
+<p>The <code class="docutils literal notranslate"><span class="pre">LOAD_FACTOR</span></code> for the hash map is internally set at <span class="math notranslate nohighlight">\(75\%\)</span>, which means at any time the map capacity of <code class="docutils literal notranslate"><span class="pre">(item,</span> <span class="pre">count)</span></code> pairs is <code class="docutils literal notranslate"><span class="pre">mapCap</span> <span class="pre">=</span> <span class="pre">0.75</span> < [...]
+The maximum capacity of <code class="docutils literal notranslate"><span class="pre">(item,</span> <span class="pre">count)</span></code> pairs of the sketch is <code class="docutils literal notranslate"><span class="pre">maxMapCap</span> <span class="pre">=</span> <span class="pre">0.75</span> <span class="pre">*</span> <span class="pre">maxMapSize</span></code>.</p>
+<p><strong>Updating the sketch with ``(item, count)`` pairs</strong></p>
+<p>If the item is found in the hash map, the mapped count field (the “counter”) is incremented by the incoming count; otherwise, a new counter <code class="docutils literal notranslate"><span class="pre">(item,</span> <span class="pre">count)</span></code> pair is created.
If the number of tracked counters reaches the maximum capacity of the hash map, the sketch decrements all of the counters (by an approximately computed median)
and removes any non-positive counters.</p>
<p><strong>Accuracy</strong></p>
-<p>If fewer than <cite>0.75 * maxMapSize</cite> different items are inserted into the sketch, the estimated frequencies returned by the sketch will be exact.</p>
+<p>If fewer than <code class="docutils literal notranslate"><span class="pre">0.75</span> <span class="pre">*</span> <span class="pre">maxMapSize</span></code> different items are inserted into the sketch, the estimated frequencies returned by the sketch will be exact.</p>
<p>The logic of the frequent items sketch is such that the stored counts and true counts are never too different.
More specifically, for any item, the sketch can return an estimate of the true frequency of item, along with upper and lower bounds on the frequency (that hold deterministically).</p>
<p>For this implementation and for a specific active item, it is guaranteed that the true frequency will be between the Upper Bound (UB) and the Lower Bound (LB) computed for that item.
-Specifically, <cite>(UB- LB) ≤ W * epsilon</cite>, where <span class="math notranslate nohighlight">\(W\)</span> denotes the sum of all item counts, and <span class="math notranslate nohighlight">\(epsilon = 3.5/M\)</span>, where <span class="math notranslate nohighlight">\(epsilon = M\)</span> is the maxMapSize.</p>
+Specifically, <code class="docutils literal notranslate"><span class="pre">(UB-</span> <span class="pre">LB)</span> <span class="pre">≤</span> <span class="pre">W</span> <span class="pre">*</span> <span class="pre">epsilon</span></code>, where <span class="math notranslate nohighlight">\(W\)</span> denotes the sum of all item counts, and <span class="math notranslate nohighlight">\(epsilon = 3.5/M\)</span>, where <span class="math notranslate nohighlight">\(epsilon = M\)</span> is the ma [...]
<p>This is a worst-case guarantee that applies to arbitrary inputs.
-For inputs typically seen in practice, <cite>(UB-LB)</cite> is usually much smaller.</p>
+For inputs typically seen in practice, <code class="docutils literal notranslate"><span class="pre">(UB-LB)</span></code> is usually much smaller.</p>
<p><strong>Background</strong></p>
<p>This code implements a variant of what is commonly known as the “Misra-Gries algorithm”.
Variants of it were discovered and rediscovered and redesigned several times over the years:</p>
@@ -159,15 +196,40 @@ Variants of it were discovered and rediscovered and redesigned several times ove
<p>For speed, we do employ some randomization that introduces a small probability that our proof of the worst-case bound might not apply to a given run.
However, we have ensured that this probability is extremely small.
For example, if the stream causes one table purge (rebuild), our proof of the worst-case bound applies with a probability of at least <cite>1 - 1E-14</cite>.
-If the stream causes <cite>1E9</cite> purges, our proof applies with a probability of at least <cite>1 - 1E-5</cite>.</p>
-<p>Parameter: <T> The type of item to be tracked by this sketch</p>
+If the stream causes <code class="docutils literal notranslate"><span class="pre">1E9</span></code> purges, our proof applies with a probability of at least <code class="docutils literal notranslate"><span class="pre">1</span> <span class="pre">-</span> <span class="pre">1E-5</span></code>.</p>
+<p>There are two flavors of Frequent Items Sketches, one with generic items (objects) and another specific to strings.
+The string version is a legacy name from before the library supported generic objects and is retained
+only for backwards compatibility.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>The <a class="reference internal" href="#datasketches.frequent_items_sketch" title="datasketches.frequent_items_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">frequent_items_sketch</span></code></a> uses an input object’s <code class="docutils literal notranslate"><span class="pre">__hash__</span></code> and <code class="docutils literal notranslate"><span class="pre">__eq__</span></code> methods.</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Serializing and deserializing the <a class="reference internal" href="#datasketches.frequent_items_sketch" title="datasketches.frequent_items_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">frequent_items_sketch</span></code></a> requires the use of a <a class="reference internal" href="serde.html#datasketches.PyObjectSerDe" title="datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSer [...]
+</div>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.frequent_items_error_type">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">frequent_items_error_type</span></span><a class="headerlink" href="#datasketches.frequent_items_error_type" title="Link to this definition"></a></dt>
+<dd><dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_items_error_type.NO_FALSE_POSITIVES">
+<span class="sig-name descname"><span class="pre">NO_FALSE_POSITIVES</span></span><em class="property"><span class="w"> </span><span class="pre">:</span> <span class="pre">Returns</span> <span class="pre">only</span> <span class="pre">true</span> <span class="pre">positives</span> <span class="pre">but</span> <span class="pre">may</span> <span class="pre">miss</span> <span class="pre">some</span> <span class="pre">heavy</span> <span class="pre">hitters.</span></em><a class="headerlink" h [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_items_error_type.NO_FALSE_NEGATIVES">
+<span class="sig-name descname"><span class="pre">NO_FALSE_NEGATIVES</span></span><em class="property"><span class="w"> </span><span class="pre">:</span> <span class="pre">Does</span> <span class="pre">not</span> <span class="pre">miss</span> <span class="pre">any</span> <span class="pre">heavy</span> <span class="pre">hitters</span> <span class="pre">but</span> <span class="pre">may</span> <span class="pre">return</span> <span class="pre">false</span> <span class="pre">positives.</span> [...]
+<dd></dd></dl>
+
+</dd></dl>
+
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.frequent_items_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">frequent_items_sketch</span></span><a class="headerlink" href="#datasketches.frequent_items_sketch" title="Link to this definition"></a></dt>
<dd><p class="rubric">Static Methods:</p>
<dl class="py method">
<dt class="sig sig-object py" id="datasketches.frequent_items_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">serde</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class=" [...]
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">serde</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="ref [...]
<dd><p>Reads a bytes object using the provided serde and returns the corresponding frequent_strings_sketch.</p>
</dd></dl>
@@ -184,6 +246,17 @@ If the stream causes <cite>1E9</cite> purges, our proof applies with a probabili
</dd></dl>
<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.frequent_items_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lg_max_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</sp [...]
+<dd><p>Creates an instance of the sketch</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>lg_max_k</strong> (<em>int</em>) – base 2 logarithm of the maximum size of the internal hash map of the sketch. Maximum capacity is 0.75 of this value, which is the maximum number of distinct items the sketch can contain.</p>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py property">
<dt class="sig sig-object py" id="datasketches.frequent_items_sketch.epsilon">
<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">epsilon</span></span><a class="headerlink" href="#datasketches.frequent_items_sketch.epsilon" title="Link to this definition"></a></dt>
@@ -264,6 +337,120 @@ Note: The true frequency of a item would be the sum of the counts as a result of
</dd></dl>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">frequent_strings_sketch</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch" title="Link to this definition"></a></dt>
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.deserialize">
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#data [...]
+<dd><p>Reads a bytes object and returns the corresponding frequent_strings_sketch.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.get_epsilon_for_lg_size">
+<span class="sig-name descname"><span class="pre">get_epsilon_for_lg_size</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lg_max_map_size</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><span class="pre">flo [...]
+<dd><p>Returns the epsilon value used to compute a priori error for a given log2(max_map_size)</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.get_apriori_error">
+<span class="sig-name descname"><span class="pre">get_apriori_error</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lg_max_map_size</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">estimated_total_weight</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </spa [...]
+<dd><p>Returns the estimated a priori error given the max_map_size for the sketch and the estimated_total_stream_weight.</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lg_max_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</sp [...]
+<dd><p>Creates an instance of the sketch</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>lg_max_k</strong> (<em>int</em>) – base 2 logarithm of the maximum size of the internal hash map of the sketch. Maximum capacity is 0.75 of this value, which is the maximum number of distinct items the sketch can contain.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.epsilon">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">epsilon</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.epsilon" title="Link to this definition"></a></dt>
+<dd><p>The epsilon value used by the sketch to compute error</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.get_estimate">
+<span class="sig-name descname"><span class="pre">get_estimate</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.get_estimate" title="Link to this definition"></a></dt>
+<dd><p>Returns the estimate of the weight (frequency) of the given item.
+Note: The true frequency of a item would be the sum of the counts as a result of the two update functions.</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.get_frequent_items">
+<span class="sig-name descname"><span class="pre">get_frequent_items</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.get_frequent_items" title="Link to this definition"></a></dt>
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.get_lower_bound">
+<span class="sig-name descname"><span class="pre">get_lower_bound</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.get_lower_bound" title="Link to this definition"></a></dt>
+<dd><p>Returns the guaranteed lower bound weight (frequency) of the given item.</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.get_serialized_size_bytes">
+<span class="sig-name descname"><span class="pre">get_serialized_size_bytes</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.get_serialized_size_bytes" title="Link to this definition"></a></dt>
+<dd><p>Computes the size needed to serialize the current state of the sketch. This can be expensive since every item needs to be looked at.</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.get_upper_bound">
+<span class="sig-name descname"><span class="pre">get_upper_bound</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.get_upper_bound" title="Link to this definition"></a></dt>
+<dd><p>Returns the guaranteed upper bound weight (frequency) of the given item.</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.is_empty">
+<span class="sig-name descname"><span class="pre">is_empty</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.is_empty" title="Link to this definition"></a></dt>
+<dd><p>Returns True if the sketch is empty, otherwise False</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.merge">
+<span class="sig-name descname"><span class="pre">merge</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.merge" title="Link to this definition"></a></dt>
+<dd><p>Merges the given sketch into this one</p>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.num_active_items">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">num_active_items</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.num_active_items" title="Link to this definition"></a></dt>
+<dd><p>The number of active items in the sketch</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.serialize">
+<span class="sig-name descname"><span class="pre">serialize</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.serialize" title="Link to this definition"></a></dt>
+<dd><p>Serializes the sketch into a bytes object.</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.to_string">
+<span class="sig-name descname"><span class="pre">to_string</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.to_string" title="Link to this definition"></a></dt>
+<dd><p>Produces a string summary of the sketch</p>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.total_weight">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">total_weight</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.total_weight" title="Link to this definition"></a></dt>
+<dd><p>The sum of the weights (frequencies) in the stream seen so far by the sketch</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.frequent_strings_sketch.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.frequent_strings_sketch.update" title="Link to this definition"></a></dt>
+<dd><p>Updates the sketch with the given string and, optionally, a weight</p>
+</dd></dl>
+
+</dd></dl>
+
</section>
diff --git a/docs/main/genindex.html b/docs/main/genindex.html
index 59b7bfc..a20e542 100644
--- a/docs/main/genindex.html
+++ b/docs/main/genindex.html
@@ -60,6 +60,17 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -89,7 +100,8 @@
<h1 id="index">Index</h1>
<div class="genindex-jumpbox">
- <a href="#A"><strong>A</strong></a>
+ <a href="#_"><strong>_</strong></a>
+ | <a href="#A"><strong>A</strong></a>
| <a href="#C"><strong>C</strong></a>
| <a href="#D"><strong>D</strong></a>
| <a href="#E"><strong>E</strong></a>
@@ -97,21 +109,100 @@
| <a href="#G"><strong>G</strong></a>
| <a href="#H"><strong>H</strong></a>
| <a href="#I"><strong>I</strong></a>
+ | <a href="#J"><strong>J</strong></a>
| <a href="#K"><strong>K</strong></a>
| <a href="#L"><strong>L</strong></a>
| <a href="#M"><strong>M</strong></a>
| <a href="#N"><strong>N</strong></a>
+ | <a href="#P"><strong>P</strong></a>
| <a href="#Q"><strong>Q</strong></a>
| <a href="#R"><strong>R</strong></a>
| <a href="#S"><strong>S</strong></a>
| <a href="#T"><strong>T</strong></a>
| <a href="#U"><strong>U</strong></a>
+ | <a href="#V"><strong>V</strong></a>
</div>
+<h2 id="_">_</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="kernel.html#datasketches.KernelFunction.__call__">__call__() (KernelFunction method)</a>
+
+ <ul>
+ <li><a href="tuple_policy.html#datasketches.TuplePolicy.__call__">(TuplePolicy method)</a>
+</li>
+ </ul></li>
+ <li><a href="theta.html#datasketches.compact_theta_sketch.__init__">__init__() (compact_theta_sketch method)</a>
+
+ <ul>
+ <li><a href="tuple.html#datasketches.compact_tuple_sketch.__init__">(compact_tuple_sketch method)</a>
+</li>
+ <li><a href="cpc.html#datasketches.cpc_sketch.__init__">(cpc_sketch method)</a>
+</li>
+ <li><a href="cpc.html#datasketches.cpc_union.__init__">(cpc_union method)</a>
+</li>
+ <li><a href="density_sketch.html#datasketches.density_sketch.__init__">(density_sketch method)</a>
+</li>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.__init__">(ebpps_sketch method)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_items_sketch.__init__">(frequent_items_sketch method)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.__init__">(frequent_strings_sketch method)</a>
+</li>
+ <li><a href="hyper_log_log.html#datasketches.hll_sketch.__init__">(hll_sketch method)</a>
+</li>
+ <li><a href="hyper_log_log.html#datasketches.hll_union.__init__">(hll_union method)</a>
+</li>
+ <li><a href="kll.html#datasketches.kll_doubles_sketch.__init__">(kll_doubles_sketch method)</a>
+</li>
+ <li><a href="kll.html#datasketches.kll_floats_sketch.__init__">(kll_floats_sketch method)</a>
+</li>
+ <li><a href="kll.html#datasketches.kll_ints_sketch.__init__">(kll_ints_sketch method)</a>
+</li>
+ <li><a href="kll.html#datasketches.kll_items_sketch.__init__">(kll_items_sketch method)</a>
+</li>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_doubles_sketch.__init__">(quantiles_doubles_sketch method)</a>
+</li>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_floats_sketch.__init__">(quantiles_floats_sketch method)</a>
+</li>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_ints_sketch.__init__">(quantiles_ints_sketch method)</a>
+</li>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_items_sketch.__init__">(quantiles_items_sketch method)</a>
+</li>
+ <li><a href="req.html#datasketches.req_floats_sketch.__init__">(req_floats_sketch method)</a>
+</li>
+ <li><a href="req.html#datasketches.req_ints_sketch.__init__">(req_ints_sketch method)</a>
+</li>
+ <li><a href="req.html#datasketches.req_items_sketch.__init__">(req_items_sketch method)</a>
+</li>
+ <li><a href="theta.html#datasketches.theta_a_not_b.__init__">(theta_a_not_b method)</a>
+</li>
+ <li><a href="theta.html#datasketches.theta_intersection.__init__">(theta_intersection method)</a>
+</li>
+ <li><a href="theta.html#datasketches.theta_union.__init__">(theta_union method)</a>
+</li>
+ <li><a href="tuple.html#datasketches.tuple_a_not_b.__init__">(tuple_a_not_b method)</a>
+</li>
+ <li><a href="tuple.html#datasketches.tuple_intersection.__init__">(tuple_intersection method)</a>
+</li>
+ <li><a href="tuple.html#datasketches.tuple_union.__init__">(tuple_union method)</a>
+</li>
+ <li><a href="theta.html#datasketches.update_theta_sketch.__init__">(update_theta_sketch method)</a>
+</li>
+ <li><a href="tuple.html#datasketches.update_tuple_sketch.__init__">(update_tuple_sketch method)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.__init__">(var_opt_sketch method)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_union.__init__">(var_opt_union method)</a>
+</li>
+ </ul></li>
+ </ul></td>
+</tr></table>
+
<h2 id="A">A</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="tuple.html#datasketches.AccumulatorPolicy">AccumulatorPolicy (class in datasketches)</a>
+ <li><a href="tuple_policy.html#datasketches.AccumulatorPolicy">AccumulatorPolicy (class in datasketches)</a>
</li>
</ul></td>
</tr></table>
@@ -119,20 +210,34 @@
<h2 id="C">C</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="count_min_sketch.html#datasketches.count_min_sketch">count_min_sketch (class in _datasketches)</a>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.c">c (ebpps_sketch property)</a>
</li>
- <li><a href="cpc.html#datasketches.cpc_sketch">cpc_sketch (class in _datasketches)</a>
+ <li><a href="theta.html#datasketches.update_theta_sketch.compact">compact (update_theta_sketch attribute)</a>
+
+ <ul>
+ <li><a href="tuple.html#datasketches.update_tuple_sketch.compact">(update_tuple_sketch attribute)</a>
+</li>
+ </ul></li>
+ <li><a href="theta.html#datasketches.compact_theta_sketch">compact_theta_sketch (class in datasketches)</a>
+</li>
+ <li><a href="tuple.html#datasketches.compact_tuple_sketch">compact_tuple_sketch (class in datasketches)</a>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="tuple.html#datasketches.AccumulatorPolicy.create_summary">create_summary() (AccumulatorPolicy method)</a>
+ <li><a href="theta.html#datasketches.theta_a_not_b.compute">compute (theta_a_not_b attribute)</a>
<ul>
- <li><a href="tuple.html#datasketches.MaxIntPolicy.create_summary">(MaxIntPolicy method)</a>
-</li>
- <li><a href="tuple.html#datasketches.MinIntPolicy.create_summary">(MinIntPolicy method)</a>
+ <li><a href="tuple.html#datasketches.tuple_a_not_b.compute">(tuple_a_not_b attribute)</a>
</li>
</ul></li>
+ <li><a href="count_min_sketch.html#datasketches.count_min_sketch">count_min_sketch (class in _datasketches)</a>
+</li>
+ <li><a href="cpc.html#datasketches.cpc_sketch">cpc_sketch (class in _datasketches)</a>
+</li>
+ <li><a href="cpc.html#datasketches.cpc_union">cpc_union (class in _datasketches)</a>
+</li>
+ <li><a href="tuple_policy.html#datasketches.TuplePolicy.create_summary">create_summary() (TuplePolicy method)</a>
+</li>
</ul></td>
</tr></table>
@@ -143,61 +248,101 @@
</li>
<li><a href="density_sketch.html#datasketches.density_sketch">density_sketch (class in datasketches)</a>
</li>
- <li><a href="density_sketch.html#datasketches.density_sketch.deserialize">deserialize (density_sketch attribute)</a>
+ <li><a href="theta.html#datasketches.compact_theta_sketch.deserialize">deserialize() (compact_theta_sketch method)</a>
<ul>
- <li><a href="kll.html#datasketches.kll_doubles_sketch.deserialize">(kll_doubles_sketch attribute)</a>
+ <li><a href="tuple.html#datasketches.compact_tuple_sketch.deserialize">(compact_tuple_sketch method)</a>
</li>
- <li><a href="kll.html#datasketches.kll_floats_sketch.deserialize">(kll_floats_sketch attribute)</a>
+ <li><a href="count_min_sketch.html#datasketches.count_min_sketch.deserialize">(count_min_sketch method)</a>
</li>
- <li><a href="kll.html#datasketches.kll_ints_sketch.deserialize">(kll_ints_sketch attribute)</a>
+ <li><a href="cpc.html#datasketches.cpc_sketch.deserialize">(cpc_sketch method)</a>
</li>
- <li><a href="kll.html#datasketches.kll_items_sketch.deserialize">(kll_items_sketch attribute)</a>
+ <li><a href="density_sketch.html#datasketches.density_sketch.deserialize">(density_sketch method)</a>
</li>
- <li><a href="quantiles_depr.html#datasketches.quantiles_doubles_sketch.deserialize">(quantiles_doubles_sketch attribute)</a>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.deserialize">(ebpps_sketch method)</a>
</li>
- <li><a href="quantiles_depr.html#datasketches.quantiles_floats_sketch.deserialize">(quantiles_floats_sketch attribute)</a>
+ <li><a href="frequent_items.html#datasketches.frequent_items_sketch.deserialize">(frequent_items_sketch method)</a>
</li>
- <li><a href="quantiles_depr.html#datasketches.quantiles_ints_sketch.deserialize">(quantiles_ints_sketch attribute)</a>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.deserialize">(frequent_strings_sketch method)</a>
</li>
- <li><a href="quantiles_depr.html#datasketches.quantiles_items_sketch.deserialize">(quantiles_items_sketch attribute)</a>
+ <li><a href="hyper_log_log.html#datasketches.hll_sketch.deserialize">(hll_sketch method)</a>
+</li>
+ <li><a href="kll.html#datasketches.kll_doubles_sketch.deserialize">(kll_doubles_sketch method)</a>
+</li>
+ <li><a href="kll.html#datasketches.kll_floats_sketch.deserialize">(kll_floats_sketch method)</a>
+</li>
+ <li><a href="kll.html#datasketches.kll_ints_sketch.deserialize">(kll_ints_sketch method)</a>
+</li>
+ <li><a href="kll.html#datasketches.kll_items_sketch.deserialize">(kll_items_sketch method)</a>
+</li>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_doubles_sketch.deserialize">(quantiles_doubles_sketch method)</a>
+</li>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_floats_sketch.deserialize">(quantiles_floats_sketch method)</a>
+</li>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_ints_sketch.deserialize">(quantiles_ints_sketch method)</a>
</li>
- <li><a href="req.html#datasketches.req_floats_sketch.deserialize">(req_floats_sketch attribute)</a>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_items_sketch.deserialize">(quantiles_items_sketch method)</a>
</li>
- <li><a href="req.html#datasketches.req_ints_sketch.deserialize">(req_ints_sketch attribute)</a>
+ <li><a href="req.html#datasketches.req_floats_sketch.deserialize">(req_floats_sketch method)</a>
</li>
- <li><a href="req.html#datasketches.req_items_sketch.deserialize">(req_items_sketch attribute)</a>
+ <li><a href="req.html#datasketches.req_ints_sketch.deserialize">(req_ints_sketch method)</a>
+</li>
+ <li><a href="req.html#datasketches.req_items_sketch.deserialize">(req_items_sketch method)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.deserialize">(var_opt_sketch method)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_union.deserialize">(var_opt_union method)</a>
</li>
</ul></li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="count_min_sketch.html#datasketches.count_min_sketch.deserialize">deserialize() (count_min_sketch method)</a>
+ <li><a href="density_sketch.html#datasketches.density_sketch.dim">dim (density_sketch property)</a>
+</li>
+ <li><a href="jaccard.html#datasketches.theta_jaccard_similarity.dissimilarity_test">dissimilarity_test() (theta_jaccard_similarity method)</a>
<ul>
- <li><a href="cpc.html#datasketches.cpc_sketch.deserialize">(cpc_sketch method)</a>
-</li>
- <li><a href="frequent_items.html#datasketches.frequent_items_sketch.deserialize">(frequent_items_sketch method)</a>
-</li>
- <li><a href="hyper_log_log.html#datasketches.hll_sketch.deserialize">(hll_sketch method)</a>
+ <li><a href="jaccard.html#datasketches.tuple_jaccard_similarity.dissimilarity_test">(tuple_jaccard_similarity method)</a>
</li>
</ul></li>
- <li><a href="density_sketch.html#datasketches.density_sketch.dim">dim (density_sketch property)</a>
-</li>
</ul></td>
</tr></table>
<h2 id="E">E</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch">ebpps_sketch (class in datasketches)</a>
+</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.epsilon">epsilon (frequent_items_sketch property)</a>
+
+ <ul>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.epsilon">(frequent_strings_sketch property)</a>
</li>
+ </ul></li>
+ </ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.estimate_subset_sum">estimate_subset_sum (var_opt_sketch attribute)</a>
+</li>
+ <li><a href="jaccard.html#datasketches.theta_jaccard_similarity.exactly_equal">exactly_equal() (theta_jaccard_similarity method)</a>
+
+ <ul>
+ <li><a href="jaccard.html#datasketches.tuple_jaccard_similarity.exactly_equal">(tuple_jaccard_similarity method)</a>
+</li>
+ </ul></li>
</ul></td>
</tr></table>
<h2 id="F">F</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="frequent_items.html#datasketches.frequent_items_sketch">frequent_items_sketch (class in _datasketches)</a>
+ <li><a href="frequent_items.html#datasketches.frequent_items_error_type">frequent_items_error_type (class in datasketches)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_items_sketch">frequent_items_sketch (class in datasketches)</a>
+</li>
+ </ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch">frequent_strings_sketch (class in datasketches)</a>
+</li>
+ <li><a href="serde.html#datasketches.PyObjectSerDe.from_bytes">from_bytes() (PyObjectSerDe method)</a>
</li>
</ul></td>
</tr></table>
@@ -205,10 +350,14 @@
<h2 id="G">G</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="density_sketch.html#datasketches.GaussianKernel">GaussianKernel (class in datasketches)</a>
+ <li><a href="kernel.html#datasketches.GaussianKernel">GaussianKernel (class in datasketches)</a>
</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.get_apriori_error">get_apriori_error() (frequent_items_sketch method)</a>
+
+ <ul>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.get_apriori_error">(frequent_strings_sketch method)</a>
</li>
+ </ul></li>
<li><a href="kll.html#datasketches.kll_doubles_sketch.get_cdf">get_cdf (kll_doubles_sketch attribute)</a>
<ul>
@@ -236,7 +385,11 @@
<li><a href="hyper_log_log.html#datasketches.hll_sketch.get_compact_serialization_bytes">get_compact_serialization_bytes (hll_sketch attribute)</a>
</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.get_epsilon_for_lg_size">get_epsilon_for_lg_size() (frequent_items_sketch method)</a>
+
+ <ul>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.get_epsilon_for_lg_size">(frequent_strings_sketch method)</a>
</li>
+ </ul></li>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.get_estimate">get_estimate (count_min_sketch attribute)</a>
<ul>
@@ -245,8 +398,12 @@
<li><a href="density_sketch.html#datasketches.density_sketch.get_estimate">(density_sketch attribute)</a>
</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.get_estimate">(frequent_items_sketch attribute)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.get_estimate">(frequent_strings_sketch attribute)</a>
</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.get_estimate">(hll_sketch attribute)</a>
+</li>
+ <li><a href="hyper_log_log.html#datasketches.hll_union.get_estimate">(hll_union attribute)</a>
</li>
<li><a href="theta.html#datasketches.theta_sketch.get_estimate">(theta_sketch attribute)</a>
</li>
@@ -254,15 +411,23 @@
</li>
</ul></li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.get_frequent_items">get_frequent_items (frequent_items_sketch attribute)</a>
+
+ <ul>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.get_frequent_items">(frequent_strings_sketch attribute)</a>
</li>
+ </ul></li>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.get_lower_bound">get_lower_bound (count_min_sketch attribute)</a>
<ul>
<li><a href="cpc.html#datasketches.cpc_sketch.get_lower_bound">(cpc_sketch attribute)</a>
</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.get_lower_bound">(frequent_items_sketch attribute)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.get_lower_bound">(frequent_strings_sketch attribute)</a>
</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.get_lower_bound">(hll_sketch attribute)</a>
+</li>
+ <li><a href="hyper_log_log.html#datasketches.hll_union.get_lower_bound">(hll_union attribute)</a>
</li>
<li><a href="theta.html#datasketches.theta_sketch.get_lower_bound">(theta_sketch attribute)</a>
</li>
@@ -319,22 +484,22 @@
<li><a href="req.html#datasketches.req_items_sketch.get_min_value">(req_items_sketch attribute)</a>
</li>
</ul></li>
- <li><a href="kll.html#datasketches.kll_doubles_sketch.get_normalized_rank_error">get_normalized_rank_error (kll_doubles_sketch attribute)</a>
+ <li><a href="kll.html#datasketches.kll_doubles_sketch.get_normalized_rank_error">get_normalized_rank_error() (kll_doubles_sketch method)</a>
<ul>
- <li><a href="kll.html#datasketches.kll_floats_sketch.get_normalized_rank_error">(kll_floats_sketch attribute)</a>
+ <li><a href="kll.html#datasketches.kll_floats_sketch.get_normalized_rank_error">(kll_floats_sketch method)</a>
</li>
- <li><a href="kll.html#datasketches.kll_ints_sketch.get_normalized_rank_error">(kll_ints_sketch attribute)</a>
+ <li><a href="kll.html#datasketches.kll_ints_sketch.get_normalized_rank_error">(kll_ints_sketch method)</a>
</li>
- <li><a href="kll.html#datasketches.kll_items_sketch.get_normalized_rank_error">(kll_items_sketch attribute)</a>
+ <li><a href="kll.html#datasketches.kll_items_sketch.get_normalized_rank_error">(kll_items_sketch method)</a>
</li>
- <li><a href="quantiles_depr.html#datasketches.quantiles_doubles_sketch.get_normalized_rank_error">(quantiles_doubles_sketch attribute)</a>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_doubles_sketch.get_normalized_rank_error">(quantiles_doubles_sketch method)</a>
</li>
- <li><a href="quantiles_depr.html#datasketches.quantiles_floats_sketch.get_normalized_rank_error">(quantiles_floats_sketch attribute)</a>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_floats_sketch.get_normalized_rank_error">(quantiles_floats_sketch method)</a>
</li>
- <li><a href="quantiles_depr.html#datasketches.quantiles_ints_sketch.get_normalized_rank_error">(quantiles_ints_sketch attribute)</a>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_ints_sketch.get_normalized_rank_error">(quantiles_ints_sketch method)</a>
</li>
- <li><a href="quantiles_depr.html#datasketches.quantiles_items_sketch.get_normalized_rank_error">(quantiles_items_sketch attribute)</a>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_items_sketch.get_normalized_rank_error">(quantiles_items_sketch method)</a>
</li>
</ul></li>
<li><a href="kll.html#datasketches.kll_doubles_sketch.get_pmf">get_pmf (kll_doubles_sketch attribute)</a>
@@ -452,15 +617,35 @@
</li>
</ul></li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.get_rel_err">get_rel_err() (hll_sketch method)</a>
+
+ <ul>
+ <li><a href="hyper_log_log.html#datasketches.hll_union.get_rel_err">(hll_union method)</a>
</li>
+ </ul></li>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.get_relative_error">get_relative_error (count_min_sketch attribute)</a>
</li>
- <li><a href="req.html#datasketches.req_floats_sketch.get_RSE">get_RSE (req_floats_sketch attribute)</a>
+ <li><a href="cpc.html#datasketches.cpc_union.get_result">get_result (cpc_union attribute)</a>
+
+ <ul>
+ <li><a href="hyper_log_log.html#datasketches.hll_union.get_result">(hll_union attribute)</a>
+</li>
+ <li><a href="theta.html#datasketches.theta_intersection.get_result">(theta_intersection attribute)</a>
+</li>
+ <li><a href="theta.html#datasketches.theta_union.get_result">(theta_union attribute)</a>
+</li>
+ <li><a href="tuple.html#datasketches.tuple_intersection.get_result">(tuple_intersection attribute)</a>
+</li>
+ <li><a href="tuple.html#datasketches.tuple_union.get_result">(tuple_union attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_union.get_result">(var_opt_union attribute)</a>
+</li>
+ </ul></li>
+ <li><a href="req.html#datasketches.req_floats_sketch.get_RSE">get_RSE() (req_floats_sketch method)</a>
<ul>
- <li><a href="req.html#datasketches.req_ints_sketch.get_RSE">(req_ints_sketch attribute)</a>
+ <li><a href="req.html#datasketches.req_ints_sketch.get_RSE">(req_ints_sketch method)</a>
</li>
- <li><a href="req.html#datasketches.req_items_sketch.get_RSE">(req_items_sketch attribute)</a>
+ <li><a href="req.html#datasketches.req_items_sketch.get_RSE">(req_items_sketch method)</a>
</li>
</ul></li>
<li><a href="theta.html#datasketches.theta_sketch.get_seed_hash">get_seed_hash (theta_sketch attribute)</a>
@@ -472,9 +657,19 @@
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.get_serialized_size_bytes">get_serialized_size_bytes (count_min_sketch attribute)</a>
<ul>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.get_serialized_size_bytes">(ebpps_sketch attribute)</a>
+</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.get_serialized_size_bytes">(frequent_items_sketch attribute)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.get_serialized_size_bytes">(frequent_strings_sketch attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.get_serialized_size_bytes">(var_opt_sketch attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_union.get_serialized_size_bytes">(var_opt_union attribute)</a>
</li>
</ul></li>
+ <li><a href="serde.html#datasketches.PyObjectSerDe.get_size">get_size() (PyObjectSerDe method)</a>
+</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.get_updatable_serialization_bytes">get_updatable_serialization_bytes (hll_sketch attribute)</a>
</li>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.get_upper_bound">get_upper_bound (count_min_sketch attribute)</a>
@@ -483,8 +678,12 @@
<li><a href="cpc.html#datasketches.cpc_sketch.get_upper_bound">(cpc_sketch attribute)</a>
</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.get_upper_bound">(frequent_items_sketch attribute)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.get_upper_bound">(frequent_strings_sketch attribute)</a>
</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.get_upper_bound">(hll_sketch attribute)</a>
+</li>
+ <li><a href="hyper_log_log.html#datasketches.hll_union.get_upper_bound">(hll_union attribute)</a>
</li>
<li><a href="theta.html#datasketches.theta_sketch.get_upper_bound">(theta_sketch attribute)</a>
</li>
@@ -497,7 +696,23 @@
<h2 id="H">H</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="theta.html#datasketches.theta_intersection.has_result">has_result (theta_intersection attribute)</a>
+
+ <ul>
+ <li><a href="tuple.html#datasketches.tuple_intersection.has_result">(tuple_intersection attribute)</a>
+</li>
+ </ul></li>
+ <li><a href="hyper_log_log.html#datasketches.tgt_hll_type.HLL_4">HLL_4 (tgt_hll_type attribute)</a>
+</li>
+ </ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="hyper_log_log.html#datasketches.tgt_hll_type.HLL_6">HLL_6 (tgt_hll_type attribute)</a>
+</li>
+ <li><a href="hyper_log_log.html#datasketches.tgt_hll_type.HLL_8">HLL_8 (tgt_hll_type attribute)</a>
+</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch">hll_sketch (class in _datasketches)</a>
+</li>
+ <li><a href="hyper_log_log.html#datasketches.hll_union">hll_union (class in _datasketches)</a>
</li>
</ul></td>
</tr></table>
@@ -513,10 +728,16 @@
<li><a href="cpc.html#datasketches.cpc_sketch.is_empty">(cpc_sketch attribute)</a>
</li>
<li><a href="density_sketch.html#datasketches.density_sketch.is_empty">(density_sketch attribute)</a>
+</li>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.is_empty">(ebpps_sketch attribute)</a>
</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.is_empty">(frequent_items_sketch attribute)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.is_empty">(frequent_strings_sketch attribute)</a>
</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.is_empty">(hll_sketch attribute)</a>
+</li>
+ <li><a href="hyper_log_log.html#datasketches.hll_union.is_empty">(hll_union attribute)</a>
</li>
<li><a href="kll.html#datasketches.kll_doubles_sketch.is_empty">(kll_doubles_sketch attribute)</a>
</li>
@@ -543,6 +764,8 @@
<li><a href="theta.html#datasketches.theta_sketch.is_empty">(theta_sketch attribute)</a>
</li>
<li><a href="tuple.html#datasketches.tuple_sketch.is_empty">(tuple_sketch attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.is_empty">(var_opt_sketch attribute)</a>
</li>
</ul></li>
</ul></td>
@@ -594,12 +817,26 @@
</ul></td>
</tr></table>
+<h2 id="J">J</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="jaccard.html#datasketches.theta_jaccard_similarity.jaccard">jaccard() (theta_jaccard_similarity method)</a>
+
+ <ul>
+ <li><a href="jaccard.html#datasketches.tuple_jaccard_similarity.jaccard">(tuple_jaccard_similarity method)</a>
+</li>
+ </ul></li>
+ </ul></td>
+</tr></table>
+
<h2 id="K">K</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="density_sketch.html#datasketches.density_sketch.k">k (density_sketch property)</a>
<ul>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.k">(ebpps_sketch property)</a>
+</li>
<li><a href="kll.html#datasketches.kll_doubles_sketch.k">(kll_doubles_sketch property)</a>
</li>
<li><a href="kll.html#datasketches.kll_floats_sketch.k">(kll_floats_sketch property)</a>
@@ -621,17 +858,23 @@
<li><a href="req.html#datasketches.req_ints_sketch.k">(req_ints_sketch property)</a>
</li>
<li><a href="req.html#datasketches.req_items_sketch.k">(req_items_sketch property)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.k">(var_opt_sketch property)</a>
</li>
</ul></li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="kll.html#datasketches.kll_doubles_sketch">kll_doubles_sketch (class in _datasketches)</a>
+ <li><a href="kernel.html#datasketches.KernelFunction">KernelFunction (class in datasketches)</a>
+</li>
+ <li><a href="kll.html#datasketches.kll_doubles_sketch">kll_doubles_sketch (class in datasketches)</a>
</li>
- <li><a href="kll.html#datasketches.kll_floats_sketch">kll_floats_sketch (class in _datasketches)</a>
+ <li><a href="kll.html#datasketches.kll_floats_sketch">kll_floats_sketch (class in datasketches)</a>
</li>
- <li><a href="kll.html#datasketches.kll_ints_sketch">kll_ints_sketch (class in _datasketches)</a>
+ <li><a href="kll.html#datasketches.kll_ints_sketch">kll_ints_sketch (class in datasketches)</a>
</li>
- <li><a href="kll.html#datasketches.kll_items_sketch">kll_items_sketch (class in _datasketches)</a>
+ <li><a href="kll.html#datasketches.kll_items_sketch">kll_items_sketch (class in datasketches)</a>
+</li>
+ <li><a href="ks_test.html#datasketches.ks_test">ks_test() (in module datasketches)</a>
</li>
</ul></td>
</tr></table>
@@ -640,7 +883,11 @@
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.lg_config_k">lg_config_k (hll_sketch property)</a>
+
+ <ul>
+ <li><a href="hyper_log_log.html#datasketches.hll_union.lg_config_k">(hll_union property)</a>
</li>
+ </ul></li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="cpc.html#datasketches.cpc_sketch.lg_k">lg_k (cpc_sketch property)</a>
@@ -651,14 +898,18 @@
<h2 id="M">M</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="tuple.html#datasketches.MaxIntPolicy">MaxIntPolicy (class in datasketches)</a>
+ <li><a href="tuple_policy.html#datasketches.MaxIntPolicy">MaxIntPolicy (class in datasketches)</a>
</li>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.merge">merge (count_min_sketch attribute)</a>
<ul>
<li><a href="density_sketch.html#datasketches.density_sketch.merge">(density_sketch attribute)</a>
+</li>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.merge">(ebpps_sketch attribute)</a>
</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.merge">(frequent_items_sketch attribute)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.merge">(frequent_strings_sketch attribute)</a>
</li>
<li><a href="kll.html#datasketches.kll_doubles_sketch.merge">(kll_doubles_sketch attribute)</a>
</li>
@@ -685,7 +936,7 @@
</ul></li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="tuple.html#datasketches.MinIntPolicy">MinIntPolicy (class in datasketches)</a>
+ <li><a href="tuple_policy.html#datasketches.MinIntPolicy">MinIntPolicy (class in datasketches)</a>
</li>
</ul></td>
</tr></table>
@@ -696,6 +947,8 @@
<li><a href="density_sketch.html#datasketches.density_sketch.n">n (density_sketch property)</a>
<ul>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.n">(ebpps_sketch property)</a>
+</li>
<li><a href="kll.html#datasketches.kll_doubles_sketch.n">(kll_doubles_sketch property)</a>
</li>
<li><a href="kll.html#datasketches.kll_floats_sketch.n">(kll_floats_sketch property)</a>
@@ -717,8 +970,14 @@
<li><a href="req.html#datasketches.req_ints_sketch.n">(req_ints_sketch property)</a>
</li>
<li><a href="req.html#datasketches.req_items_sketch.n">(req_items_sketch property)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.n">(var_opt_sketch property)</a>
</li>
</ul></li>
+ <li><a href="frequent_items.html#datasketches.frequent_items_error_type.NO_FALSE_NEGATIVES">NO_FALSE_NEGATIVES (frequent_items_error_type attribute)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_items_error_type.NO_FALSE_POSITIVES">NO_FALSE_POSITIVES (frequent_items_error_type attribute)</a>
+</li>
<li><a href="kll.html#datasketches.kll_doubles_sketch.normalized_rank_error">normalized_rank_error (kll_doubles_sketch attribute)</a>
<ul>
@@ -740,7 +999,11 @@
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.num_active_items">num_active_items (frequent_items_sketch property)</a>
+
+ <ul>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.num_active_items">(frequent_strings_sketch property)</a>
</li>
+ </ul></li>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.num_buckets">num_buckets (count_min_sketch property)</a>
</li>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.num_hashes">num_hashes (count_min_sketch property)</a>
@@ -775,38 +1038,70 @@
<li><a href="tuple.html#datasketches.tuple_sketch.num_retained">(tuple_sketch property)</a>
</li>
</ul></li>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.num_samples">num_samples (var_opt_sketch property)</a>
+</li>
</ul></td>
</tr></table>
-<h2 id="Q">Q</h2>
+<h2 id="P">P</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="quantiles_depr.html#datasketches.quantiles_doubles_sketch">quantiles_doubles_sketch (class in _datasketches)</a>
+ <li><a href="serde.html#datasketches.PyDoublesSerDe">PyDoublesSerDe (class in datasketches)</a>
</li>
- <li><a href="quantiles_depr.html#datasketches.quantiles_floats_sketch">quantiles_floats_sketch (class in _datasketches)</a>
+ <li><a href="serde.html#datasketches.PyFloatsSerDe">PyFloatsSerDe (class in datasketches)</a>
+</li>
+ <li><a href="serde.html#datasketches.PyIntsSerDe">PyIntsSerDe (class in datasketches)</a>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="quantiles_depr.html#datasketches.quantiles_ints_sketch">quantiles_ints_sketch (class in _datasketches)</a>
+ <li><a href="serde.html#datasketches.PyLongsSerDe">PyLongsSerDe (class in datasketches)</a>
+</li>
+ <li><a href="serde.html#datasketches.PyObjectSerDe">PyObjectSerDe (class in datasketches)</a>
</li>
- <li><a href="quantiles_depr.html#datasketches.quantiles_items_sketch">quantiles_items_sketch (class in _datasketches)</a>
+ <li><a href="serde.html#datasketches.PyStringsSerDe">PyStringsSerDe (class in datasketches)</a>
</li>
</ul></td>
</tr></table>
-<h2 id="R">R</h2>
+<h2 id="Q">Q</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="req.html#datasketches.req_floats_sketch">req_floats_sketch (class in _datasketches)</a>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_doubles_sketch">quantiles_doubles_sketch (class in datasketches)</a>
</li>
- <li><a href="req.html#datasketches.req_ints_sketch">req_ints_sketch (class in _datasketches)</a>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_floats_sketch">quantiles_floats_sketch (class in datasketches)</a>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="req.html#datasketches.req_items_sketch">req_items_sketch (class in _datasketches)</a>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_ints_sketch">quantiles_ints_sketch (class in datasketches)</a>
+</li>
+ <li><a href="quantiles_depr.html#datasketches.quantiles_items_sketch">quantiles_items_sketch (class in datasketches)</a>
+</li>
+ </ul></td>
+</tr></table>
+
+<h2 id="R">R</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="req.html#datasketches.req_floats_sketch">req_floats_sketch (class in datasketches)</a>
+</li>
+ <li><a href="req.html#datasketches.req_ints_sketch">req_ints_sketch (class in datasketches)</a>
+</li>
+ <li><a href="req.html#datasketches.req_items_sketch">req_items_sketch (class in datasketches)</a>
</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.reset">reset (hll_sketch attribute)</a>
+
+ <ul>
+ <li><a href="hyper_log_log.html#datasketches.hll_union.reset">(hll_union attribute)</a>
+</li>
+ <li><a href="tuple.html#datasketches.tuple_union.reset">(tuple_union attribute)</a>
</li>
+ <li><a href="theta.html#datasketches.update_theta_sketch.reset">(update_theta_sketch attribute)</a>
+</li>
+ <li><a href="tuple.html#datasketches.update_tuple_sketch.reset">(update_tuple_sketch attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_union.reset">(var_opt_union attribute)</a>
+</li>
+ </ul></li>
</ul></td>
</tr></table>
@@ -815,14 +1110,22 @@
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.seed">seed (count_min_sketch property)</a>
</li>
- <li><a href="count_min_sketch.html#datasketches.count_min_sketch.serialize">serialize (count_min_sketch attribute)</a>
+ <li><a href="theta.html#datasketches.compact_theta_sketch.serialize">serialize (compact_theta_sketch attribute)</a>
<ul>
+ <li><a href="tuple.html#datasketches.compact_tuple_sketch.serialize">(compact_tuple_sketch attribute)</a>
+</li>
+ <li><a href="count_min_sketch.html#datasketches.count_min_sketch.serialize">(count_min_sketch attribute)</a>
+</li>
<li><a href="cpc.html#datasketches.cpc_sketch.serialize">(cpc_sketch attribute)</a>
</li>
<li><a href="density_sketch.html#datasketches.density_sketch.serialize">(density_sketch attribute)</a>
+</li>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.serialize">(ebpps_sketch attribute)</a>
</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.serialize">(frequent_items_sketch attribute)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.serialize">(frequent_strings_sketch attribute)</a>
</li>
<li><a href="kll.html#datasketches.kll_doubles_sketch.serialize">(kll_doubles_sketch attribute)</a>
</li>
@@ -845,6 +1148,10 @@
<li><a href="req.html#datasketches.req_ints_sketch.serialize">(req_ints_sketch attribute)</a>
</li>
<li><a href="req.html#datasketches.req_items_sketch.serialize">(req_items_sketch attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.serialize">(var_opt_sketch attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_union.serialize">(var_opt_union attribute)</a>
</li>
</ul></li>
</ul></td>
@@ -853,6 +1160,12 @@
</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.serialize_updatable">serialize_updatable (hll_sketch attribute)</a>
</li>
+ <li><a href="jaccard.html#datasketches.theta_jaccard_similarity.similarity_test">similarity_test() (theta_jaccard_similarity method)</a>
+
+ <ul>
+ <li><a href="jaccard.html#datasketches.tuple_jaccard_similarity.similarity_test">(tuple_jaccard_similarity method)</a>
+</li>
+ </ul></li>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.suggest_num_buckets">suggest_num_buckets() (count_min_sketch method)</a>
</li>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.suggest_num_hashes">suggest_num_hashes() (count_min_sketch method)</a>
@@ -863,6 +1176,8 @@
<h2 id="T">T</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="hyper_log_log.html#datasketches.tgt_hll_type">tgt_hll_type (class in _datasketches)</a>
+</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.tgt_type">tgt_type (hll_sketch property)</a>
</li>
<li><a href="theta.html#datasketches.theta_sketch.theta">theta (theta_sketch property)</a>
@@ -877,7 +1192,17 @@
<li><a href="tuple.html#datasketches.tuple_sketch.theta64">(tuple_sketch property)</a>
</li>
</ul></li>
- <li><a href="theta.html#datasketches.theta_sketch">theta_sketch (class in _datasketches)</a>
+ <li><a href="theta.html#datasketches.theta_a_not_b">theta_a_not_b (class in datasketches)</a>
+</li>
+ <li><a href="theta.html#datasketches.theta_intersection">theta_intersection (class in datasketches)</a>
+</li>
+ <li><a href="jaccard.html#datasketches.theta_jaccard_similarity">theta_jaccard_similarity (class in datasketches)</a>
+</li>
+ <li><a href="theta.html#datasketches.theta_sketch">theta_sketch (class in datasketches)</a>
+</li>
+ <li><a href="theta.html#datasketches.theta_union">theta_union (class in datasketches)</a>
+</li>
+ <li><a href="serde.html#datasketches.PyObjectSerDe.to_bytes">to_bytes() (PyObjectSerDe method)</a>
</li>
<li><a href="count_min_sketch.html#datasketches.count_min_sketch.to_string">to_string (count_min_sketch attribute)</a>
@@ -885,8 +1210,12 @@
<li><a href="cpc.html#datasketches.cpc_sketch.to_string">(cpc_sketch attribute)</a>
</li>
<li><a href="density_sketch.html#datasketches.density_sketch.to_string">(density_sketch attribute)</a>
+</li>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.to_string">(ebpps_sketch attribute)</a>
</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.to_string">(frequent_items_sketch attribute)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.to_string">(frequent_strings_sketch attribute)</a>
</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.to_string">(hll_sketch attribute)</a>
</li>
@@ -915,6 +1244,10 @@
<li><a href="theta.html#datasketches.theta_sketch.to_string">(theta_sketch attribute)</a>
</li>
<li><a href="tuple.html#datasketches.tuple_sketch.to_string">(tuple_sketch attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.to_string">(var_opt_sketch attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_union.to_string">(var_opt_union attribute)</a>
</li>
</ul></li>
</ul></td>
@@ -923,9 +1256,27 @@
<ul>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.total_weight">(frequent_items_sketch property)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.total_weight">(frequent_strings_sketch property)</a>
</li>
</ul></li>
+ <li><a href="theta.html#datasketches.update_theta_sketch.trim">trim (update_theta_sketch attribute)</a>
+
+ <ul>
+ <li><a href="tuple.html#datasketches.update_tuple_sketch.trim">(update_tuple_sketch attribute)</a>
+</li>
+ </ul></li>
+ <li><a href="tuple.html#datasketches.tuple_a_not_b">tuple_a_not_b (class in datasketches)</a>
+</li>
+ <li><a href="tuple.html#datasketches.tuple_intersection">tuple_intersection (class in datasketches)</a>
+</li>
+ <li><a href="jaccard.html#datasketches.tuple_jaccard_similarity">tuple_jaccard_similarity (class in datasketches)</a>
+</li>
<li><a href="tuple.html#datasketches.tuple_sketch">tuple_sketch (class in datasketches)</a>
+</li>
+ <li><a href="tuple.html#datasketches.tuple_union">tuple_union (class in datasketches)</a>
+</li>
+ <li><a href="tuple_policy.html#datasketches.TuplePolicy">TuplePolicy (class in datasketches)</a>
</li>
</ul></td>
</tr></table>
@@ -937,12 +1288,20 @@
<ul>
<li><a href="cpc.html#datasketches.cpc_sketch.update">(cpc_sketch attribute)</a>
+</li>
+ <li><a href="cpc.html#datasketches.cpc_union.update">(cpc_union attribute)</a>
</li>
<li><a href="density_sketch.html#datasketches.density_sketch.update">(density_sketch attribute)</a>
+</li>
+ <li><a href="ebpps.html#datasketches.ebpps_sketch.update">(ebpps_sketch attribute)</a>
</li>
<li><a href="frequent_items.html#datasketches.frequent_items_sketch.update">(frequent_items_sketch attribute)</a>
+</li>
+ <li><a href="frequent_items.html#datasketches.frequent_strings_sketch.update">(frequent_strings_sketch attribute)</a>
</li>
<li><a href="hyper_log_log.html#datasketches.hll_sketch.update">(hll_sketch attribute)</a>
+</li>
+ <li><a href="hyper_log_log.html#datasketches.hll_union.update">(hll_union attribute)</a>
</li>
<li><a href="kll.html#datasketches.kll_doubles_sketch.update">(kll_doubles_sketch attribute)</a>
</li>
@@ -965,18 +1324,44 @@
<li><a href="req.html#datasketches.req_ints_sketch.update">(req_ints_sketch attribute)</a>
</li>
<li><a href="req.html#datasketches.req_items_sketch.update">(req_items_sketch attribute)</a>
+</li>
+ <li><a href="theta.html#datasketches.theta_intersection.update">(theta_intersection attribute)</a>
+</li>
+ <li><a href="theta.html#datasketches.theta_union.update">(theta_union attribute)</a>
+</li>
+ <li><a href="tuple.html#datasketches.tuple_intersection.update">(tuple_intersection attribute)</a>
+</li>
+ <li><a href="tuple.html#datasketches.tuple_union.update">(tuple_union attribute)</a>
+</li>
+ <li><a href="theta.html#datasketches.update_theta_sketch.update">(update_theta_sketch attribute)</a>
+</li>
+ <li><a href="tuple.html#datasketches.update_tuple_sketch.update">(update_tuple_sketch attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_sketch.update">(var_opt_sketch attribute)</a>
+</li>
+ <li><a href="varopt.html#datasketches.var_opt_union.update">(var_opt_union attribute)</a>
</li>
</ul></li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="tuple.html#datasketches.AccumulatorPolicy.update_summary">update_summary() (AccumulatorPolicy method)</a>
+ <li><a href="tuple_policy.html#datasketches.TuplePolicy.update_summary">update_summary() (TuplePolicy method)</a>
+</li>
+ <li><a href="theta.html#datasketches.update_theta_sketch">update_theta_sketch (class in datasketches)</a>
+</li>
+ <li><a href="tuple.html#datasketches.update_tuple_sketch">update_tuple_sketch (class in datasketches)</a>
+</li>
+ </ul></td>
+</tr></table>
- <ul>
- <li><a href="tuple.html#datasketches.MaxIntPolicy.update_summary">(MaxIntPolicy method)</a>
+<h2 id="V">V</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="varopt.html#datasketches.var_opt_sketch">var_opt_sketch (class in datasketches)</a>
</li>
- <li><a href="tuple.html#datasketches.MinIntPolicy.update_summary">(MinIntPolicy method)</a>
+ </ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="varopt.html#datasketches.var_opt_union">var_opt_union (class in datasketches)</a>
</li>
- </ul></li>
</ul></td>
</tr></table>
diff --git a/docs/main/hyper_log_log.html b/docs/main/hyper_log_log.html
index 7c6d808..939d947 100644
--- a/docs/main/hyper_log_log.html
+++ b/docs/main/hyper_log_log.html
@@ -46,10 +46,17 @@
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<ul class="current">
<li class="toctree-l1 current"><a class="current reference internal" href="#">HyperLogLog (HLL)</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.tgt_hll_type"><code class="docutils literal notranslate"><span class="pre">tgt_hll_type</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tgt_hll_type.HLL_4"><code class="docutils literal notranslate"><span class="pre">tgt_hll_type.HLL_4</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tgt_hll_type.HLL_6"><code class="docutils literal notranslate"><span class="pre">tgt_hll_type.HLL_6</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tgt_hll_type.HLL_8"><code class="docutils literal notranslate"><span class="pre">tgt_hll_type.HLL_8</span></code></a></li>
+</ul>
+</li>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.hll_sketch"><code class="docutils literal notranslate"><span class="pre">hll_sketch</span></code></a><ul>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">hll_sketch.deserialize()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_sketch.get_max_updatable_serialization_bytes"><code class="docutils literal notranslate"><span class="pre">hll_sketch.get_max_updatable_serialization_bytes()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_sketch.get_rel_err"><code class="docutils literal notranslate"><span class="pre">hll_sketch.get_rel_err()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">hll_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_sketch.get_compact_serialization_bytes"><code class="docutils literal notranslate"><span class="pre">hll_sketch.get_compact_serialization_bytes</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_sketch.get_estimate"><code class="docutils literal notranslate"><span class="pre">hll_sketch.get_estimate</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_sketch.get_lower_bound"><code class="docutils literal notranslate"><span class="pre">hll_sketch.get_lower_bound</span></code></a></li>
@@ -66,6 +73,19 @@
<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_sketch.update"><code class="docutils literal notranslate"><span class="pre">hll_sketch.update</span></code></a></li>
</ul>
</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.hll_union"><code class="docutils literal notranslate"><span class="pre">hll_union</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_union.get_rel_err"><code class="docutils literal notranslate"><span class="pre">hll_union.get_rel_err()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_union.__init__"><code class="docutils literal notranslate"><span class="pre">hll_union.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_union.get_estimate"><code class="docutils literal notranslate"><span class="pre">hll_union.get_estimate</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_union.get_lower_bound"><code class="docutils literal notranslate"><span class="pre">hll_union.get_lower_bound</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_union.get_result"><code class="docutils literal notranslate"><span class="pre">hll_union.get_result</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_union.get_upper_bound"><code class="docutils literal notranslate"><span class="pre">hll_union.get_upper_bound</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_union.is_empty"><code class="docutils literal notranslate"><span class="pre">hll_union.is_empty</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_union.lg_config_k"><code class="docutils literal notranslate"><span class="pre">hll_union.lg_config_k</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_union.reset"><code class="docutils literal notranslate"><span class="pre">hll_union.reset</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.hll_union.update"><code class="docutils literal notranslate"><span class="pre">hll_union.update</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="cpc.html">Compressed Probabilistic Counting (CPC)</a></li>
@@ -85,6 +105,17 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -117,21 +148,34 @@
<p>If the ONLY use case for sketching is counting uniques and merging, the HLL sketch is a reasonable choice, although the highest performing in terms of accuracy for storage space consumed is CPC (Compressed Probabilistic Counting). For large enough counts, this HLL version (with HLL_4) can be 2 to 16 times smaller than the Theta sketch family for the same accuracy.</p>
<p>This implementation offers three different types of HLL sketch, each with different trade-offs with accuracy, space and performance.
These types are specified with the target_hll_type parameter.</p>
-<p>In terms of accuracy, all three types, for the same lg_config_k, have the same error distribution as a function of n, the number of unique values fed to the sketch.
-The configuration parameter <cite>lg_config_k</cite> is the log-base-2 of <cite>K</cite>, where <cite>K</cite> is the number of buckets or slots for the sketch.</p>
-<p>During warmup, when the sketch has only received a small number of unique items (up to about 10% of <cite>K</cite>), this implementation leverages a new class of estimator algorithms with significantly better accuracy.</p>
-<p>This sketch also offers the capability of operating off-heap.
-Given a WritableMemory object created by the user, the sketch will perform all of its updates and internal phase transitions in that object, which can actually reside either on-heap or off-heap based on how it is configured.
-In large systems that must update and merge many millions of sketches, having the sketch operate off-heap avoids the serialization and deserialization costs of moving sketches to and from off-heap memory-mapped files, for example, and eliminates big garbage collection delays.</p>
+<p>In terms of accuracy, all three types, for the same lg_config_k, have the same error distribution as a function of <code class="docutils literal notranslate"><span class="pre">n</span></code>, the number of unique values fed to the sketch.
+The configuration parameter <code class="docutils literal notranslate"><span class="pre">lg_config_k</span></code> is the log-base-2 of <code class="docutils literal notranslate"><span class="pre">k</span></code>, where <code class="docutils literal notranslate"><span class="pre">k</span></code> is the number of buckets or slots for the sketch.</p>
+<p>During warmup, when the sketch has only received a small number of unique items (up to about 10% of <code class="docutils literal notranslate"><span class="pre">k</span></code>), this implementation leverages a new class of estimator algorithms with significantly better accuracy.</p>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.tgt_hll_type">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">tgt_hll_type</span></span><a class="headerlink" href="#datasketches.tgt_hll_type" title="Link to this definition"></a></dt>
+<dd><p>Target HLL flavor</p>
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.tgt_hll_type.HLL_4">
+<span class="sig-name descname"><span class="pre">HLL_4</span></span><em class="property"><span class="w"> </span><span class="pre">:</span> <span class="pre">4</span> <span class="pre">bits</span> <span class="pre">per</span> <span class="pre">entry</span></em><a class="headerlink" href="#datasketches.tgt_hll_type.HLL_4" title="Link to this definition"></a></dt>
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.tgt_hll_type.HLL_6">
+<span class="sig-name descname"><span class="pre">HLL_6</span></span><em class="property"><span class="w"> </span><span class="pre">:</span> <span class="pre">6</span> <span class="pre">bits</span> <span class="pre">per</span> <span class="pre">entry</span></em><a class="headerlink" href="#datasketches.tgt_hll_type.HLL_6" title="Link to this definition"></a></dt>
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.tgt_hll_type.HLL_8">
+<span class="sig-name descname"><span class="pre">HLL_8</span></span><em class="property"><span class="w"> </span><span class="pre">:</span> <span class="pre">8</span> <span class="pre">bits</span> <span class="pre">per</span> <span class="pre">entry</span></em><a class="headerlink" href="#datasketches.tgt_hll_type.HLL_8" title="Link to this definition"></a></dt>
+<dd></dd></dl>
+
+</dd></dl>
+
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.hll_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">hll_sketch</span></span><a class="headerlink" href="#datasketches.hll_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="field-list simple">
-<dt class="field-odd">Member-order<span class="colon">:</span></dt>
-<dd class="field-odd"><p>groupwise</p>
-</dd>
-</dl>
-<p class="rubric">Static Methods:</p>
+<dd><p class="rubric">Static Methods:</p>
<dl class="py method">
<dt class="sig sig-object py" id="datasketches.hll_sketch.deserialize">
<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#data [...]
@@ -140,7 +184,7 @@ In large systems that must update and merge many millions of sketches, having th
<dl class="py method">
<dt class="sig sig-object py" id="datasketches.hll_sketch.get_max_updatable_serialization_bytes">
-<span class="sig-name descname"><span class="pre">get_max_updatable_serialization_bytes</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lg_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tgt_type</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><sp [...]
+<span class="sig-name descname"><span class="pre">get_max_updatable_serialization_bytes</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">lg_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tgt_type</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><sp [...]
<dd><p>Provides a likely upper bound on serialization size for the given parameters</p>
</dd></dl>
@@ -151,6 +195,21 @@ In large systems that must update and merge many millions of sketches, having th
</dd></dl>
<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.hll_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lg_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">tgt_type</span></span><span class="p"><span cl [...]
+<dd><p>Constructs a new HLL sketch</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>lg_config_k</strong> (<em>int</em>) – A full sketch can hold 2^lg_config_k rows. Must be between 7 and 21, inclusive,</p></li>
+<li><p><strong>tgt_type</strong> (<a class="reference internal" href="#datasketches.tgt_hll_type" title="_datasketches.tgt_hll_type"><em>tgt_hll_type</em></a>) – The HLL mode to use, if/when the sketch reaches estimation mode</p></li>
+<li><p><strong>start_full_size</strong> (<em>bool</em>) – Indicates whether to start in HLL mode, keeping memory use constant (if HLL_6 or HLL_8) at the cost of much higher initial memory use. Default (and recommended) is False.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.hll_sketch.get_compact_serialization_bytes">
<span class="sig-name descname"><span class="pre">get_compact_serialization_bytes</span></span><a class="headerlink" href="#datasketches.hll_sketch.get_compact_serialization_bytes" title="Link to this definition"></a></dt>
@@ -249,6 +308,94 @@ In large systems that must update and merge many millions of sketches, having th
</dd></dl>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.hll_union">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">hll_union</span></span><a class="headerlink" href="#datasketches.hll_union" title="Link to this definition"></a></dt>
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.hll_union.get_rel_err">
+<span class="sig-name descname"><span class="pre">get_rel_err</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">upper_bound</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">unioned</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span [...]
+<dd><p>Returns the a priori relative error bound for the given parameters</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.hll_union.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lg_max_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</sp [...]
+<dd><p>Construct an hll_union object if the given size.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>lg_max_k</strong> (<em>int</em>) – The maximum size, in log2, of k. Must be between 7 and 21, inclusive.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.hll_union.get_estimate">
+<span class="sig-name descname"><span class="pre">get_estimate</span></span><a class="headerlink" href="#datasketches.hll_union.get_estimate" title="Link to this definition"></a></dt>
+<dd><p>Estimate of the distinct count of the input stream</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.hll_union.get_lower_bound">
+<span class="sig-name descname"><span class="pre">get_lower_bound</span></span><a class="headerlink" href="#datasketches.hll_union.get_lower_bound" title="Link to this definition"></a></dt>
+<dd><p>Returns the approximate lower error bound given the specified number of standard deviations in {1, 2, 3}</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.hll_union.get_result">
+<span class="sig-name descname"><span class="pre">get_result</span></span><a class="headerlink" href="#datasketches.hll_union.get_result" title="Link to this definition"></a></dt>
+<dd><p>Returns a sketch of the target type representing the current union state</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.hll_union.get_upper_bound">
+<span class="sig-name descname"><span class="pre">get_upper_bound</span></span><a class="headerlink" href="#datasketches.hll_union.get_upper_bound" title="Link to this definition"></a></dt>
+<dd><p>Returns the approximate upper error bound given the specified number of standard deviations in {1, 2, 3}</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.hll_union.is_empty">
+<span class="sig-name descname"><span class="pre">is_empty</span></span><a class="headerlink" href="#datasketches.hll_union.is_empty" title="Link to this definition"></a></dt>
+<dd><p>True if the union is empty, otherwise False</p>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="datasketches.hll_union.lg_config_k">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">lg_config_k</span></span><a class="headerlink" href="#datasketches.hll_union.lg_config_k" title="Link to this definition"></a></dt>
+<dd><p>Configured lg_k value for the union</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.hll_union.reset">
+<span class="sig-name descname"><span class="pre">reset</span></span><a class="headerlink" href="#datasketches.hll_union.reset" title="Link to this definition"></a></dt>
+<dd><p>Resets the union to the empty state</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.hll_union.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.hll_union.update" title="Link to this definition"></a></dt>
+<dd><p>Overloaded function.</p>
+<ol class="arabic simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">update(self,</span> <span class="pre">sketch:</span> <span class="pre">_datasketches.hll_sketch)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Updates the union with the given HLL sketch</p>
+<ol class="arabic simple" start="2">
+<li><p><code class="docutils literal notranslate"><span class="pre">update(self,</span> <span class="pre">datum:</span> <span class="pre">int)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Updates the union with the given integral value</p>
+<ol class="arabic simple" start="3">
+<li><p><code class="docutils literal notranslate"><span class="pre">update(self,</span> <span class="pre">datum:</span> <span class="pre">float)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Updates the union with the given floating point value</p>
+<ol class="arabic simple" start="4">
+<li><p><code class="docutils literal notranslate"><span class="pre">update(self,</span> <span class="pre">datum:</span> <span class="pre">str)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Updates the union with the given string value</p>
+</dd></dl>
+
+</dd></dl>
+
</section>
diff --git a/docs/main/index.html b/docs/main/index.html
index bdf532c..7b537a2 100644
--- a/docs/main/index.html
+++ b/docs/main/index.html
@@ -62,6 +62,17 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -136,6 +147,30 @@
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
</ul>
</div>
+</section>
+<section id="random-sampling">
+<h2>Random Sampling<a class="headerlink" href="#random-sampling" title="Link to this heading"></a></h2>
+<div class="toctree-wrapper compound">
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+</div>
+</section>
+<section id="helper-classes">
+<h2>Helper Classes<a class="headerlink" href="#helper-classes" title="Link to this heading"></a></h2>
+<div class="toctree-wrapper compound">
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
+</ul>
+</div>
+<p>These classes are required for certain sketches or specific functionality withing sketches.
+Some of them are abstract base classes, but in those cases there is at least one reference example of a
+concrete class.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>This project is under active development.</p>
diff --git a/docs/main/jaccard.html b/docs/main/jaccard.html
new file mode 100644
index 0000000..c9ff2bd
--- /dev/null
+++ b/docs/main/jaccard.html
@@ -0,0 +1,217 @@
+<!DOCTYPE html>
+<html class="writer-html5" lang="en" data-content_root="./">
+<head>
+ <meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
+
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <title>Jaccard Similarity — datasketches 0.1 documentation</title>
+ <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
+ <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+
+
+ <!--[if lt IE 9]>
+ <script src="_static/js/html5shiv.min.js"></script>
+ <![endif]-->
+
+ <script src="_static/jquery.js?v=5d32c60e"></script>
+ <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
+ <script src="_static/documentation_options.js?v=2709fde1"></script>
+ <script src="_static/doctools.js?v=888ff710"></script>
+ <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+ <script src="_static/js/theme.js"></script>
+ <link rel="index" title="Index" href="genindex.html" />
+ <link rel="search" title="Search" href="search.html" />
+ <link rel="next" title="Tuple Policy" href="tuple_policy.html" />
+ <link rel="prev" title="Kernel Function" href="kernel.html" />
+</head>
+
+<body class="wy-body-for-nav">
+ <div class="wy-grid-for-nav">
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-scroll">
+ <div class="wy-side-nav-search" >
+
+
+
+ <a href="index.html" class="icon icon-home">
+ datasketches
+ </a>
+<div role="search">
+ <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
+ <ul>
+<li class="toctree-l1"><a class="reference internal" href="hyper_log_log.html">HyperLogLog (HLL)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="cpc.html">Compressed Probabilistic Counting (CPC)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="theta.html">Theta Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple.html">Tuple Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="density_sketch.html">Density Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="count_min_sketch.html">CountMin Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="frequent_items.html">Frequent Items</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1 current"><a class="current reference internal" href="#">Jaccard Similarity</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.theta_jaccard_similarity"><code class="docutils literal notranslate"><span class="pre">theta_jaccard_similarity</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_jaccard_similarity.jaccard"><code class="docutils literal notranslate"><span class="pre">theta_jaccard_similarity.jaccard()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_jaccard_similarity.exactly_equal"><code class="docutils literal notranslate"><span class="pre">theta_jaccard_similarity.exactly_equal()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_jaccard_similarity.similarity_test"><code class="docutils literal notranslate"><span class="pre">theta_jaccard_similarity.similarity_test()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_jaccard_similarity.dissimilarity_test"><code class="docutils literal notranslate"><span class="pre">theta_jaccard_similarity.dissimilarity_test()</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.tuple_jaccard_similarity"><code class="docutils literal notranslate"><span class="pre">tuple_jaccard_similarity</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_jaccard_similarity.jaccard"><code class="docutils literal notranslate"><span class="pre">tuple_jaccard_similarity.jaccard()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_jaccard_similarity.exactly_equal"><code class="docutils literal notranslate"><span class="pre">tuple_jaccard_similarity.exactly_equal()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_jaccard_similarity.similarity_test"><code class="docutils literal notranslate"><span class="pre">tuple_jaccard_similarity.similarity_test()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_jaccard_similarity.dissimilarity_test"><code class="docutils literal notranslate"><span class="pre">tuple_jaccard_similarity.dissimilarity_test()</span></code></a></li>
+</ul>
+</li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
+</ul>
+
+ </div>
+ </div>
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">datasketches</a>
+ </nav>
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="Page navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
+ <li class="breadcrumb-item active">Jaccard Similarity</li>
+ <li class="wy-breadcrumbs-aside">
+ <a href="_sources/jaccard.rst.txt" rel="nofollow"> View page source</a>
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+ <div itemprop="articleBody">
+
+ <section id="jaccard-similarity">
+<h1>Jaccard Similarity<a class="headerlink" href="#jaccard-similarity" title="Link to this heading"></a></h1>
+<p>These objects provide measures related to the <a class="reference external" href="https://en.wikipedia.org/wiki/Jaccard_similarity">Jaccard similarity</a>
+of <a class="reference internal" href="theta.html#datasketches.theta_sketch" title="datasketches.theta_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">theta_sketch</span></code></a> and <a class="reference internal" href="tuple.html#datasketches.tuple_sketch" title="datasketches.tuple_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">tuple_sketch</span></code></a> objects.</p>
+<p>Note that there are separate classes to be used for theta and tuple sketches.</p>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.theta_jaccard_similarity">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">theta_jaccard_similarity</span></span><a class="headerlink" href="#datasketches.theta_jaccard_similarity" title="Link to this definition"></a></dt>
+<dd><p>An object to help compute Jaccard similarity between theta sketches.</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.theta_jaccard_similarity.jaccard">
+<span class="sig-name descname"><span class="pre">jaccard</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sketch_a</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="theta.html#datasketches.theta_sketch" title="_datasketches.theta_sketch"><span class="pre">_datasketches.theta_sketch</span></a></span></em>, <em class="sig-param"><span class="n"><span [...]
+<dd><p>Returns a list with {lower_bound, estimate, upper_bound} of the Jaccard similarity between sketches</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.theta_jaccard_similarity.exactly_equal">
+<span class="sig-name descname"><span class="pre">exactly_equal</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sketch_a</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="theta.html#datasketches.theta_sketch" title="_datasketches.theta_sketch"><span class="pre">_datasketches.theta_sketch</span></a></span></em>, <em class="sig-param"><span class="n"> [...]
+<dd><p>Returns True if sketch_a and sketch_b are equivalent, otherwise False</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.theta_jaccard_similarity.similarity_test">
+<span class="sig-name descname"><span class="pre">similarity_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">actual</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="theta.html#datasketches.theta_sketch" title="_datasketches.theta_sketch"><span class="pre">_datasketches.theta_sketch</span></a></span></em>, <em class="sig-param"><span class="n"> [...]
+<dd><p>Tests similarity of an actual sketch against an expected sketch. Computers the lower bound of the Jaccard index J_{LB} of the actual and expected sketches. If J_{LB} >= threshold, then the sketches are considered to be similar with a confidence of 97.7% and returns True, otherwise False.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.theta_jaccard_similarity.dissimilarity_test">
+<span class="sig-name descname"><span class="pre">dissimilarity_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">actual</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="theta.html#datasketches.theta_sketch" title="_datasketches.theta_sketch"><span class="pre">_datasketches.theta_sketch</span></a></span></em>, <em class="sig-param"><span class=" [...]
+<dd><p>Tests dissimilarity of an actual sketch against an expected sketch. Computers the lower bound of the Jaccard index J_{UB} of the actual and expected sketches. If J_{UB} <= threshold, then the sketches are considered to be dissimilar with a confidence of 97.7% and returns True, otherwise False.</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.tuple_jaccard_similarity">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">tuple_jaccard_similarity</span></span><a class="headerlink" href="#datasketches.tuple_jaccard_similarity" title="Link to this definition"></a></dt>
+<dd><p>An object to help compute Jaccard similarity between tuple sketches.</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.tuple_jaccard_similarity.jaccard">
+<span class="sig-name descname"><span class="pre">jaccard</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sketch_a</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="tuple.html#datasketches.tuple_sketch" title="_datasketches.tuple_sketch"><span class="pre">_datasketches.tuple_sketch</span></a></span></em>, <em class="sig-param"><span class="n"><span [...]
+<dd><p>Returns a list with {lower_bound, estimate, upper_bound} of the Jaccard similarity between sketches</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.tuple_jaccard_similarity.exactly_equal">
+<span class="sig-name descname"><span class="pre">exactly_equal</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sketch_a</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="tuple.html#datasketches.tuple_sketch" title="_datasketches.tuple_sketch"><span class="pre">_datasketches.tuple_sketch</span></a></span></em>, <em class="sig-param"><span class="n"> [...]
+<dd><p>Returns True if sketch_a and sketch_b are equivalent, otherwise False</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.tuple_jaccard_similarity.similarity_test">
+<span class="sig-name descname"><span class="pre">similarity_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">actual</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="tuple.html#datasketches.tuple_sketch" title="_datasketches.tuple_sketch"><span class="pre">_datasketches.tuple_sketch</span></a></span></em>, <em class="sig-param"><span class="n"> [...]
+<dd><p>Tests similarity of an actual sketch against an expected sketch. Computes the lower bound of the Jaccard index J_{LB} of the actual and expected sketches. If J_{LB} >= threshold, then the sketches are considered to be similar with a confidence of 97.7% and returns True, otherwise False.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.tuple_jaccard_similarity.dissimilarity_test">
+<span class="sig-name descname"><span class="pre">dissimilarity_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">actual</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="tuple.html#datasketches.tuple_sketch" title="_datasketches.tuple_sketch"><span class="pre">_datasketches.tuple_sketch</span></a></span></em>, <em class="sig-param"><span class=" [...]
+<dd><p>Tests dissimilarity of an actual sketch against an expected sketch. Computes the upper bound of the Jaccard index J_{UB} of the actual and expected sketches. If J_{UB} <= threshold, then the sketches are considered to be dissimilar with a confidence of 97.7% and returns True, otherwise False.</p>
+</dd></dl>
+
+</dd></dl>
+
+</section>
+
+
+ </div>
+ </div>
+ <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
+ <a href="kernel.html" class="btn btn-neutral float-left" title="Kernel Function" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+ <a href="tuple_policy.html" class="btn btn-neutral float-right" title="Tuple Policy" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
+ </div>
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>© Copyright 2023.</p>
+ </div>
+
+ Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
+ <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
+ provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+
+</footer>
+ </div>
+ </div>
+ </section>
+ </div>
+ <script>
+ jQuery(function () {
+ SphinxRtdTheme.Navigation.enable(true);
+ });
+ </script>
+
+</body>
+</html>
\ No newline at end of file
diff --git a/docs/main/kernel.html b/docs/main/kernel.html
new file mode 100644
index 0000000..5e8ab1f
--- /dev/null
+++ b/docs/main/kernel.html
@@ -0,0 +1,191 @@
+<!DOCTYPE html>
+<html class="writer-html5" lang="en" data-content_root="./">
+<head>
+ <meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
+
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <title>Kernel Function — datasketches 0.1 documentation</title>
+ <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
+ <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+
+
+ <!--[if lt IE 9]>
+ <script src="_static/js/html5shiv.min.js"></script>
+ <![endif]-->
+
+ <script src="_static/jquery.js?v=5d32c60e"></script>
+ <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
+ <script src="_static/documentation_options.js?v=2709fde1"></script>
+ <script src="_static/doctools.js?v=888ff710"></script>
+ <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+ <script src="_static/js/theme.js"></script>
+ <link rel="index" title="Index" href="genindex.html" />
+ <link rel="search" title="Search" href="search.html" />
+ <link rel="next" title="Jaccard Similarity" href="jaccard.html" />
+ <link rel="prev" title="Serialize/Deserialize (SerDe)" href="serde.html" />
+</head>
+
+<body class="wy-body-for-nav">
+ <div class="wy-grid-for-nav">
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-scroll">
+ <div class="wy-side-nav-search" >
+
+
+
+ <a href="index.html" class="icon icon-home">
+ datasketches
+ </a>
+<div role="search">
+ <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
+ <ul>
+<li class="toctree-l1"><a class="reference internal" href="hyper_log_log.html">HyperLogLog (HLL)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="cpc.html">Compressed Probabilistic Counting (CPC)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="theta.html">Theta Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple.html">Tuple Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="density_sketch.html">Density Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="count_min_sketch.html">CountMin Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="frequent_items.html">Frequent Items</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1 current"><a class="current reference internal" href="#">Kernel Function</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.KernelFunction"><code class="docutils literal notranslate"><span class="pre">KernelFunction</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.KernelFunction.__call__"><code class="docutils literal notranslate"><span class="pre">KernelFunction.__call__()</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.GaussianKernel"><code class="docutils literal notranslate"><span class="pre">GaussianKernel</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
+</ul>
+
+ </div>
+ </div>
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">datasketches</a>
+ </nav>
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="Page navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
+ <li class="breadcrumb-item active">Kernel Function</li>
+ <li class="wy-breadcrumbs-aside">
+ <a href="_sources/kernel.rst.txt" rel="nofollow"> View page source</a>
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+ <div itemprop="articleBody">
+
+ <section id="kernel-function">
+<h1>Kernel Function<a class="headerlink" href="#kernel-function" title="Link to this heading"></a></h1>
+<p>A <a class="reference external" href="https://en.wikipedia.org/wiki/Positive-definite_kernel">kernel function</a> is a specific type of
+mathematical funciton that is particularly useful in certain machine learning and pattern recognition
+contexts. The <a class="reference internal" href="density_sketch.html#datasketches.density_sketch" title="datasketches.density_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">density_sketch</span></code></a> performs approximate
+<a class="reference external" href="https://en.wikipedia.org/wiki/Kernel_density_estimation">kernel density estimation</a> which, unsurprisingly,
+relies on the use of such a kernel function.</p>
+<p>The library provides an abstract base class <a class="reference internal" href="#datasketches.KernelFunction" title="datasketches.KernelFunction"><code class="xref py py-class docutils literal notranslate"><span class="pre">KernelFunction</span></code></a> and an example implementation of a
+Gaussian (also known as a Radial Basis Function) kernel. Custom classes must override the base class
+and provide a floating point value as a score indicating the similarity of two input vectors.</p>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.KernelFunction">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">KernelFunction</span></span><a class="headerlink" href="#datasketches.KernelFunction" title="Link to this definition"></a></dt>
+<dd><p>A generic base class from which user-defined kernels must inherit.</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.KernelFunction.__call__">
+<span class="sig-name descname"><span class="pre">__call__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">a</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">object</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">b</span></span><span class="p"><span class="pr [...]
+<dd><p>A method to evaluate a kernel with given inputs a and b.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>a</strong> (<em>numpy array</em>) – An input vector</p></li>
+<li><p><strong>b</strong> (<em>numpy array</em>) – An input vector</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns<span class="colon">:</span></dt>
+<dd class="field-even"><p>A vector similarity score</p>
+</dd>
+<dt class="field-odd">Return type<span class="colon">:</span></dt>
+<dd class="field-odd"><p>float</p>
+</dd>
+</dl>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.GaussianKernel">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">GaussianKernel</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bandwidth</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span> [...]
+<dd><p>Bases: <a class="reference internal" href="#datasketches.KernelFunction" title="_datasketches.KernelFunction"><code class="xref py py-class docutils literal notranslate"><span class="pre">KernelFunction</span></code></a></p>
+<p>Implements a basic Gaussian kernel</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>bandwidth</strong> (<em>float</em>) – The kernel bandwidth, default 1.0</p>
+</dd>
+</dl>
+</dd></dl>
+
+</section>
+
+
+ </div>
+ </div>
+ <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
+ <a href="serde.html" class="btn btn-neutral float-left" title="Serialize/Deserialize (SerDe)" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+ <a href="jaccard.html" class="btn btn-neutral float-right" title="Jaccard Similarity" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
+ </div>
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>© Copyright 2023.</p>
+ </div>
+
+ Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
+ <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
+ provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+
+</footer>
+ </div>
+ </div>
+ </section>
+ </div>
+ <script>
+ jQuery(function () {
+ SphinxRtdTheme.Navigation.enable(true);
+ });
+ </script>
+
+</body>
+</html>
\ No newline at end of file
diff --git a/docs/main/kll.html b/docs/main/kll.html
index 5af2928..b36411f 100644
--- a/docs/main/kll.html
+++ b/docs/main/kll.html
@@ -62,11 +62,12 @@
<ul class="current">
<li class="toctree-l1 current"><a class="current reference internal" href="#">KLL Sketch</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.kll_ints_sketch"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.deserialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.get_normalized_rank_error()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.get_min_value</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.get_normalized_rank_error</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.get_pmf"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.get_pmf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.get_quantile"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.get_quantile</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_ints_sketch.get_quantiles"><code class="docutils literal notranslate"><span class="pre">kll_ints_sketch.get_quantiles</span></code></a></li>
@@ -84,11 +85,12 @@
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.kll_floats_sketch"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.deserialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.get_normalized_rank_error()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.get_min_value</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.get_normalized_rank_error</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.get_pmf"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.get_pmf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.get_quantile"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.get_quantile</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_floats_sketch.get_quantiles"><code class="docutils literal notranslate"><span class="pre">kll_floats_sketch.get_quantiles</span></code></a></li>
@@ -106,11 +108,12 @@
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.kll_doubles_sketch"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.deserialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.get_normalized_rank_error()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.get_min_value</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.get_normalized_rank_error</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.get_pmf"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.get_pmf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.get_quantile"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.get_quantile</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_doubles_sketch.get_quantiles"><code class="docutils literal notranslate"><span class="pre">kll_doubles_sketch.get_quantiles</span></code></a></li>
@@ -128,11 +131,12 @@
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.kll_items_sketch"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.deserialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.get_normalized_rank_error()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.get_min_value</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.get_normalized_rank_error</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.get_pmf"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.get_pmf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.get_quantile"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.get_quantile</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.kll_items_sketch.get_quantiles"><code class="docutils literal notranslate"><span class="pre">kll_items_sketch.get_quantiles</span></code></a></li>
@@ -153,6 +157,17 @@
</li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -268,13 +283,44 @@ Additionally, the interval may be quite large for certain distributions.
- Let <cite>q_lo = estimated quantile of rank (r - eps)</cite>.
- Let <cite>q_hi = estimated quantile of rank (r + eps)</cite>.
- Then <cite>q_lo ≤ q ≤ q_hi</cite>, with 99% confidence.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>For the <a class="reference internal" href="#datasketches.kll_items_sketch" title="datasketches.kll_items_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">kll_items_sketch</span></code></a>, objects must be comparable with <code class="docutils literal notranslate"><span class="pre">__lt__</span></code>.</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Serializing and deserializing a <a class="reference internal" href="#datasketches.kll_items_sketch" title="datasketches.kll_items_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">kll_items_sketch</span></code></a> requires the use of a <a class="reference internal" href="serde.html#datasketches.PyObjectSerDe" title="datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code>< [...]
+</div>
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.kll_ints_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">kll_ints_sketch</span></span><a class="headerlink" href="#datasketches.kll_ints_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.kll_ints_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.kll_ints_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#data [...]
+<dd><p>Deserializes the sketch from a bytes object.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.kll_ints_sketch.get_normalized_rank_error">
+<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">as_pmf</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><spa [...]
+<dd><p>Gets the normalized rank error given parameters k and the pmf flag.
+If pmf is True, returns the ‘double-sided’ normalized rank error for the get_PMF() function.
+Otherwise, it is the ‘single-sided’ normalized rank error for all the other queries.
+Constants were derived as the best fit to 99 percentile empirically measured max error in thousands of trials</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.kll_ints_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates a KLL sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 200.</p>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.kll_ints_sketch.get_cdf">
@@ -300,11 +346,6 @@ It is not necessary to include either the min or max values in these split point
<dd><p>Returns the minimum value from the stream. If empty, kll_floats_sketch returns nan; kll_ints_sketch throws a RuntimeError</p>
</dd></dl>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datasketches.kll_ints_sketch.get_normalized_rank_error">
-<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.kll_ints_sketch.get_normalized_rank_error" title="Link to this definition"></a></dt>
-<dd></dd></dl>
-
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.kll_ints_sketch.get_pmf">
<span class="sig-name descname"><span class="pre">get_pmf</span></span><a class="headerlink" href="#datasketches.kll_ints_sketch.get_pmf" title="Link to this definition"></a></dt>
@@ -416,10 +457,33 @@ Constants were derived as the best fit to 99 percentile empirically measured max
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.kll_floats_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">kll_floats_sketch</span></span><a class="headerlink" href="#datasketches.kll_floats_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.kll_floats_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.kll_floats_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#data [...]
+<dd><p>Deserializes the sketch from a bytes object.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.kll_floats_sketch.get_normalized_rank_error">
+<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">as_pmf</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><spa [...]
+<dd><p>Gets the normalized rank error given parameters k and the pmf flag.
+If pmf is True, returns the ‘double-sided’ normalized rank error for the get_PMF() function.
+Otherwise, it is the ‘single-sided’ normalized rank error for all the other queries.
+Constants were derived as the best fit to 99 percentile empirically measured max error in thousands of trials</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.kll_floats_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates a KLL sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 200.</p>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.kll_floats_sketch.get_cdf">
@@ -445,11 +509,6 @@ It is not necessary to include either the min or max values in these split point
<dd><p>Returns the minimum value from the stream. If empty, kll_floats_sketch returns nan; kll_ints_sketch throws a RuntimeError</p>
</dd></dl>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datasketches.kll_floats_sketch.get_normalized_rank_error">
-<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.kll_floats_sketch.get_normalized_rank_error" title="Link to this definition"></a></dt>
-<dd></dd></dl>
-
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.kll_floats_sketch.get_pmf">
<span class="sig-name descname"><span class="pre">get_pmf</span></span><a class="headerlink" href="#datasketches.kll_floats_sketch.get_pmf" title="Link to this definition"></a></dt>
@@ -561,10 +620,33 @@ Constants were derived as the best fit to 99 percentile empirically measured max
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.kll_doubles_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">kll_doubles_sketch</span></span><a class="headerlink" href="#datasketches.kll_doubles_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.kll_doubles_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.kll_doubles_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#data [...]
+<dd><p>Deserializes the sketch from a bytes object.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.kll_doubles_sketch.get_normalized_rank_error">
+<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">as_pmf</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><spa [...]
+<dd><p>Gets the normalized rank error given parameters k and the pmf flag.
+If pmf is True, returns the ‘double-sided’ normalized rank error for the get_PMF() function.
+Otherwise, it is the ‘single-sided’ normalized rank error for all the other queries.
+Constants were derived as the best fit to 99 percentile empirically measured max error in thousands of trials</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.kll_doubles_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates a KLL sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 200.</p>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.kll_doubles_sketch.get_cdf">
@@ -590,11 +672,6 @@ It is not necessary to include either the min or max values in these split point
<dd><p>Returns the minimum value from the stream. If empty, kll_floats_sketch returns nan; kll_ints_sketch throws a RuntimeError</p>
</dd></dl>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datasketches.kll_doubles_sketch.get_normalized_rank_error">
-<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.kll_doubles_sketch.get_normalized_rank_error" title="Link to this definition"></a></dt>
-<dd></dd></dl>
-
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.kll_doubles_sketch.get_pmf">
<span class="sig-name descname"><span class="pre">get_pmf</span></span><a class="headerlink" href="#datasketches.kll_doubles_sketch.get_pmf" title="Link to this definition"></a></dt>
@@ -706,10 +783,33 @@ Constants were derived as the best fit to 99 percentile empirically measured max
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.kll_items_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">kll_items_sketch</span></span><a class="headerlink" href="#datasketches.kll_items_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.kll_items_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.kll_items_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">serde</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="ref [...]
+<dd><p>Deserializes the sketch from a bytes object using the provided serde.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.kll_items_sketch.get_normalized_rank_error">
+<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">as_pmf</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><spa [...]
+<dd><p>Gets the normalized rank error given parameters k and the pmf flag.
+If pmf is True, returns the ‘double-sided’ normalized rank error for the get_PMF() function.
+Otherwise, it is the ‘single-sided’ normalized rank error for all the other queries.
+Constants were derived as the best fit to 99 percentile empirically measured max error in thousands of trials</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.kll_items_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates a KLL sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 200.</p>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.kll_items_sketch.get_cdf">
@@ -735,11 +835,6 @@ It is not necessary to include either the min or max values in these split point
<dd><p>Returns the minimum value from the stream. If empty, kll_floats_sketch returns nan; kll_ints_sketch throws a RuntimeError</p>
</dd></dl>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datasketches.kll_items_sketch.get_normalized_rank_error">
-<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.kll_items_sketch.get_normalized_rank_error" title="Link to this definition"></a></dt>
-<dd></dd></dl>
-
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.kll_items_sketch.get_pmf">
<span class="sig-name descname"><span class="pre">get_pmf</span></span><a class="headerlink" href="#datasketches.kll_items_sketch.get_pmf" title="Link to this definition"></a></dt>
diff --git a/docs/main/ks_test.html b/docs/main/ks_test.html
new file mode 100644
index 0000000..f227f8f
--- /dev/null
+++ b/docs/main/ks_test.html
@@ -0,0 +1,213 @@
+<!DOCTYPE html>
+<html class="writer-html5" lang="en" data-content_root="./">
+<head>
+ <meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
+
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <title>Kolmogorov-Smirnov Test — datasketches 0.1 documentation</title>
+ <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
+ <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+
+
+ <!--[if lt IE 9]>
+ <script src="_static/js/html5shiv.min.js"></script>
+ <![endif]-->
+
+ <script src="_static/jquery.js?v=5d32c60e"></script>
+ <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
+ <script src="_static/documentation_options.js?v=2709fde1"></script>
+ <script src="_static/doctools.js?v=888ff710"></script>
+ <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+ <script src="_static/js/theme.js"></script>
+ <link rel="index" title="Index" href="genindex.html" />
+ <link rel="search" title="Search" href="search.html" />
+ <link rel="prev" title="Tuple Policy" href="tuple_policy.html" />
+</head>
+
+<body class="wy-body-for-nav">
+ <div class="wy-grid-for-nav">
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-scroll">
+ <div class="wy-side-nav-search" >
+
+
+
+ <a href="index.html" class="icon icon-home">
+ datasketches
+ </a>
+<div role="search">
+ <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
+ <ul>
+<li class="toctree-l1"><a class="reference internal" href="hyper_log_log.html">HyperLogLog (HLL)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="cpc.html">Compressed Probabilistic Counting (CPC)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="theta.html">Theta Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple.html">Tuple Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="density_sketch.html">Density Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="count_min_sketch.html">CountMin Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="frequent_items.html">Frequent Items</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1 current"><a class="current reference internal" href="#">Kolmogorov-Smirnov Test</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.ks_test"><code class="docutils literal notranslate"><span class="pre">ks_test()</span></code></a></li>
+</ul>
+</li>
+</ul>
+
+ </div>
+ </div>
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">datasketches</a>
+ </nav>
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="Page navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
+ <li class="breadcrumb-item active">Kolmogorov-Smirnov Test</li>
+ <li class="wy-breadcrumbs-aside">
+ <a href="_sources/ks_test.rst.txt" rel="nofollow"> View page source</a>
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+ <div itemprop="articleBody">
+
+ <section id="kolmogorov-smirnov-test">
+<h1>Kolmogorov-Smirnov Test<a class="headerlink" href="#kolmogorov-smirnov-test" title="Link to this heading"></a></h1>
+<p>A <cite>Kolmogorov-Smirnov Test <https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test>`_</cite> is
+a test of equality of two distributions to determine if they are likely to have come from the same
+underlying distribution.
+The DataSketches library provides a modified form of the test that takes into account the error
+in each underlying sketch in the analysis.</p>
+<p>Currently, the test assumes both input sketches are of the same family and data type.</p>
+<dl class="py function">
+<dt class="sig sig-object py" id="datasketches.ks_test">
+<span class="sig-name descname"><span class="pre">ks_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sk_1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="kll.html#datasketches.kll_ints_sketch" title="_datasketches.kll_ints_sketch"><span class="pre">_datasketches.kll_ints_sketch</span></a></span></em>, <em class="sig-param"><span class="n"><sp [...]
+<dt class="sig sig-object py">
+<span class="sig-name descname"><span class="pre">ks_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sk_1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="kll.html#datasketches.kll_floats_sketch" title="_datasketches.kll_floats_sketch"><span class="pre">_datasketches.kll_floats_sketch</span></a></span></em>, <em class="sig-param"><span class=" [...]
+<dt class="sig sig-object py">
+<span class="sig-name descname"><span class="pre">ks_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sk_1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="kll.html#datasketches.kll_doubles_sketch" title="_datasketches.kll_doubles_sketch"><span class="pre">_datasketches.kll_doubles_sketch</span></a></span></em>, <em class="sig-param"><span clas [...]
+<dt class="sig sig-object py">
+<span class="sig-name descname"><span class="pre">ks_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sk_1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="kll.html#datasketches.kll_items_sketch" title="_datasketches.kll_items_sketch"><span class="pre">_datasketches.kll_items_sketch</span></a></span></em>, <em class="sig-param"><span class="n"> [...]
+<dt class="sig sig-object py">
+<span class="sig-name descname"><span class="pre">ks_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sk_1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="quantiles_depr.html#datasketches.quantiles_ints_sketch" title="_datasketches.quantiles_ints_sketch"><span class="pre">_datasketches.quantiles_ints_sketch</span></a></span></em>, <em class="s [...]
+<dt class="sig sig-object py">
+<span class="sig-name descname"><span class="pre">ks_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sk_1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="quantiles_depr.html#datasketches.quantiles_floats_sketch" title="_datasketches.quantiles_floats_sketch"><span class="pre">_datasketches.quantiles_floats_sketch</span></a></span></em>, <em cl [...]
+<dt class="sig sig-object py">
+<span class="sig-name descname"><span class="pre">ks_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sk_1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="quantiles_depr.html#datasketches.quantiles_doubles_sketch" title="_datasketches.quantiles_doubles_sketch"><span class="pre">_datasketches.quantiles_doubles_sketch</span></a></span></em>, <em [...]
+<dt class="sig sig-object py">
+<span class="sig-name descname"><span class="pre">ks_test</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sk_1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="quantiles_depr.html#datasketches.quantiles_items_sketch" title="_datasketches.quantiles_items_sketch"><span class="pre">_datasketches.quantiles_items_sketch</span></a></span></em>, <em class [...]
+<dd><p>Overloaded function.</p>
+<ol class="arabic simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">ks_test(sk_1:</span> <span class="pre">_datasketches.kll_ints_sketch,</span> <span class="pre">sk_2:</span> <span class="pre">_datasketches.kll_ints_sketch,</span> <span class="pre">p:</span> <span class="pre">float)</span> <span class="pre">-></span> <span class="pre">bool</span></code></p></li>
+</ol>
+<p>Performs the Kolmogorov-Smirnov Test for <code class="code docutils literal notranslate"><span class="pre">kll_ints_sketch</span></code> pairs.
+Note: if the given sketches have insufficient data or if the sketch sizes are too small, this will return false.
+Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying distribution) using the provided p-value, otherwise False.</p>
+<ol class="arabic simple" start="2">
+<li><p><code class="docutils literal notranslate"><span class="pre">ks_test(sk_1:</span> <span class="pre">_datasketches.kll_floats_sketch,</span> <span class="pre">sk_2:</span> <span class="pre">_datasketches.kll_floats_sketch,</span> <span class="pre">p:</span> <span class="pre">float)</span> <span class="pre">-></span> <span class="pre">bool</span></code></p></li>
+</ol>
+<p>Performs the Kolmogorov-Smirnov Test for <code class="code docutils literal notranslate"><span class="pre">kll_floats_sketch</span></code> pairs.
+Note: if the given sketches have insufficient data or if the sketch sizes are too small, this will return false.
+Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying distribution) using the provided p-value, otherwise False.</p>
+<ol class="arabic simple" start="3">
+<li><p><code class="docutils literal notranslate"><span class="pre">ks_test(sk_1:</span> <span class="pre">_datasketches.kll_doubles_sketch,</span> <span class="pre">sk_2:</span> <span class="pre">_datasketches.kll_doubles_sketch,</span> <span class="pre">p:</span> <span class="pre">float)</span> <span class="pre">-></span> <span class="pre">bool</span></code></p></li>
+</ol>
+<p>Performs the Kolmogorov-Smirnov Test for <code class="code docutils literal notranslate"><span class="pre">kll_doubles_sketch</span></code> pairs.
+Note: if the given sketches have insufficient data or if the sketch sizes are too small, this will return false.
+Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying distribution) using the provided p-value, otherwise False.</p>
+<ol class="arabic simple" start="4">
+<li><p><code class="docutils literal notranslate"><span class="pre">ks_test(sk_1:</span> <span class="pre">_datasketches.kll_items_sketch,</span> <span class="pre">sk_2:</span> <span class="pre">_datasketches.kll_items_sketch,</span> <span class="pre">p:</span> <span class="pre">float)</span> <span class="pre">-></span> <span class="pre">bool</span></code></p></li>
+</ol>
+<p>Performs the Kolmogorov-Smirnov Test for <code class="code docutils literal notranslate"><span class="pre">kll_items_sketch</span></code> pairs.
+Note: if the given sketches have insufficient data or if the sketch sizes are too small, this will return false.
+Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying distribution) using the provided p-value, otherwise False.</p>
+<ol class="arabic simple" start="5">
+<li><p><code class="docutils literal notranslate"><span class="pre">ks_test(sk_1:</span> <span class="pre">_datasketches.quantiles_ints_sketch,</span> <span class="pre">sk_2:</span> <span class="pre">_datasketches.quantiles_ints_sketch,</span> <span class="pre">p:</span> <span class="pre">float)</span> <span class="pre">-></span> <span class="pre">bool</span></code></p></li>
+</ol>
+<p>Performs the Kolmogorov-Smirnov Test for <code class="code docutils literal notranslate"><span class="pre">quantiles_ints_sketch</span></code> pairs.
+Note: if the given sketches have insufficient data or if the sketch sizes are too small, this will return false.
+Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying distribution) using the provided p-value, otherwise False.</p>
+<ol class="arabic simple" start="6">
+<li><p><code class="docutils literal notranslate"><span class="pre">ks_test(sk_1:</span> <span class="pre">_datasketches.quantiles_floats_sketch,</span> <span class="pre">sk_2:</span> <span class="pre">_datasketches.quantiles_floats_sketch,</span> <span class="pre">p:</span> <span class="pre">float)</span> <span class="pre">-></span> <span class="pre">bool</span></code></p></li>
+</ol>
+<p>Performs the Kolmogorov-Smirnov Test for <code class="code docutils literal notranslate"><span class="pre">quantiles_floats_sketch</span></code> pairs.
+Note: if the given sketches have insufficient data or if the sketch sizes are too small, this will return false.
+:Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying distribution) using the provided p-value, otherwise False.</p>
+<ol class="arabic simple" start="7">
+<li><p><code class="docutils literal notranslate"><span class="pre">ks_test(sk_1:</span> <span class="pre">_datasketches.quantiles_doubles_sketch,</span> <span class="pre">sk_2:</span> <span class="pre">_datasketches.quantiles_doubles_sketch,</span> <span class="pre">p:</span> <span class="pre">float)</span> <span class="pre">-></span> <span class="pre">bool</span></code></p></li>
+</ol>
+<p>Performs the Kolmogorov-Smirnov Test for <code class="code docutils literal notranslate"><span class="pre">quantiles_doubles_sketch</span></code> pairs.
+Note: if the given sketches have insufficient data or if the sketch sizes are too small, this will return false.
+Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying distribution) using the provided p-value, otherwise False.</p>
+<ol class="arabic simple" start="8">
+<li><p><code class="docutils literal notranslate"><span class="pre">ks_test(sk_1:</span> <span class="pre">_datasketches.quantiles_items_sketch,</span> <span class="pre">sk_2:</span> <span class="pre">_datasketches.quantiles_items_sketch,</span> <span class="pre">p:</span> <span class="pre">float)</span> <span class="pre">-></span> <span class="pre">bool</span></code></p></li>
+</ol>
+<p>Performs the Kolmogorov-Smirnov Test for <code class="code docutils literal notranslate"><span class="pre">quantiles_items_sketch</span></code> pairs.
+Note: if the given sketches have insufficient data or if the sketch sizes are too small, this will return false.
+Returns True if we can reject the null hypothesis (that the sketches reflect the same underlying distribution) using the provided p-value, otherwise False.</p>
+</dd></dl>
+
+</section>
+
+
+ </div>
+ </div>
+ <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
+ <a href="tuple_policy.html" class="btn btn-neutral float-left" title="Tuple Policy" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+ </div>
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>© Copyright 2023.</p>
+ </div>
+
+ Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
+ <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
+ provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+
+</footer>
+ </div>
+ </div>
+ </section>
+ </div>
+ <script>
+ jQuery(function () {
+ SphinxRtdTheme.Navigation.enable(true);
+ });
+ </script>
+
+</body>
+</html>
\ No newline at end of file
diff --git a/docs/main/objects.inv b/docs/main/objects.inv
index 8f89e97..5022c13 100644
Binary files a/docs/main/objects.inv and b/docs/main/objects.inv differ
diff --git a/docs/main/quantiles_depr.html b/docs/main/quantiles_depr.html
index b929942..bf98fac 100644
--- a/docs/main/quantiles_depr.html
+++ b/docs/main/quantiles_depr.html
@@ -21,6 +21,7 @@
<script src="_static/js/theme.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
+ <link rel="next" title="Variance Optimal Sampling (VarOpt)" href="varopt.html" />
<link rel="prev" title="Relative Error Quantiles (REQ) Sketch" href="req.html" />
</head>
@@ -63,11 +64,12 @@
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Quantiles Sketch (Deprecated)</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.quantiles_ints_sketch"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.deserialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.get_normalized_rank_error()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.get_min_value</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.get_normalized_rank_error</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.get_pmf"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.get_pmf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.get_quantile"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.get_quantile</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_ints_sketch.get_quantiles"><code class="docutils literal notranslate"><span class="pre">quantiles_ints_sketch.get_quantiles</span></code></a></li>
@@ -85,11 +87,12 @@
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.quantiles_floats_sketch"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.deserialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.get_normalized_rank_error()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.get_min_value</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.get_normalized_rank_error</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.get_pmf"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.get_pmf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.get_quantile"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.get_quantile</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_floats_sketch.get_quantiles"><code class="docutils literal notranslate"><span class="pre">quantiles_floats_sketch.get_quantiles</span></code></a></li>
@@ -107,11 +110,12 @@
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.deserialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.get_normalized_rank_error()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.get_min_value</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.get_normalized_rank_error</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.get_pmf"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.get_pmf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.get_quantile"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.get_quantile</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_doubles_sketch.get_quantiles"><code class="docutils literal notranslate"><span class="pre">quantiles_doubles_sketch.get_quantiles</span></code></a></li>
@@ -129,11 +133,12 @@
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.quantiles_items_sketch"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.deserialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.get_normalized_rank_error()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.get_min_value</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.get_normalized_rank_error"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.get_normalized_rank_error</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.get_pmf"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.get_pmf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.get_quantile"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.get_quantile</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.quantiles_items_sketch.get_quantiles"><code class="docutils literal notranslate"><span class="pre">quantiles_items_sketch.get_quantiles</span></code></a></li>
@@ -152,6 +157,17 @@
</li>
</ul>
</li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -180,7 +196,9 @@
<section id="quantiles-sketch-deprecated">
<h1>Quantiles Sketch (Deprecated)<a class="headerlink" href="#quantiles-sketch-deprecated" title="Link to this heading"></a></h1>
-<p>This is a deprecated quantiles sketch that is included for cross-language compatibility.</p>
+<p>This is a deprecated quantiles sketch that is included for cross-language compatibility.
+Most new projects will favor the KLL sketch over this one, or the REQ sketch for higher accuracy
+at the very edge of a distribution.</p>
<p>This is a stochastic streaming sketch that enables near-real time analysis of the
approximate distribution from a very large stream in a single pass.
The analysis is obtained using <cite>get_rank()</cite> and <cite>get_quantile()</cite> functions,
@@ -212,13 +230,44 @@ the normalized rank. A <cite>k</cite> of 128 produces a normalized, rank error o
For example, the median item returned from <cite>get_quantile(0.5)</cite> will be between the actual items
from the hypothetically sorted array of input items at normalized ranks of 0.483 and 0.517, with
a confidence of about 99%.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>For the <a class="reference internal" href="#datasketches.quantiles_items_sketch" title="datasketches.quantiles_items_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">quantiles_items_sketch</span></code></a>, objects must be comparable with <code class="docutils literal notranslate"><span class="pre">__lt__</span></code>.</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Serializing and deserializing a <a class="reference internal" href="#datasketches.quantiles_items_sketch" title="datasketches.quantiles_items_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">quantiles_items_sketch</span></code></a> requires the use of a <a class="reference internal" href="serde.html#datasketches.PyObjectSerDe" title="datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSe [...]
+</div>
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.quantiles_ints_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">quantiles_ints_sketch</span></span><a class="headerlink" href="#datasketches.quantiles_ints_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.quantiles_ints_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.quantiles_ints_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#data [...]
+<dd><p>Deserializes the sketch from a bytes object.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.quantiles_ints_sketch.get_normalized_rank_error">
+<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">as_pmf</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><spa [...]
+<dd><p>Gets the normalized rank error given parameters k and the pmf flag.
+If pmf is True, returns the ‘double-sided’ normalized rank error for the get_PMF() function.
+Otherwise, it is the ‘single-sided’ normalized rank error for all the other queries.
+Constants were derived as the best fit to 99 percentile empirically measured max error in thousands of trials</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.quantiles_ints_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates a classic quantiles sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 128.</p>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.quantiles_ints_sketch.get_cdf">
@@ -243,11 +292,6 @@ It is not necessary to include either the min or max values in these split point
<dd><p>Returns the minimum value from the stream. If empty, quantiles_floats_sketch returns nan; quantiles_ints_sketch throws a RuntimeError</p>
</dd></dl>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datasketches.quantiles_ints_sketch.get_normalized_rank_error">
-<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.quantiles_ints_sketch.get_normalized_rank_error" title="Link to this definition"></a></dt>
-<dd></dd></dl>
-
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.quantiles_ints_sketch.get_pmf">
<span class="sig-name descname"><span class="pre">get_pmf</span></span><a class="headerlink" href="#datasketches.quantiles_ints_sketch.get_pmf" title="Link to this definition"></a></dt>
@@ -358,10 +402,33 @@ Constants were derived as the best fit to 99 percentile empirically measured max
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.quantiles_floats_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">quantiles_floats_sketch</span></span><a class="headerlink" href="#datasketches.quantiles_floats_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.quantiles_floats_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.quantiles_floats_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#data [...]
+<dd><p>Deserializes the sketch from a bytes object.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.quantiles_floats_sketch.get_normalized_rank_error">
+<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">as_pmf</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><spa [...]
+<dd><p>Gets the normalized rank error given parameters k and the pmf flag.
+If pmf is True, returns the ‘double-sided’ normalized rank error for the get_PMF() function.
+Otherwise, it is the ‘single-sided’ normalized rank error for all the other queries.
+Constants were derived as the best fit to 99 percentile empirically measured max error in thousands of trials</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.quantiles_floats_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates a classic quantiles sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 128.</p>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.quantiles_floats_sketch.get_cdf">
@@ -386,11 +453,6 @@ It is not necessary to include either the min or max values in these split point
<dd><p>Returns the minimum value from the stream. If empty, quantiles_floats_sketch returns nan; quantiles_ints_sketch throws a RuntimeError</p>
</dd></dl>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datasketches.quantiles_floats_sketch.get_normalized_rank_error">
-<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.quantiles_floats_sketch.get_normalized_rank_error" title="Link to this definition"></a></dt>
-<dd></dd></dl>
-
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.quantiles_floats_sketch.get_pmf">
<span class="sig-name descname"><span class="pre">get_pmf</span></span><a class="headerlink" href="#datasketches.quantiles_floats_sketch.get_pmf" title="Link to this definition"></a></dt>
@@ -501,10 +563,33 @@ Constants were derived as the best fit to 99 percentile empirically measured max
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.quantiles_doubles_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">quantiles_doubles_sketch</span></span><a class="headerlink" href="#datasketches.quantiles_doubles_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.quantiles_doubles_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.quantiles_doubles_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#data [...]
+<dd><p>Deserializes the sketch from a bytes object.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.quantiles_doubles_sketch.get_normalized_rank_error">
+<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">as_pmf</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><spa [...]
+<dd><p>Gets the normalized rank error given parameters k and the pmf flag.
+If pmf is True, returns the ‘double-sided’ normalized rank error for the get_PMF() function.
+Otherwise, it is the ‘single-sided’ normalized rank error for all the other queries.
+Constants were derived as the best fit to 99 percentile empirically measured max error in thousands of trials</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.quantiles_doubles_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates a classic quantiles sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 128.</p>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.quantiles_doubles_sketch.get_cdf">
@@ -529,11 +614,6 @@ It is not necessary to include either the min or max values in these split point
<dd><p>Returns the minimum value from the stream. If empty, quantiles_floats_sketch returns nan; quantiles_ints_sketch throws a RuntimeError</p>
</dd></dl>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datasketches.quantiles_doubles_sketch.get_normalized_rank_error">
-<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.quantiles_doubles_sketch.get_normalized_rank_error" title="Link to this definition"></a></dt>
-<dd></dd></dl>
-
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.quantiles_doubles_sketch.get_pmf">
<span class="sig-name descname"><span class="pre">get_pmf</span></span><a class="headerlink" href="#datasketches.quantiles_doubles_sketch.get_pmf" title="Link to this definition"></a></dt>
@@ -644,10 +724,33 @@ Constants were derived as the best fit to 99 percentile empirically measured max
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.quantiles_items_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">quantiles_items_sketch</span></span><a class="headerlink" href="#datasketches.quantiles_items_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.quantiles_items_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.quantiles_items_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">serde</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="ref [...]
+<dd><p>Deserializes the sketch from a bytes object using the provided serde.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.quantiles_items_sketch.get_normalized_rank_error">
+<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">as_pmf</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><spa [...]
+<dd><p>Gets the normalized rank error given parameters k and the pmf flag.
+If pmf is True, returns the ‘double-sided’ normalized rank error for the get_PMF() function.
+Otherwise, it is the ‘single-sided’ normalized rank error for all the other queries.
+Constants were derived as the best fit to 99 percentile empirically measured max error in thousands of trials</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.quantiles_items_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates a classic quantiles sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 128.</p>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.quantiles_items_sketch.get_cdf">
@@ -672,11 +775,6 @@ It is not necessary to include either the min or max values in these split point
<dd><p>Returns the minimum value from the stream. If empty, quantiles_floats_sketch returns nan; quantiles_ints_sketch throws a RuntimeError</p>
</dd></dl>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datasketches.quantiles_items_sketch.get_normalized_rank_error">
-<span class="sig-name descname"><span class="pre">get_normalized_rank_error</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.quantiles_items_sketch.get_normalized_rank_error" title="Link to this definition"></a></dt>
-<dd></dd></dl>
-
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.quantiles_items_sketch.get_pmf">
<span class="sig-name descname"><span class="pre">get_pmf</span></span><a class="headerlink" href="#datasketches.quantiles_items_sketch.get_pmf" title="Link to this definition"></a></dt>
@@ -783,6 +881,7 @@ Constants were derived as the best fit to 99 percentile empirically measured max
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="req.html" class="btn btn-neutral float-left" title="Relative Error Quantiles (REQ) Sketch" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+ <a href="varopt.html" class="btn btn-neutral float-right" title="Variance Optimal Sampling (VarOpt)" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
diff --git a/docs/main/req.html b/docs/main/req.html
index 94b45a3..ce3ad19 100644
--- a/docs/main/req.html
+++ b/docs/main/req.html
@@ -63,8 +63,9 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Relative Error Quantiles (REQ) Sketch</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.req_ints_sketch"><code class="docutils literal notranslate"><span class="pre">req_ints_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_ints_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">req_ints_sketch.deserialize</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_ints_sketch.get_RSE"><code class="docutils literal notranslate"><span class="pre">req_ints_sketch.get_RSE</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_ints_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">req_ints_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_ints_sketch.get_RSE"><code class="docutils literal notranslate"><span class="pre">req_ints_sketch.get_RSE()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_ints_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">req_ints_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_ints_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">req_ints_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_ints_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">req_ints_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_ints_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">req_ints_sketch.get_min_value</span></code></a></li>
@@ -87,8 +88,9 @@
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.req_floats_sketch"><code class="docutils literal notranslate"><span class="pre">req_floats_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_floats_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">req_floats_sketch.deserialize</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_floats_sketch.get_RSE"><code class="docutils literal notranslate"><span class="pre">req_floats_sketch.get_RSE</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_floats_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">req_floats_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_floats_sketch.get_RSE"><code class="docutils literal notranslate"><span class="pre">req_floats_sketch.get_RSE()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_floats_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">req_floats_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_floats_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">req_floats_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_floats_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">req_floats_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_floats_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">req_floats_sketch.get_min_value</span></code></a></li>
@@ -111,8 +113,9 @@
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="#datasketches.req_items_sketch"><code class="docutils literal notranslate"><span class="pre">req_items_sketch</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_items_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">req_items_sketch.deserialize</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_items_sketch.get_RSE"><code class="docutils literal notranslate"><span class="pre">req_items_sketch.get_RSE</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_items_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">req_items_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_items_sketch.get_RSE"><code class="docutils literal notranslate"><span class="pre">req_items_sketch.get_RSE()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_items_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">req_items_sketch.__init__()</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_items_sketch.get_cdf"><code class="docutils literal notranslate"><span class="pre">req_items_sketch.get_cdf</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_items_sketch.get_max_value"><code class="docutils literal notranslate"><span class="pre">req_items_sketch.get_max_value</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#datasketches.req_items_sketch.get_min_value"><code class="docutils literal notranslate"><span class="pre">req_items_sketch.get_min_value</span></code></a></li>
@@ -137,6 +140,17 @@
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -186,18 +200,45 @@ The Python prototype only implemented a comparison criterion of <cite>INCLUSIVE<
This implementation allows the user to use both the <cite>INCLUSIVE</cite> criterion and the <cite>EXCLUSIVE</cite> criterion.
This implementation provides extensive debug visibility into the operation of the sketch with two levels of detail output.
This is not only useful for debugging, but is a powerful tool to help users understand how the sketch works.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>For the <a class="reference internal" href="#datasketches.req_items_sketch" title="datasketches.req_items_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">req_items_sketch</span></code></a>, objects must be comparable with <code class="docutils literal notranslate"><span class="pre">__lt__</span></code>.</p>
+</div>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Serializing and deserializing a <a class="reference internal" href="#datasketches.req_items_sketch" title="datasketches.req_items_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">req_items_sketch</span></code></a> requires the use of a <a class="reference internal" href="serde.html#datasketches.PyObjectSerDe" title="datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code>< [...]
+</div>
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.req_ints_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">req_ints_sketch</span></span><a class="headerlink" href="#datasketches.req_ints_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.req_ints_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.req_ints_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#data [...]
+<dd><p>Deserializes the sketch from a bytes object.</p>
+</dd></dl>
-<dl class="py attribute">
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.req_ints_sketch.get_RSE">
-<span class="sig-name descname"><span class="pre">get_RSE</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.req_ints_sketch.get_RSE" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">get_RSE</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">rank</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float< [...]
+<dd><p>Returns an a priori estimate of relative standard error (RSE, expressed as a number in [0,1]). Derived from Lemma 12 in <a class="reference external" href="http://arxiv.org/abs/2004.01668v2">http://arxiv.org/abs/2004.01668v2</a>, but the constant factors have been modified based on empirical measurements, for a given value of parameter k.
+Normalized rank must be a value between 0.0 and 1.0 (inclusive). If is_hra is True, uses high rank accuracy mode, else low rank accuracy. N is an estimate of the total number of points provided to the sketch.</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.req_ints_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates an REQ sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 12.</p></li>
+<li><p><strong>is_hra</strong> (<em>bool</em><em>, </em><em>optional</em>) – Specifies whether the skech has High Rank Accuracy (True) or Low Rank Accuracy. Default True.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.req_ints_sketch.get_cdf">
@@ -345,15 +386,34 @@ Normalized rank must be a value between 0.0 and 1.0 (inclusive); the number of s
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.req_floats_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">req_floats_sketch</span></span><a class="headerlink" href="#datasketches.req_floats_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.req_floats_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.req_floats_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#data [...]
+<dd><p>Deserializes the sketch from a bytes object.</p>
+</dd></dl>
-<dl class="py attribute">
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.req_floats_sketch.get_RSE">
-<span class="sig-name descname"><span class="pre">get_RSE</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.req_floats_sketch.get_RSE" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">get_RSE</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">rank</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float< [...]
+<dd><p>Returns an a priori estimate of relative standard error (RSE, expressed as a number in [0,1]). Derived from Lemma 12 in <a class="reference external" href="http://arxiv.org/abs/2004.01668v2">http://arxiv.org/abs/2004.01668v2</a>, but the constant factors have been modified based on empirical measurements, for a given value of parameter k.
+Normalized rank must be a value between 0.0 and 1.0 (inclusive). If is_hra is True, uses high rank accuracy mode, else low rank accuracy. N is an estimate of the total number of points provided to the sketch.</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.req_floats_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates an REQ sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 12.</p></li>
+<li><p><strong>is_hra</strong> (<em>bool</em><em>, </em><em>optional</em>) – Specifies whether the skech has High Rank Accuracy (True) or Low Rank Accuracy. Default True.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.req_floats_sketch.get_cdf">
@@ -501,15 +561,34 @@ Normalized rank must be a value between 0.0 and 1.0 (inclusive); the number of s
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.req_items_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">req_items_sketch</span></span><a class="headerlink" href="#datasketches.req_items_sketch" title="Link to this definition"></a></dt>
-<dd><dl class="py attribute">
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.req_items_sketch.deserialize">
-<span class="sig-name descname"><span class="pre">deserialize</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.req_items_sketch.deserialize" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">serde</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="ref [...]
+<dd><p>Deserializes the sketch from a bytes object using the provided serde.</p>
+</dd></dl>
-<dl class="py attribute">
+<dl class="py method">
<dt class="sig sig-object py" id="datasketches.req_items_sketch.get_RSE">
-<span class="sig-name descname"><span class="pre">get_RSE</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre"><nanobind.nb_func</span> <span class="pre">object></span></em><a class="headerlink" href="#datasketches.req_items_sketch.get_RSE" title="Link to this definition"></a></dt>
-<dd></dd></dl>
+<span class="sig-name descname"><span class="pre">get_RSE</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">rank</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float< [...]
+<dd><p>Returns an a priori estimate of relative standard error (RSE, expressed as a number in [0,1]). Derived from Lemma 12 in <a class="reference external" href="http://arxiv.org/abs/2004.01668v2">http://arxiv.org/abs/2004.01668v2</a>, but the constant factors have been modified based on empirical measurements, for a given value of parameter k.
+Normalized rank must be a value between 0.0 and 1.0 (inclusive). If is_hra is True, uses high rank accuracy mode, else low rank accuracy. N is an estimate of the total number of points provided to the sketch.</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.req_items_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="de [...]
+<dd><p>Creates an REQ sketch instance with the given value of k.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>k</strong> (<em>int</em><em>, </em><em>optional</em>) – Controls the size/accuracy trade-off of the sketch. Default is 12.</p></li>
+<li><p><strong>is_hra</strong> (<em>bool</em><em>, </em><em>optional</em>) – Specifies whether the skech has High Rank Accuracy (True) or Low Rank Accuracy. Default True.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="datasketches.req_items_sketch.get_cdf">
diff --git a/docs/main/search.html b/docs/main/search.html
index 6a23597..1dcbbbb 100644
--- a/docs/main/search.html
+++ b/docs/main/search.html
@@ -63,6 +63,17 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
diff --git a/docs/main/searchindex.js b/docs/main/searchindex.js
index e19a842..da6df17 100644
--- a/docs/main/searchindex.js
+++ b/docs/main/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["count_min_sketch", "cpc", "density_sketch", "frequent_items", "hyper_log_log", "index", "kll", "quantiles_depr", "req", "theta", "tuple"], "filenames": ["count_min_sketch.rst", "cpc.rst", "density_sketch.rst", "frequent_items.rst", "hyper_log_log.rst", "index.rst", "kll.rst", "quantiles_depr.rst", "req.rst", "theta.rst", "tuple.rst"], "titles": ["CountMin Sketch", "Compressed Probabilistic Counting (CPC)", "Density Sketch", "Frequent Items", "HyperLogLog (H [...]
\ No newline at end of file
+Search.setIndex({"docnames": ["count_min_sketch", "cpc", "density_sketch", "ebpps", "frequent_items", "hyper_log_log", "index", "jaccard", "kernel", "kll", "ks_test", "quantiles_depr", "req", "serde", "theta", "tuple", "tuple_policy", "varopt"], "filenames": ["count_min_sketch.rst", "cpc.rst", "density_sketch.rst", "ebpps.rst", "frequent_items.rst", "hyper_log_log.rst", "index.rst", "jaccard.rst", "kernel.rst", "kll.rst", "ks_test.rst", "quantiles_depr.rst", "req.rst", "serde.rst", "thet [...]
\ No newline at end of file
diff --git a/docs/main/serde.html b/docs/main/serde.html
new file mode 100644
index 0000000..e290c47
--- /dev/null
+++ b/docs/main/serde.html
@@ -0,0 +1,258 @@
+<!DOCTYPE html>
+<html class="writer-html5" lang="en" data-content_root="./">
+<head>
+ <meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
+
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <title>Serialize/Deserialize (SerDe) — datasketches 0.1 documentation</title>
+ <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
+ <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+
+
+ <!--[if lt IE 9]>
+ <script src="_static/js/html5shiv.min.js"></script>
+ <![endif]-->
+
+ <script src="_static/jquery.js?v=5d32c60e"></script>
+ <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
+ <script src="_static/documentation_options.js?v=2709fde1"></script>
+ <script src="_static/doctools.js?v=888ff710"></script>
+ <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+ <script src="_static/js/theme.js"></script>
+ <link rel="index" title="Index" href="genindex.html" />
+ <link rel="search" title="Search" href="search.html" />
+ <link rel="next" title="Kernel Function" href="kernel.html" />
+ <link rel="prev" title="Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling" href="ebpps.html" />
+</head>
+
+<body class="wy-body-for-nav">
+ <div class="wy-grid-for-nav">
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-scroll">
+ <div class="wy-side-nav-search" >
+
+
+
+ <a href="index.html" class="icon icon-home">
+ datasketches
+ </a>
+<div role="search">
+ <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
+ <ul>
+<li class="toctree-l1"><a class="reference internal" href="hyper_log_log.html">HyperLogLog (HLL)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="cpc.html">Compressed Probabilistic Counting (CPC)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="theta.html">Theta Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple.html">Tuple Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="density_sketch.html">Density Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="count_min_sketch.html">CountMin Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="frequent_items.html">Frequent Items</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul class="current">
+<li class="toctree-l1 current"><a class="current reference internal" href="#">Serialize/Deserialize (SerDe)</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.PyObjectSerDe"><code class="docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.PyObjectSerDe.get_size"><code class="docutils literal notranslate"><span class="pre">PyObjectSerDe.get_size()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.PyObjectSerDe.to_bytes"><code class="docutils literal notranslate"><span class="pre">PyObjectSerDe.to_bytes()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.PyObjectSerDe.from_bytes"><code class="docutils literal notranslate"><span class="pre">PyObjectSerDe.from_bytes()</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.PyStringsSerDe"><code class="docutils literal notranslate"><span class="pre">PyStringsSerDe</span></code></a></li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.PyIntsSerDe"><code class="docutils literal notranslate"><span class="pre">PyIntsSerDe</span></code></a></li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.PyLongsSerDe"><code class="docutils literal notranslate"><span class="pre">PyLongsSerDe</span></code></a></li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.PyFloatsSerDe"><code class="docutils literal notranslate"><span class="pre">PyFloatsSerDe</span></code></a></li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.PyDoublesSerDe"><code class="docutils literal notranslate"><span class="pre">PyDoublesSerDe</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
+</ul>
+
+ </div>
+ </div>
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">datasketches</a>
+ </nav>
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="Page navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
+ <li class="breadcrumb-item active">Serialize/Deserialize (SerDe)</li>
+ <li class="wy-breadcrumbs-aside">
+ <a href="_sources/serde.rst.txt" rel="nofollow"> View page source</a>
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+ <div itemprop="articleBody">
+
+ <section id="serialize-deserialize-serde">
+<h1>Serialize/Deserialize (SerDe)<a class="headerlink" href="#serialize-deserialize-serde" title="Link to this heading"></a></h1>
+<p>A SerDe is a class used to serialize items sketches to a <code class="xref py py-class docutils literal notranslate"><span class="pre">bytes</span></code> object in binary.
+Several example SerDes are provided as references.</p>
+<p>The use of binary-compatible SerDes in different languages is critical for cross-language compatibility.</p>
+<p>Each implementation must extend the <a class="reference internal" href="#datasketches.PyObjectSerDe" title="datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code></a> class and override all three of its methods.</p>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.PyObjectSerDe">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">PyObjectSerDe</span></span><a class="headerlink" href="#datasketches.PyObjectSerDe" title="Link to this definition"></a></dt>
+<dd><p>An abstract base class for serde objects. All custom serdes must extend this class.</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.PyObjectSerDe.get_size">
+<span class="sig-name descname"><span class="pre">get_size</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">item</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">object</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</spa [...]
+<dd><p>Returns the size in bytes of an item</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>item</strong> (<em>object</em>) – The specified object</p>
+</dd>
+<dt class="field-even">Returns<span class="colon">:</span></dt>
+<dd class="field-even"><p>The size of the item in bytes</p>
+</dd>
+<dt class="field-odd">Return type<span class="colon">:</span></dt>
+<dd class="field-odd"><p>int</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.PyObjectSerDe.to_bytes">
+<span class="sig-name descname"><span class="pre">to_bytes</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">item</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">object</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</spa [...]
+<dd><p>Retuns a bytes object with a serialized version of an item</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>item</strong> (<em>object</em>) – The specified object</p>
+</dd>
+<dt class="field-even">Returns<span class="colon">:</span></dt>
+<dd class="field-even"><p>A <code class="xref py py-class docutils literal notranslate"><span class="pre">bytes</span></code> object with the serialized object</p>
+</dd>
+<dt class="field-odd">Return type<span class="colon">:</span></dt>
+<dd class="field-odd"><p>bytes</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.PyObjectSerDe.from_bytes">
+<span class="sig-name descname"><span class="pre">from_bytes</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">data</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">offset</span></span><span class="p"><span [...]
+<dd><p>Reads a bytes object starting from the given offest and returns a tuple of the reconstructed object and the number of additional bytes read</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>data</strong> (<em>bytes</em>) – A <code class="xref py py-class docutils literal notranslate"><span class="pre">bytes</span></code> object from which to deserialize</p></li>
+<li><p><strong>offset</strong> (<em>int</em>) – The offset, in bytes, at which to start reading</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns<span class="colon">:</span></dt>
+<dd class="field-even"><p>A <code class="xref py py-class docutils literal notranslate"><span class="pre">tuple</span></code> with the reconstructed object and the number of bytes read</p>
+</dd>
+<dt class="field-odd">Return type<span class="colon">:</span></dt>
+<dd class="field-odd"><p>tuple(object, int)</p>
+</dd>
+</dl>
+</dd></dl>
+
+</dd></dl>
+
+<p>The provided SerDes are:</p>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.PyStringsSerDe">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">PyStringsSerDe</span></span><a class="headerlink" href="#datasketches.PyStringsSerDe" title="Link to this definition"></a></dt>
+<dd><p>Bases: <a class="reference internal" href="#datasketches.PyObjectSerDe" title="_datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code></a></p>
+<p>Implements a simple string-encoding scheme where a string is
+written as <cite><num_bytes> <string_contents></cite>, with no null termination.
+This format allows pre-allocating each string, at the cost of
+additional storage. Using this format, the serialized string consumes
+<code class="docutils literal notranslate"><span class="pre">4</span> <span class="pre">+</span> <span class="pre">len(item)</span></code> bytes.</p>
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.PyIntsSerDe">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">PyIntsSerDe</span></span><a class="headerlink" href="#datasketches.PyIntsSerDe" title="Link to this definition"></a></dt>
+<dd><p>Bases: <a class="reference internal" href="#datasketches.PyObjectSerDe" title="_datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code></a></p>
+<p>Implements an integer encoding scheme where each integer is written
+as a 32-bit (4 byte) little-endian value.</p>
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.PyLongsSerDe">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">PyLongsSerDe</span></span><a class="headerlink" href="#datasketches.PyLongsSerDe" title="Link to this definition"></a></dt>
+<dd><p>Bases: <a class="reference internal" href="#datasketches.PyObjectSerDe" title="_datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code></a></p>
+<p>Implements an integer encoding scheme where each integer is written
+as a 64-bit (8 byte) little-endian value.</p>
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.PyFloatsSerDe">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">PyFloatsSerDe</span></span><a class="headerlink" href="#datasketches.PyFloatsSerDe" title="Link to this definition"></a></dt>
+<dd><p>Bases: <a class="reference internal" href="#datasketches.PyObjectSerDe" title="_datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code></a></p>
+<p>Implements a floating point encoding scheme where each value is written
+as a 32-bit floating point value.</p>
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.PyDoublesSerDe">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">PyDoublesSerDe</span></span><a class="headerlink" href="#datasketches.PyDoublesSerDe" title="Link to this definition"></a></dt>
+<dd><p>Implements a floating point encoding scheme where each value is written
+as a 64-bit floating point value.</p>
+</dd></dl>
+
+</section>
+
+
+ </div>
+ </div>
+ <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
+ <a href="ebpps.html" class="btn btn-neutral float-left" title="Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+ <a href="kernel.html" class="btn btn-neutral float-right" title="Kernel Function" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
+ </div>
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>© Copyright 2023.</p>
+ </div>
+
+ Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
+ <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
+ provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+
+</footer>
+ </div>
+ </div>
+ </section>
+ </div>
+ <script>
+ jQuery(function () {
+ SphinxRtdTheme.Navigation.enable(true);
+ });
+ </script>
+
+</body>
+</html>
\ No newline at end of file
diff --git a/docs/main/theta.html b/docs/main/theta.html
index 8d5984b..36eb286 100644
--- a/docs/main/theta.html
+++ b/docs/main/theta.html
@@ -62,6 +62,38 @@
<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_sketch.to_string"><code class="docutils literal notranslate"><span class="pre">theta_sketch.to_string</span></code></a></li>
</ul>
</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.update_theta_sketch"><code class="docutils literal notranslate"><span class="pre">update_theta_sketch</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.update_theta_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">update_theta_sketch.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.update_theta_sketch.compact"><code class="docutils literal notranslate"><span class="pre">update_theta_sketch.compact</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.update_theta_sketch.reset"><code class="docutils literal notranslate"><span class="pre">update_theta_sketch.reset</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.update_theta_sketch.trim"><code class="docutils literal notranslate"><span class="pre">update_theta_sketch.trim</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.update_theta_sketch.update"><code class="docutils literal notranslate"><span class="pre">update_theta_sketch.update</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.compact_theta_sketch"><code class="docutils literal notranslate"><span class="pre">compact_theta_sketch</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.compact_theta_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">compact_theta_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.compact_theta_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">compact_theta_sketch.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.compact_theta_sketch.serialize"><code class="docutils literal notranslate"><span class="pre">compact_theta_sketch.serialize</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.theta_union"><code class="docutils literal notranslate"><span class="pre">theta_union</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_union.__init__"><code class="docutils literal notranslate"><span class="pre">theta_union.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_union.get_result"><code class="docutils literal notranslate"><span class="pre">theta_union.get_result</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_union.update"><code class="docutils literal notranslate"><span class="pre">theta_union.update</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.theta_intersection"><code class="docutils literal notranslate"><span class="pre">theta_intersection</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_intersection.__init__"><code class="docutils literal notranslate"><span class="pre">theta_intersection.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_intersection.get_result"><code class="docutils literal notranslate"><span class="pre">theta_intersection.get_result</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_intersection.has_result"><code class="docutils literal notranslate"><span class="pre">theta_intersection.has_result</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_intersection.update"><code class="docutils literal notranslate"><span class="pre">theta_intersection.update</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.theta_a_not_b"><code class="docutils literal notranslate"><span class="pre">theta_a_not_b</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_a_not_b.__init__"><code class="docutils literal notranslate"><span class="pre">theta_a_not_b.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.theta_a_not_b.compute"><code class="docutils literal notranslate"><span class="pre">theta_a_not_b.compute</span></code></a></li>
+</ul>
+</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="tuple.html">Tuple Sketch</a></li>
@@ -79,6 +111,17 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -107,6 +150,7 @@
<section id="theta-sketch">
<h1>Theta Sketch<a class="headerlink" href="#theta-sketch" title="Link to this heading"></a></h1>
+<p>Theta sketches are used for distinct counting.</p>
<p>The theta package contains the basic sketch classes that are members of the <a class="reference external" href="https://datasketches.apache.org/docs/Theta/ThetaSketchFramework.html">Theta Sketch Framework</a>.
There is a separate Tuple package for many of the sketches that are derived from the same algorithms defined in the Theta Sketch Framework paper.</p>
<p>The <em>Theta Sketch</em> sketch is a space-efficient method for estimating cardinalities of sets.
@@ -114,6 +158,9 @@ It can also easily handle set operations (such as union, intersection, differenc
Theta sketch is a practical variant of the K-Minimum Values sketch which avoids the need to sort the stored
hash values on every insertion to the sketch.
It has better error properties than the HyperLogLog sketch for set operations beyond the simple union.</p>
+<p>Set operations (union, intersection, A-not-B) are performed through the use of dedicated objects.</p>
+<p>Several <a class="reference external" href="https://en.wikipedia.org/wiki/Jaccard_similarity">Jaccard similarity</a>
+measures can be computed between theta sketches with the <a class="reference internal" href="jaccard.html#datasketches.theta_jaccard_similarity" title="datasketches.theta_jaccard_similarity"><code class="xref py py-class docutils literal notranslate"><span class="pre">theta_jaccard_similarity</span></code></a> class.</p>
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.theta_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">theta_sketch</span></span><a class="headerlink" href="#datasketches.theta_sketch" title="Link to this definition"></a></dt>
@@ -186,6 +233,183 @@ It has better error properties than the HyperLogLog sketch for set operations be
</dd></dl>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.update_theta_sketch">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">update_theta_sketch</span></span><a class="headerlink" href="#datasketches.update_theta_sketch" title="Link to this definition"></a></dt>
+<dd><dl class="py method">
+<dt class="sig sig-object py" id="datasketches.update_theta_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lg_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class= [...]
+<dd><p>Creates an update_theta_sketch using the provided parameters</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>lg_k</strong> (<em>int</em><em>, </em><em>optional</em>) – base 2 logarithm of the maximum size of the sketch. Default 12.</p></li>
+<li><p><strong>p</strong> (<em>float</em><em>, </em><em>optional</em>) – an initial sampling rate to use. Default 1.0</p></li>
+<li><p><strong>seed</strong> (<em>int</em><em>, </em><em>optional</em>) – the seed to use when hashing values</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.update_theta_sketch.compact">
+<span class="sig-name descname"><span class="pre">compact</span></span><a class="headerlink" href="#datasketches.update_theta_sketch.compact" title="Link to this definition"></a></dt>
+<dd><p>Returns a compacted form of the sketch, optionally sorting it</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.update_theta_sketch.reset">
+<span class="sig-name descname"><span class="pre">reset</span></span><a class="headerlink" href="#datasketches.update_theta_sketch.reset" title="Link to this definition"></a></dt>
+<dd><p>Resets the sketch to the initial empty state</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.update_theta_sketch.trim">
+<span class="sig-name descname"><span class="pre">trim</span></span><a class="headerlink" href="#datasketches.update_theta_sketch.trim" title="Link to this definition"></a></dt>
+<dd><p>Removes retained entries in excess of the nominal size k (if any)</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.update_theta_sketch.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.update_theta_sketch.update" title="Link to this definition"></a></dt>
+<dd><p>Overloaded function.</p>
+<ol class="arabic simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">update(self,</span> <span class="pre">datum:</span> <span class="pre">int)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Updates the sketch with the given integral value</p>
+<ol class="arabic simple" start="2">
+<li><p><code class="docutils literal notranslate"><span class="pre">update(self,</span> <span class="pre">datum:</span> <span class="pre">float)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Updates the sketch with the given floating point value</p>
+<ol class="arabic simple" start="3">
+<li><p><code class="docutils literal notranslate"><span class="pre">update(self,</span> <span class="pre">datum:</span> <span class="pre">str)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Updates the sketch with the given string</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.compact_theta_sketch">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">compact_theta_sketch</span></span><a class="headerlink" href="#datasketches.compact_theta_sketch" title="Link to this definition"></a></dt>
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.compact_theta_sketch.deserialize">
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">seed</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="p [...]
+<dd><p>Reads a bytes object and returns the corresponding compact_theta_sketch</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.compact_theta_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">arg0</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datasketches.theta_sketch" title="_datasketches.theta_sketch"><span class="pre">_datasketches.theta_sketch< [...]
+<dd><p>Creates a compact_theta_sketch from an existing theta_sketch.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>other</strong> (<a class="reference internal" href="#datasketches.theta_sketch" title="datasketches.theta_sketch"><em>theta_sketch</em></a>) – a source theta_sketch</p></li>
+<li><p><strong>ordered</strong> (<em>bool</em>) – whether the incoming sketch entries are sorted. Default True</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.compact_theta_sketch.serialize">
+<span class="sig-name descname"><span class="pre">serialize</span></span><a class="headerlink" href="#datasketches.compact_theta_sketch.serialize" title="Link to this definition"></a></dt>
+<dd><p>Serializes the sketch into a bytes object</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.theta_union">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">theta_union</span></span><a class="headerlink" href="#datasketches.theta_union" title="Link to this definition"></a></dt>
+<dd><dl class="py method">
+<dt class="sig sig-object py" id="datasketches.theta_union.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lg_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class= [...]
+<dd><p>Creates a theta_union using the provided parameters</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>lg_k</strong> (<em>int</em><em>, </em><em>optional</em>) – base 2 logarithm of the maximum size of the union. Default 12.</p></li>
+<li><p><strong>p</strong> (<em>float</em><em>, </em><em>optional</em>) – an initial sampling rate to use. Default 1.0</p></li>
+<li><p><strong>seed</strong> (<em>int</em><em>, </em><em>optional</em>) – the seed to use when hashing values. Must match all sketch seeds.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.theta_union.get_result">
+<span class="sig-name descname"><span class="pre">get_result</span></span><a class="headerlink" href="#datasketches.theta_union.get_result" title="Link to this definition"></a></dt>
+<dd><p>Returns the sketch corresponding to the union result</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.theta_union.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.theta_union.update" title="Link to this definition"></a></dt>
+<dd><p>Updates the union with the given sketch</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.theta_intersection">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">theta_intersection</span></span><a class="headerlink" href="#datasketches.theta_intersection" title="Link to this definition"></a></dt>
+<dd><dl class="py method">
+<dt class="sig sig-object py" id="datasketches.theta_intersection.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">seed</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class= [...]
+<dd><p>Creates a theta_intersection using the provided parameters</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>seed</strong> (<em>int</em><em>, </em><em>optional</em>) – the seed to use when hashing values. Must match all sketch seeds</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.theta_intersection.get_result">
+<span class="sig-name descname"><span class="pre">get_result</span></span><a class="headerlink" href="#datasketches.theta_intersection.get_result" title="Link to this definition"></a></dt>
+<dd><p>Returns the sketch corresponding to the intersection result</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.theta_intersection.has_result">
+<span class="sig-name descname"><span class="pre">has_result</span></span><a class="headerlink" href="#datasketches.theta_intersection.has_result" title="Link to this definition"></a></dt>
+<dd><p>Returns True if the intersection has a valid result, otherwise False</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.theta_intersection.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.theta_intersection.update" title="Link to this definition"></a></dt>
+<dd><p>Intersections the provided sketch with the current intersection state</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.theta_a_not_b">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">theta_a_not_b</span></span><a class="headerlink" href="#datasketches.theta_a_not_b" title="Link to this definition"></a></dt>
+<dd><dl class="py method">
+<dt class="sig sig-object py" id="datasketches.theta_a_not_b.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">seed</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class= [...]
+<dd><p>Creates a tuple_a_not_b object</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>seed</strong> (<em>int</em><em>, </em><em>optional</em>) – the seed to use when hashing values. Must match all sketch seeds.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.theta_a_not_b.compute">
+<span class="sig-name descname"><span class="pre">compute</span></span><a class="headerlink" href="#datasketches.theta_a_not_b.compute" title="Link to this definition"></a></dt>
+<dd><p>Returns a sketch with the result of applying the A-not-B operation on the given inputs</p>
+</dd></dl>
+
+</dd></dl>
+
</section>
diff --git a/docs/main/tuple.html b/docs/main/tuple.html
index b31b84f..8a21e75 100644
--- a/docs/main/tuple.html
+++ b/docs/main/tuple.html
@@ -64,19 +64,37 @@
<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_sketch.to_string"><code class="docutils literal notranslate"><span class="pre">tuple_sketch.to_string</span></code></a></li>
</ul>
</li>
-<li class="toctree-l2"><a class="reference internal" href="#datasketches.AccumulatorPolicy"><code class="docutils literal notranslate"><span class="pre">AccumulatorPolicy</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.AccumulatorPolicy.create_summary"><code class="docutils literal notranslate"><span class="pre">AccumulatorPolicy.create_summary()</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.AccumulatorPolicy.update_summary"><code class="docutils literal notranslate"><span class="pre">AccumulatorPolicy.update_summary()</span></code></a></li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.update_tuple_sketch"><code class="docutils literal notranslate"><span class="pre">update_tuple_sketch</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.update_tuple_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">update_tuple_sketch.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.update_tuple_sketch.compact"><code class="docutils literal notranslate"><span class="pre">update_tuple_sketch.compact</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.update_tuple_sketch.reset"><code class="docutils literal notranslate"><span class="pre">update_tuple_sketch.reset</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.update_tuple_sketch.trim"><code class="docutils literal notranslate"><span class="pre">update_tuple_sketch.trim</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.update_tuple_sketch.update"><code class="docutils literal notranslate"><span class="pre">update_tuple_sketch.update</span></code></a></li>
</ul>
</li>
-<li class="toctree-l2"><a class="reference internal" href="#datasketches.MaxIntPolicy"><code class="docutils literal notranslate"><span class="pre">MaxIntPolicy</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.MaxIntPolicy.create_summary"><code class="docutils literal notranslate"><span class="pre">MaxIntPolicy.create_summary()</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.MaxIntPolicy.update_summary"><code class="docutils literal notranslate"><span class="pre">MaxIntPolicy.update_summary()</span></code></a></li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.compact_tuple_sketch"><code class="docutils literal notranslate"><span class="pre">compact_tuple_sketch</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.compact_tuple_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">compact_tuple_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.compact_tuple_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">compact_tuple_sketch.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.compact_tuple_sketch.serialize"><code class="docutils literal notranslate"><span class="pre">compact_tuple_sketch.serialize</span></code></a></li>
</ul>
</li>
-<li class="toctree-l2"><a class="reference internal" href="#datasketches.MinIntPolicy"><code class="docutils literal notranslate"><span class="pre">MinIntPolicy</span></code></a><ul>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.MinIntPolicy.create_summary"><code class="docutils literal notranslate"><span class="pre">MinIntPolicy.create_summary()</span></code></a></li>
-<li class="toctree-l3"><a class="reference internal" href="#datasketches.MinIntPolicy.update_summary"><code class="docutils literal notranslate"><span class="pre">MinIntPolicy.update_summary()</span></code></a></li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.tuple_union"><code class="docutils literal notranslate"><span class="pre">tuple_union</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_union.__init__"><code class="docutils literal notranslate"><span class="pre">tuple_union.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_union.get_result"><code class="docutils literal notranslate"><span class="pre">tuple_union.get_result</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_union.reset"><code class="docutils literal notranslate"><span class="pre">tuple_union.reset</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_union.update"><code class="docutils literal notranslate"><span class="pre">tuple_union.update</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.tuple_intersection"><code class="docutils literal notranslate"><span class="pre">tuple_intersection</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_intersection.__init__"><code class="docutils literal notranslate"><span class="pre">tuple_intersection.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_intersection.get_result"><code class="docutils literal notranslate"><span class="pre">tuple_intersection.get_result</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_intersection.has_result"><code class="docutils literal notranslate"><span class="pre">tuple_intersection.has_result</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_intersection.update"><code class="docutils literal notranslate"><span class="pre">tuple_intersection.update</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.tuple_a_not_b"><code class="docutils literal notranslate"><span class="pre">tuple_a_not_b</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_a_not_b.__init__"><code class="docutils literal notranslate"><span class="pre">tuple_a_not_b.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.tuple_a_not_b.compute"><code class="docutils literal notranslate"><span class="pre">tuple_a_not_b.compute</span></code></a></li>
</ul>
</li>
</ul>
@@ -95,6 +113,17 @@
<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
</ul>
</div>
@@ -123,8 +152,19 @@
<section id="tuple-sketch">
<h1>Tuple Sketch<a class="headerlink" href="#tuple-sketch" title="Link to this heading"></a></h1>
-<p>Tuple sketches are an extension of Theta sketch that allow the keeping of arbitrary summaries associated with each retained key
-(for example, a count for every key).</p>
+<p>Tuple sketches are an extension of Theta sketches, meaning they provide estimate of distinct counts, that
+allow the keeping of arbitrary summaries associated with each retained key
+(for example, a count for every key). The use of a <a class="reference internal" href="#datasketches.tuple_sketch" title="datasketches.tuple_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">tuple_sketch</span></code></a> requires a <a class="reference internal" href="tuple_policy.html#datasketches.TuplePolicy" title="datasketches.TuplePolicy"><code class="xref py py-class docutils literal notranslate"><span class="pre">TuplePolicy</span></code></a> which
+defines how summaries are created, updated, merged, or intersected. The library provides a few basic
+examples of <a class="reference internal" href="tuple_policy.html#datasketches.TuplePolicy" title="datasketches.TuplePolicy"><code class="xref py py-class docutils literal notranslate"><span class="pre">TuplePolicy</span></code></a> implementations, but the right custom summary and policy can allow very
+complicated analysis to be performed quite easily.</p>
+<p>Set operations (union, intersection, A-not-B) are performed through the use of dedicated objects.</p>
+<p>Several <a class="reference external" href="https://en.wikipedia.org/wiki/Jaccard_similarity">Jaccard similarity</a>
+measures can be computed between theta sketches with the <a class="reference internal" href="jaccard.html#datasketches.tuple_jaccard_similarity" title="datasketches.tuple_jaccard_similarity"><code class="xref py py-class docutils literal notranslate"><span class="pre">tuple_jaccard_similarity</span></code></a> class.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Serializing and deserializing this sketch requires the use of a <a class="reference internal" href="serde.html#datasketches.PyObjectSerDe" title="datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code></a>.</p>
+</div>
<dl class="py class">
<dt class="sig sig-object py" id="datasketches.tuple_sketch">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">tuple_sketch</span></span><a class="headerlink" href="#datasketches.tuple_sketch" title="Link to this definition"></a></dt>
@@ -203,52 +243,207 @@
</dd></dl>
<dl class="py class">
-<dt class="sig sig-object py" id="datasketches.AccumulatorPolicy">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">AccumulatorPolicy</span></span><a class="headerlink" href="#datasketches.AccumulatorPolicy" title="Link to this definition"></a></dt>
+<dt class="sig sig-object py" id="datasketches.update_tuple_sketch">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">update_tuple_sketch</span></span><a class="headerlink" href="#datasketches.update_tuple_sketch" title="Link to this definition"></a></dt>
<dd><dl class="py method">
-<dt class="sig sig-object py" id="datasketches.AccumulatorPolicy.create_summary">
-<span class="sig-name descname"><span class="pre">create_summary</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><span class="pre">object</span></span></span><a class="headerlink" href="#datasketches.AccumulatorPolicy.create_summary" title="Link to this definition"></a></dt>
-<dd><p>Creates a new Summary object</p>
+<dt class="sig sig-object py" id="datasketches.update_tuple_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">policy</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="tuple_policy.html#datasketches.TuplePolicy" title="_datasketches.TuplePolicy"><span class="pre">_datasketc [...]
+<dd><p>Creates an update_tuple_sketch using the provided parameters</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>policy</strong> (<a class="reference internal" href="tuple_policy.html#datasketches.TuplePolicy" title="datasketches.TuplePolicy"><em>TuplePolicy</em></a>) – a policy to use when updating</p></li>
+<li><p><strong>lg_k</strong> (<em>int</em><em>, </em><em>optional</em>) – base 2 logarithm of the maximum size of the sketch. Default 12.</p></li>
+<li><p><strong>p</strong> (<em>float</em><em>, </em><em>optional</em>) – an initial sampling rate to use. Default 1.0</p></li>
+<li><p><strong>seed</strong> (<em>int</em><em>, </em><em>optional</em>) – the seed to use when hashing values</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.update_tuple_sketch.compact">
+<span class="sig-name descname"><span class="pre">compact</span></span><a class="headerlink" href="#datasketches.update_tuple_sketch.compact" title="Link to this definition"></a></dt>
+<dd><p>Returns a compacted form of the sketch, optionally sorting it</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.update_tuple_sketch.reset">
+<span class="sig-name descname"><span class="pre">reset</span></span><a class="headerlink" href="#datasketches.update_tuple_sketch.reset" title="Link to this definition"></a></dt>
+<dd><p>Resets the sketch to the initial empty state</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.update_tuple_sketch.trim">
+<span class="sig-name descname"><span class="pre">trim</span></span><a class="headerlink" href="#datasketches.update_tuple_sketch.trim" title="Link to this definition"></a></dt>
+<dd><p>Removes retained entries in excess of the nominal size k (if any)</p>
</dd></dl>
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.update_tuple_sketch.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.update_tuple_sketch.update" title="Link to this definition"></a></dt>
+<dd><p>Overloaded function.</p>
+<ol class="arabic simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">update(self,</span> <span class="pre">datum:</span> <span class="pre">int,</span> <span class="pre">value:</span> <span class="pre">object)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Updates the sketch with the given integral item and summary value</p>
+<ol class="arabic simple" start="2">
+<li><p><code class="docutils literal notranslate"><span class="pre">update(self,</span> <span class="pre">datum:</span> <span class="pre">float,</span> <span class="pre">value:</span> <span class="pre">object)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Updates the sketch with the given floating point item and summary value</p>
+<ol class="arabic simple" start="3">
+<li><p><code class="docutils literal notranslate"><span class="pre">update(self,</span> <span class="pre">datum:</span> <span class="pre">str,</span> <span class="pre">value:</span> <span class="pre">object)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Updates the sketch with the given string item and summary value</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.compact_tuple_sketch">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">compact_tuple_sketch</span></span><a class="headerlink" href="#datasketches.compact_tuple_sketch" title="Link to this definition"></a></dt>
+<dd><p class="rubric">Static Methods:</p>
<dl class="py method">
-<dt class="sig sig-object py" id="datasketches.AccumulatorPolicy.update_summary">
-<span class="sig-name descname"><span class="pre">update_summary</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">summary</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">object</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">update</span></span><span class="p [...]
-<dd><p>applies the relevant policy to update the provided summary with the data in update.</p>
+<dt class="sig sig-object py" id="datasketches.compact_tuple_sketch.deserialize">
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">serde</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="ref [...]
+<dd><p>Reads a bytes object and returns the corresponding compact_tuple_sketch</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.compact_tuple_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">other</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datasketches.tuple_sketch" title="_datasketches.tuple_sketch"><span class="pre">_datasketches.tuple_sketch [...]
+<dt class="sig sig-object py">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">other</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="theta.html#datasketches.theta_sketch" title="_datasketches.theta_sketch"><span class="pre">_datasketches.th [...]
+<dd><p>Overloaded function.</p>
+<ol class="arabic simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">__init__(self,</span> <span class="pre">other:</span> <span class="pre">_datasketches.tuple_sketch,</span> <span class="pre">ordered:</span> <span class="pre">bool</span> <span class="pre">=</span> <span class="pre">True)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Creates a compact_tuple_sketch from an existing tuple_sketch.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>other</strong> (<a class="reference internal" href="#datasketches.tuple_sketch" title="datasketches.tuple_sketch"><em>tuple_sketch</em></a>) – a sourch tuple_sketch</p></li>
+<li><p><strong>ordered</strong> (<em>bool</em><em>, </em><em>optional</em>) – whether the incoming sketch entries are sorted. Default True</p></li>
+</ul>
+</dd>
+</dl>
+<ol class="arabic simple" start="2">
+<li><p><code class="docutils literal notranslate"><span class="pre">__init__(self,</span> <span class="pre">other:</span> <span class="pre">_datasketches.theta_sketch,</span> <span class="pre">summary:</span> <span class="pre">object)</span> <span class="pre">-></span> <span class="pre">None</span></code></p></li>
+</ol>
+<p>Creates a compact_tuple_sketch from a theta_sketch using a fixed summary value.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>other</strong> (<a class="reference internal" href="theta.html#datasketches.theta_sketch" title="datasketches.theta_sketch"><em>theta_sketch</em></a>) – a source theta sketch</p></li>
+<li><p><strong>summary</strong> (<em>object</em>) – a summary to use for every sketch entry</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.compact_tuple_sketch.serialize">
+<span class="sig-name descname"><span class="pre">serialize</span></span><a class="headerlink" href="#datasketches.compact_tuple_sketch.serialize" title="Link to this definition"></a></dt>
+<dd><p>Serializes the sketch into a bytes object</p>
</dd></dl>
</dd></dl>
<dl class="py class">
-<dt class="sig sig-object py" id="datasketches.MaxIntPolicy">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">MaxIntPolicy</span></span><a class="headerlink" href="#datasketches.MaxIntPolicy" title="Link to this definition"></a></dt>
+<dt class="sig sig-object py" id="datasketches.tuple_union">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">tuple_union</span></span><a class="headerlink" href="#datasketches.tuple_union" title="Link to this definition"></a></dt>
<dd><dl class="py method">
-<dt class="sig sig-object py" id="datasketches.MaxIntPolicy.create_summary">
-<span class="sig-name descname"><span class="pre">create_summary</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><span class="pre">object</span></span></span><a class="headerlink" href="#datasketches.MaxIntPolicy.create_summary" title="Link to this definition"></a></dt>
-<dd><p>Creates a new Summary object</p>
+<dt class="sig sig-object py" id="datasketches.tuple_union.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">policy</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="tuple_policy.html#datasketches.TuplePolicy" title="_datasketches.TuplePolicy"><span class="pre">_datasketc [...]
+<dd><p>Creates a tuple_union using the provided parameters</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>policy</strong> (<a class="reference internal" href="tuple_policy.html#datasketches.TuplePolicy" title="datasketches.TuplePolicy"><em>TuplePolicy</em></a>) – a policy to use when unioning entries</p></li>
+<li><p><strong>lg_k</strong> (<em>int</em><em>, </em><em>optional</em>) – base 2 logarithm of the maximum size of the union. Default 12.</p></li>
+<li><p><strong>p</strong> (<em>float</em><em>, </em><em>optional</em>) – an initial sampling rate to use. Default 1.0</p></li>
+<li><p><strong>seed</strong> (<em>int</em><em>, </em><em>optional</em>) – the seed to use when hashing values. Must match any sketch seeds.</p></li>
+</ul>
+</dd>
+</dl>
</dd></dl>
-<dl class="py method">
-<dt class="sig sig-object py" id="datasketches.MaxIntPolicy.update_summary">
-<span class="sig-name descname"><span class="pre">update_summary</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">summary</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">object</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">update</span></span><span class="p [...]
-<dd><p>applies the relevant policy to update the provided summary with the data in update.</p>
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.tuple_union.get_result">
+<span class="sig-name descname"><span class="pre">get_result</span></span><a class="headerlink" href="#datasketches.tuple_union.get_result" title="Link to this definition"></a></dt>
+<dd><p>Returns the sketch corresponding to the union result</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.tuple_union.reset">
+<span class="sig-name descname"><span class="pre">reset</span></span><a class="headerlink" href="#datasketches.tuple_union.reset" title="Link to this definition"></a></dt>
+<dd><p>Resets the sketch to the initial empty</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.tuple_union.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.tuple_union.update" title="Link to this definition"></a></dt>
+<dd><p>Updates the union with the given sketch</p>
</dd></dl>
</dd></dl>
<dl class="py class">
-<dt class="sig sig-object py" id="datasketches.MinIntPolicy">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">MinIntPolicy</span></span><a class="headerlink" href="#datasketches.MinIntPolicy" title="Link to this definition"></a></dt>
+<dt class="sig sig-object py" id="datasketches.tuple_intersection">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">tuple_intersection</span></span><a class="headerlink" href="#datasketches.tuple_intersection" title="Link to this definition"></a></dt>
<dd><dl class="py method">
-<dt class="sig sig-object py" id="datasketches.MinIntPolicy.create_summary">
-<span class="sig-name descname"><span class="pre">create_summary</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><span class="pre">object</span></span></span><a class="headerlink" href="#datasketches.MinIntPolicy.create_summary" title="Link to this definition"></a></dt>
-<dd><p>Creates a new Summary object</p>
+<dt class="sig sig-object py" id="datasketches.tuple_intersection.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">policy</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="tuple_policy.html#datasketches.TuplePolicy" title="_datasketches.TuplePolicy"><span class="pre">_datasketc [...]
+<dd><p>Creates a tuple_intersection using the provided parameters</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>policy</strong> (<a class="reference internal" href="tuple_policy.html#datasketches.TuplePolicy" title="datasketches.TuplePolicy"><em>TuplePolicy</em></a>) – a policy to use when intersecting entries</p></li>
+<li><p><strong>seed</strong> (<em>int</em><em>, </em><em>optional</em>) – the seed to use when hashing values. Must match any sketch seeds</p></li>
+</ul>
+</dd>
+</dl>
</dd></dl>
-<dl class="py method">
-<dt class="sig sig-object py" id="datasketches.MinIntPolicy.update_summary">
-<span class="sig-name descname"><span class="pre">update_summary</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">summary</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">object</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">update</span></span><span class="p [...]
-<dd><p>applies the relevant policy to update the provided summary with the data in update.</p>
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.tuple_intersection.get_result">
+<span class="sig-name descname"><span class="pre">get_result</span></span><a class="headerlink" href="#datasketches.tuple_intersection.get_result" title="Link to this definition"></a></dt>
+<dd><p>Returns the sketch corresponding to the intersection result</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.tuple_intersection.has_result">
+<span class="sig-name descname"><span class="pre">has_result</span></span><a class="headerlink" href="#datasketches.tuple_intersection.has_result" title="Link to this definition"></a></dt>
+<dd><p>Returns True if the intersection has a valid result, otherwise False</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.tuple_intersection.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.tuple_intersection.update" title="Link to this definition"></a></dt>
+<dd><p>Intersects the provided sketch with the current intersection state</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.tuple_a_not_b">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">tuple_a_not_b</span></span><a class="headerlink" href="#datasketches.tuple_a_not_b" title="Link to this definition"></a></dt>
+<dd><dl class="py method">
+<dt class="sig sig-object py" id="datasketches.tuple_a_not_b.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">seed</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class= [...]
+<dd><p>Creates a tuple_a_not_b object</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>seed</strong> (<em>int</em><em>, </em><em>optional</em>) – the seed to use when hashing values. Must match any sketch seeds.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.tuple_a_not_b.compute">
+<span class="sig-name descname"><span class="pre">compute</span></span><a class="headerlink" href="#datasketches.tuple_a_not_b.compute" title="Link to this definition"></a></dt>
+<dd><p>Returns a sketch with the result of applying the A-not-B operation on the given inputs</p>
</dd></dl>
</dd></dl>
diff --git a/docs/main/tuple_policy.html b/docs/main/tuple_policy.html
new file mode 100644
index 0000000..68226b1
--- /dev/null
+++ b/docs/main/tuple_policy.html
@@ -0,0 +1,235 @@
+<!DOCTYPE html>
+<html class="writer-html5" lang="en" data-content_root="./">
+<head>
+ <meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
+
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <title>Tuple Policy — datasketches 0.1 documentation</title>
+ <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
+ <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+
+
+ <!--[if lt IE 9]>
+ <script src="_static/js/html5shiv.min.js"></script>
+ <![endif]-->
+
+ <script src="_static/jquery.js?v=5d32c60e"></script>
+ <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
+ <script src="_static/documentation_options.js?v=2709fde1"></script>
+ <script src="_static/doctools.js?v=888ff710"></script>
+ <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+ <script src="_static/js/theme.js"></script>
+ <link rel="index" title="Index" href="genindex.html" />
+ <link rel="search" title="Search" href="search.html" />
+ <link rel="next" title="Kolmogorov-Smirnov Test" href="ks_test.html" />
+ <link rel="prev" title="Jaccard Similarity" href="jaccard.html" />
+</head>
+
+<body class="wy-body-for-nav">
+ <div class="wy-grid-for-nav">
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-scroll">
+ <div class="wy-side-nav-search" >
+
+
+
+ <a href="index.html" class="icon icon-home">
+ datasketches
+ </a>
+<div role="search">
+ <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
+ <ul>
+<li class="toctree-l1"><a class="reference internal" href="hyper_log_log.html">HyperLogLog (HLL)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="cpc.html">Compressed Probabilistic Counting (CPC)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="theta.html">Theta Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple.html">Tuple Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="density_sketch.html">Density Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="count_min_sketch.html">CountMin Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="frequent_items.html">Frequent Items</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="varopt.html">Variance Optimal Sampling (VarOpt)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul class="current">
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1 current"><a class="current reference internal" href="#">Tuple Policy</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.TuplePolicy"><code class="docutils literal notranslate"><span class="pre">TuplePolicy</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.TuplePolicy.create_summary"><code class="docutils literal notranslate"><span class="pre">TuplePolicy.create_summary()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.TuplePolicy.update_summary"><code class="docutils literal notranslate"><span class="pre">TuplePolicy.update_summary()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.TuplePolicy.__call__"><code class="docutils literal notranslate"><span class="pre">TuplePolicy.__call__()</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.AccumulatorPolicy"><code class="docutils literal notranslate"><span class="pre">AccumulatorPolicy</span></code></a></li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.MinIntPolicy"><code class="docutils literal notranslate"><span class="pre">MinIntPolicy</span></code></a></li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.MaxIntPolicy"><code class="docutils literal notranslate"><span class="pre">MaxIntPolicy</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
+</ul>
+
+ </div>
+ </div>
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">datasketches</a>
+ </nav>
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="Page navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
+ <li class="breadcrumb-item active">Tuple Policy</li>
+ <li class="wy-breadcrumbs-aside">
+ <a href="_sources/tuple_policy.rst.txt" rel="nofollow"> View page source</a>
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+ <div itemprop="articleBody">
+
+ <section id="tuple-policy">
+<h1>Tuple Policy<a class="headerlink" href="#tuple-policy" title="Link to this heading"></a></h1>
+<p>A Tuple Policy is needed when using a <a class="reference internal" href="tuple.html#datasketches.tuple_sketch" title="datasketches.tuple_sketch"><code class="xref py py-class docutils literal notranslate"><span class="pre">tuple_sketch</span></code></a> in order to specify how the
+summary values should be created, updated, combined, or intersected. A summary can consist of
+any python object.</p>
+<p>Each implementation must extend the abstract base class <a class="reference internal" href="#datasketches.TuplePolicy" title="datasketches.TuplePolicy"><code class="xref py py-class docutils literal notranslate"><span class="pre">TuplePolicy</span></code></a>.</p>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.TuplePolicy">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">TuplePolicy</span></span><a class="headerlink" href="#datasketches.TuplePolicy" title="Link to this definition"></a></dt>
+<dd><p>An abstract base class for Tuple Policy objects. All custom policies must extend this class.</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.TuplePolicy.create_summary">
+<span class="sig-name descname"><span class="pre">create_summary</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><span class="pre">object</span></span></span><a class="headerlink" href="#datasketches.TuplePolicy.create_summary" title="Link to this definition"></a></dt>
+<dd><p>Creates a new Summary object</p>
+<dl class="field-list simple">
+<dt class="field-odd">Returns<span class="colon">:</span></dt>
+<dd class="field-odd"><p>a Summary object</p>
+</dd>
+<dt class="field-even">Return type<span class="colon">:</span></dt>
+<dd class="field-even"><p><code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.TuplePolicy.update_summary">
+<span class="sig-name descname"><span class="pre">update_summary</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">summary</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">object</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">update</span></span><span class="p [...]
+<dd><p>Applies the relevant policy to update the provided summary with the data in update.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>summary</strong> (<code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code>) – An existing Summary</p></li>
+<li><p><strong>update</strong> (<code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code>) – An update to apply to the Summary</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns<span class="colon">:</span></dt>
+<dd class="field-even"><p>The updated Summary</p>
+</dd>
+<dt class="field-odd">Return type<span class="colon">:</span></dt>
+<dd class="field-odd"><p><code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.TuplePolicy.__call__">
+<span class="sig-name descname"><span class="pre">__call__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">summary</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">object</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">update</span></span><span class="p"><spa [...]
+<dd><p>Similar to update_summary but allows a different implementation for set operations (union and intersection)</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>summary</strong> (<code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code>) – An existing Summary</p></li>
+<li><p><strong>update</strong> (<code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code>) – An update to apply to the Summary</p></li>
+</ul>
+</dd>
+<dt class="field-even">Returns<span class="colon">:</span></dt>
+<dd class="field-even"><p>The updated Summary</p>
+</dd>
+<dt class="field-odd">Return type<span class="colon">:</span></dt>
+<dd class="field-odd"><p><code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></p>
+</dd>
+</dl>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.AccumulatorPolicy">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">AccumulatorPolicy</span></span><a class="headerlink" href="#datasketches.AccumulatorPolicy" title="Link to this definition"></a></dt>
+<dd><p>Bases: <a class="reference internal" href="#datasketches.TuplePolicy" title="_datasketches.TuplePolicy"><code class="xref py py-class docutils literal notranslate"><span class="pre">TuplePolicy</span></code></a></p>
+<p>Implements an accumulatory summary policy, where new values
+are added to the existing value.</p>
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.MinIntPolicy">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">MinIntPolicy</span></span><a class="headerlink" href="#datasketches.MinIntPolicy" title="Link to this definition"></a></dt>
+<dd><p>Bases: <a class="reference internal" href="#datasketches.TuplePolicy" title="_datasketches.TuplePolicy"><code class="xref py py-class docutils literal notranslate"><span class="pre">TuplePolicy</span></code></a></p>
+<p>Implements a MIN rule, where the smallest integer value is always kept.</p>
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.MaxIntPolicy">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">MaxIntPolicy</span></span><a class="headerlink" href="#datasketches.MaxIntPolicy" title="Link to this definition"></a></dt>
+<dd><p>Bases: <a class="reference internal" href="#datasketches.TuplePolicy" title="_datasketches.TuplePolicy"><code class="xref py py-class docutils literal notranslate"><span class="pre">TuplePolicy</span></code></a></p>
+<p>Implements a MAX rule, where the largest integer value is always kept.</p>
+</dd></dl>
+
+</section>
+
+
+ </div>
+ </div>
+ <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
+ <a href="jaccard.html" class="btn btn-neutral float-left" title="Jaccard Similarity" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+ <a href="ks_test.html" class="btn btn-neutral float-right" title="Kolmogorov-Smirnov Test" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
+ </div>
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>© Copyright 2023.</p>
+ </div>
+
+ Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
+ <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
+ provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+
+</footer>
+ </div>
+ </div>
+ </section>
+ </div>
+ <script>
+ jQuery(function () {
+ SphinxRtdTheme.Navigation.enable(true);
+ });
+ </script>
+
+</body>
+</html>
\ No newline at end of file
diff --git a/docs/main/varopt.html b/docs/main/varopt.html
new file mode 100644
index 0000000..5584b0f
--- /dev/null
+++ b/docs/main/varopt.html
@@ -0,0 +1,308 @@
+<!DOCTYPE html>
+<html class="writer-html5" lang="en" data-content_root="./">
+<head>
+ <meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
+
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <title>Variance Optimal Sampling (VarOpt) — datasketches 0.1 documentation</title>
+ <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
+ <link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=19f00094" />
+
+
+ <!--[if lt IE 9]>
+ <script src="_static/js/html5shiv.min.js"></script>
+ <![endif]-->
+
+ <script src="_static/jquery.js?v=5d32c60e"></script>
+ <script src="_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script>
+ <script src="_static/documentation_options.js?v=2709fde1"></script>
+ <script src="_static/doctools.js?v=888ff710"></script>
+ <script src="_static/sphinx_highlight.js?v=dc90522c"></script>
+ <script src="_static/js/theme.js"></script>
+ <link rel="index" title="Index" href="genindex.html" />
+ <link rel="search" title="Search" href="search.html" />
+ <link rel="next" title="Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling" href="ebpps.html" />
+ <link rel="prev" title="Quantiles Sketch (Deprecated)" href="quantiles_depr.html" />
+</head>
+
+<body class="wy-body-for-nav">
+ <div class="wy-grid-for-nav">
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-scroll">
+ <div class="wy-side-nav-search" >
+
+
+
+ <a href="index.html" class="icon icon-home">
+ datasketches
+ </a>
+<div role="search">
+ <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
+ <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
+ <ul>
+<li class="toctree-l1"><a class="reference internal" href="hyper_log_log.html">HyperLogLog (HLL)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="cpc.html">Compressed Probabilistic Counting (CPC)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="theta.html">Theta Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple.html">Tuple Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="density_sketch.html">Density Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="count_min_sketch.html">CountMin Sketch</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="frequent_items.html">Frequent Items</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="kll.html">KLL Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="req.html">Relative Error Quantiles (REQ) Sketch</a></li>
+<li class="toctree-l1"><a class="reference internal" href="quantiles_depr.html">Quantiles Sketch (Deprecated)</a></li>
+</ul>
+<ul class="current">
+<li class="toctree-l1 current"><a class="current reference internal" href="#">Variance Optimal Sampling (VarOpt)</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.var_opt_sketch"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.deserialize"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.__init__"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.estimate_subset_sum"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.estimate_subset_sum</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.get_serialized_size_bytes"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.get_serialized_size_bytes</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.is_empty"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.is_empty</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.k"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.k</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.n"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.n</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.num_samples"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.num_samples</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.serialize"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.serialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.to_string"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.to_string</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_sketch.update"><code class="docutils literal notranslate"><span class="pre">var_opt_sketch.update</span></code></a></li>
+</ul>
+</li>
+<li class="toctree-l2"><a class="reference internal" href="#datasketches.var_opt_union"><code class="docutils literal notranslate"><span class="pre">var_opt_union</span></code></a><ul>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_union.deserialize"><code class="docutils literal notranslate"><span class="pre">var_opt_union.deserialize()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_union.__init__"><code class="docutils literal notranslate"><span class="pre">var_opt_union.__init__()</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_union.get_result"><code class="docutils literal notranslate"><span class="pre">var_opt_union.get_result</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_union.get_serialized_size_bytes"><code class="docutils literal notranslate"><span class="pre">var_opt_union.get_serialized_size_bytes</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_union.reset"><code class="docutils literal notranslate"><span class="pre">var_opt_union.reset</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_union.serialize"><code class="docutils literal notranslate"><span class="pre">var_opt_union.serialize</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_union.to_string"><code class="docutils literal notranslate"><span class="pre">var_opt_union.to_string</span></code></a></li>
+<li class="toctree-l3"><a class="reference internal" href="#datasketches.var_opt_union.update"><code class="docutils literal notranslate"><span class="pre">var_opt_union.update</span></code></a></li>
+</ul>
+</li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="ebpps.html">Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling</a></li>
+</ul>
+<ul>
+<li class="toctree-l1"><a class="reference internal" href="serde.html">Serialize/Deserialize (SerDe)</a></li>
+<li class="toctree-l1"><a class="reference internal" href="kernel.html">Kernel Function</a></li>
+<li class="toctree-l1"><a class="reference internal" href="jaccard.html">Jaccard Similarity</a></li>
+<li class="toctree-l1"><a class="reference internal" href="tuple_policy.html">Tuple Policy</a></li>
+<li class="toctree-l1"><a class="reference internal" href="ks_test.html">Kolmogorov-Smirnov Test</a></li>
+</ul>
+
+ </div>
+ </div>
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">datasketches</a>
+ </nav>
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="Page navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html" class="icon icon-home" aria-label="Home"></a></li>
+ <li class="breadcrumb-item active">Variance Optimal Sampling (VarOpt)</li>
+ <li class="wy-breadcrumbs-aside">
+ <a href="_sources/varopt.rst.txt" rel="nofollow"> View page source</a>
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
+ <div itemprop="articleBody">
+
+ <section id="variance-optimal-sampling-varopt">
+<h1>Variance Optimal Sampling (VarOpt)<a class="headerlink" href="#variance-optimal-sampling-varopt" title="Link to this heading"></a></h1>
+<p>A VarOpt sketch samples data from a stream of items. The sketch is desinged for optimal (minimum)
+variance when querying the sketch to estimate subset sums of items matching a provided predicate.
+The sketch will produce a sample of size <cite>k</cite> (or smaller if fewer items have been presented), with
+the probability of including an item roughly corresponding it the item’s weight relative to the total
+weight of all items presented to the sketch.</p>
+<p>VarOpt sampling is related to reservoir sampling, with improved error bounds for subset sum estimation.
+Feeding the sketch items with a uniform weight value will produce a sample equivalent to reservoir sampling.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Serializing and deserializing this sketch requires the use of a <a class="reference internal" href="serde.html#datasketches.PyObjectSerDe" title="datasketches.PyObjectSerDe"><code class="xref py py-class docutils literal notranslate"><span class="pre">PyObjectSerDe</span></code></a>.</p>
+</div>
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">var_opt_sketch</span></span><a class="headerlink" href="#datasketches.var_opt_sketch" title="Link to this definition"></a></dt>
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.deserialize">
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">serde</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="ref [...]
+<dd><p>Reads a bytes object and returns the corresponding var opt sketch</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <sp [...]
+<dd><p>Creates a new Var Opt sketch instance</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>k</strong> (<em>int</em>) – Maximum number of samples in the sketch</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.estimate_subset_sum">
+<span class="sig-name descname"><span class="pre">estimate_subset_sum</span></span><a class="headerlink" href="#datasketches.var_opt_sketch.estimate_subset_sum" title="Link to this definition"></a></dt>
+<dd><p>Applies a provided predicate to the sketch and returns the estimated total weight matching the predicate, as well as upper and lower bounds on the estimate and the total weight processed by the sketch</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.get_serialized_size_bytes">
+<span class="sig-name descname"><span class="pre">get_serialized_size_bytes</span></span><a class="headerlink" href="#datasketches.var_opt_sketch.get_serialized_size_bytes" title="Link to this definition"></a></dt>
+<dd><p>Computes the size in bytes needed to serialize the current sketch</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.is_empty">
+<span class="sig-name descname"><span class="pre">is_empty</span></span><a class="headerlink" href="#datasketches.var_opt_sketch.is_empty" title="Link to this definition"></a></dt>
+<dd><p>Returns True if the sketch is empty, otherwise False</p>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.k">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">k</span></span><a class="headerlink" href="#datasketches.var_opt_sketch.k" title="Link to this definition"></a></dt>
+<dd><p>Returns the sketch’s maximum configured sample size</p>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.n">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">n</span></span><a class="headerlink" href="#datasketches.var_opt_sketch.n" title="Link to this definition"></a></dt>
+<dd><p>Returns the total stream length</p>
+</dd></dl>
+
+<dl class="py property">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.num_samples">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">num_samples</span></span><a class="headerlink" href="#datasketches.var_opt_sketch.num_samples" title="Link to this definition"></a></dt>
+<dd><p>Returns the number of samples currently in the sketch</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.serialize">
+<span class="sig-name descname"><span class="pre">serialize</span></span><a class="headerlink" href="#datasketches.var_opt_sketch.serialize" title="Link to this definition"></a></dt>
+<dd><p>Serializes the sketch into a bytes object</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.to_string">
+<span class="sig-name descname"><span class="pre">to_string</span></span><a class="headerlink" href="#datasketches.var_opt_sketch.to_string" title="Link to this definition"></a></dt>
+<dd><p>Produces a string summary of the sketch and optionally prints the items</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_sketch.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.var_opt_sketch.update" title="Link to this definition"></a></dt>
+<dd><p>Updates the sketch with the given value and weight</p>
+</dd></dl>
+
+</dd></dl>
+
+<dl class="py class">
+<dt class="sig sig-object py" id="datasketches.var_opt_union">
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">var_opt_union</span></span><a class="headerlink" href="#datasketches.var_opt_union" title="Link to this definition"></a></dt>
+<dd><p class="rubric">Static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.var_opt_union.deserialize">
+<span class="sig-name descname"><span class="pre">deserialize</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">bytes</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bytes</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">serde</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="ref [...]
+<dd><p>Constructs a var opt union from the given bytes using the provided serde</p>
+</dd></dl>
+
+<p class="rubric">Non-static Methods:</p>
+<dl class="py method">
+<dt class="sig sig-object py" id="datasketches.var_opt_union.__init__">
+<span class="sig-name descname"><span class="pre">__init__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">self</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_k</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> [...]
+<dd></dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_union.get_result">
+<span class="sig-name descname"><span class="pre">get_result</span></span><a class="headerlink" href="#datasketches.var_opt_union.get_result" title="Link to this definition"></a></dt>
+<dd><p>Returns a sketch corresponding to the union result</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_union.get_serialized_size_bytes">
+<span class="sig-name descname"><span class="pre">get_serialized_size_bytes</span></span><a class="headerlink" href="#datasketches.var_opt_union.get_serialized_size_bytes" title="Link to this definition"></a></dt>
+<dd><p>Computes the size in bytes needed to serialize the current union</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_union.reset">
+<span class="sig-name descname"><span class="pre">reset</span></span><a class="headerlink" href="#datasketches.var_opt_union.reset" title="Link to this definition"></a></dt>
+<dd><p>Resets the union to the empty state</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_union.serialize">
+<span class="sig-name descname"><span class="pre">serialize</span></span><a class="headerlink" href="#datasketches.var_opt_union.serialize" title="Link to this definition"></a></dt>
+<dd><p>Serializes the union into a bytes object with the provided serde</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_union.to_string">
+<span class="sig-name descname"><span class="pre">to_string</span></span><a class="headerlink" href="#datasketches.var_opt_union.to_string" title="Link to this definition"></a></dt>
+<dd><p>Produces a string summary of the sketch</p>
+</dd></dl>
+
+<dl class="py attribute">
+<dt class="sig sig-object py" id="datasketches.var_opt_union.update">
+<span class="sig-name descname"><span class="pre">update</span></span><a class="headerlink" href="#datasketches.var_opt_union.update" title="Link to this definition"></a></dt>
+<dd><p>Updates the union with the given sketch</p>
+</dd></dl>
+
+</dd></dl>
+
+</section>
+
+
+ </div>
+ </div>
+ <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
+ <a href="quantiles_depr.html" class="btn btn-neutral float-left" title="Quantiles Sketch (Deprecated)" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
+ <a href="ebpps.html" class="btn btn-neutral float-right" title="Exact and Bounded, Probabilitiy Proportional to Size (EBPPS) Sampling" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
+ </div>
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>© Copyright 2023.</p>
+ </div>
+
+ Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
+ <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
+ provided by <a href="https://readthedocs.org">Read the Docs</a>.
+
+
+</footer>
+ </div>
+ </div>
+ </section>
+ </div>
+ <script>
+ jQuery(function () {
+ SphinxRtdTheme.Navigation.enable(true);
+ });
+ </script>
+
+</body>
+</html>
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org