[GitHub] [datasketches-java] breandan commented on issue #357: Comparison between DataSketches, DDSketch, t-digests et al.

[GitBox <gi...@apache.org>]

breandan commented on issue #357:
URL: https://github.com/apache/datasketches-java/issues/357#issuecomment-890004278


   I share your frustration and am beginning to appreciate the difficulty of conducting a fair comparison: on one hand, these sketches are built with different guarantees in mind (e.g. relative error or rank error), and a single metric may not accurately capture the benefits they provide (e.g. latency, memory) or data types they support (e.g. categorical, ordinal, numerical).
   
   On the other hand, we may be splitting hairs a little. You have a mergeable summary of some probability space. If we choose,
   
   1.  a fixed data type (e.g. ordinal)
   2. a fixed hardware (e.g. 8-cores, merged)
   3. a fixed probability metric (e.g. Kolmogorov–Smirnov, Kantorovich-Rubenstein, Lévy–Prokhorov)
   4. a set of performance criteria (e.g. latency, memory, precision) 
   
   and compare the sketch with an exact summary - then rinse and repeat for each algorithm (LDMQS, REQ/KLL, DDSketch, t-digest et al.), and report the pareto efficiency across all performance criteria - does that seem reasonable?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datasketches.apache.org
For additional commands, e-mail: commits-help@datasketches.apache.org