You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datasketches.apache.org by "geonove (via GitHub)" <gi...@apache.org> on 2025/04/27 22:07:05 UTC
[PR] Count-Min Sketch [datasketches-go]
geonove opened a new pull request, #42:
URL: https://github.com/apache/datasketches-go/pull/42
# Port Count-Min Sketch from datasketches-cpp to datasketches-go
## Description
This PR introduces a Go port of the Count-Min Sketch data structure, based on the original implementation in [datasketches-cpp](https://github.com/apache/datasketches-cpp).
The implementation includes methods for updating and querying estimates for different types (e.g., `int`, `uint64`, `string`), following the structure of the C++ version.
In addition, I have developed a second version of this port using Go generics. This alternative approach removes the need to manually implement `Update` and `GetEstimate` methods for each type individually.
If preferred, I can submit the generic version for review instead.
---
Looking forward to any feedback or suggestions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] Count-Min Sketch [datasketches-go]
Posted by "geonove (via GitHub)" <gi...@apache.org>.
geonove commented on code in PR #42:
URL: https://github.com/apache/datasketches-go/pull/42#discussion_r2098806722
##########
count/count_min_sketch.go:
##########
@@ -0,0 +1,344 @@
+package count
+
+import (
+ "bytes"
+ "encoding/binary"
+ "errors"
+ "io"
+ "math"
+ "math/rand"
+
+ "github.com/apache/datasketches-go/internal"
+)
+
+type CountMinSketch struct {
+ numBuckets int32 // counter array size for each of the hashing function
+ numHashes int8 // number of hashing functions
+ sketchSlice []int64
+ seed int64
+ totalWeight int64
+ hashSeeds []int64
+}
+
+func NewCountMinSketch(numHashes int8, numBuckets int32, seed int64) (*CountMinSketch, error) {
Review Comment:
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] Count-Min Sketch [datasketches-go]
Posted by "freakyzoidberg (via GitHub)" <gi...@apache.org>.
freakyzoidberg commented on PR #42:
URL: https://github.com/apache/datasketches-go/pull/42#issuecomment-2841724545
LGTM
Can you add the license headers to each files - and if you can add a cross test deserialising various count min sketch from cpp in Go (look at **_serialization_test.go in other modules)
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] Count-Min Sketch [datasketches-go]
Posted by "freakyzoidberg (via GitHub)" <gi...@apache.org>.
freakyzoidberg commented on code in PR #42:
URL: https://github.com/apache/datasketches-go/pull/42#discussion_r2068511299
##########
count/count_min_sketch.go:
##########
@@ -0,0 +1,344 @@
+package count
+
+import (
+ "bytes"
+ "encoding/binary"
+ "errors"
+ "io"
+ "math"
+ "math/rand"
+
+ "github.com/apache/datasketches-go/internal"
+)
+
+type CountMinSketch struct {
+ numBuckets int32 // counter array size for each of the hashing function
+ numHashes int8 // number of hashing functions
+ sketchSlice []int64
+ seed int64
+ totalWeight int64
+ hashSeeds []int64
+}
+
+func NewCountMinSketch(numHashes int8, numBuckets int32, seed int64) (*CountMinSketch, error) {
Review Comment:
some comments for the public method would be nice :)
##########
count/utils.go:
##########
@@ -0,0 +1,43 @@
+package count
+
+import (
+ "errors"
+ "math"
+
+ "golang.org/x/exp/constraints"
+)
+
+func Min[T constraints.Ordered](a, b T) T {
+ if a < b {
+ return a
+ }
+ return b
+}
+
+func SuggestNumBuckets(relativeError float64) (int32, error) {
+ if relativeError <= 0 {
+ return 0, errors.New("relative error must be greater than 0.0")
+ }
+ return int32(math.Ceil(math.Exp(1.0) / relativeError)), nil
+}
+
+func SuggestNumHashes(confidence float64) (int8, error) {
+ if confidence < 0 || confidence > 1.0 {
+ return 0, errors.New("confidence must be between 0 and 1.0 (inclusive)")
+ }
+ return Min(int8(math.Ceil(math.Log(1.0/(1.0-confidence)))), int8(math.MaxInt8)), nil
+}
+
+func checkHeaderValidity(preamble, serVer, familyID, flagsByte byte) error {
+ return nil
+}
+
+const (
+ PreambleLongsShort = 2
+ SerialVersion1 = 1
+ FamilyId = 18
Review Comment:
Can you move that familyId in https://github.com/apache/datasketches-go/blob/04e51ea1586a0a87371baee010dcaeb8a3f1a62b/internal/family.go ? thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] Count-Min Sketch [datasketches-go]
Posted by "freakyzoidberg (via GitHub)" <gi...@apache.org>.
freakyzoidberg commented on PR #42:
URL: https://github.com/apache/datasketches-go/pull/42#issuecomment-2908818091
LGTM thanks !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] Count-Min Sketch [datasketches-go]
Posted by "freakyzoidberg (via GitHub)" <gi...@apache.org>.
freakyzoidberg merged PR #42:
URL: https://github.com/apache/datasketches-go/pull/42
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org