You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datasketches.apache.org by "ariesdevil (via GitHub)" <gi...@apache.org> on 2026/06/23 13:12:45 UTC
[PR] feat: add tuple theta implementation and tests (datasketches-rust)
ariesdevil opened a new pull request, #138:
URL: https://github.com/apache/datasketches-rust/pull/138
## Summary
Implement the **Tuple sketch** behind a new `tuple` feature (built on top of `theta`). A Tuple sketch extends the Theta sketch by attaching a user-defined summary to every retained key, merging summaries on duplicate keys and during set operations.
Summary behavior is injected externally via policies (C++-style) rather than baked into the type.
## Note
This implementation is highly inspired by the C++ version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
tisonkun commented on PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#issuecomment-4884854877
Thanks for your contribution @ariesdevil! I'll review this in the next week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on code in PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#discussion_r3551839612
##########
datasketches/src/tuple/union.rs:
##########
@@ -0,0 +1,495 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Tuple sketch union.
+//!
+//! [`TupleUnion`] computes the union (set OR) of any number of Tuple sketches. Like the Theta union
+//! it keeps an internal "gadget" hash table plus a running `union_theta`, and updates it from each
+//! input sketch. The only Tuple-specific behavior is that when an incoming key already exists in
+//! the gadget, the two summaries are combined with a [`SummaryCombinePolicy`] instead of one being
+//! dropped.
+
+use std::marker::PhantomData;
+
+use crate::common::ResizeFactor;
+use crate::error::Error;
+use crate::hash::DEFAULT_UPDATE_SEED;
+use crate::theta::DEFAULT_LG_K;
+use crate::theta::MAX_LG_K;
+use crate::theta::MIN_LG_K;
+use crate::tuple::hash_table::TupleHashTable;
+use crate::tuple::policy::DefaultUnionPolicy;
+use crate::tuple::policy::SummaryCombinePolicy;
+use crate::tuple::sketch::CompactTupleSketch;
+use crate::tuple::sketch::TupleSketchView;
+
+/// Union (set OR) of Tuple sketches.
+///
+/// `S` is the summary type and `P` is the [`SummaryCombinePolicy`] applied when a key is present in
+/// more than one input. For additive summaries the default [`DefaultUnionPolicy`] is used.
+///
+/// # Examples
+///
+/// ```
+/// # use datasketches::tuple::{TupleUnion, UpdatableTupleSketch};
+/// let mut a = UpdatableTupleSketch::<u64>::builder().build();
+/// a.update("apple", 1);
+/// a.update("banana", 1);
+///
+/// let mut b = UpdatableTupleSketch::<u64>::builder().build();
+/// b.update("banana", 1);
+/// b.update("cherry", 1);
+///
+/// let mut union = TupleUnion::<u64>::builder().build();
+/// union.update(&a).unwrap();
+/// union.update(&b).unwrap();
+///
+/// let result = union.result(true);
+/// assert_eq!(result.num_retained(), 3); // apple, banana, cherry
+/// ```
+#[derive(Debug)]
+pub struct TupleUnion<S, P = DefaultUnionPolicy> {
Review Comment:
We can separate out `RawHashTable` as a PR first. After merge it, #145 can be rewrite as generic type version. And then tuple union can reuse it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ariesdevil (via GitHub)" <gi...@apache.org>.
ariesdevil closed pull request #138: feat: add tuple theta implementation and tests
URL: https://github.com/apache/datasketches-rust/pull/138
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ariesdevil (via GitHub)" <gi...@apache.org>.
ariesdevil commented on PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#issuecomment-4789300267
@tisonkun @PsiACE PTAL if you have time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on code in PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#discussion_r3530833211
##########
datasketches/src/tuple/hash_table.rs:
##########
@@ -0,0 +1,780 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::hash::Hash;
+
+use crate::common::ResizeFactor;
+use crate::hash::MurmurHash3X64128;
+use crate::hash::compute_seed_hash;
+use crate::theta::HASH_TABLE_REBUILD_THRESHOLD;
+use crate::theta::HASH_TABLE_RESIZE_THRESHOLD;
+use crate::theta::MAX_THETA;
+use crate::theta::MIN_LG_K;
+use crate::theta::STRIDE_MASK;
+use crate::theta::starting_sub_multiple;
+use crate::theta::starting_theta_from_sampling_probability;
+
+/// A retained entry: a hash key together with its associated summary.
+#[derive(Debug)]
+struct TupleEntry<S> {
+ hash: u64,
+ summary: S,
+}
+
+/// Specific hash table for tuple sketch.
+///
+/// This is the Theta sketch hash table extended so that each retained key carries a user-defined
+/// summary. It maintains an array with capacity up to 2^lg_max_size:
+/// * Before it reaches the max capacity, it will extend the array based on resize_factor.
+/// * After it reaches the capacity bigger than 2^lg_nom_size, every time the number of entries
+/// exceeds the threshold, it will rebuild the table: only keep the min 2^lg_nom_size entries and
+/// update the theta to the k-th smallest entry.
+///
+/// Unlike the Theta hash table, when a key is inserted that already exists, the incoming update is
+/// merged into the existing summary rather than discarded.
+#[derive(Debug)]
+pub(super) struct TupleHashTable<S> {
Review Comment:
Since this hash table is same as `ThetaHashTable`. Like cpp, maybe we should refactor it to use Generics type and avoid duplicate implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#issuecomment-4895834554
Based on https://github.com/apache/datasketches-rust/pull/138#discussion_r3530833211 , looks like it's better to refactor ThetaHashTable, related set operation to generic version first. So that we can reuse them in tuple implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ariesdevil (via GitHub)" <gi...@apache.org>.
ariesdevil commented on code in PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#discussion_r3540401243
##########
datasketches/src/tuple/union.rs:
##########
@@ -0,0 +1,495 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Tuple sketch union.
+//!
+//! [`TupleUnion`] computes the union (set OR) of any number of Tuple sketches. Like the Theta union
+//! it keeps an internal "gadget" hash table plus a running `union_theta`, and updates it from each
+//! input sketch. The only Tuple-specific behavior is that when an incoming key already exists in
+//! the gadget, the two summaries are combined with a [`SummaryCombinePolicy`] instead of one being
+//! dropped.
+
+use std::marker::PhantomData;
+
+use crate::common::ResizeFactor;
+use crate::error::Error;
+use crate::hash::DEFAULT_UPDATE_SEED;
+use crate::theta::DEFAULT_LG_K;
+use crate::theta::MAX_LG_K;
+use crate::theta::MIN_LG_K;
+use crate::tuple::hash_table::TupleHashTable;
+use crate::tuple::policy::DefaultUnionPolicy;
+use crate::tuple::policy::SummaryCombinePolicy;
+use crate::tuple::sketch::CompactTupleSketch;
+use crate::tuple::sketch::TupleSketchView;
+
+/// Union (set OR) of Tuple sketches.
+///
+/// `S` is the summary type and `P` is the [`SummaryCombinePolicy`] applied when a key is present in
+/// more than one input. For additive summaries the default [`DefaultUnionPolicy`] is used.
+///
+/// # Examples
+///
+/// ```
+/// # use datasketches::tuple::{TupleUnion, UpdatableTupleSketch};
+/// let mut a = UpdatableTupleSketch::<u64>::builder().build();
+/// a.update("apple", 1);
+/// a.update("banana", 1);
+///
+/// let mut b = UpdatableTupleSketch::<u64>::builder().build();
+/// b.update("banana", 1);
+/// b.update("cherry", 1);
+///
+/// let mut union = TupleUnion::<u64>::builder().build();
+/// union.update(&a).unwrap();
+/// union.update(&b).unwrap();
+///
+/// let result = union.result(true);
+/// assert_eq!(result.num_retained(), 3); // apple, banana, cherry
+/// ```
+#[derive(Debug)]
+pub struct TupleUnion<S, P = DefaultUnionPolicy> {
Review Comment:
So we should merge #145 then I'll rebase on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#issuecomment-4904678762
Thanks @ariesdevil ! This PR contains several part. How about separate this PR into smaller so that it can be more easier to review e.g.
- theta refactor + tuple sketch
- set op(union, intersection, ...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ariesdevil (via GitHub)" <gi...@apache.org>.
ariesdevil commented on code in PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#discussion_r3537257708
##########
datasketches/src/tuple/union.rs:
##########
@@ -0,0 +1,495 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Tuple sketch union.
+//!
+//! [`TupleUnion`] computes the union (set OR) of any number of Tuple sketches. Like the Theta union
+//! it keeps an internal "gadget" hash table plus a running `union_theta`, and updates it from each
+//! input sketch. The only Tuple-specific behavior is that when an incoming key already exists in
+//! the gadget, the two summaries are combined with a [`SummaryCombinePolicy`] instead of one being
+//! dropped.
+
+use std::marker::PhantomData;
+
+use crate::common::ResizeFactor;
+use crate::error::Error;
+use crate::hash::DEFAULT_UPDATE_SEED;
+use crate::theta::DEFAULT_LG_K;
+use crate::theta::MAX_LG_K;
+use crate::theta::MIN_LG_K;
+use crate::tuple::hash_table::TupleHashTable;
+use crate::tuple::policy::DefaultUnionPolicy;
+use crate::tuple::policy::SummaryCombinePolicy;
+use crate::tuple::sketch::CompactTupleSketch;
+use crate::tuple::sketch::TupleSketchView;
+
+/// Union (set OR) of Tuple sketches.
+///
+/// `S` is the summary type and `P` is the [`SummaryCombinePolicy`] applied when a key is present in
+/// more than one input. For additive summaries the default [`DefaultUnionPolicy`] is used.
+///
+/// # Examples
+///
+/// ```
+/// # use datasketches::tuple::{TupleUnion, UpdatableTupleSketch};
+/// let mut a = UpdatableTupleSketch::<u64>::builder().build();
+/// a.update("apple", 1);
+/// a.update("banana", 1);
+///
+/// let mut b = UpdatableTupleSketch::<u64>::builder().build();
+/// b.update("banana", 1);
+/// b.update("cherry", 1);
+///
+/// let mut union = TupleUnion::<u64>::builder().build();
+/// union.update(&a).unwrap();
+/// union.update(&b).unwrap();
+///
+/// let result = union.result(true);
+/// assert_eq!(result.num_retained(), 3); // apple, banana, cherry
+/// ```
+#[derive(Debug)]
+pub struct TupleUnion<S, P = DefaultUnionPolicy> {
Review Comment:
Seems theta has no union implementation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ariesdevil (via GitHub)" <gi...@apache.org>.
ariesdevil commented on PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#issuecomment-4899797364
Fixed, PTAL again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on code in PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#discussion_r3537407241
##########
datasketches/src/tuple/union.rs:
##########
@@ -0,0 +1,495 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Tuple sketch union.
+//!
+//! [`TupleUnion`] computes the union (set OR) of any number of Tuple sketches. Like the Theta union
+//! it keeps an internal "gadget" hash table plus a running `union_theta`, and updates it from each
+//! input sketch. The only Tuple-specific behavior is that when an incoming key already exists in
+//! the gadget, the two summaries are combined with a [`SummaryCombinePolicy`] instead of one being
+//! dropped.
+
+use std::marker::PhantomData;
+
+use crate::common::ResizeFactor;
+use crate::error::Error;
+use crate::hash::DEFAULT_UPDATE_SEED;
+use crate::theta::DEFAULT_LG_K;
+use crate::theta::MAX_LG_K;
+use crate::theta::MIN_LG_K;
+use crate::tuple::hash_table::TupleHashTable;
+use crate::tuple::policy::DefaultUnionPolicy;
+use crate::tuple::policy::SummaryCombinePolicy;
+use crate::tuple::sketch::CompactTupleSketch;
+use crate::tuple::sketch::TupleSketchView;
+
+/// Union (set OR) of Tuple sketches.
+///
+/// `S` is the summary type and `P` is the [`SummaryCombinePolicy`] applied when a key is present in
+/// more than one input. For additive summaries the default [`DefaultUnionPolicy`] is used.
+///
+/// # Examples
+///
+/// ```
+/// # use datasketches::tuple::{TupleUnion, UpdatableTupleSketch};
+/// let mut a = UpdatableTupleSketch::<u64>::builder().build();
+/// a.update("apple", 1);
+/// a.update("banana", 1);
+///
+/// let mut b = UpdatableTupleSketch::<u64>::builder().build();
+/// b.update("banana", 1);
+/// b.update("cherry", 1);
+///
+/// let mut union = TupleUnion::<u64>::builder().build();
+/// union.update(&a).unwrap();
+/// union.update(&b).unwrap();
+///
+/// let result = union.result(true);
+/// assert_eq!(result.num_retained(), 3); // apple, banana, cherry
+/// ```
+#[derive(Debug)]
+pub struct TupleUnion<S, P = DefaultUnionPolicy> {
Review Comment:
Yes. I send a implementation : https://github.com/apache/datasketches-rust/pull/145 . But it's a intermediate state for theta. I think in final we should have something like `ThetaUnion<E, P = DefaultUnionPolicy>` (or `RawThetaUnion<E, P = DefaultUnionPolicy>`) so that tuple union and theta union can reuse it in both.
In https://github.com/apache/datasketches-rust/pull/145 I didn't implement it in this way is because we need to have RawHashTable first. 🤣
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ZENOTME (via GitHub)" <gi...@apache.org>.
ZENOTME commented on code in PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#discussion_r3537054441
##########
datasketches/src/tuple/union.rs:
##########
@@ -0,0 +1,495 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Tuple sketch union.
+//!
+//! [`TupleUnion`] computes the union (set OR) of any number of Tuple sketches. Like the Theta union
+//! it keeps an internal "gadget" hash table plus a running `union_theta`, and updates it from each
+//! input sketch. The only Tuple-specific behavior is that when an incoming key already exists in
+//! the gadget, the two summaries are combined with a [`SummaryCombinePolicy`] instead of one being
+//! dropped.
+
+use std::marker::PhantomData;
+
+use crate::common::ResizeFactor;
+use crate::error::Error;
+use crate::hash::DEFAULT_UPDATE_SEED;
+use crate::theta::DEFAULT_LG_K;
+use crate::theta::MAX_LG_K;
+use crate::theta::MIN_LG_K;
+use crate::tuple::hash_table::TupleHashTable;
+use crate::tuple::policy::DefaultUnionPolicy;
+use crate::tuple::policy::SummaryCombinePolicy;
+use crate::tuple::sketch::CompactTupleSketch;
+use crate::tuple::sketch::TupleSketchView;
+
+/// Union (set OR) of Tuple sketches.
+///
+/// `S` is the summary type and `P` is the [`SummaryCombinePolicy`] applied when a key is present in
+/// more than one input. For additive summaries the default [`DefaultUnionPolicy`] is used.
+///
+/// # Examples
+///
+/// ```
+/// # use datasketches::tuple::{TupleUnion, UpdatableTupleSketch};
+/// let mut a = UpdatableTupleSketch::<u64>::builder().build();
+/// a.update("apple", 1);
+/// a.update("banana", 1);
+///
+/// let mut b = UpdatableTupleSketch::<u64>::builder().build();
+/// b.update("banana", 1);
+/// b.update("cherry", 1);
+///
+/// let mut union = TupleUnion::<u64>::builder().build();
+/// union.update(&a).unwrap();
+/// union.update(&b).unwrap();
+///
+/// let result = union.result(true);
+/// assert_eq!(result.num_retained(), 3); // apple, banana, cherry
+/// ```
+#[derive(Debug)]
+pub struct TupleUnion<S, P = DefaultUnionPolicy> {
Review Comment:
This also can be `ThetaUnion<E, P = DefaultUnionPolicy>` and reuse between tuple and theta.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org
Re: [PR] feat: add tuple theta implementation and tests (datasketches-rust)
Posted by "ariesdevil (via GitHub)" <gi...@apache.org>.
ariesdevil commented on code in PR #138:
URL: https://github.com/apache/datasketches-rust/pull/138#discussion_r3551975431
##########
datasketches/src/tuple/union.rs:
##########
@@ -0,0 +1,495 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Tuple sketch union.
+//!
+//! [`TupleUnion`] computes the union (set OR) of any number of Tuple sketches. Like the Theta union
+//! it keeps an internal "gadget" hash table plus a running `union_theta`, and updates it from each
+//! input sketch. The only Tuple-specific behavior is that when an incoming key already exists in
+//! the gadget, the two summaries are combined with a [`SummaryCombinePolicy`] instead of one being
+//! dropped.
+
+use std::marker::PhantomData;
+
+use crate::common::ResizeFactor;
+use crate::error::Error;
+use crate::hash::DEFAULT_UPDATE_SEED;
+use crate::theta::DEFAULT_LG_K;
+use crate::theta::MAX_LG_K;
+use crate::theta::MIN_LG_K;
+use crate::tuple::hash_table::TupleHashTable;
+use crate::tuple::policy::DefaultUnionPolicy;
+use crate::tuple::policy::SummaryCombinePolicy;
+use crate::tuple::sketch::CompactTupleSketch;
+use crate::tuple::sketch::TupleSketchView;
+
+/// Union (set OR) of Tuple sketches.
+///
+/// `S` is the summary type and `P` is the [`SummaryCombinePolicy`] applied when a key is present in
+/// more than one input. For additive summaries the default [`DefaultUnionPolicy`] is used.
+///
+/// # Examples
+///
+/// ```
+/// # use datasketches::tuple::{TupleUnion, UpdatableTupleSketch};
+/// let mut a = UpdatableTupleSketch::<u64>::builder().build();
+/// a.update("apple", 1);
+/// a.update("banana", 1);
+///
+/// let mut b = UpdatableTupleSketch::<u64>::builder().build();
+/// b.update("banana", 1);
+/// b.update("cherry", 1);
+///
+/// let mut union = TupleUnion::<u64>::builder().build();
+/// union.update(&a).unwrap();
+/// union.update(&b).unwrap();
+///
+/// let result = union.result(true);
+/// assert_eq!(result.num_retained(), 3); // apple, banana, cherry
+/// ```
+#[derive(Debug)]
+pub struct TupleUnion<S, P = DefaultUnionPolicy> {
Review Comment:
Done at #146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@datasketches.apache.org
For additional commands, e-mail: dev-help@datasketches.apache.org