You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Rachelint (via GitHub)" <gi...@apache.org> on 2024/10/18 09:35:15 UTC
[PR] POC: Vectorize append value [datafusion]
Rachelint opened a new pull request, #12996:
URL: https://github.com/apache/datafusion/pull/12996
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
Closes #.
## Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2451399396
> Amazing!!!
>
> > 1.24x slower
>
> Worth profiling / checking this regression before merging I think
๐ค I plan to try to check it today or tomorrow, I suspect if it is the low cardinality case that many repated rows exist in the input cols?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1822704701
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +144,84 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ group_indices: &[usize],
+ array: &ArrayRef,
+ rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
+
+ for (idx, &lhs_row) in group_indices.iter().enumerate() {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2447041980
I think this pr is ready now!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823755412
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
Review Comment:
I think it is because we need to update the bucket in the hash table ๐
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826167201
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -356,3 +1048,461 @@ impl GroupValues for GroupValuesColumn {
self.hashes_buffer.shrink_to(count);
}
}
+
+/// Returns true if [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+/// supported for the specified schema
+pub fn supported_schema(schema: &Schema) -> bool {
+ schema
+ .fields()
+ .iter()
+ .map(|f| f.data_type())
+ .all(supported_type)
+}
+
+/// Returns true if the specified data type is supported by
+/// [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+///
+/// In order to be supported, there must be a specialized implementation of
+/// [`GroupColumn`] for the data type, instantiated in [`GroupValuesColumn::intern`]
+/// or [`VectorizedGroupValuesColumn::intern`]
+fn supported_type(data_type: &DataType) -> bool {
+ matches!(
+ *data_type,
+ DataType::Int8
+ | DataType::Int16
+ | DataType::Int32
+ | DataType::Int64
+ | DataType::UInt8
+ | DataType::UInt16
+ | DataType::UInt32
+ | DataType::UInt64
+ | DataType::Float32
+ | DataType::Float64
+ | DataType::Utf8
+ | DataType::LargeUtf8
+ | DataType::Binary
+ | DataType::LargeBinary
+ | DataType::Date32
+ | DataType::Date64
+ | DataType::Utf8View
+ | DataType::BinaryView
+ )
+}
+
+#[cfg(test)]
+mod tests {
+ use std::sync::Arc;
+
+ use arrow::{compute::concat_batches, util::pretty::pretty_format_batches};
+ use arrow_array::{ArrayRef, Int64Array, RecordBatch, StringArray, StringViewArray};
+ use arrow_schema::{DataType, Field, Schema, SchemaRef};
+ use datafusion_expr::EmitTo;
+
+ use crate::aggregates::group_values::{
+ column::VectorizedGroupValuesColumn, GroupValues,
+ };
+
+ #[test]
+ fn test_intern_for_vectorized_group_values() {
+ let data_set = VectorizedTestDataSet::new();
+ let mut group_values =
+ VectorizedGroupValuesColumn::try_new(data_set.schema()).unwrap();
+
+ data_set.load_to_group_values(&mut group_values);
+ let actual_batch = group_values.emit(EmitTo::All).unwrap();
+ let actual_batch = RecordBatch::try_new(data_set.schema(), actual_batch).unwrap();
+
+ check_result(&actual_batch, &data_set.expected_batch);
+ }
+
+ #[test]
+ fn test_emit_first_n_for_vectorized_group_values() {
Review Comment:
https://github.com/apache/datafusion/pull/12996#discussion_r1824623183 has been finished, now we can totally avoid the unnecessary memory allocations for modifying `group_index_lists`!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2453401637
BTW I think this code is fairly well covered by the aggregate fuzz tester (also added by @Rachelint :))
Also, @LeslieKid is adding additional data type coverage which is great: https://github.com/apache/datafusion/pull/13226
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2457883692
I am giving this a final review now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1818154221
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -125,6 +233,292 @@ impl GroupValuesColumn {
| DataType::BinaryView
)
}
+
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
Review Comment:
It should be slightly faster to not do both `get` and `insert`, you could use `find_or_find_insert_slot` and `insert_in_slot` if this is expensive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1818601807
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better. I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
>
> Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
Make sense to me, and support it in `arrow` seems really good.
I plan to sort out codes in this pr , and then try to improve the primitive case using `take` + `eq`.
I guess some case like `q32` in clickbench is possible to get benefit from it.
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better. I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
>
> Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
Make sense to me, and support it in `arrow` seems really good.
I plan to sort out codes in this pr , and then try to improve the primitive case using `take` + `eq`.
I guess some case like `q32` in clickbench is possible to get benefit from it.
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better. I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
>
> Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
Make sense to me, and support it in `arrow` seems really good.
I plan to sort out codes in this pr , and then try to improve the primitive case using `take` + `eq`.
I guess some case like `q32` in clickbench is possible to get benefit from it.
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better. I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
>
> Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
Make sense to me, and support it in `arrow` seems really good.
I plan to sort out codes in this pr , and then try to improve the primitive case using `take` + `eq`.
I guess some case like `q32` in clickbench is possible to get benefit from it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1818156207
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
In hash join implementation the equal to is basically using the arrow kernels `take` (for rows to check) + `eq` + `and` (if multiple columns), which is I think pretty fast, at least for primitives (and string views, byte/string might be slower as it needs some more copying). This performed better than the row-by-row check.
I didn't look to much at this implementation, but does it seem worth trying to you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819914951
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -58,6 +63,17 @@ pub trait GroupColumn: Send + Sync {
fn equal_to(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool;
/// Appends the row at `row` in `array` to this builder
fn append_val(&mut self, array: &ArrayRef, row: usize);
+
+ fn vectorized_equal_to(
Review Comment:
Are these vectorized_xxx SIMD? or they are just processing the rows all at once?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2450735735
๐คฉ
```
Comparing main_base and vectorize-append-value
--------------------
Benchmark clickbench_1.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main_base โ vectorize-append-value โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 0.69ms โ 0.67ms โ no change โ
โ QQuery 1 โ 69.53ms โ 69.81ms โ no change โ
โ QQuery 2 โ 125.56ms โ 125.06ms โ no change โ
โ QQuery 3 โ 134.49ms โ 133.53ms โ no change โ
โ QQuery 4 โ 986.06ms โ 1002.09ms โ no change โ
โ QQuery 5 โ 1092.56ms โ 1111.32ms โ no change โ
โ QQuery 6 โ 67.90ms โ 66.92ms โ no change โ
โ QQuery 7 โ 82.12ms โ 80.79ms โ no change โ
โ QQuery 8 โ 1381.07ms โ 1055.63ms โ +1.31x faster โ
โ QQuery 9 โ 1376.13ms โ 1371.51ms โ no change โ
โ QQuery 10 โ 466.36ms โ 435.31ms โ +1.07x faster โ
โ QQuery 11 โ 508.85ms โ 474.68ms โ +1.07x faster โ
โ QQuery 12 โ 1245.08ms โ 1216.74ms โ no change โ
โ QQuery 13 โ 1919.50ms โ 1746.68ms โ +1.10x faster โ
โ QQuery 14 โ 1371.00ms โ 1238.46ms โ +1.11x faster โ
โ QQuery 15 โ 1156.13ms โ 1123.78ms โ no change โ
โ QQuery 16 โ 2593.20ms โ 2187.34ms โ +1.19x faster โ
โ QQuery 17 โ 2374.51ms โ 2008.88ms โ +1.18x faster โ
โ QQuery 18 โ 5049.66ms โ 4169.48ms โ +1.21x faster โ
โ QQuery 19 โ 125.17ms โ 119.76ms โ no change โ
โ QQuery 20 โ 1650.50ms โ 1642.21ms โ no change โ
โ QQuery 21 โ 2100.16ms โ 2082.53ms โ no change โ
โ QQuery 22 โ 5039.82ms โ 4996.73ms โ no change โ
โ QQuery 23 โ 11902.50ms โ 11835.25ms โ no change โ
โ QQuery 24 โ 810.90ms โ 797.78ms โ no change โ
โ QQuery 25 โ 728.72ms โ 699.22ms โ no change โ
โ QQuery 26 โ 868.88ms โ 891.65ms โ no change โ
โ QQuery 27 โ 2627.37ms โ 2600.82ms โ no change โ
โ QQuery 28 โ 15172.21ms โ 15297.59ms โ no change โ
โ QQuery 29 โ 572.83ms โ 571.91ms โ no change โ
โ QQuery 30 โ 1239.14ms โ 1141.57ms โ +1.09x faster โ
โ QQuery 31 โ 1277.14ms โ 1206.06ms โ +1.06x faster โ
โ QQuery 32 โ 4198.97ms โ 3978.68ms โ +1.06x faster โ
โ QQuery 33 โ 5184.58ms โ 5281.99ms โ no change โ
โ QQuery 34 โ 5334.25ms โ 5234.23ms โ no change โ
โ QQuery 35 โ 1799.37ms โ 1351.66ms โ +1.33x faster โ
โ QQuery 36 โ 319.76ms โ 312.16ms โ no change โ
โ QQuery 37 โ 220.55ms โ 220.76ms โ no change โ
โ QQuery 38 โ 197.20ms โ 195.72ms โ no change โ
โ QQuery 39 โ 810.69ms โ 732.04ms โ +1.11x faster โ
โ QQuery 40 โ 91.81ms โ 84.87ms โ +1.08x faster โ
โ QQuery 41 โ 81.72ms โ 77.45ms โ +1.06x faster โ
โ QQuery 42 โ 94.61ms โ 94.34ms โ no change โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (main_base) โ 84449.27ms โ
โ Total Time (vectorize-append-value) โ 81065.67ms โ
โ Average Time (main_base) โ 1963.94ms โ
โ Average Time (vectorize-append-value) โ 1885.25ms โ
โ Queries Faster โ 15 โ
โ Queries Slower โ 0 โ
โ Queries with No Change โ 28 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโ
--------------------
Benchmark clickbench_extended.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main_base โ vectorize-append-value โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 2736.57ms โ 2712.16ms โ no change โ
โ QQuery 1 โ 772.15ms โ 769.86ms โ no change โ
โ QQuery 2 โ 1624.12ms โ 1639.77ms โ no change โ
โ QQuery 3 โ 827.73ms โ 694.66ms โ +1.19x faster โ
โ QQuery 4 โ 12716.33ms โ 12364.86ms โ no change โ
โ QQuery 5 โ 19095.23ms โ 19058.86ms โ no change โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (main_base) โ 37772.14ms โ
โ Total Time (vectorize-append-value) โ 37240.16ms โ
โ Average Time (main_base) โ 6295.36ms โ
โ Average Time (vectorize-append-value) โ 6206.69ms โ
โ Queries Faster โ 1 โ
โ Queries Slower โ 0 โ
โ Queries with No Change โ 5 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโ
--------------------
Benchmark clickbench_partitioned.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main_base โ vectorize-append-value โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 2.25ms โ 2.27ms โ no change โ
โ QQuery 1 โ 40.78ms โ 38.83ms โ no change โ
โ QQuery 2 โ 98.26ms โ 95.76ms โ no change โ
โ QQuery 3 โ 108.58ms โ 106.87ms โ no change โ
โ QQuery 4 โ 922.34ms โ 917.62ms โ no change โ
โ QQuery 5 โ 980.37ms โ 938.35ms โ no change โ
โ QQuery 6 โ 38.42ms โ 39.18ms โ no change โ
โ QQuery 7 โ 43.00ms โ 43.24ms โ no change โ
โ QQuery 8 โ 1359.05ms โ 1009.35ms โ +1.35x faster โ
โ QQuery 9 โ 1348.60ms โ 1323.55ms โ no change โ
โ QQuery 10 โ 424.82ms โ 406.74ms โ no change โ
โ QQuery 11 โ 475.00ms โ 451.86ms โ no change โ
โ QQuery 12 โ 1090.45ms โ 1106.96ms โ no change โ
โ QQuery 13 โ 1577.54ms โ 1508.58ms โ no change โ
โ QQuery 14 โ 1259.77ms โ 1086.88ms โ +1.16x faster โ
โ QQuery 15 โ 1086.47ms โ 1081.46ms โ no change โ
โ QQuery 16 โ 2449.02ms โ 2030.13ms โ +1.21x faster โ
โ QQuery 17 โ 2329.89ms โ 1911.21ms โ +1.22x faster โ
โ QQuery 18 โ 5004.89ms โ 3989.09ms โ +1.25x faster โ
โ QQuery 19 โ 97.70ms โ 98.64ms โ no change โ
โ QQuery 20 โ 1474.31ms โ 1478.79ms โ no change โ
โ QQuery 21 โ 1808.23ms โ 1810.91ms โ no change โ
โ QQuery 22 โ 3132.66ms โ 3128.20ms โ no change โ
โ QQuery 23 โ 10113.15ms โ 10129.78ms โ no change โ
โ QQuery 24 โ 656.18ms โ 651.62ms โ no change โ
โ QQuery 25 โ 518.58ms โ 524.60ms โ no change โ
โ QQuery 26 โ 719.15ms โ 725.11ms โ no change โ
โ QQuery 27 โ 2266.93ms โ 2264.89ms โ no change โ
โ QQuery 28 โ 14304.45ms โ 14598.41ms โ no change โ
โ QQuery 29 โ 542.95ms โ 540.96ms โ no change โ
โ QQuery 30 โ 1091.63ms โ 1000.47ms โ +1.09x faster โ
โ QQuery 31 โ 1140.61ms โ 1070.64ms โ +1.07x faster โ
โ QQuery 32 โ 4198.09ms โ 3928.13ms โ +1.07x faster โ
โ QQuery 33 โ 5178.24ms โ 5025.69ms โ no change โ
โ QQuery 34 โ 5065.45ms โ 5110.93ms โ no change โ
โ QQuery 35 โ 1869.61ms โ 1300.99ms โ +1.44x faster โ
โ QQuery 36 โ 274.68ms โ 270.33ms โ no change โ
โ QQuery 37 โ 125.61ms โ 125.59ms โ no change โ
โ QQuery 38 โ 150.35ms โ 144.38ms โ no change โ
โ QQuery 39 โ 732.00ms โ 689.60ms โ +1.06x faster โ
โ QQuery 40 โ 57.04ms โ 60.17ms โ 1.05x slower โ
โ QQuery 41 โ 47.52ms โ 47.56ms โ no change โ
โ QQuery 42 โ 65.98ms โ 64.11ms โ no change โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (main_base) โ 76270.60ms โ
โ Total Time (vectorize-append-value) โ 72878.40ms โ
โ Average Time (main_base) โ 1773.73ms โ
โ Average Time (vectorize-append-value) โ 1694.85ms โ
โ Queries Faster โ 10 โ
โ Queries Slower โ 1 โ
โ Queries with No Change โ 32 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโ
--------------------
Benchmark tpch_sf1.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main_base โ vectorize-append-value โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 1 โ 203.49ms โ 201.33ms โ no change โ
โ QQuery 2 โ 115.50ms โ 120.92ms โ no change โ
โ QQuery 3 โ 129.08ms โ 119.50ms โ +1.08x faster โ
โ QQuery 4 โ 87.14ms โ 83.75ms โ no change โ
โ QQuery 5 โ 161.47ms โ 159.55ms โ no change โ
โ QQuery 6 โ 48.73ms โ 42.26ms โ +1.15x faster โ
โ QQuery 7 โ 208.57ms โ 200.37ms โ no change โ
โ QQuery 8 โ 157.16ms โ 153.09ms โ no change โ
โ QQuery 9 โ 244.82ms โ 247.43ms โ no change โ
โ QQuery 10 โ 226.83ms โ 238.14ms โ no change โ
โ QQuery 11 โ 86.91ms โ 91.82ms โ 1.06x slower โ
โ QQuery 12 โ 128.79ms โ 128.85ms โ no change โ
โ QQuery 13 โ 225.47ms โ 219.07ms โ no change โ
โ QQuery 14 โ 72.15ms โ 89.80ms โ 1.24x slower โ
โ QQuery 15 โ 115.99ms โ 107.62ms โ +1.08x faster โ
โ QQuery 16 โ 80.43ms โ 74.99ms โ +1.07x faster โ
โ QQuery 17 โ 218.58ms โ 218.94ms โ no change โ
โ QQuery 18 โ 329.54ms โ 321.20ms โ no change โ
โ QQuery 19 โ 131.86ms โ 129.84ms โ no change โ
โ QQuery 20 โ 136.33ms โ 121.60ms โ +1.12x faster โ
โ QQuery 21 โ 272.83ms โ 283.72ms โ no change โ
โ QQuery 22 โ 66.11ms โ 63.99ms โ no change โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (main_base) โ 3447.78ms โ
โ Total Time (vectorize-append-value) โ 3417.79ms โ
โ Average Time (main_base) โ 156.72ms โ
โ Average Time (vectorize-append-value) โ 155.35ms โ
โ Queries Faster โ 5 โ
โ Queries Slower โ 2 โ
โ Queries with No Change โ 15 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโ
```
Wow very nice @Rachelint -- I will plan on checking this PR out in more detail tomorrow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2453401222
> ๐ค I personally prefer the second one? What do you think about it @alamb ?
I think this makes sense -- thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823150363
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -125,6 +233,292 @@ impl GroupValuesColumn {
| DataType::BinaryView
)
}
+
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
Review Comment:
Probably it's not a very hot path at the moment, was also surprised to see for hash join that other parts (e.g. traversal of matching rows / equality check) are much more expensive than the lookup of the map.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823132193
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -125,6 +233,292 @@ impl GroupValuesColumn {
| DataType::BinaryView
)
}
+
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
Review Comment:
Seems not obvious improvement same as I tried in some previous experiments...
๐ค Really strange... I am confused about why the high cardinality query can not get improvement from it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826584391
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -35,29 +37,113 @@ use datafusion_common::{not_impl_err, DataFusionError, Result};
use datafusion_execution::memory_pool::proxy::{RawTableAllocExt, VecAllocExt};
use datafusion_expr::EmitTo;
use datafusion_physical_expr::binary_map::OutputType;
+
use hashbrown::raw::RawTable;
-use std::mem::size_of;
-/// A [`GroupValues`] that stores multiple columns of group values.
+const NON_INLINED_FLAG: u64 = 0x8000000000000000;
+const VALUE_MASK: u64 = 0x7FFFFFFFFFFFFFFF;
+
+/// The view of indices pointing to the actual values in `GroupValues`
///
+/// If only single `group index` represented by view,
+/// value of view is just the `group index`, and we call it a `inlined view`.
///
-pub struct GroupValuesColumn {
+/// If multiple `group indices` represented by view,
+/// value of view is the actually the index pointing to `group indices`,
+/// and we call it `non-inlined view`.
+///
+/// The view(a u64) format is like:
+/// +---------------------+---------------------------------------------+
+/// | inlined flag(1bit) | group index / index to group indices(63bit) |
+/// +---------------------+---------------------------------------------+
+///
+/// `inlined flag`: 1 represents `non-inlined`, and 0 represents `inlined`
+///
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+struct GroupIndexView(u64);
+
+impl GroupIndexView {
+ #[inline]
+ pub fn is_non_inlined(&self) -> bool {
+ (self.0 & NON_INLINED_FLAG) > 0
+ }
+
+ #[inline]
+ pub fn new_inlined(group_index: u64) -> Self {
+ Self(group_index)
+ }
+
+ #[inline]
+ pub fn new_non_inlined(list_offset: u64) -> Self {
+ let non_inlined_value = list_offset | NON_INLINED_FLAG;
+ Self(non_inlined_value)
+ }
+
+ #[inline]
+ pub fn value(&self) -> u64 {
+ self.0 & VALUE_MASK
+ }
+}
+
+/// A [`GroupValues`] that stores multiple columns of group values,
+/// and supports vectorized operators for them
+///
+pub struct VectorizedGroupValuesColumn {
/// The output schema
schema: SchemaRef,
/// Logically maps group values to a group_index in
/// [`Self::group_values`] and in each accumulator
///
- /// Uses the raw API of hashbrown to avoid actually storing the
- /// keys (group values) in the table
+ /// It is a `hashtable` based on `hashbrown`.
///
- /// keys: u64 hashes of the GroupValue
- /// values: (hash, group_index)
- map: RawTable<(u64, usize)>,
+ /// Key and value in the `hashtable`:
+ /// - The `key` is `hash value(u64)` of the `group value`
+ /// - The `value` is the `group values` with the same `hash value`
+ ///
+ /// We don't really store the actual `group values` in `hashtable`,
+ /// instead we store the `group indices` pointing to values in `GroupValues`.
+ /// And we use [`GroupIndexView`] to represent such `group indices` in table.
+ ///
+ ///
+ map: RawTable<(u64, GroupIndexView)>,
Review Comment:
Is it the case
1. If group1 and group2 have exactly the same hash value, `GroupIndexView` will use chaining to resolve the collision
2. If group1 and group2 have different hash values but map to the same slot in hash table, `hashbrown` will handle the collision for you with probing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1807857538
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -160,11 +162,112 @@ macro_rules! instantiate_primitive {
};
}
-fn append_col_value<C>(mut core: C, array: &ArrayRef, row: usize)
-where
- C: FnMut(&ArrayRef, usize),
-{
- core(array, row);
+struct AggregationHashTable<T: AggregationHashTableEntry> {
+ /// Raw table storing values in a `Vec`
+ raw_table: Vec<T>,
Review Comment:
The reason I think is that the lookup is incredibly well optimized using the swiss table design and you get fewer 'false" candidates to check for, while we can still use the vectorized/type specialized equality check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1807821784
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -160,11 +162,112 @@ macro_rules! instantiate_primitive {
};
}
-fn append_col_value<C>(mut core: C, array: &ArrayRef, row: usize)
-where
- C: FnMut(&ArrayRef, usize),
-{
- core(array, row);
+struct AggregationHashTable<T: AggregationHashTableEntry> {
+ /// Raw table storing values in a `Vec`
+ raw_table: Vec<T>,
Review Comment:
Based on some experiments in changing hash join algorithm, I think it's likely `hashbrown` performs much better than implementing a hashtable ourselves although I would like to be surprised ๐
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1855477417
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better. I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
>
> Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
I try `take + eq` approach , but see no improvement... I think it may be due to the extra copys.
code is here https://github.com/Rachelint/arrow-datafusion/tree/optimize-vectorized-operations-bak
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1806247830
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
Review Comment:
This could be optimized to append nulls for entire batch instead of value
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
Review Comment:
This could be optimized to append nulls for entire batch instead of per value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1807293142
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
Review Comment:
I add the `append_batch` function to support vectorized append more better.
But the improvement seems still not obvious. https://github.com/apache/datafusion/pull/12996#issuecomment-2423766327
๐ค I guess, it is likely due the new introduced branch of `equal_to`:
```
if *group_idx < group_values_len {
for (i, group_val) in self.group_values.iter().enumerate() {
if !check_row_equal(group_val.as_ref(), *group_idx, &cols[i], row)
{
return false;
}
}
} else {
let row_idx_offset = group_idx - group_values_len;
let row_idx = self.append_rows_buffer[row_idx_offset];
return is_rows_eq(cols, row, cols, row_idx).unwrap();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823793289
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -212,17 +330,92 @@ where
self.offsets.push(O::usize_as(offset));
} else {
self.nulls.append(false);
- let value: &[u8] = arr.value(row).as_ref();
- self.buffer.append_slice(value);
- self.offsets.push(O::usize_as(self.buffer.len()));
+ self.do_append_val_inner(arr, row);
}
}
- fn equal_to_inner<B>(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool
- where
+ fn vectorized_equal_to_inner<B>(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) where
B: ByteArrayType,
{
let array = array.as_bytes::<B>();
+
+ let iter = lhs_rows
+ .iter()
+ .zip(rhs_rows.iter())
+ .zip(equal_to_results.iter_mut());
Review Comment:
NIce, fixed
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -212,17 +330,92 @@ where
self.offsets.push(O::usize_as(offset));
} else {
self.nulls.append(false);
- let value: &[u8] = arr.value(row).as_ref();
- self.buffer.append_slice(value);
- self.offsets.push(O::usize_as(self.buffer.len()));
+ self.do_append_val_inner(arr, row);
}
}
- fn equal_to_inner<B>(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool
- where
+ fn vectorized_equal_to_inner<B>(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) where
B: ByteArrayType,
{
let array = array.as_bytes::<B>();
+
+ let iter = lhs_rows
+ .iter()
+ .zip(rhs_rows.iter())
+ .zip(equal_to_results.iter_mut());
Review Comment:
Nice, fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826586152
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -35,29 +37,113 @@ use datafusion_common::{not_impl_err, DataFusionError, Result};
use datafusion_execution::memory_pool::proxy::{RawTableAllocExt, VecAllocExt};
use datafusion_expr::EmitTo;
use datafusion_physical_expr::binary_map::OutputType;
+
use hashbrown::raw::RawTable;
-use std::mem::size_of;
-/// A [`GroupValues`] that stores multiple columns of group values.
+const NON_INLINED_FLAG: u64 = 0x8000000000000000;
+const VALUE_MASK: u64 = 0x7FFFFFFFFFFFFFFF;
+
+/// The view of indices pointing to the actual values in `GroupValues`
///
+/// If only single `group index` represented by view,
+/// value of view is just the `group index`, and we call it a `inlined view`.
///
-pub struct GroupValuesColumn {
+/// If multiple `group indices` represented by view,
+/// value of view is the actually the index pointing to `group indices`,
+/// and we call it `non-inlined view`.
+///
+/// The view(a u64) format is like:
+/// +---------------------+---------------------------------------------+
+/// | inlined flag(1bit) | group index / index to group indices(63bit) |
+/// +---------------------+---------------------------------------------+
+///
+/// `inlined flag`: 1 represents `non-inlined`, and 0 represents `inlined`
+///
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+struct GroupIndexView(u64);
+
+impl GroupIndexView {
+ #[inline]
+ pub fn is_non_inlined(&self) -> bool {
+ (self.0 & NON_INLINED_FLAG) > 0
+ }
+
+ #[inline]
+ pub fn new_inlined(group_index: u64) -> Self {
+ Self(group_index)
+ }
+
+ #[inline]
+ pub fn new_non_inlined(list_offset: u64) -> Self {
+ let non_inlined_value = list_offset | NON_INLINED_FLAG;
+ Self(non_inlined_value)
+ }
+
+ #[inline]
+ pub fn value(&self) -> u64 {
+ self.0 & VALUE_MASK
+ }
+}
+
+/// A [`GroupValues`] that stores multiple columns of group values,
+/// and supports vectorized operators for them
+///
+pub struct VectorizedGroupValuesColumn {
/// The output schema
schema: SchemaRef,
/// Logically maps group values to a group_index in
/// [`Self::group_values`] and in each accumulator
///
- /// Uses the raw API of hashbrown to avoid actually storing the
- /// keys (group values) in the table
+ /// It is a `hashtable` based on `hashbrown`.
///
- /// keys: u64 hashes of the GroupValue
- /// values: (hash, group_index)
- map: RawTable<(u64, usize)>,
+ /// Key and value in the `hashtable`:
+ /// - The `key` is `hash value(u64)` of the `group value`
+ /// - The `value` is the `group values` with the same `hash value`
+ ///
+ /// We don't really store the actual `group values` in `hashtable`,
+ /// instead we store the `group indices` pointing to values in `GroupValues`.
+ /// And we use [`GroupIndexView`] to represent such `group indices` in table.
+ ///
+ ///
+ map: RawTable<(u64, GroupIndexView)>,
Review Comment:
Totally right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2450647555
Nice -- I am running some benchmarks on this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823718252
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
Review Comment:
I'm unclear on the meaning of the placeholders here, but I understand why we need to append values first. Groups that require an equality check may need to compare against the newly added groups.
In collect_vectorized_process_context, new groups only update the hash in the hash table without appending values yet. After checking the hash, rows are allocated to groups requiring an equality check. Therefore, we need to append the new groups first so that equality checks can include the newly appended groups
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2455731485
This is top of my list to review tomorrow morning
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1820061845
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -58,6 +63,17 @@ pub trait GroupColumn: Send + Sync {
fn equal_to(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool;
/// Appends the row at `row` in `array` to this builder
fn append_val(&mut self, array: &ArrayRef, row: usize);
+
+ fn vectorized_equal_to(
Review Comment:
It only switch the process logic from `row oriented` to `col oriented`.
The original logic is like:
```
eq col1[0], col2[0], col3[0]
eq col1[1], col2[1], col3[1]
...
eq col1[n], col2[n], col3[n]
```
And current logic:
```
eq col1[0] ~ col1[n]
eq col2[0] ~ col2[n]
eq col3[0] ~ col3[[n]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819944790
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
+ let entry = map.get_mut(target_hash, |(exist_hash, group_index_view)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ // Only `rows` having the same hash value with `exist rows` but different value
+ // will be process in `scalarized_intern`.
+ // So related `buckets` in `map` is ensured to be `Some`.
+ let Some((_, group_index_view)) = entry else {
+ unreachable!()
+ };
+
+ // Perform scalarized equal to
+ if self.scalarized_equal_to(&group_index_view, cols, row, groups) {
+ // Found the row actually exists in group values,
+ // don't need to create new group for it.
+ continue;
+ }
+
+ // Insert the `row` to `group_values` before checking `next row`
+ let group_idx = self.group_values[0].len();
+ let mut checklen = 0;
+ for (i, group_value) in self.group_values.iter_mut().enumerate() {
+ group_value.append_val(&cols[i], row);
+ let len = group_value.len();
+ if i == 0 {
+ checklen = len;
+ } else {
+ debug_assert_eq!(checklen, len);
+ }
+ }
+
+ // Check if the `view` is `inlined` or `non-inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, get `group_index_list` from `group_index_lists`,
+ // then add the new `group` with the same hash values into it.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &mut self.group_index_lists[list_offset];
+ group_index_list.push(group_idx);
+ } else {
+ // Inlined case
+ let list_offset = self.group_index_lists.len();
+
+ // Create new `group_index_list` including
+ // `exist group index` + `new group index`.
+ // Add new `group_index_list` into ``group_index_lists`.
+ let exist_group_index = group_index_view.value() as usize;
+ let new_group_index_list = vec![exist_group_index, group_idx];
Review Comment:
Do we need `exist_group_index`? I think it is the one that has the same value and different value for other rows, so we don't need to check the value once again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1820123250
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
Review Comment:
Can we do `vectorized_equal_to` check here directly? I think we don't need to wait until `vectorized_append` are done since their hash are all different.
If we do equality check here the logic is more close together and can avoid to store information like `row` and `group_index`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1855477417
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better. I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
>
> Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
I try `take + eq` approach , but see no improvement... code is here https://github.com/Rachelint/arrow-datafusion/tree/optimize-vectorized-operations-bak
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1830886618
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +157,89 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
+
+ let iter = izip!(
+ lhs_rows.iter(),
+ rhs_rows.iter(),
+ equal_to_results.iter_mut(),
+ );
+
+ for (&lhs_row, &rhs_row, equal_to_result) in iter {
+ // Has found not equal to in previous column, don't need to check
+ if !*equal_to_result {
+ continue;
+ }
+
+ // Perf: skip null check (by short circuit) if input is not nullable
+ if NULLABLE {
+ let exist_null = self.nulls.is_null(lhs_row);
+ let input_null = array.is_null(rhs_row);
+ if let Some(result) = nulls_equal_to(exist_null, input_null) {
+ *equal_to_result = result;
+ continue;
+ }
+ // Otherwise, we need to check their values
+ }
+
+ *equal_to_result = self.group_values[lhs_row] == array.value(rhs_row);
+ }
+ }
+
+ fn vectorized_append(&mut self, array: &ArrayRef, rows: &[usize]) {
+ let arr = array.as_primitive::<T>();
+
+ let null_count = array.null_count();
+ let num_rows = array.len();
+ let all_null_or_non_null = if null_count == 0 {
+ Some(true)
+ } else if null_count == num_rows {
+ Some(false)
+ } else {
+ None
+ };
+
+ match (NULLABLE, all_null_or_non_null) {
+ (true, None) => {
+ for &row in rows {
+ if array.is_null(row) {
+ self.nulls.append(true);
+ self.group_values.push(T::default_value());
+ } else {
+ self.nulls.append(false);
+ self.group_values.push(arr.value(row));
+ }
+ }
+ }
+
+ (true, Some(true)) => {
+ self.nulls.append_n(rows.len(), false);
+ for &row in rows {
+ self.group_values.push(arr.value(row));
+ }
+ }
+
+ (true, Some(false)) => {
+ self.nulls.append_n(rows.len(), true);
+ self.group_values
+ .extend(iter::repeat(T::default_value()).take(rows.len()));
+ }
+
+ (false, _) => {
+ for &row in rows {
+ self.group_values.push(arr.value(row));
Review Comment:
> I think we already could use extend instead of `push`? `extend` on `Vec` is somewhat faster than push as the capacity check / allocation is done once instead of once per value.
Ok, I got it, it is really simple to do it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1807857648
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -160,11 +162,112 @@ macro_rules! instantiate_primitive {
};
}
-fn append_col_value<C>(mut core: C, array: &ArrayRef, row: usize)
-where
- C: FnMut(&ArrayRef, usize),
-{
- core(array, row);
+struct AggregationHashTable<T: AggregationHashTableEntry> {
+ /// Raw table storing values in a `Vec`
+ raw_table: Vec<T>,
Review Comment:
Make sense, thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1820076357
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
+ let entry = map.get_mut(target_hash, |(exist_hash, group_index_view)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ // Only `rows` having the same hash value with `exist rows` but different value
+ // will be process in `scalarized_intern`.
+ // So related `buckets` in `map` is ensured to be `Some`.
+ let Some((_, group_index_view)) = entry else {
+ unreachable!()
+ };
+
+ // Perform scalarized equal to
+ if self.scalarized_equal_to(&group_index_view, cols, row, groups) {
+ // Found the row actually exists in group values,
+ // don't need to create new group for it.
+ continue;
+ }
+
+ // Insert the `row` to `group_values` before checking `next row`
+ let group_idx = self.group_values[0].len();
+ let mut checklen = 0;
+ for (i, group_value) in self.group_values.iter_mut().enumerate() {
+ group_value.append_val(&cols[i], row);
+ let len = group_value.len();
+ if i == 0 {
+ checklen = len;
+ } else {
+ debug_assert_eq!(checklen, len);
+ }
+ }
+
+ // Check if the `view` is `inlined` or `non-inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, get `group_index_list` from `group_index_lists`,
+ // then add the new `group` with the same hash values into it.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &mut self.group_index_lists[list_offset];
+ group_index_list.push(group_idx);
+ } else {
+ // Inlined case
+ let list_offset = self.group_index_lists.len();
+
+ // Create new `group_index_list` including
+ // `exist group index` + `new group index`.
+ // Add new `group_index_list` into ``group_index_lists`.
+ let exist_group_index = group_index_view.value() as usize;
+ let new_group_index_list = vec![exist_group_index, group_idx];
Review Comment:
The `map` is a bit different with origin.
The original `map` can be seen as `HashMap<(hash value, group value), ()>`, and each bucket only points to `single group`.
And current `map` is actullay like `HashMap<hash value, Vec<group value>>`, each bucket will point to a list of `groups with the same hash value(hash collision)`.
In this branch, the `new group with the same hash value` found, so we need to switch `inlined group view(the actual and single group index)` to `non-inlined group view(a index to gourp indices list)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2450744226
Amazing!!!
> 1.24x slower
Worth profiling / checking this regression before merging I think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826380362
##########
datafusion/physical-plan/src/aggregates/group_values/mod.rs:
##########
@@ -143,8 +148,12 @@ pub fn new_group_values(schema: SchemaRef) -> Result<Box<dyn GroupValues>> {
}
}
- if GroupValuesColumn::supported_schema(schema.as_ref()) {
- Ok(Box::new(GroupValuesColumn::try_new(schema)?))
+ if column::supported_schema(schema.as_ref()) {
Review Comment:
> Is it because the VectorizedGroupValuesColumn doesn't keep the groups in order?
Yes, because we iterate all the rows at once, some rows are append beforehand so they are not kept in order.
> If that is the case, it seems like maybe emit_n would never be called
`emit_early_if_necessary` may be called
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1830737096
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +157,89 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
+
+ let iter = izip!(
+ lhs_rows.iter(),
+ rhs_rows.iter(),
+ equal_to_results.iter_mut(),
+ );
+
+ for (&lhs_row, &rhs_row, equal_to_result) in iter {
+ // Has found not equal to in previous column, don't need to check
+ if !*equal_to_result {
+ continue;
+ }
+
+ // Perf: skip null check (by short circuit) if input is not nullable
+ if NULLABLE {
+ let exist_null = self.nulls.is_null(lhs_row);
+ let input_null = array.is_null(rhs_row);
+ if let Some(result) = nulls_equal_to(exist_null, input_null) {
+ *equal_to_result = result;
+ continue;
+ }
+ // Otherwise, we need to check their values
+ }
+
+ *equal_to_result = self.group_values[lhs_row] == array.value(rhs_row);
+ }
+ }
+
+ fn vectorized_append(&mut self, array: &ArrayRef, rows: &[usize]) {
+ let arr = array.as_primitive::<T>();
+
+ let null_count = array.null_count();
+ let num_rows = array.len();
+ let all_null_or_non_null = if null_count == 0 {
+ Some(true)
+ } else if null_count == num_rows {
+ Some(false)
+ } else {
+ None
+ };
+
+ match (NULLABLE, all_null_or_non_null) {
+ (true, None) => {
+ for &row in rows {
+ if array.is_null(row) {
+ self.nulls.append(true);
+ self.group_values.push(T::default_value());
+ } else {
+ self.nulls.append(false);
+ self.group_values.push(arr.value(row));
+ }
+ }
+ }
+
+ (true, Some(true)) => {
+ self.nulls.append_n(rows.len(), false);
+ for &row in rows {
+ self.group_values.push(arr.value(row));
+ }
+ }
+
+ (true, Some(false)) => {
+ self.nulls.append_n(rows.len(), true);
+ self.group_values
+ .extend(iter::repeat(T::default_value()).take(rows.len()));
+ }
+
+ (false, _) => {
+ for &row in rows {
+ self.group_values.push(arr.value(row));
Review Comment:
I think we already could use extend instead of `push`? `extend` on `Vec` is somewhat faster than push as the capacity check / allocation is done once instead of once per value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1830911653
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +157,89 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
+
+ let iter = izip!(
+ lhs_rows.iter(),
+ rhs_rows.iter(),
+ equal_to_results.iter_mut(),
+ );
+
+ for (&lhs_row, &rhs_row, equal_to_result) in iter {
+ // Has found not equal to in previous column, don't need to check
+ if !*equal_to_result {
+ continue;
+ }
+
+ // Perf: skip null check (by short circuit) if input is not nullable
+ if NULLABLE {
+ let exist_null = self.nulls.is_null(lhs_row);
+ let input_null = array.is_null(rhs_row);
+ if let Some(result) = nulls_equal_to(exist_null, input_null) {
+ *equal_to_result = result;
+ continue;
+ }
+ // Otherwise, we need to check their values
+ }
+
+ *equal_to_result = self.group_values[lhs_row] == array.value(rhs_row);
+ }
+ }
+
+ fn vectorized_append(&mut self, array: &ArrayRef, rows: &[usize]) {
+ let arr = array.as_primitive::<T>();
+
+ let null_count = array.null_count();
+ let num_rows = array.len();
+ let all_null_or_non_null = if null_count == 0 {
+ Some(true)
+ } else if null_count == num_rows {
+ Some(false)
+ } else {
+ None
+ };
+
+ match (NULLABLE, all_null_or_non_null) {
+ (true, None) => {
+ for &row in rows {
+ if array.is_null(row) {
+ self.nulls.append(true);
+ self.group_values.push(T::default_value());
+ } else {
+ self.nulls.append(false);
+ self.group_values.push(arr.value(row));
+ }
+ }
+ }
+
+ (true, Some(true)) => {
+ self.nulls.append_n(rows.len(), false);
+ for &row in rows {
+ self.group_values.push(arr.value(row));
+ }
+ }
+
+ (true, Some(false)) => {
+ self.nulls.append_n(rows.len(), true);
+ self.group_values
+ .extend(iter::repeat(T::default_value()).take(rows.len()));
+ }
+
+ (false, _) => {
+ for &row in rows {
+ self.group_values.push(arr.value(row));
Review Comment:
> I think there are several things that could be done to make the append even faster:
>
> 1. `extend_from_slice` `if rows.len() == array.len()`
>
> 2. use `extend` rather than `push` for values
>
> 3. Speed up appending nulls (don't append bits one by one)
I filed an issue to tracking the potential improvements for vecotrized operations.
https://github.com/apache/datafusion/issues/13275
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2451212600
<img width="1220" alt="Screenshot 2024-11-01 at 11 10 46โฏAM" src="https://github.com/user-attachments/assets/b2b10d9e-8755-429e-b11d-35f6a1f459a8">
Parquet read dominates ๐ค
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823736082
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
+ // `group_values` before performing append.
+ //
+ // 3. Perform `vectorized_equal_to` for `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`. If found some rows in input `cols`
+ // not equal to `exist rows` in `group_values`, place them in `scalarized_indices`
+ // and perform `scalarized_intern` for them similar as what in [`GroupValuesColumn`]
+ // after.
+ //
+ // 4. Perform `scalarized_intern` for rows mentioned above, when we process like this
+ // can see the comments of `scalarized_intern`.
+ //
+
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`
+ self.collect_vectorized_process_context(&batch_hashes, groups);
+
+ // 2. Perform `vectorized_append`
+ self.vectorized_append(cols);
+
+ // 3. Perform `vectorized_equal_to`
+ self.vectorized_equal_to(cols, groups);
+
+ // 4. Perform `scalarized_intern`
+ self.scalarized_intern(cols, &batch_hashes, groups);
+
+ self.hashes_buffer = batch_hashes;
+
+ Ok(())
+ }
+
+ fn size(&self) -> usize {
+ let group_values_size: usize = self.group_values.iter().map(|v| v.size()).sum();
+ group_values_size + self.map_size + self.hashes_buffer.allocated_size()
+ }
+
+ fn is_empty(&self) -> bool {
+ self.len() == 0
+ }
+
+ fn len(&self) -> usize {
+ if self.group_values.is_empty() {
+ return 0;
+ }
+
+ self.group_values[0].len()
+ }
+
+ fn emit(&mut self, emit_to: EmitTo) -> Result<Vec<ArrayRef>> {
+ let mut output = match emit_to {
+ EmitTo::All => {
+ let group_values = mem::take(&mut self.group_values);
+ debug_assert!(self.group_values.is_empty());
+
+ group_values
+ .into_iter()
+ .map(|v| v.build())
+ .collect::<Vec<_>>()
+ }
+ EmitTo::First(n) => {
+ let output = self
Review Comment:
~I think first n is only called in `emit_early_if_necessary` in `GroupOrdering::None` case, but it seems there is no test coverage of this. If I add panic here, the existing test passes too.~
I saw the test below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823761465
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
Review Comment:
๐ฅ Yes, the alternative is to store the bucket as I tried in previous.
But we need to reserve to ensure the hashtable big enough at the beginning of `intern`.
Otherwise when reserve and rehash happend, the bucket(actually a pointer to hashtable slot) will not point to the expected `(hash, group_index_view)` anymore.
I think it actually a bit danger to do so, and give up the try in later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823746929
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -212,17 +330,92 @@ where
self.offsets.push(O::usize_as(offset));
} else {
self.nulls.append(false);
- let value: &[u8] = arr.value(row).as_ref();
- self.buffer.append_slice(value);
- self.offsets.push(O::usize_as(self.buffer.len()));
+ self.do_append_val_inner(arr, row);
}
}
- fn equal_to_inner<B>(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool
- where
+ fn vectorized_equal_to_inner<B>(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) where
B: ByteArrayType,
{
let array = array.as_bytes::<B>();
+
+ let iter = lhs_rows
+ .iter()
+ .zip(rhs_rows.iter())
+ .zip(equal_to_results.iter_mut());
Review Comment:
Does izip! help here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1818601803
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better. I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
>
> Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
Make sense to me, and support it in `arrow` seems really good.
I plan to sort out codes in this pr , and then try to improve the primitive case using `take` + `eq`.
I guess some case like `q32` in clickbench is possible to get benefit from it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819225765
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +144,84 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ group_indices: &[usize],
+ array: &ArrayRef,
+ rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
Review Comment:
Here we can check the input arrays to be non-null and optimize for that case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2439225271
The logic is a bit complex, I plan to finish and do benchmark for it today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1818601807
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better. I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
>
> Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
Make sense for me, and support it in `arrow` seems really good.
I plan to sort out codes in this pr firstly, and then try to improve the primitive case using `take` + `eq`.
I guess some case like `q32` in clickbench is possible to get benefit from it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2440105531
It is really to see that the vectorized approach is promising!
The rest work is to find why it get `q28` slower.
```
--------------------
Benchmark clickbench_1.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main โ vectorize-append-value โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 0.78ms โ 0.66ms โ +1.18x faster โ
โ QQuery 1 โ 68.16ms โ 67.25ms โ no change โ
โ QQuery 2 โ 161.83ms โ 161.80ms โ no change โ
โ QQuery 3 โ 186.20ms โ 179.59ms โ no change โ
โ QQuery 4 โ 1581.81ms โ 1601.15ms โ no change โ
โ QQuery 5 โ 1556.95ms โ 1504.38ms โ no change โ
โ QQuery 6 โ 59.36ms โ 58.69ms โ no change โ
โ QQuery 7 โ 79.29ms โ 75.72ms โ no change โ
โ QQuery 8 โ 1945.74ms โ 1730.29ms โ +1.12x faster โ
โ QQuery 9 โ 1958.91ms โ 1912.77ms โ no change โ
โ QQuery 10 โ 519.94ms โ 496.55ms โ no change โ
โ QQuery 11 โ 592.06ms โ 573.94ms โ no change โ
โ QQuery 12 โ 1834.06ms โ 1818.58ms โ no change โ
โ QQuery 13 โ 2963.09ms โ 2937.11ms โ no change โ
โ QQuery 14 โ 2053.23ms โ 1959.00ms โ no change โ
โ QQuery 15 โ 1902.87ms โ 1873.43ms โ no change โ
โ QQuery 16 โ 4042.73ms โ 3721.92ms โ +1.09x faster โ
โ QQuery 17 โ 3649.27ms โ 3413.73ms โ +1.07x faster โ
โ QQuery 18 โ 8286.60ms โ 7555.43ms โ +1.10x faster โ
โ QQuery 19 โ 148.06ms โ 143.84ms โ no change โ
โ QQuery 20 โ 3241.10ms โ 3225.91ms โ no change โ
โ QQuery 21 โ 3879.00ms โ 3893.55ms โ no change โ
โ QQuery 22 โ 9109.29ms โ 9306.23ms โ no change โ
โ QQuery 23 โ 23670.14ms โ 23590.65ms โ no change โ
โ QQuery 24 โ 1166.75ms โ 1126.80ms โ no change โ
โ QQuery 25 โ 998.82ms โ 1006.37ms โ no change โ
โ QQuery 26 โ 1320.64ms โ 1307.47ms โ no change โ
โ QQuery 27 โ 4772.51ms โ 4675.67ms โ no change โ
โ QQuery 28 โ 22374.44ms โ 24834.06ms โ 1.11x slower โ
โ QQuery 29 โ 902.94ms โ 896.11ms โ no change โ
โ QQuery 30 โ 1829.63ms โ 1739.30ms โ no change โ
โ QQuery 31 โ 2059.05ms โ 1951.76ms โ +1.05x faster โ
โ QQuery 32 โ 7460.76ms โ 7609.67ms โ no change โ
โ QQuery 33 โ 9834.20ms โ 9681.51ms โ no change โ
โ QQuery 34 โ 9759.46ms โ 9556.88ms โ no change โ
โ QQuery 35 โ 2786.28ms โ 2258.62ms โ +1.23x faster โ
โ QQuery 36 โ 257.69ms โ 250.87ms โ no change โ
โ QQuery 37 โ 155.01ms โ 151.16ms โ no change โ
โ QQuery 38 โ 154.50ms โ 147.32ms โ no change โ
โ QQuery 39 โ 635.06ms โ 596.76ms โ +1.06x faster โ
โ QQuery 40 โ 60.37ms โ 59.58ms โ no change โ
โ QQuery 41 โ 53.82ms โ 53.27ms โ no change โ
โ QQuery 42 โ 66.93ms โ 63.49ms โ +1.05x faster โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (main) โ 140139.34ms โ
โ Total Time (vectorize-append-value) โ 139768.84ms โ
โ Average Time (main) โ 3259.05ms โ
โ Average Time (vectorize-append-value) โ 3250.44ms โ
โ Queries Faster โ 9 โ
โ Queries Slower โ 1 โ
โ Queries with No Change โ 33 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโ
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2440105534
It is really to see that the vectorized approach is promising!
The rest work is to find why it get `q28` slower.
```
--------------------
Benchmark clickbench_1.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main โ vectorize-append-value โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 0.78ms โ 0.66ms โ +1.18x faster โ
โ QQuery 1 โ 68.16ms โ 67.25ms โ no change โ
โ QQuery 2 โ 161.83ms โ 161.80ms โ no change โ
โ QQuery 3 โ 186.20ms โ 179.59ms โ no change โ
โ QQuery 4 โ 1581.81ms โ 1601.15ms โ no change โ
โ QQuery 5 โ 1556.95ms โ 1504.38ms โ no change โ
โ QQuery 6 โ 59.36ms โ 58.69ms โ no change โ
โ QQuery 7 โ 79.29ms โ 75.72ms โ no change โ
โ QQuery 8 โ 1945.74ms โ 1730.29ms โ +1.12x faster โ
โ QQuery 9 โ 1958.91ms โ 1912.77ms โ no change โ
โ QQuery 10 โ 519.94ms โ 496.55ms โ no change โ
โ QQuery 11 โ 592.06ms โ 573.94ms โ no change โ
โ QQuery 12 โ 1834.06ms โ 1818.58ms โ no change โ
โ QQuery 13 โ 2963.09ms โ 2937.11ms โ no change โ
โ QQuery 14 โ 2053.23ms โ 1959.00ms โ no change โ
โ QQuery 15 โ 1902.87ms โ 1873.43ms โ no change โ
โ QQuery 16 โ 4042.73ms โ 3721.92ms โ +1.09x faster โ
โ QQuery 17 โ 3649.27ms โ 3413.73ms โ +1.07x faster โ
โ QQuery 18 โ 8286.60ms โ 7555.43ms โ +1.10x faster โ
โ QQuery 19 โ 148.06ms โ 143.84ms โ no change โ
โ QQuery 20 โ 3241.10ms โ 3225.91ms โ no change โ
โ QQuery 21 โ 3879.00ms โ 3893.55ms โ no change โ
โ QQuery 22 โ 9109.29ms โ 9306.23ms โ no change โ
โ QQuery 23 โ 23670.14ms โ 23590.65ms โ no change โ
โ QQuery 24 โ 1166.75ms โ 1126.80ms โ no change โ
โ QQuery 25 โ 998.82ms โ 1006.37ms โ no change โ
โ QQuery 26 โ 1320.64ms โ 1307.47ms โ no change โ
โ QQuery 27 โ 4772.51ms โ 4675.67ms โ no change โ
โ QQuery 28 โ 22374.44ms โ 24834.06ms โ 1.11x slower โ
โ QQuery 29 โ 902.94ms โ 896.11ms โ no change โ
โ QQuery 30 โ 1829.63ms โ 1739.30ms โ no change โ
โ QQuery 31 โ 2059.05ms โ 1951.76ms โ +1.05x faster โ
โ QQuery 32 โ 7460.76ms โ 7609.67ms โ no change โ
โ QQuery 33 โ 9834.20ms โ 9681.51ms โ no change โ
โ QQuery 34 โ 9759.46ms โ 9556.88ms โ no change โ
โ QQuery 35 โ 2786.28ms โ 2258.62ms โ +1.23x faster โ
โ QQuery 36 โ 257.69ms โ 250.87ms โ no change โ
โ QQuery 37 โ 155.01ms โ 151.16ms โ no change โ
โ QQuery 38 โ 154.50ms โ 147.32ms โ no change โ
โ QQuery 39 โ 635.06ms โ 596.76ms โ +1.06x faster โ
โ QQuery 40 โ 60.37ms โ 59.58ms โ no change โ
โ QQuery 41 โ 53.82ms โ 53.27ms โ no change โ
โ QQuery 42 โ 66.93ms โ 63.49ms โ +1.05x faster โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (main) โ 140139.34ms โ
โ Total Time (vectorize-append-value) โ 139768.84ms โ
โ Average Time (main) โ 3259.05ms โ
โ Average Time (vectorize-append-value) โ 3250.44ms โ
โ Queries Faster โ 9 โ
โ Queries Slower โ 1 โ
โ Queries with No Change โ 33 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโ
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819948314
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
Review Comment:
Since we need the `group_index_view` of a specific row, which we already have in `collect_vectorized_process_context`, we could store `group_index_view` in another Vec. By doing this, we avoid using an additional map lookup.
```rust
self.map.insert_accounted(
(target_hash, group_index_view),
|(hash, _)| *hash,
&mut self.map_size,
);
// Add row index to `vectorized_append_row_indices`
self.vectorized_append_row_indices.push(row);
// Something similar to this
group_view_indexes[row] = group_index_view
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1818601807
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better. I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
>
> Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
Make sense for me, and support it in `arrow` seems really good.
I plan to sort out codes in this pr , and then try to improve the primitive case using `take` + `eq`.
I guess some case like `q32` in clickbench is possible to get benefit from it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1820076357
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
+ let entry = map.get_mut(target_hash, |(exist_hash, group_index_view)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ // Only `rows` having the same hash value with `exist rows` but different value
+ // will be process in `scalarized_intern`.
+ // So related `buckets` in `map` is ensured to be `Some`.
+ let Some((_, group_index_view)) = entry else {
+ unreachable!()
+ };
+
+ // Perform scalarized equal to
+ if self.scalarized_equal_to(&group_index_view, cols, row, groups) {
+ // Found the row actually exists in group values,
+ // don't need to create new group for it.
+ continue;
+ }
+
+ // Insert the `row` to `group_values` before checking `next row`
+ let group_idx = self.group_values[0].len();
+ let mut checklen = 0;
+ for (i, group_value) in self.group_values.iter_mut().enumerate() {
+ group_value.append_val(&cols[i], row);
+ let len = group_value.len();
+ if i == 0 {
+ checklen = len;
+ } else {
+ debug_assert_eq!(checklen, len);
+ }
+ }
+
+ // Check if the `view` is `inlined` or `non-inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, get `group_index_list` from `group_index_lists`,
+ // then add the new `group` with the same hash values into it.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &mut self.group_index_lists[list_offset];
+ group_index_list.push(group_idx);
+ } else {
+ // Inlined case
+ let list_offset = self.group_index_lists.len();
+
+ // Create new `group_index_list` including
+ // `exist group index` + `new group index`.
+ // Add new `group_index_list` into ``group_index_lists`.
+ let exist_group_index = group_index_view.value() as usize;
+ let new_group_index_list = vec![exist_group_index, group_idx];
Review Comment:
The `map` is a bit different with origin.
The original `map` can be seen as `HashMap<(hash value, group value), ()>`, and each bucket only points to `single group`.
And current `map` is actullay like `HashMap<hash value, Vec<group value>>`, each bucket will point to a list of `groups with the same hash value(hash collision)`.
In this branch, the `new group with the same hash value` found, so we need to switch `inlined group view(the actual and single group index)` to `non-inlined group view(a index to gourp indices list)`.
And `exist_group_index` and `group_idx` are the two hash collision groups needed to be stored in the list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2451402632
Tpch q14 doesn't seem to run through the change of this PR -- `groupBy` is empty in `AggregateExec `, I also doesn't see any print out in `VectorizedGroupValuesColumn`. I think this change is not the reason of slowdown ๐ค
```
query TT
explain select
100.00 * sum(case
when p_type like 'PROMO%'
then l_extendedprice * (1 - l_discount)
else 0
end) / sum(l_extendedprice * (1 - l_discount)) as promo_revenue
from
lineitem,
part
where
l_partkey = p_partkey
and l_shipdate >= date '1995-09-01'
and l_shipdate < date '1995-10-01';
----
logical_plan
01)Projection: Float64(100) * CAST(sum(CASE WHEN part.p_type LIKE Utf8("PROMO%") THEN lineitem.l_extendedprice * Int64(1) - lineitem.l_discount ELSE Int64(0) END) AS Float64) / CAST(sum(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount) AS Float64) AS promo_revenue
02)--Aggregate: groupBy=[[]], aggr=[[sum(CASE WHEN part.p_type LIKE Utf8("PROMO%") THEN __common_expr_1 ELSE Decimal128(Some(0),38,4) END) AS sum(CASE WHEN part.p_type LIKE Utf8("PROMO%") THEN lineitem.l_extendedprice * Int64(1) - lineitem.l_discount ELSE Int64(0) END), sum(__common_expr_1) AS sum(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount)]]
03)----Projection: lineitem.l_extendedprice * (Decimal128(Some(1),20,0) - lineitem.l_discount) AS __common_expr_1, part.p_type
04)------Inner Join: lineitem.l_partkey = part.p_partkey
05)--------Projection: lineitem.l_partkey, lineitem.l_extendedprice, lineitem.l_discount
06)----------Filter: lineitem.l_shipdate >= Date32("1995-09-01") AND lineitem.l_shipdate < Date32("1995-10-01")
07)------------TableScan: lineitem projection=[l_partkey, l_extendedprice, l_discount, l_shipdate], partial_filters=[lineitem.l_shipdate >= Date32("1995-09-01"), lineitem.l_shipdate < Date32("1995-10-01")]
08)--------TableScan: part projection=[p_partkey, p_type]
physical_plan
01)ProjectionExec: expr=[100 * CAST(sum(CASE WHEN part.p_type LIKE Utf8("PROMO%") THEN lineitem.l_extendedprice * Int64(1) - lineitem.l_discount ELSE Int64(0) END)@0 AS Float64) / CAST(sum(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount)@1 AS Float64) as promo_revenue]
02)--AggregateExec: mode=Final, gby=[], aggr=[sum(CASE WHEN part.p_type LIKE Utf8("PROMO%") THEN lineitem.l_extendedprice * Int64(1) - lineitem.l_discount ELSE Int64(0) END), sum(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount)]
03)----CoalescePartitionsExec
04)------AggregateExec: mode=Partial, gby=[], aggr=[sum(CASE WHEN part.p_type LIKE Utf8("PROMO%") THEN lineitem.l_extendedprice * Int64(1) - lineitem.l_discount ELSE Int64(0) END), sum(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount)]
05)--------ProjectionExec: expr=[l_extendedprice@0 * (Some(1),20,0 - l_discount@1) as __common_expr_1, p_type@2 as p_type]
06)----------CoalesceBatchesExec: target_batch_size=8192
07)------------HashJoinExec: mode=Partitioned, join_type=Inner, on=[(l_partkey@0, p_partkey@0)], projection=[l_extendedprice@1, l_discount@2, p_type@4]
08)--------------CoalesceBatchesExec: target_batch_size=8192
09)----------------RepartitionExec: partitioning=Hash([l_partkey@0], 4), input_partitions=4
10)------------------CoalesceBatchesExec: target_batch_size=8192
11)--------------------FilterExec: l_shipdate@3 >= 1995-09-01 AND l_shipdate@3 < 1995-10-01, projection=[l_partkey@0, l_extendedprice@1, l_discount@2]
12)----------------------CsvExec: file_groups={4 groups: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:0..18561749], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:18561749..37123498], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:37123498..55685247], [WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/lineitem.tbl:55685247..74246996]]}, projection=[l_partkey, l_extendedprice, l_discount, l_shipdate], has_header=false
13)--------------CoalesceBatchesExec: target_batch_size=8192
14)----------------RepartitionExec: partitioning=Hash([p_partkey@0], 4), input_partitions=4
15)------------------RepartitionExec: partitioning=RoundRobinBatch(4), input_partitions=1
16)--------------------CsvExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/tpch/data/part.tbl]]}, projection=[p_partkey, p_type], has_header=false
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2452991852
> Is it possible to somehow unify GroupValuesColumn and VectorizedGroupValuesColumn ?
๐คI think It can unify simply, `VectorizedGroupValuesColumn::scalarized_intern` is similar as `GroupValuesColumn::intern`.
But its logic is a bit more complex, I am afraid performance regression of `streaming aggregation`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823736082
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
+ // `group_values` before performing append.
+ //
+ // 3. Perform `vectorized_equal_to` for `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`. If found some rows in input `cols`
+ // not equal to `exist rows` in `group_values`, place them in `scalarized_indices`
+ // and perform `scalarized_intern` for them similar as what in [`GroupValuesColumn`]
+ // after.
+ //
+ // 4. Perform `scalarized_intern` for rows mentioned above, when we process like this
+ // can see the comments of `scalarized_intern`.
+ //
+
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`
+ self.collect_vectorized_process_context(&batch_hashes, groups);
+
+ // 2. Perform `vectorized_append`
+ self.vectorized_append(cols);
+
+ // 3. Perform `vectorized_equal_to`
+ self.vectorized_equal_to(cols, groups);
+
+ // 4. Perform `scalarized_intern`
+ self.scalarized_intern(cols, &batch_hashes, groups);
+
+ self.hashes_buffer = batch_hashes;
+
+ Ok(())
+ }
+
+ fn size(&self) -> usize {
+ let group_values_size: usize = self.group_values.iter().map(|v| v.size()).sum();
+ group_values_size + self.map_size + self.hashes_buffer.allocated_size()
+ }
+
+ fn is_empty(&self) -> bool {
+ self.len() == 0
+ }
+
+ fn len(&self) -> usize {
+ if self.group_values.is_empty() {
+ return 0;
+ }
+
+ self.group_values[0].len()
+ }
+
+ fn emit(&mut self, emit_to: EmitTo) -> Result<Vec<ArrayRef>> {
+ let mut output = match emit_to {
+ EmitTo::All => {
+ let group_values = mem::take(&mut self.group_values);
+ debug_assert!(self.group_values.is_empty());
+
+ group_values
+ .into_iter()
+ .map(|v| v.build())
+ .collect::<Vec<_>>()
+ }
+ EmitTo::First(n) => {
+ let output = self
Review Comment:
I think first n is also called in `emit_early_if_necessary` in `GroupOrdering::None` case. but it seems there is no test coverage of this. If I add panic here, the existing test is passed too.
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
+ // `group_values` before performing append.
+ //
+ // 3. Perform `vectorized_equal_to` for `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`. If found some rows in input `cols`
+ // not equal to `exist rows` in `group_values`, place them in `scalarized_indices`
+ // and perform `scalarized_intern` for them similar as what in [`GroupValuesColumn`]
+ // after.
+ //
+ // 4. Perform `scalarized_intern` for rows mentioned above, when we process like this
+ // can see the comments of `scalarized_intern`.
+ //
+
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`
+ self.collect_vectorized_process_context(&batch_hashes, groups);
+
+ // 2. Perform `vectorized_append`
+ self.vectorized_append(cols);
+
+ // 3. Perform `vectorized_equal_to`
+ self.vectorized_equal_to(cols, groups);
+
+ // 4. Perform `scalarized_intern`
+ self.scalarized_intern(cols, &batch_hashes, groups);
+
+ self.hashes_buffer = batch_hashes;
+
+ Ok(())
+ }
+
+ fn size(&self) -> usize {
+ let group_values_size: usize = self.group_values.iter().map(|v| v.size()).sum();
+ group_values_size + self.map_size + self.hashes_buffer.allocated_size()
+ }
+
+ fn is_empty(&self) -> bool {
+ self.len() == 0
+ }
+
+ fn len(&self) -> usize {
+ if self.group_values.is_empty() {
+ return 0;
+ }
+
+ self.group_values[0].len()
+ }
+
+ fn emit(&mut self, emit_to: EmitTo) -> Result<Vec<ArrayRef>> {
+ let mut output = match emit_to {
+ EmitTo::All => {
+ let group_values = mem::take(&mut self.group_values);
+ debug_assert!(self.group_values.is_empty());
+
+ group_values
+ .into_iter()
+ .map(|v| v.build())
+ .collect::<Vec<_>>()
+ }
+ EmitTo::First(n) => {
+ let output = self
Review Comment:
I think first n is only called in `emit_early_if_necessary` in `GroupOrdering::None` case. but it seems there is no test coverage of this. If I add panic here, the existing test is passed too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823736082
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
+ // `group_values` before performing append.
+ //
+ // 3. Perform `vectorized_equal_to` for `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`. If found some rows in input `cols`
+ // not equal to `exist rows` in `group_values`, place them in `scalarized_indices`
+ // and perform `scalarized_intern` for them similar as what in [`GroupValuesColumn`]
+ // after.
+ //
+ // 4. Perform `scalarized_intern` for rows mentioned above, when we process like this
+ // can see the comments of `scalarized_intern`.
+ //
+
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`
+ self.collect_vectorized_process_context(&batch_hashes, groups);
+
+ // 2. Perform `vectorized_append`
+ self.vectorized_append(cols);
+
+ // 3. Perform `vectorized_equal_to`
+ self.vectorized_equal_to(cols, groups);
+
+ // 4. Perform `scalarized_intern`
+ self.scalarized_intern(cols, &batch_hashes, groups);
+
+ self.hashes_buffer = batch_hashes;
+
+ Ok(())
+ }
+
+ fn size(&self) -> usize {
+ let group_values_size: usize = self.group_values.iter().map(|v| v.size()).sum();
+ group_values_size + self.map_size + self.hashes_buffer.allocated_size()
+ }
+
+ fn is_empty(&self) -> bool {
+ self.len() == 0
+ }
+
+ fn len(&self) -> usize {
+ if self.group_values.is_empty() {
+ return 0;
+ }
+
+ self.group_values[0].len()
+ }
+
+ fn emit(&mut self, emit_to: EmitTo) -> Result<Vec<ArrayRef>> {
+ let mut output = match emit_to {
+ EmitTo::All => {
+ let group_values = mem::take(&mut self.group_values);
+ debug_assert!(self.group_values.is_empty());
+
+ group_values
+ .into_iter()
+ .map(|v| v.build())
+ .collect::<Vec<_>>()
+ }
+ EmitTo::First(n) => {
+ let output = self
Review Comment:
I think first n is only called in `emit_early_if_necessary` in `GroupOrdering::None` case, but it seems there is no test coverage of this. If I add panic here, the existing test passes too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826582961
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -35,29 +37,113 @@ use datafusion_common::{not_impl_err, DataFusionError, Result};
use datafusion_execution::memory_pool::proxy::{RawTableAllocExt, VecAllocExt};
use datafusion_expr::EmitTo;
use datafusion_physical_expr::binary_map::OutputType;
+
use hashbrown::raw::RawTable;
-use std::mem::size_of;
-/// A [`GroupValues`] that stores multiple columns of group values.
+const NON_INLINED_FLAG: u64 = 0x8000000000000000;
+const VALUE_MASK: u64 = 0x7FFFFFFFFFFFFFFF;
+
+/// The view of indices pointing to the actual values in `GroupValues`
///
+/// If only single `group index` represented by view,
+/// value of view is just the `group index`, and we call it a `inlined view`.
///
-pub struct GroupValuesColumn {
+/// If multiple `group indices` represented by view,
+/// value of view is the actually the index pointing to `group indices`,
+/// and we call it `non-inlined view`.
+///
+/// The view(a u64) format is like:
+/// +---------------------+---------------------------------------------+
+/// | inlined flag(1bit) | group index / index to group indices(63bit) |
+/// +---------------------+---------------------------------------------+
+///
+/// `inlined flag`: 1 represents `non-inlined`, and 0 represents `inlined`
+///
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+struct GroupIndexView(u64);
+
+impl GroupIndexView {
+ #[inline]
+ pub fn is_non_inlined(&self) -> bool {
+ (self.0 & NON_INLINED_FLAG) > 0
+ }
+
+ #[inline]
+ pub fn new_inlined(group_index: u64) -> Self {
+ Self(group_index)
+ }
+
+ #[inline]
+ pub fn new_non_inlined(list_offset: u64) -> Self {
+ let non_inlined_value = list_offset | NON_INLINED_FLAG;
+ Self(non_inlined_value)
+ }
+
+ #[inline]
+ pub fn value(&self) -> u64 {
+ self.0 & VALUE_MASK
+ }
+}
+
+/// A [`GroupValues`] that stores multiple columns of group values,
+/// and supports vectorized operators for them
+///
+pub struct VectorizedGroupValuesColumn {
/// The output schema
schema: SchemaRef,
/// Logically maps group values to a group_index in
/// [`Self::group_values`] and in each accumulator
///
- /// Uses the raw API of hashbrown to avoid actually storing the
- /// keys (group values) in the table
+ /// It is a `hashtable` based on `hashbrown`.
///
- /// keys: u64 hashes of the GroupValue
- /// values: (hash, group_index)
- map: RawTable<(u64, usize)>,
+ /// Key and value in the `hashtable`:
+ /// - The `key` is `hash value(u64)` of the `group value`
+ /// - The `value` is the `group values` with the same `hash value`
+ ///
+ /// We don't really store the actual `group values` in `hashtable`,
+ /// instead we store the `group indices` pointing to values in `GroupValues`.
+ /// And we use [`GroupIndexView`] to represent such `group indices` in table.
+ ///
+ ///
+ map: RawTable<(u64, GroupIndexView)>,
Review Comment:
Is it the case
1. If group1 and group2 have exactly the same hash value, `GroupIndexView` will use chaining to resolve the collision
2. If group1 and group2 have different hash values but map to the same slot in hash table, `hashbrown` will handle the collision for you with probing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2451667527
My results after merging main show no regressions ๐
Comparing main and vectorized_aggregate
--------------------
Benchmark clickbench_1.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main โ vectorized_aggregate โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 0.33ms โ 0.39ms โ 1.17x slower โ
โ QQuery 1 โ 35.24ms โ 35.03ms โ no change โ
โ QQuery 2 โ 82.54ms โ 83.07ms โ no change โ
โ QQuery 3 โ 76.90ms โ 65.62ms โ +1.17x faster โ
โ QQuery 4 โ 521.48ms โ 491.44ms โ +1.06x faster โ
โ QQuery 5 โ 809.76ms โ 768.98ms โ +1.05x faster โ
โ QQuery 6 โ 33.27ms โ 32.65ms โ no change โ
โ QQuery 7 โ 39.26ms โ 39.52ms โ no change โ
โ QQuery 8 โ 766.34ms โ 568.72ms โ +1.35x faster โ
โ QQuery 9 โ 821.86ms โ 823.16ms โ no change โ
โ QQuery 10 โ 217.90ms โ 207.96ms โ no change โ
โ QQuery 11 โ 247.04ms โ 230.55ms โ +1.07x faster โ
โ QQuery 12 โ 839.06ms โ 814.24ms โ no change โ
โ QQuery 13 โ 1123.51ms โ 1102.40ms โ no change โ
โ QQuery 14 โ 874.87ms โ 771.94ms โ +1.13x faster โ
โ QQuery 15 โ 607.72ms โ 615.06ms โ no change โ
โ QQuery 16 โ 1573.81ms โ 1332.64ms โ +1.18x faster โ
โ QQuery 17 โ 1454.94ms โ 1220.57ms โ +1.19x faster โ
โ QQuery 18 โ 3348.40ms โ 2936.64ms โ +1.14x faster โ
โ QQuery 19 โ 59.96ms โ 59.11ms โ no change โ
โ QQuery 20 โ 943.24ms โ 964.36ms โ no change โ
โ QQuery 21 โ 1291.59ms โ 1299.99ms โ no change โ
โ QQuery 22 โ 3798.86ms โ 3812.87ms โ no change โ
โ QQuery 23 โ 8083.11ms โ 7952.95ms โ no change โ
โ QQuery 24 โ 553.23ms โ 554.28ms โ no change โ
โ QQuery 25 โ 530.46ms โ 534.44ms โ no change โ
โ QQuery 26 โ 610.97ms โ 606.59ms โ no change โ
โ QQuery 27 โ 1558.28ms โ 1584.92ms โ no change โ
โ QQuery 28 โ 12837.43ms โ 12852.93ms โ no change โ
โ QQuery 29 โ 448.12ms โ 446.98ms โ no change โ
โ QQuery 30 โ 824.09ms โ 756.85ms โ +1.09x faster โ
โ QQuery 31 โ 796.62ms โ 752.91ms โ +1.06x faster โ
โ QQuery 32 โ 3502.80ms โ 2829.34ms โ +1.24x faster โ
โ QQuery 33 โ 2775.26ms โ 2729.67ms โ no change โ
โ QQuery 34 โ 2748.59ms โ 2738.52ms โ no change โ
โ QQuery 35 โ 1194.59ms โ 840.66ms โ +1.42x faster โ
โ QQuery 36 โ 150.59ms โ 146.20ms โ no change โ
โ QQuery 37 โ 102.54ms โ 104.18ms โ no change โ
โ QQuery 38 โ 105.60ms โ 104.55ms โ no change โ
โ QQuery 39 โ 279.20ms โ 264.85ms โ +1.05x faster โ
โ QQuery 40 โ 34.25ms โ 32.64ms โ no change โ
โ QQuery 41 โ 31.88ms โ 31.48ms โ no change โ
โ QQuery 42 โ 40.98ms โ 41.15ms โ no change โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (main) โ 56776.48ms โ
โ Total Time (vectorized_aggregate) โ 54183.02ms โ
โ Average Time (main) โ 1320.38ms โ
โ Average Time (vectorized_aggregate) โ 1260.07ms โ
โ Queries Faster โ 14 โ
โ Queries Slower โ 1 โ
โ Queries with No Change โ 28 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโ
--------------------
Benchmark tpch_sf1.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main โ vectorized_aggregate โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 1 โ 122.91ms โ 120.39ms โ no change โ
โ QQuery 2 โ 55.64ms โ 55.18ms โ no change โ
โ QQuery 3 โ 56.02ms โ 54.98ms โ no change โ
โ QQuery 4 โ 39.93ms โ 39.85ms โ no change โ
โ QQuery 5 โ 75.72ms โ 75.51ms โ no change โ
โ QQuery 6 โ 20.83ms โ 20.81ms โ no change โ
โ QQuery 7 โ 87.12ms โ 90.26ms โ no change โ
โ QQuery 8 โ 75.97ms โ 75.86ms โ no change โ
โ QQuery 9 โ 111.84ms โ 111.82ms โ no change โ
โ QQuery 10 โ 101.88ms โ 101.16ms โ no change โ
โ QQuery 11 โ 41.21ms โ 40.45ms โ no change โ
โ QQuery 12 โ 53.79ms โ 53.63ms โ no change โ
โ QQuery 13 โ 120.02ms โ 123.73ms โ no change โ
โ QQuery 14 โ 37.49ms โ 38.08ms โ no change โ
โ QQuery 15 โ 45.55ms โ 46.17ms โ no change โ
โ QQuery 16 โ 36.87ms โ 33.76ms โ +1.09x faster โ
โ QQuery 17 โ 99.41ms โ 100.12ms โ no change โ
โ QQuery 18 โ 146.85ms โ 149.60ms โ no change โ
โ QQuery 19 โ 65.43ms โ 65.90ms โ no change โ
โ QQuery 20 โ 61.97ms โ 58.57ms โ +1.06x faster โ
โ QQuery 21 โ 121.33ms โ 122.14ms โ no change โ
โ QQuery 22 โ 34.58ms โ 34.67ms โ no change โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2441185485
I profile to get flamegraph of `q28` and found it actually not go the path in this pr.
The slower may be due to some enviorment reason in my local, I am re-running the bench on `main` now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #12996:
URL: https://github.com/apache/datafusion/pull/12996
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2455797228
> This is top of my list to review tomorrow morning
I am sorry -- I am just finding other PRs like https://github.com/apache/datafusion/pull/12978 and https://github.com/apache/datafusion/pull/13133 very subtle and take a long time to review (aka write tests for / help make sure they are still correct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823151542
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +144,84 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ group_indices: &[usize],
+ array: &ArrayRef,
+ rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
Review Comment:
I plan to try it together with https://github.com/apache/datafusion/pull/12996#discussion_r1818601807 on follow up prs.
Seems code changes in this one are already a bit large...
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +144,84 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ group_indices: &[usize],
+ array: &ArrayRef,
+ rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
Review Comment:
I plan to try it together with https://github.com/apache/datafusion/pull/12996#discussion_r1818601807 on follow up prs.
Seems code changes in this one are already a bit large.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826554952
##########
datafusion/physical-plan/src/aggregates/group_values/mod.rs:
##########
@@ -143,8 +148,12 @@ pub fn new_group_values(schema: SchemaRef) -> Result<Box<dyn GroupValues>> {
}
}
- if GroupValuesColumn::supported_schema(schema.as_ref()) {
- Ok(Box::new(GroupValuesColumn::try_new(schema)?))
+ if column::supported_schema(schema.as_ref()) {
Review Comment:
> Can you explain here why GroupOrdering::None is required? Is it because the `VectorizedGroupValuesColumn` doesn't keep the groups in order?
>
> If that is the case, it seems like maybe `emit_n` would never be called ๐ค
The situation is just as @jayzhan211 mentioned, and the detail about why `GroupOrdering::None` needed can also see:
https://github.com/Rachelint/arrow-datafusion/blob/406acb4983efe0c2072c5d7759674eec9db9404a/datafusion/physical-plan/src/aggregates/group_values/column.rs#L792-L834
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823733539
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +154,350 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut [usize],
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self
+ .map
+ .get(target_hash, |(exist_hash, _)| target_hash == *exist_hash);
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
- ///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
+ }
+
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
+ ///
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut [usize]) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ self.scalarized_indices.clear();
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
Review Comment:
Alternative way is to store the offset of the certain row instead of pushing equivalent row in `vectorized_equal_to_row_indices`. The benefit is we can reduce memory in low cardinality case which has many equality check.
Current:
vectorized_equal_to_row_indices: [row1, row1, row1, row2, row2, row2, row2]
vectorized_equal_to_group_indices: [gid1,gid2,gid3 ...]
Offset approach:
vectorized_equal_to_row_indices: [row1, row2]
vectorized_equal_to_offset: [0, 3, 7]
vectorized_equal_to_group_indices: [gid1,gid2,gid3 ...]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823772169
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +154,350 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut [usize],
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self
+ .map
+ .get(target_hash, |(exist_hash, _)| target_hash == *exist_hash);
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
- ///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
+ }
+
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
+ ///
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut [usize]) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ self.scalarized_indices.clear();
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
Review Comment:
Good point!
I plan to continue optimizing the `vectorized_equal_to` together with https://github.com/apache/datafusion/pull/12996#discussion_r1818601807 and https://github.com/apache/datafusion/pull/12996#discussion_r1823151542
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826380362
##########
datafusion/physical-plan/src/aggregates/group_values/mod.rs:
##########
@@ -143,8 +148,12 @@ pub fn new_group_values(schema: SchemaRef) -> Result<Box<dyn GroupValues>> {
}
}
- if GroupValuesColumn::supported_schema(schema.as_ref()) {
- Ok(Box::new(GroupValuesColumn::try_new(schema)?))
+ if column::supported_schema(schema.as_ref()) {
Review Comment:
> Is it because the VectorizedGroupValuesColumn doesn't keep the groups in order?
Yes, because we now process all the rows at once (not one by one like before), some rows are appended beforehand so they are not kept in order.
> If that is the case, it seems like maybe emit_n would never be called
`emit_early_if_necessary` may be called
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1822530341
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
Review Comment:
The situations may be like:
```
// Input batch
row1 (found a empty bucket in `map`, the future group index will be inserted in `map` as a placeholder)
row1 (found the placeholder bucket, and need to equal to the future group index)
row1 (same as above)
...
row2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823790860
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +156,88 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
+
+ let iter = lhs_rows
+ .iter()
+ .zip(rhs_rows.iter())
+ .zip(equal_to_results.iter_mut());
+
+ for ((&lhs_row, &rhs_row), equal_to_result) in iter {
+ // Has found not equal to, don't need to check
Review Comment:
```suggestion
// Has found not equal to in previous column, don't need to check
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1825482475
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -356,3 +1048,461 @@ impl GroupValues for GroupValuesColumn {
self.hashes_buffer.shrink_to(count);
}
}
+
+/// Returns true if [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+/// supported for the specified schema
+pub fn supported_schema(schema: &Schema) -> bool {
+ schema
+ .fields()
+ .iter()
+ .map(|f| f.data_type())
+ .all(supported_type)
+}
+
+/// Returns true if the specified data type is supported by
+/// [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+///
+/// In order to be supported, there must be a specialized implementation of
+/// [`GroupColumn`] for the data type, instantiated in [`GroupValuesColumn::intern`]
+/// or [`VectorizedGroupValuesColumn::intern`]
+fn supported_type(data_type: &DataType) -> bool {
+ matches!(
+ *data_type,
+ DataType::Int8
+ | DataType::Int16
+ | DataType::Int32
+ | DataType::Int64
+ | DataType::UInt8
+ | DataType::UInt16
+ | DataType::UInt32
+ | DataType::UInt64
+ | DataType::Float32
+ | DataType::Float64
+ | DataType::Utf8
+ | DataType::LargeUtf8
+ | DataType::Binary
+ | DataType::LargeBinary
+ | DataType::Date32
+ | DataType::Date64
+ | DataType::Utf8View
+ | DataType::BinaryView
+ )
+}
+
+#[cfg(test)]
+mod tests {
+ use std::sync::Arc;
+
+ use arrow::{compute::concat_batches, util::pretty::pretty_format_batches};
+ use arrow_array::{ArrayRef, Int64Array, RecordBatch, StringArray, StringViewArray};
+ use arrow_schema::{DataType, Field, Schema, SchemaRef};
+ use datafusion_expr::EmitTo;
+
+ use crate::aggregates::group_values::{
+ column::VectorizedGroupValuesColumn, GroupValues,
+ };
+
+ #[test]
+ fn test_intern_for_vectorized_group_values() {
+ let data_set = VectorizedTestDataSet::new();
+ let mut group_values =
+ VectorizedGroupValuesColumn::try_new(data_set.schema()).unwrap();
+
+ data_set.load_to_group_values(&mut group_values);
+ let actual_batch = group_values.emit(EmitTo::All).unwrap();
+ let actual_batch = RecordBatch::try_new(data_set.schema(), actual_batch).unwrap();
+
+ check_result(&actual_batch, &data_set.expected_batch);
+ }
+
+ #[test]
+ fn test_emit_first_n_for_vectorized_group_values() {
Review Comment:
I add the test to cover all the cases in first n. Could check it out https://github.com/Rachelint/arrow-datafusion/pull/2/files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1807856591
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -160,11 +162,112 @@ macro_rules! instantiate_primitive {
};
}
-fn append_col_value<C>(mut core: C, array: &ArrayRef, row: usize)
-where
- C: FnMut(&ArrayRef, usize),
-{
- core(array, row);
+struct AggregationHashTable<T: AggregationHashTableEntry> {
+ /// Raw table storing values in a `Vec`
+ raw_table: Vec<T>,
Review Comment:
We can still do "vectorized compare" by doing the lookup in the hashtable (based on hash value only) and the vectorized equality check separately. That way you still can use the fast hashtable, but move the equality check to a separate/vectorized step.
That's at least what is done in the vectorized hash join implementation :). I changed it before to use a `Vec`-based index like you did here, but that performed significantly worse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1807856591
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -160,11 +162,112 @@ macro_rules! instantiate_primitive {
};
}
-fn append_col_value<C>(mut core: C, array: &ArrayRef, row: usize)
-where
- C: FnMut(&ArrayRef, usize),
-{
- core(array, row);
+struct AggregationHashTable<T: AggregationHashTableEntry> {
+ /// Raw table storing values in a `Vec`
+ raw_table: Vec<T>,
Review Comment:
We can still do "vectorized compare" by doing the lookup in the hashtable (based on hash value only) and the vectorized equality check separately. That way you still can use the fast hashtable, but move the equality check to a separate/vectorized step.
That's at least what is done in the vectorized join implementation :). I changed it before to use a `Vec`-based index like you did here, but that performed significantly worse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819191554
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> The same combination of `take` of some subset of rows is basically what the arrow filter kernel does. However curently it only accepts a single array as input --> single array as output.
>
> I wonder if we could implement some version of `filter` that that can be invoked multiple times or something each time adding to the in-progress array.
>
> That might be the low level building block that generalized `take` and `filter` and `interleave` (and we could use it as prt of `Repartition`) ๐ค
๐ค It seems somethinng like `TakenArrayBuilder`?
And we input multiple `Array` and `taken indices` into it, and we call `build` to get the final `Array`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819216243
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
Hmm... ๐ค I think `filter` is a subset of `take` in what it supports as you can repeat values / use them out of order in `take` but can't do that in `filter`.
Somehow generalizing them sounds reasonable, as `filter` even uses the same strategy whenever the predicate is sparse (converting to indices).
Appending to a in-progress array seems like it could be very useful to aggregates yeah, I think that will mostly remove the need for the `GroupColumn` code...
For the equality check however we would need to take in two arrays of indices and/or boolean predicate followed by a equality operation (ideally in one go without copying the input arrays).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819245889
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
I tried `interleave` as well before for `RepartitionExec`, but that has some more overhead, as it needs to index both into the arrays by index as the values in the arrays (so `take` + `concat` seemed to be faster as far as I remember).
> ๐ค It seems somethinng like TakenArrayBuilder?
> And we input multiple Array and taken indices pairs into it, and we call build to get the final Array?
I guess yeah that could work both for aggregate and `RepartitionExec`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1820061845
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -58,6 +63,17 @@ pub trait GroupColumn: Send + Sync {
fn equal_to(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool;
/// Appends the row at `row` in `array` to this builder
fn append_val(&mut self, array: &ArrayRef, row: usize);
+
+ fn vectorized_equal_to(
Review Comment:
It only switch the process logic from `row oriented` to `col oriented` now.
The original logic is like:
```
eq col1[0], col2[0], col3[0]
eq col1[1], col2[1], col3[1]
...
eq col1[n], col2[n], col3[n]
```
And current logic:
```
eq col1[0] ~ col1[n]
eq col2[0] ~ col2[n]
eq col3[0] ~ col3[[n]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1820062958
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -58,6 +63,17 @@ pub trait GroupColumn: Send + Sync {
fn equal_to(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool;
/// Appends the row at `row` in `array` to this builder
fn append_val(&mut self, array: &ArrayRef, row: usize);
+
+ fn vectorized_equal_to(
Review Comment:
It can get performance improvement due to eliminating the unnecessary downcasting for each row.
And maybe can reduce the random memory accesses.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2423766327
```
$ bash bench.sh compare main vectorize-append-value
Comparing main and vectorize-append-value
--------------------
Benchmark clickbench_1.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main โ vectorize-append-value โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 0.78ms โ 0.71ms โ +1.10x faster โ
โ QQuery 1 โ 68.16ms โ 67.61ms โ no change โ
โ QQuery 2 โ 161.83ms โ 164.41ms โ no change โ
โ QQuery 3 โ 186.20ms โ 179.03ms โ no change โ
โ QQuery 4 โ 1581.81ms โ 1584.04ms โ no change โ
โ QQuery 5 โ 1556.95ms โ 1531.27ms โ no change โ
โ QQuery 6 โ 59.36ms โ 60.04ms โ no change โ
โ QQuery 7 โ 79.29ms โ 75.62ms โ no change โ
โ QQuery 8 โ 1945.74ms โ 2101.13ms โ 1.08x slower โ
โ QQuery 9 โ 1958.91ms โ 1912.39ms โ no change โ
โ QQuery 10 โ 519.94ms โ 522.07ms โ no change โ
โ QQuery 11 โ 592.06ms โ 599.92ms โ no change โ
โ QQuery 12 โ 1834.06ms โ 1794.95ms โ no change โ
โ QQuery 13 โ 2963.09ms โ 2877.52ms โ no change โ
โ QQuery 14 โ 2053.23ms โ 2000.01ms โ no change โ
โ QQuery 15 โ 1902.87ms โ 1872.49ms โ no change โ
โ QQuery 16 โ 4042.73ms โ 4017.57ms โ no change โ
โ QQuery 17 โ 3649.27ms โ 3650.16ms โ no change โ
โ QQuery 18 โ 8286.60ms โ 7886.01ms โ no change โ
โ QQuery 19 โ 148.06ms โ 145.58ms โ no change โ
โ QQuery 20 โ 3241.10ms โ 3239.72ms โ no change โ
โ QQuery 21 โ 3879.00ms โ 3910.91ms โ no change โ
โ QQuery 22 โ 9109.29ms โ 9086.56ms โ no change โ
โ QQuery 23 โ 23670.14ms โ 23864.02ms โ no change โ
โ QQuery 24 โ 1166.75ms โ 1126.00ms โ no change โ
โ QQuery 25 โ 998.82ms โ 998.04ms โ no change โ
โ QQuery 26 โ 1320.64ms โ 1297.67ms โ no change โ
โ QQuery 27 โ 4772.51ms โ 4704.92ms โ no change โ
โ QQuery 28 โ 22374.44ms โ 23790.97ms โ 1.06x slower โ
โ QQuery 29 โ 902.94ms โ 895.34ms โ no change โ
โ QQuery 30 โ 1829.63ms โ 1779.65ms โ no change โ
โ QQuery 31 โ 2059.05ms โ 1950.41ms โ +1.06x faster โ
โ QQuery 32 โ 7460.76ms โ 7516.17ms โ no change โ
โ QQuery 33 โ 9834.20ms โ 9577.03ms โ no change โ
โ QQuery 34 โ 9759.46ms โ 9734.09ms โ no change โ
โ QQuery 35 โ 2786.28ms โ 2869.86ms โ no change โ
โ QQuery 36 โ 257.69ms โ 257.19ms โ no change โ
โ QQuery 37 โ 155.01ms โ 154.12ms โ no change โ
โ QQuery 38 โ 154.50ms โ 155.42ms โ no change โ
โ QQuery 39 โ 635.06ms โ 616.96ms โ no change โ
โ QQuery 40 โ 60.37ms โ 60.21ms โ no change โ
โ QQuery 41 โ 53.82ms โ 53.67ms โ no change โ
โ QQuery 42 โ 66.93ms โ 65.83ms โ no change โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (main) โ 140139.34ms โ
โ Total Time (vectorize-append-value) โ 140747.28ms โ
โ Average Time (main) โ 3259.05ms โ
โ Average Time (vectorize-append-value) โ 3273.19ms โ
โ Queries Faster โ 2 โ
โ Queries Slower โ 2 โ
โ Queries with No Change โ 39 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโ
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1806255315
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
Review Comment:
(i.e. remove it here and call it in such a way we use https://docs.rs/arrow/latest/arrow/array/struct.BooleanBufferBuilder.html#method.append_n
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1818406542
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
๐ค Yes, I agree with the row by row checking is indeed not efficient enough, and switching the similar implementation in `hash_join` may be really worth trying.
Maybe better to try it in the follow on pr? Following points are still not clear for me, and I want to experiment about them:
- If we need a reusable buffer to hold the taken values?
- It seems that the `cmp` for some arrays like `StringArray` and `StringViewArray` is expansive?
Is it better to just check row by row for skipping some unnecessary checkings (if `row` not equal in `col a`, actually we don't need to check it again in `col b`)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2633686313
Update here is that this is looking like it results in some sweet clickbench improvements:
- https://github.com/apache/datafusion/issues/13983#issuecomment-2632053433
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2457985415
Performance results:
```
--------------------
Benchmark clickbench_partitioned.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main_base โ vectorize-append-value โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 2.30ms โ 2.33ms โ no change โ
โ QQuery 1 โ 40.32ms โ 40.38ms โ no change โ
โ QQuery 2 โ 96.98ms โ 97.06ms โ no change โ
โ QQuery 3 โ 106.77ms โ 108.36ms โ no change โ
โ QQuery 4 โ 912.91ms โ 923.46ms โ no change โ
โ QQuery 5 โ 957.04ms โ 943.27ms โ no change โ
โ QQuery 6 โ 36.29ms โ 35.65ms โ no change โ
โ QQuery 7 โ 44.35ms โ 44.05ms โ no change โ
โ QQuery 8 โ 1374.70ms โ 1026.74ms โ +1.34x faster โ
โ QQuery 9 โ 1349.37ms โ 1354.92ms โ no change โ
โ QQuery 10 โ 308.40ms โ 287.66ms โ +1.07x faster โ
โ QQuery 11 โ 358.62ms โ 321.64ms โ +1.11x faster โ
โ QQuery 12 โ 1003.91ms โ 981.15ms โ no change โ
โ QQuery 13 โ 1542.79ms โ 1470.92ms โ no change โ
โ QQuery 14 โ 1076.66ms โ 913.23ms โ +1.18x faster โ
โ QQuery 15 โ 1080.68ms โ 1107.04ms โ no change โ
โ QQuery 16 โ 2434.58ms โ 1986.02ms โ +1.23x faster โ
โ QQuery 17 โ 2243.82ms โ 1854.36ms โ +1.21x faster โ
โ QQuery 18 โ 5145.29ms โ 4294.07ms โ +1.20x faster โ
โ QQuery 19 โ 98.01ms โ 100.58ms โ no change โ
โ QQuery 20 โ 1259.01ms โ 1273.34ms โ no change โ
โ QQuery 21 โ 1524.57ms โ 1495.15ms โ no change โ
โ QQuery 22 โ 2711.65ms โ 2661.01ms โ no change โ
โ QQuery 23 โ 8991.12ms โ 8565.66ms โ no change โ
โ QQuery 24 โ 521.71ms โ 515.62ms โ no change โ
โ QQuery 25 โ 434.70ms โ 423.71ms โ no change โ
โ QQuery 26 โ 594.60ms โ 584.15ms โ no change โ
โ QQuery 27 โ 1884.39ms โ 1857.91ms โ no change โ
โ QQuery 28 โ 12978.56ms โ 13103.89ms โ no change โ
โ QQuery 29 โ 530.69ms โ 538.63ms โ no change โ
โ QQuery 30 โ 1023.13ms โ 897.27ms โ +1.14x faster โ
โ QQuery 31 โ 1044.11ms โ 956.21ms โ +1.09x faster โ
โ QQuery 32 โ 4300.17ms โ 4064.21ms โ +1.06x faster โ
โ QQuery 33 โ 4063.10ms โ 4043.20ms โ no change โ
โ QQuery 34 โ 4084.94ms โ 4073.62ms โ no change โ
โ QQuery 35 โ 1926.27ms โ 1355.58ms โ +1.42x faster โ
โ QQuery 36 โ 239.44ms โ 231.08ms โ no change โ
โ QQuery 37 โ 96.47ms โ 97.42ms โ no change โ
โ QQuery 38 โ 140.95ms โ 142.52ms โ no change โ
โ QQuery 39 โ 513.28ms โ 443.86ms โ +1.16x faster โ
โ QQuery 40 โ 57.47ms โ 55.78ms โ no change โ
โ QQuery 41 โ 48.74ms โ 50.94ms โ no change โ
โ QQuery 42 โ 62.29ms โ 63.98ms โ no change โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (main_base) โ 69245.17ms โ
โ Total Time (vectorize-append-value) โ 65387.65ms โ
โ Average Time (main_base) โ 1610.35ms โ
โ Average Time (vectorize-append-value) โ 1520.64ms โ
โ Queries Faster โ 12 โ
โ Queries Slower โ 0 โ
โ Queries with No Change โ 31 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโ
--------------------
Benchmark clickbench_extended.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main_base โ vectorize-append-value โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 0 โ 2783.88ms โ 2821.00ms โ no change โ
โ QQuery 1 โ 690.58ms โ 679.33ms โ no change โ
โ QQuery 2 โ 1435.85ms โ 1364.87ms โ no change โ
โ QQuery 3 โ 781.53ms โ 708.00ms โ +1.10x faster โ
โ QQuery 4 โ 12395.12ms โ 12441.79ms โ no change โ
โ QQuery 5 โ 19443.67ms โ 19077.87ms โ no change โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโ
โ Benchmark Summary โ โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ Total Time (main_base) โ 37530.62ms โ
โ Total Time (vectorize-append-value) โ 37092.86ms โ
โ Average Time (main_base) โ 6255.10ms โ
โ Average Time (vectorize-append-value) โ 6182.14ms โ
โ Queries Faster โ 1 โ
โ Queries Slower โ 0 โ
โ Queries with No Change โ 5 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโ
--------------------
Benchmark tpch_sf1.json
--------------------
โโโโโโโโโโโโโโโโณโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโ
โ Query โ main_base โ vectorize-append-value โ Change โ
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
โ QQuery 1 โ 227.03ms โ 223.21ms โ no change โ
โ QQuery 2 โ 117.37ms โ 118.32ms โ no change โ
โ QQuery 3 โ 131.23ms โ 112.98ms โ +1.16x faster โ
โ QQuery 4 โ 80.11ms โ 82.46ms โ no change โ
โ QQuery 5 โ 161.69ms โ 157.51ms โ no change โ
โ QQuery 6 โ 43.57ms โ 43.28ms โ no change โ
โ QQuery 7 โ 208.38ms โ 195.13ms โ +1.07x faster โ
โ QQuery 8 โ 168.85ms โ 163.41ms โ no change โ
โ QQuery 9 โ 246.40ms โ 245.05ms โ no change โ
โ QQuery 10 โ 203.72ms โ 205.29ms โ no change โ
โ QQuery 11 โ 94.72ms โ 92.35ms โ no change โ
โ QQuery 12 โ 100.17ms โ 115.46ms โ 1.15x slower โ
โ QQuery 13 โ 212.30ms โ 208.76ms โ no change โ
โ QQuery 14 โ 83.73ms โ 70.51ms โ +1.19x faster โ
โ QQuery 15 โ 104.72ms โ 112.26ms โ 1.07x slower โ
โ QQuery 16 โ 72.68ms โ 69.40ms โ no change โ
โ QQuery 17 โ 202.88ms โ 208.88ms โ no change โ
โ QQuery 18 โ 309.89ms โ 322.50ms โ no change โ
โ QQuery 19 โ 121.13ms โ 118.12ms โ no change โ
โ QQuery 20 โ 139.03ms โ 122.02ms โ +1.14x faster โ
โ QQuery 21 โ 260.01ms โ 253.21ms โ no change โ
โ QQuery 22 โ 67.71ms โ 67.18ms โ no change โ
โโโโโโโโโโโโโโโโดโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโ
```
๐
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2460194107
I don't think we need to wait on this PR anymore, let's merge it in and keep moving forward. Thank you everyone again!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1820104867
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
Review Comment:
It seems that `vectorized_equal_to_row_indices` doesn't push the same row twice. If so, this branch should always be true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2450998196
> Worth profiling / checking this regression before merging I think
For my own future self, it is this tpch query:
> โ QQuery 14 โ 72.15ms โ 89.80ms โ 1.24x slower โ
I think just profiling what is going on would be helpful. I can maybe help over the next few days . Here is an example of the kind of ticket we could potentially file: https://github.com/apache/datafusion/issues/13188
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2451408576
Hm maybe some improvement landed on `main` not yet on this branch? Maybe rebase and profile again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2454980334
> > ๐ค I personally prefer the second one? What do you think about it @alamb ?
>
> I think this makes sense -- thank you
Have unified the `VectorizedGroupValuesColumn` and `GroupValuesColumn` through the way mentioned in https://github.com/apache/datafusion/pull/12996#issuecomment-2452991852
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1822523760
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
Review Comment:
I guess maybe we need to perform `vectorized_append` before `vectorized_equal_to`.
The process is like what in `duckdb` https://github.com/duckdb/duckdb/blob/4ba2e66277a7576f58318c1aac112faa67c47b11/src/execution/aggregate_hashtable.cpp#L379-L394
When a `new group` directly found in `hashtable` (`get` return None), the `future group index still not point to a group` will be inserted into `hashtable`.
And if next some `rows with the same hash value` in the same `input arrays` exist, such `future group index` will be picked into `vectorized_equal_to_group_indices`.
So will need to perform `vectorized_append` firstly to fill such group, and after that the `future group index` can really point to a actual `group` in `group values`.
And the `vectorized_equal_to` can be performed successfully rather than panic when encounter a `group index` pointing to no group.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1807293142
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
Review Comment:
I add the `append_batch` function to support vectorized append more better.
But the improvement seems still not obvious.
๐ค I guess, it is likely due the new introduced branch of `equal_to`:
```
if *group_idx < group_values_len {
for (i, group_val) in self.group_values.iter().enumerate() {
if !check_row_equal(group_val.as_ref(), *group_idx, &cols[i], row)
{
return false;
}
}
} else {
let row_idx_offset = group_idx - group_values_len;
let row_idx = self.append_rows_buffer[row_idx_offset];
return is_rows_eq(cols, row, cols, row_idx).unwrap();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819224348
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +144,84 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ group_indices: &[usize],
+ array: &ArrayRef,
+ rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
+
+ for (idx, &lhs_row) in group_indices.iter().enumerate() {
Review Comment:
can use `group_indices.iter().zip(rows.iter())` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819112999
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
The same combination of `take` of some subset of rows is basically what the arrow filter kernel does. However curently it only accepts a single array as input --> single array as output.
I wonder if we could implement some version of `filter` that that can be invoked multiple times or something each time adding to the in-progress array.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819948314
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
Review Comment:
Since we need the `group_index_view` of a specific row, which we already have in `collect_vectorized_process_context`, we could store `group_index_view` in another Vec. By doing this, we eliminate the need of additional map lookup.
```rust
self.map.insert_accounted(
(target_hash, group_index_view),
|(hash, _)| *hash,
&mut self.map_size,
);
// Add row index to `vectorized_append_row_indices`
self.vectorized_append_row_indices.push(row);
// Something similar to this
group_view_indexes[row] = group_index_view
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819948314
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
Review Comment:
Since we need the `group_index_view` of a specific row, which we already have in `collect_vectorized_process_context`, we could store `group_index_view` in another Vec? By doing this, we avoid using an additional map lookup.
```rust
self.map.insert_accounted(
(target_hash, group_index_view),
|(hash, _)| *hash,
&mut self.map_size,
);
// Add row index to `vectorized_append_row_indices`
self.vectorized_append_row_indices.push(row);
// Something similar to this
group_view_indexes[row] = group_index_view
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1820076357
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
+ let entry = map.get_mut(target_hash, |(exist_hash, group_index_view)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ // Only `rows` having the same hash value with `exist rows` but different value
+ // will be process in `scalarized_intern`.
+ // So related `buckets` in `map` is ensured to be `Some`.
+ let Some((_, group_index_view)) = entry else {
+ unreachable!()
+ };
+
+ // Perform scalarized equal to
+ if self.scalarized_equal_to(&group_index_view, cols, row, groups) {
+ // Found the row actually exists in group values,
+ // don't need to create new group for it.
+ continue;
+ }
+
+ // Insert the `row` to `group_values` before checking `next row`
+ let group_idx = self.group_values[0].len();
+ let mut checklen = 0;
+ for (i, group_value) in self.group_values.iter_mut().enumerate() {
+ group_value.append_val(&cols[i], row);
+ let len = group_value.len();
+ if i == 0 {
+ checklen = len;
+ } else {
+ debug_assert_eq!(checklen, len);
+ }
+ }
+
+ // Check if the `view` is `inlined` or `non-inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, get `group_index_list` from `group_index_lists`,
+ // then add the new `group` with the same hash values into it.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &mut self.group_index_lists[list_offset];
+ group_index_list.push(group_idx);
+ } else {
+ // Inlined case
+ let list_offset = self.group_index_lists.len();
+
+ // Create new `group_index_list` including
+ // `exist group index` + `new group index`.
+ // Add new `group_index_list` into ``group_index_lists`.
+ let exist_group_index = group_index_view.value() as usize;
+ let new_group_index_list = vec![exist_group_index, group_idx];
Review Comment:
The `map` is a bit different with origin.
The original `map` can be seen as `HashMap<(hash value, group value), ()>`, and each bucket only points to `single group`.
And cureent `map` is actullay like `HashMap<hash value, Vec<group value>>`, each bucket will point to a list of `groups with the same hash value(hash collision)`.
In this branch, the `new group with the same hash value` found, so we need to switch `inlined group view(the actual and single group index)` to `non-inlined group view(a index to gourp indices list)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823718252
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
Review Comment:
I'm unclear on the meaning of the placeholders here, but I understand why we need to append values first now. Groups that require an equality check may need to compare against the newly added groups.
In `collect_vectorized_process_context`, new groups only update the hash in the hash table without appending values yet. After checking the hash, some rows are added to groups requiring an equality check. Therefore, we need to append the new groups first so that equality checks can include the newly appended groups
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1824335722
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
+ // `group_values` before performing append.
+ //
+ // 3. Perform `vectorized_equal_to` for `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`. If found some rows in input `cols`
+ // not equal to `exist rows` in `group_values`, place them in `scalarized_indices`
+ // and perform `scalarized_intern` for them similar as what in [`GroupValuesColumn`]
+ // after.
+ //
+ // 4. Perform `scalarized_intern` for rows mentioned above, when we process like this
+ // can see the comments of `scalarized_intern`.
+ //
+
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`
+ self.collect_vectorized_process_context(&batch_hashes, groups);
+
+ // 2. Perform `vectorized_append`
+ self.vectorized_append(cols);
+
+ // 3. Perform `vectorized_equal_to`
+ self.vectorized_equal_to(cols, groups);
+
+ // 4. Perform `scalarized_intern`
+ self.scalarized_intern(cols, &batch_hashes, groups);
+
+ self.hashes_buffer = batch_hashes;
+
+ Ok(())
+ }
+
+ fn size(&self) -> usize {
+ let group_values_size: usize = self.group_values.iter().map(|v| v.size()).sum();
+ group_values_size + self.map_size + self.hashes_buffer.allocated_size()
+ }
+
+ fn is_empty(&self) -> bool {
+ self.len() == 0
+ }
+
+ fn len(&self) -> usize {
+ if self.group_values.is_empty() {
+ return 0;
+ }
+
+ self.group_values[0].len()
+ }
+
+ fn emit(&mut self, emit_to: EmitTo) -> Result<Vec<ArrayRef>> {
+ let mut output = match emit_to {
+ EmitTo::All => {
+ let group_values = mem::take(&mut self.group_values);
+ debug_assert!(self.group_values.is_empty());
+
+ group_values
+ .into_iter()
+ .map(|v| v.build())
+ .collect::<Vec<_>>()
+ }
+ EmitTo::First(n) => {
+ let output = self
+ .group_values
+ .iter_mut()
+ .map(|v| v.take_n(n))
+ .collect::<Vec<_>>();
+ let new_group_index_lists =
+ Vec::with_capacity(self.group_index_lists.len());
+ let old_group_index_lists =
+ mem::replace(&mut self.group_index_lists, new_group_index_lists);
+
+ // SAFETY: self.map outlives iterator and is not modified concurrently
+ unsafe {
+ for bucket in self.map.iter() {
+ // Check if it is `inlined` or `non-inlined`
+ if bucket.as_ref().1.is_non_inlined() {
+ // Non-inlined case
+ // We take `group_index_list` from `old_group_index_lists`
+ let list_offset = bucket.as_ref().1.value() as usize;
+ let old_group_index_list =
+ &old_group_index_lists[list_offset];
+
+ let mut new_group_index_list = Vec::new();
+ for &group_index in old_group_index_list {
+ if let Some(remaining) = group_index.checked_sub(n) {
+ new_group_index_list.push(remaining);
+ }
+ }
+
+ // The possible results:
+ // - `new_group_index_list` is empty, we should erase this bucket
+ // - only one value in `new_group_index_list`, switch the `view` to `inlined`
+ // - still multiple values in `new_group_index_list`, build and set the new `unlined view`
+ if new_group_index_list.is_empty() {
+ self.map.erase(bucket);
+ } else if new_group_index_list.len() == 1 {
+ let group_index = new_group_index_list.first().unwrap();
+ bucket.as_mut().1 =
+ GroupIndexView::new_inlined(*group_index as u64);
+ } else {
+ let new_list_offset = self.group_index_lists.len();
+ self.group_index_lists.push(new_group_index_list);
Review Comment:
I have another idea like https://github.com/Rachelint/arrow-datafusion/pull/2/files, but given the test doesn't seem to cover the code here (I didn't see the list offset print out), so we need to strengthen test to make sure the change makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1824623183
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -356,3 +1048,461 @@ impl GroupValues for GroupValuesColumn {
self.hashes_buffer.shrink_to(count);
}
}
+
+/// Returns true if [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+/// supported for the specified schema
+pub fn supported_schema(schema: &Schema) -> bool {
+ schema
+ .fields()
+ .iter()
+ .map(|f| f.data_type())
+ .all(supported_type)
+}
+
+/// Returns true if the specified data type is supported by
+/// [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+///
+/// In order to be supported, there must be a specialized implementation of
+/// [`GroupColumn`] for the data type, instantiated in [`GroupValuesColumn::intern`]
+/// or [`VectorizedGroupValuesColumn::intern`]
+fn supported_type(data_type: &DataType) -> bool {
+ matches!(
+ *data_type,
+ DataType::Int8
+ | DataType::Int16
+ | DataType::Int32
+ | DataType::Int64
+ | DataType::UInt8
+ | DataType::UInt16
+ | DataType::UInt32
+ | DataType::UInt64
+ | DataType::Float32
+ | DataType::Float64
+ | DataType::Utf8
+ | DataType::LargeUtf8
+ | DataType::Binary
+ | DataType::LargeBinary
+ | DataType::Date32
+ | DataType::Date64
+ | DataType::Utf8View
+ | DataType::BinaryView
+ )
+}
+
+#[cfg(test)]
+mod tests {
+ use std::sync::Arc;
+
+ use arrow::{compute::concat_batches, util::pretty::pretty_format_batches};
+ use arrow_array::{ArrayRef, Int64Array, RecordBatch, StringArray, StringViewArray};
+ use arrow_schema::{DataType, Field, Schema, SchemaRef};
+ use datafusion_expr::EmitTo;
+
+ use crate::aggregates::group_values::{
+ column::VectorizedGroupValuesColumn, GroupValues,
+ };
+
+ #[test]
+ fn test_intern_for_vectorized_group_values() {
+ let data_set = VectorizedTestDataSet::new();
+ let mut group_values =
+ VectorizedGroupValuesColumn::try_new(data_set.schema()).unwrap();
+
+ data_set.load_to_group_values(&mut group_values);
+ let actual_batch = group_values.emit(EmitTo::All).unwrap();
+ let actual_batch = RecordBatch::try_new(data_set.schema(), actual_batch).unwrap();
+
+ check_result(&actual_batch, &data_set.expected_batch);
+ }
+
+ #[test]
+ fn test_emit_first_n_for_vectorized_group_values() {
Review Comment:
๐ค But the problem is that, it just make all gourp's hash value to 0, leading to only 1 entry will exist in the hashtable and the group_index_lists.
Maybe we need a specific test for the complex index decreasing here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826129052
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -356,3 +1048,461 @@ impl GroupValues for GroupValuesColumn {
self.hashes_buffer.shrink_to(count);
}
}
+
+/// Returns true if [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+/// supported for the specified schema
+pub fn supported_schema(schema: &Schema) -> bool {
+ schema
+ .fields()
+ .iter()
+ .map(|f| f.data_type())
+ .all(supported_type)
+}
+
+/// Returns true if the specified data type is supported by
+/// [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+///
+/// In order to be supported, there must be a specialized implementation of
+/// [`GroupColumn`] for the data type, instantiated in [`GroupValuesColumn::intern`]
+/// or [`VectorizedGroupValuesColumn::intern`]
+fn supported_type(data_type: &DataType) -> bool {
+ matches!(
+ *data_type,
+ DataType::Int8
+ | DataType::Int16
+ | DataType::Int32
+ | DataType::Int64
+ | DataType::UInt8
+ | DataType::UInt16
+ | DataType::UInt32
+ | DataType::UInt64
+ | DataType::Float32
+ | DataType::Float64
+ | DataType::Utf8
+ | DataType::LargeUtf8
+ | DataType::Binary
+ | DataType::LargeBinary
+ | DataType::Date32
+ | DataType::Date64
+ | DataType::Utf8View
+ | DataType::BinaryView
+ )
+}
+
+#[cfg(test)]
+mod tests {
+ use std::sync::Arc;
+
+ use arrow::{compute::concat_batches, util::pretty::pretty_format_batches};
+ use arrow_array::{ArrayRef, Int64Array, RecordBatch, StringArray, StringViewArray};
+ use arrow_schema::{DataType, Field, Schema, SchemaRef};
+ use datafusion_expr::EmitTo;
+
+ use crate::aggregates::group_values::{
+ column::VectorizedGroupValuesColumn, GroupValues,
+ };
+
+ #[test]
+ fn test_intern_for_vectorized_group_values() {
+ let data_set = VectorizedTestDataSet::new();
+ let mut group_values =
+ VectorizedGroupValuesColumn::try_new(data_set.schema()).unwrap();
+
+ data_set.load_to_group_values(&mut group_values);
+ let actual_batch = group_values.emit(EmitTo::All).unwrap();
+ let actual_batch = RecordBatch::try_new(data_set.schema(), actual_batch).unwrap();
+
+ check_result(&actual_batch, &data_set.expected_batch);
+ }
+
+ #[test]
+ fn test_emit_first_n_for_vectorized_group_values() {
Review Comment:
Really thanks, I have merged https://github.com/Rachelint/arrow-datafusion/pull/2 , and tried to continue improving such tests based on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1825486942
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -356,3 +1048,461 @@ impl GroupValues for GroupValuesColumn {
self.hashes_buffer.shrink_to(count);
}
}
+
+/// Returns true if [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+/// supported for the specified schema
+pub fn supported_schema(schema: &Schema) -> bool {
+ schema
+ .fields()
+ .iter()
+ .map(|f| f.data_type())
+ .all(supported_type)
+}
+
+/// Returns true if the specified data type is supported by
+/// [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+///
+/// In order to be supported, there must be a specialized implementation of
+/// [`GroupColumn`] for the data type, instantiated in [`GroupValuesColumn::intern`]
+/// or [`VectorizedGroupValuesColumn::intern`]
+fn supported_type(data_type: &DataType) -> bool {
+ matches!(
+ *data_type,
+ DataType::Int8
+ | DataType::Int16
+ | DataType::Int32
+ | DataType::Int64
+ | DataType::UInt8
+ | DataType::UInt16
+ | DataType::UInt32
+ | DataType::UInt64
+ | DataType::Float32
+ | DataType::Float64
+ | DataType::Utf8
+ | DataType::LargeUtf8
+ | DataType::Binary
+ | DataType::LargeBinary
+ | DataType::Date32
+ | DataType::Date64
+ | DataType::Utf8View
+ | DataType::BinaryView
+ )
+}
+
+#[cfg(test)]
+mod tests {
+ use std::sync::Arc;
+
+ use arrow::{compute::concat_batches, util::pretty::pretty_format_batches};
+ use arrow_array::{ArrayRef, Int64Array, RecordBatch, StringArray, StringViewArray};
+ use arrow_schema::{DataType, Field, Schema, SchemaRef};
+ use datafusion_expr::EmitTo;
+
+ use crate::aggregates::group_values::{
+ column::VectorizedGroupValuesColumn, GroupValues,
+ };
+
+ #[test]
+ fn test_intern_for_vectorized_group_values() {
+ let data_set = VectorizedTestDataSet::new();
+ let mut group_values =
+ VectorizedGroupValuesColumn::try_new(data_set.schema()).unwrap();
+
+ data_set.load_to_group_values(&mut group_values);
+ let actual_batch = group_values.emit(EmitTo::All).unwrap();
+ let actual_batch = RecordBatch::try_new(data_set.schema(), actual_batch).unwrap();
+
+ check_result(&actual_batch, &data_set.expected_batch);
+ }
+
+ #[test]
+ fn test_emit_first_n_for_vectorized_group_values() {
Review Comment:
> I add the test to cover all the cases in first n. Could check it out https://github.com/Rachelint/arrow-datafusion/pull/2/files
๐ Thanks @jayzhan211 , I plan to check it tonight and try to continue implemening the possible optimization https://github.com/apache/datafusion/pull/12996#discussion_r1823813010 based on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823772169
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +154,350 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut [usize],
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self
+ .map
+ .get(target_hash, |(exist_hash, _)| target_hash == *exist_hash);
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
- ///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
+ }
+
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
+ ///
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut [usize]) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ self.scalarized_indices.clear();
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
Review Comment:
Good point!
I plan to continue optimizing the `vectorized_equal_to` together with https://github.com/apache/datafusion/pull/12996#discussion_r1818601807 and https://github.com/apache/datafusion/pull/12996#discussion_r1819225765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823784019
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
+ // `group_values` before performing append.
+ //
+ // 3. Perform `vectorized_equal_to` for `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`. If found some rows in input `cols`
+ // not equal to `exist rows` in `group_values`, place them in `scalarized_indices`
+ // and perform `scalarized_intern` for them similar as what in [`GroupValuesColumn`]
+ // after.
+ //
+ // 4. Perform `scalarized_intern` for rows mentioned above, when we process like this
+ // can see the comments of `scalarized_intern`.
+ //
+
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`
+ self.collect_vectorized_process_context(&batch_hashes, groups);
+
+ // 2. Perform `vectorized_append`
+ self.vectorized_append(cols);
+
+ // 3. Perform `vectorized_equal_to`
+ self.vectorized_equal_to(cols, groups);
+
+ // 4. Perform `scalarized_intern`
+ self.scalarized_intern(cols, &batch_hashes, groups);
+
+ self.hashes_buffer = batch_hashes;
+
+ Ok(())
+ }
+
+ fn size(&self) -> usize {
+ let group_values_size: usize = self.group_values.iter().map(|v| v.size()).sum();
+ group_values_size + self.map_size + self.hashes_buffer.allocated_size()
+ }
+
+ fn is_empty(&self) -> bool {
+ self.len() == 0
+ }
+
+ fn len(&self) -> usize {
+ if self.group_values.is_empty() {
+ return 0;
+ }
+
+ self.group_values[0].len()
+ }
+
+ fn emit(&mut self, emit_to: EmitTo) -> Result<Vec<ArrayRef>> {
+ let mut output = match emit_to {
+ EmitTo::All => {
+ let group_values = mem::take(&mut self.group_values);
+ debug_assert!(self.group_values.is_empty());
+
+ group_values
+ .into_iter()
+ .map(|v| v.build())
+ .collect::<Vec<_>>()
+ }
+ EmitTo::First(n) => {
+ let output = self
+ .group_values
+ .iter_mut()
+ .map(|v| v.take_n(n))
+ .collect::<Vec<_>>();
+ let new_group_index_lists =
+ Vec::with_capacity(self.group_index_lists.len());
+ let old_group_index_lists =
+ mem::replace(&mut self.group_index_lists, new_group_index_lists);
+
+ // SAFETY: self.map outlives iterator and is not modified concurrently
+ unsafe {
+ for bucket in self.map.iter() {
+ // Check if it is `inlined` or `non-inlined`
+ if bucket.as_ref().1.is_non_inlined() {
+ // Non-inlined case
+ // We take `group_index_list` from `old_group_index_lists`
+ let list_offset = bucket.as_ref().1.value() as usize;
+ let old_group_index_list =
+ &old_group_index_lists[list_offset];
+
+ let mut new_group_index_list = Vec::new();
+ for &group_index in old_group_index_list {
+ if let Some(remaining) = group_index.checked_sub(n) {
+ new_group_index_list.push(remaining);
+ }
+ }
+
+ // The possible results:
+ // - `new_group_index_list` is empty, we should erase this bucket
+ // - only one value in `new_group_index_list`, switch the `view` to `inlined`
+ // - still multiple values in `new_group_index_list`, build and set the new `unlined view`
+ if new_group_index_list.is_empty() {
+ self.map.erase(bucket);
+ } else if new_group_index_list.len() == 1 {
+ let group_index = new_group_index_list.first().unwrap();
+ bucket.as_mut().1 =
+ GroupIndexView::new_inlined(*group_index as u64);
+ } else {
+ let new_list_offset = self.group_index_lists.len();
+ self.group_index_lists.push(new_group_index_list);
Review Comment:
I wonder could we replace the `old_group_index_list` with `new_group_index_list`? The old group index becomes *garbage*, if we replace it with the new group indexes we don't need garbage collection for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823761465
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
Review Comment:
๐ฅ Yes, the alternative is to store the bucket as I tried in previous.
But we need to reserve to ensure the hashtable big enough at the beginning of `intern`.
Otherwise when reserve and rehash happens, the bucket(actually a pointer to hashtable slot) will not point to the expected `(hash, group_index_view)` anymore.
I think it actually a bit danger to do so, and give up the try in later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1830810815
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +157,89 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
+
+ let iter = izip!(
+ lhs_rows.iter(),
+ rhs_rows.iter(),
+ equal_to_results.iter_mut(),
+ );
+
+ for (&lhs_row, &rhs_row, equal_to_result) in iter {
+ // Has found not equal to in previous column, don't need to check
+ if !*equal_to_result {
+ continue;
+ }
+
+ // Perf: skip null check (by short circuit) if input is not nullable
+ if NULLABLE {
+ let exist_null = self.nulls.is_null(lhs_row);
+ let input_null = array.is_null(rhs_row);
+ if let Some(result) = nulls_equal_to(exist_null, input_null) {
+ *equal_to_result = result;
+ continue;
+ }
+ // Otherwise, we need to check their values
+ }
+
+ *equal_to_result = self.group_values[lhs_row] == array.value(rhs_row);
+ }
+ }
+
+ fn vectorized_append(&mut self, array: &ArrayRef, rows: &[usize]) {
+ let arr = array.as_primitive::<T>();
+
+ let null_count = array.null_count();
+ let num_rows = array.len();
+ let all_null_or_non_null = if null_count == 0 {
+ Some(true)
+ } else if null_count == num_rows {
+ Some(false)
+ } else {
+ None
+ };
+
+ match (NULLABLE, all_null_or_non_null) {
+ (true, None) => {
+ for &row in rows {
+ if array.is_null(row) {
+ self.nulls.append(true);
+ self.group_values.push(T::default_value());
+ } else {
+ self.nulls.append(false);
+ self.group_values.push(arr.value(row));
+ }
+ }
+ }
+
+ (true, Some(true)) => {
+ self.nulls.append_n(rows.len(), false);
+ for &row in rows {
+ self.group_values.push(arr.value(row));
+ }
+ }
+
+ (true, Some(false)) => {
+ self.nulls.append_n(rows.len(), true);
+ self.group_values
+ .extend(iter::repeat(T::default_value()).take(rows.len()));
+ }
+
+ (false, _) => {
+ for &row in rows {
+ self.group_values.push(arr.value(row));
Review Comment:
I think there are several things that could be done to make the append even faster:
1. `extend_from_slice` `if rows.len() == array.len()`
2. use `extend` rather than `push` for values
3. Speed up appending nulls (don't append bits one by one)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1829848355
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +148,653 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+/// Buffers to store intermediate results in `vectorized_append`
+/// and `vectorized_equal_to`, for reducing memory allocation
+#[derive(Default)]
+struct VectorizedOperationBuffers {
+ /// The `vectorized append` row indices buffer
+ append_row_indices: Vec<usize>,
+
+ /// The `vectorized_equal_to` row indices buffer
+ equal_to_row_indices: Vec<usize>,
+
+ /// The `vectorized_equal_to` group indices buffer
+ equal_to_group_indices: Vec<usize>,
+
+ /// The `vectorized_equal_to` result buffer
+ equal_to_results: Vec<bool>,
+
+ /// The buffer for storing row indices found not equal to
+ /// exist groups in `group_values` in `vectorized_equal_to`.
+ /// We will perform `scalarized_intern` for such rows.
+ remaining_row_indices: Vec<usize>,
+}
+
+impl VectorizedOperationBuffers {
+ fn clear(&mut self) {
+ self.append_row_indices.clear();
+ self.equal_to_row_indices.clear();
+ self.equal_to_group_indices.clear();
+ self.equal_to_results.clear();
+ self.remaining_row_indices.clear();
+ }
+}
+
+impl<const STREAMING: bool> GroupValuesColumn<STREAMING> {
+ // ========================================================================
+ // Initialization functions
+ // ========================================================================
+
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Review Comment:
This `with_capacity` can probably be improved (as a follow on PR) to avoid some smaller allocations
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -56,14 +59,40 @@ pub trait GroupColumn: Send + Sync {
///
/// Note that this comparison returns true if both elements are NULL
fn equal_to(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool;
+
/// Appends the row at `row` in `array` to this builder
fn append_val(&mut self, array: &ArrayRef, row: usize);
+
+ /// The vectorized version equal to
+ ///
+ /// When found nth row stored in this builder at `lhs_row`
+ /// is equal to the row in `array` at `rhs_row`,
+ /// it will record the `true` result at the corresponding
+ /// position in `equal_to_results`.
+ ///
+ /// And if found nth result in `equal_to_results` is already
Review Comment:
this is quite clever to pass in the existing "is equal to results"
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +148,653 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+/// Buffers to store intermediate results in `vectorized_append`
+/// and `vectorized_equal_to`, for reducing memory allocation
+#[derive(Default)]
+struct VectorizedOperationBuffers {
+ /// The `vectorized append` row indices buffer
+ append_row_indices: Vec<usize>,
+
+ /// The `vectorized_equal_to` row indices buffer
+ equal_to_row_indices: Vec<usize>,
+
+ /// The `vectorized_equal_to` group indices buffer
+ equal_to_group_indices: Vec<usize>,
+
+ /// The `vectorized_equal_to` result buffer
+ equal_to_results: Vec<bool>,
+
+ /// The buffer for storing row indices found not equal to
+ /// exist groups in `group_values` in `vectorized_equal_to`.
+ /// We will perform `scalarized_intern` for such rows.
+ remaining_row_indices: Vec<usize>,
+}
+
+impl VectorizedOperationBuffers {
+ fn clear(&mut self) {
+ self.append_row_indices.clear();
+ self.equal_to_row_indices.clear();
+ self.equal_to_group_indices.clear();
+ self.equal_to_results.clear();
+ self.remaining_row_indices.clear();
+ }
+}
+
+impl<const STREAMING: bool> GroupValuesColumn<STREAMING> {
+ // ========================================================================
+ // Initialization functions
+ // ========================================================================
+
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ emit_group_index_list_buffer: Vec::new(),
+ vectorized_operation_buffers: VectorizedOperationBuffers::default(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
+ // ========================================================================
+ // Scalarized intern
+ // ========================================================================
+
+ /// Scalarized intern
+ ///
+ /// This is used only for `streaming aggregation`, because `streaming aggregation`
+ /// depends on the order between `input rows` and their corresponding `group indices`.
+ ///
+ /// For example, assuming `input rows` in `cols` with 4 new rows
+ /// (not equal to `exist rows` in `group_values`, and need to create
+ /// new groups for them):
+ ///
+ /// ```text
+ /// row1 (hash collision with the exist rows)
+ /// row2
+ /// row3 (hash collision with the exist rows)
+ /// row4
+ /// ```
+ ///
+ /// # In `scalarized_intern`, their `group indices` will be
+ ///
+ /// ```text
+ /// row1 --> 0
+ /// row2 --> 1
+ /// row3 --> 2
+ /// row4 --> 3
+ /// ```
+ ///
+ /// `Group indices` order agrees with their input order, and the `streaming aggregation`
+ /// depends on this.
+ ///
+ /// # However In `vectorized_intern`, their `group indices` will be
+ ///
+ /// ```text
+ /// row1 --> 2
+ /// row2 --> 0
+ /// row3 --> 3
+ /// row4 --> 1
+ /// ```
+ ///
+ /// `Group indices` order are against with their input order, and this will lead to error
+ /// in `streaming aggregation`.
+ ///
+ fn scalarized_intern(
Review Comment:
this is basically the same as `GroupValuesColumn::intern` was previously, which makes sense to me
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +148,653 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+/// Buffers to store intermediate results in `vectorized_append`
Review Comment:
๐
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -56,14 +59,40 @@ pub trait GroupColumn: Send + Sync {
///
/// Note that this comparison returns true if both elements are NULL
fn equal_to(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool;
+
/// Appends the row at `row` in `array` to this builder
fn append_val(&mut self, array: &ArrayRef, row: usize);
Review Comment:
Maybe as a follow on we can consider removing `append_val` and `equal_to` and simpl change all codepaths to use the vectorized version
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
What i have been dreaming about with @XiangpengHao is maybe something like adding `take` / `filter` to arrow array builders
I took this opportunity to write up the idea (finally) for your amusement:
- https://github.com/apache/arrow-rs/issues/6692
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +157,89 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
+
+ let iter = izip!(
+ lhs_rows.iter(),
+ rhs_rows.iter(),
+ equal_to_results.iter_mut(),
+ );
+
+ for (&lhs_row, &rhs_row, equal_to_result) in iter {
+ // Has found not equal to in previous column, don't need to check
+ if !*equal_to_result {
+ continue;
+ }
+
+ // Perf: skip null check (by short circuit) if input is not nullable
+ if NULLABLE {
+ let exist_null = self.nulls.is_null(lhs_row);
+ let input_null = array.is_null(rhs_row);
+ if let Some(result) = nulls_equal_to(exist_null, input_null) {
+ *equal_to_result = result;
+ continue;
+ }
+ // Otherwise, we need to check their values
+ }
+
+ *equal_to_result = self.group_values[lhs_row] == array.value(rhs_row);
+ }
+ }
+
+ fn vectorized_append(&mut self, array: &ArrayRef, rows: &[usize]) {
+ let arr = array.as_primitive::<T>();
+
+ let null_count = array.null_count();
+ let num_rows = array.len();
+ let all_null_or_non_null = if null_count == 0 {
+ Some(true)
+ } else if null_count == num_rows {
+ Some(false)
+ } else {
+ None
+ };
+
+ match (NULLABLE, all_null_or_non_null) {
+ (true, None) => {
+ for &row in rows {
+ if array.is_null(row) {
+ self.nulls.append(true);
+ self.group_values.push(T::default_value());
+ } else {
+ self.nulls.append(false);
+ self.group_values.push(arr.value(row));
+ }
+ }
+ }
+
+ (true, Some(true)) => {
+ self.nulls.append_n(rows.len(), false);
+ for &row in rows {
+ self.group_values.push(arr.value(row));
+ }
+ }
+
+ (true, Some(false)) => {
+ self.nulls.append_n(rows.len(), true);
+ self.group_values
+ .extend(iter::repeat(T::default_value()).take(rows.len()));
+ }
+
+ (false, _) => {
+ for &row in rows {
+ self.group_values.push(arr.value(row));
Review Comment:
that uf possible the inner loop just looks like this (memcopy!)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1830886618
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +157,89 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
+
+ let iter = izip!(
+ lhs_rows.iter(),
+ rhs_rows.iter(),
+ equal_to_results.iter_mut(),
+ );
+
+ for (&lhs_row, &rhs_row, equal_to_result) in iter {
+ // Has found not equal to in previous column, don't need to check
+ if !*equal_to_result {
+ continue;
+ }
+
+ // Perf: skip null check (by short circuit) if input is not nullable
+ if NULLABLE {
+ let exist_null = self.nulls.is_null(lhs_row);
+ let input_null = array.is_null(rhs_row);
+ if let Some(result) = nulls_equal_to(exist_null, input_null) {
+ *equal_to_result = result;
+ continue;
+ }
+ // Otherwise, we need to check their values
+ }
+
+ *equal_to_result = self.group_values[lhs_row] == array.value(rhs_row);
+ }
+ }
+
+ fn vectorized_append(&mut self, array: &ArrayRef, rows: &[usize]) {
+ let arr = array.as_primitive::<T>();
+
+ let null_count = array.null_count();
+ let num_rows = array.len();
+ let all_null_or_non_null = if null_count == 0 {
+ Some(true)
+ } else if null_count == num_rows {
+ Some(false)
+ } else {
+ None
+ };
+
+ match (NULLABLE, all_null_or_non_null) {
+ (true, None) => {
+ for &row in rows {
+ if array.is_null(row) {
+ self.nulls.append(true);
+ self.group_values.push(T::default_value());
+ } else {
+ self.nulls.append(false);
+ self.group_values.push(arr.value(row));
+ }
+ }
+ }
+
+ (true, Some(true)) => {
+ self.nulls.append_n(rows.len(), false);
+ for &row in rows {
+ self.group_values.push(arr.value(row));
+ }
+ }
+
+ (true, Some(false)) => {
+ self.nulls.append_n(rows.len(), true);
+ self.group_values
+ .extend(iter::repeat(T::default_value()).take(rows.len()));
+ }
+
+ (false, _) => {
+ for &row in rows {
+ self.group_values.push(arr.value(row));
Review Comment:
> I think we already could use extend instead of `push`? `extend` on `Vec` is somewhat faster than push as the capacity check / allocation is done once instead of once per value.
Ok, I got it, I think again and found it indeed simple to do it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1807827680
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -160,11 +162,112 @@ macro_rules! instantiate_primitive {
};
}
-fn append_col_value<C>(mut core: C, array: &ArrayRef, row: usize)
-where
- C: FnMut(&ArrayRef, usize),
-{
- core(array, row);
+struct AggregationHashTable<T: AggregationHashTableEntry> {
+ /// Raw table storing values in a `Vec`
+ raw_table: Vec<T>,
Review Comment:
> Based on some experiments in changing hash join algorithm, I think it's likely `hashbrown` performs much better than implementing a hashtable ourselves although I would like to be surprised ๐
๐ค Even if we can perform something like `vectorized compare` or `vectorized append` in our hashtable?
I found in `multi group by` case, we will perform the `compare` for each row leading to the array downcasting again and again... And actually the `downcast` operation will be compiled to many asm codes....
And I foudn we can't eliminate it and perform the `vectorized compare` with `hashbrown`...
```
fn equal_to_inner(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool {
let array = array.as_byte_view::<B>();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1824410780
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -356,3 +1048,461 @@ impl GroupValues for GroupValuesColumn {
self.hashes_buffer.shrink_to(count);
}
}
+
+/// Returns true if [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+/// supported for the specified schema
+pub fn supported_schema(schema: &Schema) -> bool {
+ schema
+ .fields()
+ .iter()
+ .map(|f| f.data_type())
+ .all(supported_type)
+}
+
+/// Returns true if the specified data type is supported by
+/// [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+///
+/// In order to be supported, there must be a specialized implementation of
+/// [`GroupColumn`] for the data type, instantiated in [`GroupValuesColumn::intern`]
+/// or [`VectorizedGroupValuesColumn::intern`]
+fn supported_type(data_type: &DataType) -> bool {
+ matches!(
+ *data_type,
+ DataType::Int8
+ | DataType::Int16
+ | DataType::Int32
+ | DataType::Int64
+ | DataType::UInt8
+ | DataType::UInt16
+ | DataType::UInt32
+ | DataType::UInt64
+ | DataType::Float32
+ | DataType::Float64
+ | DataType::Utf8
+ | DataType::LargeUtf8
+ | DataType::Binary
+ | DataType::LargeBinary
+ | DataType::Date32
+ | DataType::Date64
+ | DataType::Utf8View
+ | DataType::BinaryView
+ )
+}
+
+#[cfg(test)]
+mod tests {
+ use std::sync::Arc;
+
+ use arrow::{compute::concat_batches, util::pretty::pretty_format_batches};
+ use arrow_array::{ArrayRef, Int64Array, RecordBatch, StringArray, StringViewArray};
+ use arrow_schema::{DataType, Field, Schema, SchemaRef};
+ use datafusion_expr::EmitTo;
+
+ use crate::aggregates::group_values::{
+ column::VectorizedGroupValuesColumn, GroupValues,
+ };
+
+ #[test]
+ fn test_intern_for_vectorized_group_values() {
+ let data_set = VectorizedTestDataSet::new();
+ let mut group_values =
+ VectorizedGroupValuesColumn::try_new(data_set.schema()).unwrap();
+
+ data_set.load_to_group_values(&mut group_values);
+ let actual_batch = group_values.emit(EmitTo::All).unwrap();
+ let actual_batch = RecordBatch::try_new(data_set.schema(), actual_batch).unwrap();
+
+ check_result(&actual_batch, &data_set.expected_batch);
+ }
+
+ #[test]
+ fn test_emit_first_n_for_vectorized_group_values() {
Review Comment:
It is covered by `hash collision` tests.
Maybe can try:
```
cargo test --features=force_hash_collisions --workspace --package datafusion-physical-plan --lib -- aggregates::group_values::column::tests::test_emit_first_n_for_vectorized_group_values --exact --show-output
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826582961
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -35,29 +37,113 @@ use datafusion_common::{not_impl_err, DataFusionError, Result};
use datafusion_execution::memory_pool::proxy::{RawTableAllocExt, VecAllocExt};
use datafusion_expr::EmitTo;
use datafusion_physical_expr::binary_map::OutputType;
+
use hashbrown::raw::RawTable;
-use std::mem::size_of;
-/// A [`GroupValues`] that stores multiple columns of group values.
+const NON_INLINED_FLAG: u64 = 0x8000000000000000;
+const VALUE_MASK: u64 = 0x7FFFFFFFFFFFFFFF;
+
+/// The view of indices pointing to the actual values in `GroupValues`
///
+/// If only single `group index` represented by view,
+/// value of view is just the `group index`, and we call it a `inlined view`.
///
-pub struct GroupValuesColumn {
+/// If multiple `group indices` represented by view,
+/// value of view is the actually the index pointing to `group indices`,
+/// and we call it `non-inlined view`.
+///
+/// The view(a u64) format is like:
+/// +---------------------+---------------------------------------------+
+/// | inlined flag(1bit) | group index / index to group indices(63bit) |
+/// +---------------------+---------------------------------------------+
+///
+/// `inlined flag`: 1 represents `non-inlined`, and 0 represents `inlined`
+///
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+struct GroupIndexView(u64);
+
+impl GroupIndexView {
+ #[inline]
+ pub fn is_non_inlined(&self) -> bool {
+ (self.0 & NON_INLINED_FLAG) > 0
+ }
+
+ #[inline]
+ pub fn new_inlined(group_index: u64) -> Self {
+ Self(group_index)
+ }
+
+ #[inline]
+ pub fn new_non_inlined(list_offset: u64) -> Self {
+ let non_inlined_value = list_offset | NON_INLINED_FLAG;
+ Self(non_inlined_value)
+ }
+
+ #[inline]
+ pub fn value(&self) -> u64 {
+ self.0 & VALUE_MASK
+ }
+}
+
+/// A [`GroupValues`] that stores multiple columns of group values,
+/// and supports vectorized operators for them
+///
+pub struct VectorizedGroupValuesColumn {
/// The output schema
schema: SchemaRef,
/// Logically maps group values to a group_index in
/// [`Self::group_values`] and in each accumulator
///
- /// Uses the raw API of hashbrown to avoid actually storing the
- /// keys (group values) in the table
+ /// It is a `hashtable` based on `hashbrown`.
///
- /// keys: u64 hashes of the GroupValue
- /// values: (hash, group_index)
- map: RawTable<(u64, usize)>,
+ /// Key and value in the `hashtable`:
+ /// - The `key` is `hash value(u64)` of the `group value`
+ /// - The `value` is the `group values` with the same `hash value`
+ ///
+ /// We don't really store the actual `group values` in `hashtable`,
+ /// instead we store the `group indices` pointing to values in `GroupValues`.
+ /// And we use [`GroupIndexView`] to represent such `group indices` in table.
+ ///
+ ///
+ map: RawTable<(u64, GroupIndexView)>,
Review Comment:
Is it the case
1. If group1 and group2 have exactly the same hash value, `GroupIndexView` will use chaining to resolve the collision
2. If group1 and group2 have different hash values but map to the same slot in hash table, `hashbrown` will handle the collision for you with probing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823718252
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
Review Comment:
I'm unclear on the meaning of the placeholders here, but I understand why we need to append values first now. Groups that require an equality check may need to compare against the newly added groups.
In collect_vectorized_process_context, new groups only update the hash in the hash table without appending values yet. After checking the hash, rows are allocated to groups requiring an equality check. Therefore, we need to append the new groups first so that equality checks can include the newly appended groups
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823746334
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
Review Comment:
I found some corner cases exist, and seems we can't store the `group_index_view`...
One case is `repeated rows` in `input batch`:
- all of them will be correspoding with same `group_index_view` in `collect_vectorized_process_context`.
- and assume that after we process the first of them, the rest stored `group_index_views` will be inconsistent.
The situation may be like:
```text
// Before process
row1 --> group_index_view (inlined, point to a group index)
row1 --> group_index_view (inlined, point to a group index)
row1 --> group_index_view (inlined, point to a group index)
// After process the first one
row1 --> group_index_view (uninline, point to group indices)
row1 --> group_index_view (inlined, inconsistent)
row1 --> group_index_view (inlined, inconsistent)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823765258
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
Review Comment:
Totally right! Seems the new placeholders indeed not easy to understand, i will try to improvement comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826380362
##########
datafusion/physical-plan/src/aggregates/group_values/mod.rs:
##########
@@ -143,8 +148,12 @@ pub fn new_group_values(schema: SchemaRef) -> Result<Box<dyn GroupValues>> {
}
}
- if GroupValuesColumn::supported_schema(schema.as_ref()) {
- Ok(Box::new(GroupValuesColumn::try_new(schema)?))
+ if column::supported_schema(schema.as_ref()) {
Review Comment:
> Is it because the VectorizedGroupValuesColumn doesn't keep the groups in order?
Yes, because we iterate all the rows at once, some rows are appended beforehand so they are not kept in order.
> If that is the case, it seems like maybe emit_n would never be called
`emit_early_if_necessary` may be called
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1820078967
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -58,6 +63,17 @@ pub trait GroupColumn: Send + Sync {
fn equal_to(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool;
/// Appends the row at `row` in `array` to this builder
fn append_val(&mut self, array: &ArrayRef, row: usize);
+
+ fn vectorized_equal_to(
Review Comment:
It is really a good point to improve later.
After refactoring to `col oriented` approach, more things we can try.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819191554
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> The same combination of `take` of some subset of rows is basically what the arrow filter kernel does. However curently it only accepts a single array as input --> single array as output.
>
> I wonder if we could implement some version of `filter` that that can be invoked multiple times or something each time adding to the in-progress array.
>
> That might be the low level building block that generalized `take` and `filter` and `interleave` (and we could use it as prt of `Repartition`) ๐ค
๐ค It seems somethinng like `TakenArrayBuilder`?
And we input multiple `Array` and `taken indices` pairs into it, and we call `build` to get the final `Array`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1820104867
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
Review Comment:
It seems that `vectorized_equal_to_row_indices` doesn't push the same row twice. If so, this branch should always be true
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1822904111
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
Review Comment:
Have added comments to explain why `vectorized_append` must before `vectorized_equal_to`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1806253429
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
Review Comment:
Yes, I plan to refactor the interface for supporting input a `rows: &[usize]`.
And make all parts' appending vectorized, and see the performance again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1806248679
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
+ self.group_values.push(array.as_primitive::<T>().value(row));
Review Comment:
This could be optimized to extend group values in one go from a slice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1806248679
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
+ self.group_values.push(array.as_primitive::<T>().value(row));
Review Comment:
This could be optimized to extend group values in one go from a slice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1806256535
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
Review Comment:
Cool :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1830485718
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +157,89 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ lhs_rows: &[usize],
+ array: &ArrayRef,
+ rhs_rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
+
+ let iter = izip!(
+ lhs_rows.iter(),
+ rhs_rows.iter(),
+ equal_to_results.iter_mut(),
+ );
+
+ for (&lhs_row, &rhs_row, equal_to_result) in iter {
+ // Has found not equal to in previous column, don't need to check
+ if !*equal_to_result {
+ continue;
+ }
+
+ // Perf: skip null check (by short circuit) if input is not nullable
+ if NULLABLE {
+ let exist_null = self.nulls.is_null(lhs_row);
+ let input_null = array.is_null(rhs_row);
+ if let Some(result) = nulls_equal_to(exist_null, input_null) {
+ *equal_to_result = result;
+ continue;
+ }
+ // Otherwise, we need to check their values
+ }
+
+ *equal_to_result = self.group_values[lhs_row] == array.value(rhs_row);
+ }
+ }
+
+ fn vectorized_append(&mut self, array: &ArrayRef, rows: &[usize]) {
+ let arr = array.as_primitive::<T>();
+
+ let null_count = array.null_count();
+ let num_rows = array.len();
+ let all_null_or_non_null = if null_count == 0 {
+ Some(true)
+ } else if null_count == num_rows {
+ Some(false)
+ } else {
+ None
+ };
+
+ match (NULLABLE, all_null_or_non_null) {
+ (true, None) => {
+ for &row in rows {
+ if array.is_null(row) {
+ self.nulls.append(true);
+ self.group_values.push(T::default_value());
+ } else {
+ self.nulls.append(false);
+ self.group_values.push(arr.value(row));
+ }
+ }
+ }
+
+ (true, Some(true)) => {
+ self.nulls.append_n(rows.len(), false);
+ for &row in rows {
+ self.group_values.push(arr.value(row));
+ }
+ }
+
+ (true, Some(false)) => {
+ self.nulls.append_n(rows.len(), true);
+ self.group_values
+ .extend(iter::repeat(T::default_value()).take(rows.len()));
+ }
+
+ (false, _) => {
+ for &row in rows {
+ self.group_values.push(arr.value(row));
Review Comment:
๐ I think we can even do more, like check `if rows.len() == array.len()`, if so we just perform `extend`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1831127446
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -56,14 +59,40 @@ pub trait GroupColumn: Send + Sync {
///
/// Note that this comparison returns true if both elements are NULL
fn equal_to(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool;
+
/// Appends the row at `row` in `array` to this builder
fn append_val(&mut self, array: &ArrayRef, row: usize);
Review Comment:
A good thing to benchmark (as a follow on PR) perhaps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1855477417
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
> At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better. I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
>
> Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
I try `take + eq` approach , but see no improvement... I think it may be due to the `extra copys`, and `memory allocation` of the result bool array.
code is here https://github.com/Rachelint/arrow-datafusion/tree/optimize-vectorized-operations-bak
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1818475481
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
At least `take` for `StringArray` is expensive as it needs to copy the string data, recheck the offsets, etc. For string / byte views this will probably be much better.
I think some approach for using `take` + `eq` for primitives while using a custom one for string arrays will likely be best overal.
Even better would be to write a optimized implementation (an arrow kernel that performs equality of two arrays by indices efficiently while applying the same tricks done by take / eq kernels). The join implementation can benefit from this as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819950324
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -15,33 +15,107 @@
// specific language governing permissions and limitations
// under the License.
+use std::ops::Sub;
+use std::{iter, mem, usize};
+
use crate::aggregates::group_values::group_column::{
ByteGroupValueBuilder, ByteViewGroupValueBuilder, GroupColumn,
PrimitiveGroupValueBuilder,
};
use crate::aggregates::group_values::GroupValues;
use ahash::RandomState;
-use arrow::compute::cast;
+use arrow::compute::{self, cast};
use arrow::datatypes::{
BinaryViewType, Date32Type, Date64Type, Float32Type, Float64Type, Int16Type,
Int32Type, Int64Type, Int8Type, StringViewType, UInt16Type, UInt32Type, UInt64Type,
UInt8Type,
};
use arrow::record_batch::RecordBatch;
-use arrow_array::{Array, ArrayRef};
-use arrow_schema::{DataType, Schema, SchemaRef};
+use arrow_array::{
+ Array, ArrayRef, BinaryArray, BinaryViewArray, BooleanArray, Date32Array,
+ Date64Array, Decimal128Array, Float32Array, Float64Array, Int16Array, Int32Array,
+ Int64Array, Int8Array, LargeStringArray, StringArray, StringViewArray,
+ TimestampMicrosecondArray, TimestampMillisecondArray, TimestampNanosecondArray,
+ TimestampSecondArray, UInt16Array, UInt32Array, UInt64Array, UInt8Array,
+};
+use arrow_schema::{DataType, Schema, SchemaRef, TimeUnit};
use datafusion_common::hash_utils::create_hashes;
use datafusion_common::{not_impl_err, DataFusionError, Result};
use datafusion_execution::memory_pool::proxy::{RawTableAllocExt, VecAllocExt};
use datafusion_expr::EmitTo;
use datafusion_physical_expr::binary_map::OutputType;
-use hashbrown::raw::RawTable;
+use datafusion_physical_expr_common::datum::compare_with_eq;
+use hashbrown::raw::{Bucket, RawTable};
-/// A [`GroupValues`] that stores multiple columns of group values.
+const NON_INLINED_FLAG: u64 = 0x8000000000000000;
+const VALUE_MASK: u64 = 0x7FFFFFFFFFFFFFFF;
+
+/// `BucketContext` is a packed struct
///
+/// ### Format:
///
-pub struct GroupValuesColumn {
+/// +---------------------+--------------------+
+/// | checking flag(1bit) | group index(63bit) |
+/// +---------------------+--------------------+
+///
+/// ### Checking flag
+///
+/// It is possible that rows with same hash values exist in `input cols`.
+/// And if we `vectorized_equal_to` and `vectorized append` them
+/// in the same round, some fault cases will occur especially when
+/// they are totally the repeated rows...
+///
+/// For example:
+/// - Two repeated rows exist in `input cols`.
+///
+/// - We found their hash values equal to one exist group
+///
+/// - We then perform `vectorized_equal_to` for them to the exist group,
+/// and found their values not equal to the exist one
+///
+/// - Finally when perform `vectorized append`, we decide to build two
+/// respective new groups for them, even we actually just need one
+/// new group...
+///
+/// So for solving such cases simply, if some rows with same hash value
+/// in `input cols`, just allow to process one of them in a round,
+/// and this flag is used to represent that one of them is processing
+/// in current round.
+///
+/// ### Group index
+///
+/// The group's index in group values
+///
+#[derive(Debug, Clone, Copy)]
+struct GroupIndexView(u64);
+
+impl GroupIndexView {
+ #[inline]
+ pub fn is_non_inlined(&self) -> bool {
Review Comment:
`inline` is a bit confusing to understand at first. I believe the main idea is to determine whether the group value already exists (is inline) or not. Would exists be a better name for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819950324
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -15,33 +15,107 @@
// specific language governing permissions and limitations
// under the License.
+use std::ops::Sub;
+use std::{iter, mem, usize};
+
use crate::aggregates::group_values::group_column::{
ByteGroupValueBuilder, ByteViewGroupValueBuilder, GroupColumn,
PrimitiveGroupValueBuilder,
};
use crate::aggregates::group_values::GroupValues;
use ahash::RandomState;
-use arrow::compute::cast;
+use arrow::compute::{self, cast};
use arrow::datatypes::{
BinaryViewType, Date32Type, Date64Type, Float32Type, Float64Type, Int16Type,
Int32Type, Int64Type, Int8Type, StringViewType, UInt16Type, UInt32Type, UInt64Type,
UInt8Type,
};
use arrow::record_batch::RecordBatch;
-use arrow_array::{Array, ArrayRef};
-use arrow_schema::{DataType, Schema, SchemaRef};
+use arrow_array::{
+ Array, ArrayRef, BinaryArray, BinaryViewArray, BooleanArray, Date32Array,
+ Date64Array, Decimal128Array, Float32Array, Float64Array, Int16Array, Int32Array,
+ Int64Array, Int8Array, LargeStringArray, StringArray, StringViewArray,
+ TimestampMicrosecondArray, TimestampMillisecondArray, TimestampNanosecondArray,
+ TimestampSecondArray, UInt16Array, UInt32Array, UInt64Array, UInt8Array,
+};
+use arrow_schema::{DataType, Schema, SchemaRef, TimeUnit};
use datafusion_common::hash_utils::create_hashes;
use datafusion_common::{not_impl_err, DataFusionError, Result};
use datafusion_execution::memory_pool::proxy::{RawTableAllocExt, VecAllocExt};
use datafusion_expr::EmitTo;
use datafusion_physical_expr::binary_map::OutputType;
-use hashbrown::raw::RawTable;
+use datafusion_physical_expr_common::datum::compare_with_eq;
+use hashbrown::raw::{Bucket, RawTable};
-/// A [`GroupValues`] that stores multiple columns of group values.
+const NON_INLINED_FLAG: u64 = 0x8000000000000000;
+const VALUE_MASK: u64 = 0x7FFFFFFFFFFFFFFF;
+
+/// `BucketContext` is a packed struct
///
+/// ### Format:
///
-pub struct GroupValuesColumn {
+/// +---------------------+--------------------+
+/// | checking flag(1bit) | group index(63bit) |
+/// +---------------------+--------------------+
+///
+/// ### Checking flag
+///
+/// It is possible that rows with same hash values exist in `input cols`.
+/// And if we `vectorized_equal_to` and `vectorized append` them
+/// in the same round, some fault cases will occur especially when
+/// they are totally the repeated rows...
+///
+/// For example:
+/// - Two repeated rows exist in `input cols`.
+///
+/// - We found their hash values equal to one exist group
+///
+/// - We then perform `vectorized_equal_to` for them to the exist group,
+/// and found their values not equal to the exist one
+///
+/// - Finally when perform `vectorized append`, we decide to build two
+/// respective new groups for them, even we actually just need one
+/// new group...
+///
+/// So for solving such cases simply, if some rows with same hash value
+/// in `input cols`, just allow to process one of them in a round,
+/// and this flag is used to represent that one of them is processing
+/// in current round.
+///
+/// ### Group index
+///
+/// The group's index in group values
+///
+#[derive(Debug, Clone, Copy)]
+struct GroupIndexView(u64);
+
+impl GroupIndexView {
+ #[inline]
+ pub fn is_non_inlined(&self) -> bool {
Review Comment:
`inline` is a bit confusing to understand at first. I believe the main idea is to determine whether the group value already exists (is inline) or not. Would exists be a better name for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorized hashtable for aggregation [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1819112999
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -287,6 +469,63 @@ where
};
}
+ fn vectorized_equal_to(
Review Comment:
The same combination of `take` of some subset of rows is basically what the arrow filter kernel does. However curently it only accepts a single array as input --> single array as output.
I wonder if we could implement some version of `filter` that that can be invoked multiple times or something each time adding to the in-progress array.
That might be the low level building block that generalized `take` and `filter` and `interleave` (and we could use it as prt of `Repartition`) ๐ค
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1822902720
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
+ let entry = map.get_mut(target_hash, |(exist_hash, group_index_view)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ // Only `rows` having the same hash value with `exist rows` but different value
+ // will be process in `scalarized_intern`.
+ // So related `buckets` in `map` is ensured to be `Some`.
+ let Some((_, group_index_view)) = entry else {
+ unreachable!()
+ };
+
+ // Perform scalarized equal to
+ if self.scalarized_equal_to(&group_index_view, cols, row, groups) {
+ // Found the row actually exists in group values,
+ // don't need to create new group for it.
+ continue;
+ }
+
+ // Insert the `row` to `group_values` before checking `next row`
+ let group_idx = self.group_values[0].len();
+ let mut checklen = 0;
+ for (i, group_value) in self.group_values.iter_mut().enumerate() {
+ group_value.append_val(&cols[i], row);
+ let len = group_value.len();
+ if i == 0 {
+ checklen = len;
+ } else {
+ debug_assert_eq!(checklen, len);
+ }
+ }
+
+ // Check if the `view` is `inlined` or `non-inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, get `group_index_list` from `group_index_lists`,
+ // then add the new `group` with the same hash values into it.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &mut self.group_index_lists[list_offset];
+ group_index_list.push(group_idx);
+ } else {
+ // Inlined case
+ let list_offset = self.group_index_lists.len();
+
+ // Create new `group_index_list` including
+ // `exist group index` + `new group index`.
+ // Add new `group_index_list` into ``group_index_lists`.
+ let exist_group_index = group_index_view.value() as usize;
+ let new_group_index_list = vec![exist_group_index, group_idx];
Review Comment:
I have added comments to explain the new `hashtable` in `VectorizedGroupValuesColumn`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823761465
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -75,55 +176,317 @@ pub struct GroupValuesColumn {
random_state: RandomState,
}
-impl GroupValuesColumn {
+impl VectorizedGroupValuesColumn {
/// Create a new instance of GroupValuesColumn if supported for the specified schema
pub fn try_new(schema: SchemaRef) -> Result<Self> {
let map = RawTable::with_capacity(0);
Ok(Self {
schema,
map,
+ group_index_lists: Vec::new(),
+ index_lists_updates: Vec::new(),
map_size: 0,
group_values: vec![],
hashes_buffer: Default::default(),
random_state: Default::default(),
+ scalarized_indices: Default::default(),
+ vectorized_equal_to_row_indices: Default::default(),
+ vectorized_equal_to_group_indices: Default::default(),
+ vectorized_equal_to_results: Default::default(),
+ vectorized_append_row_indices: Default::default(),
})
}
- /// Returns true if [`GroupValuesColumn`] supported for the specified schema
- pub fn supported_schema(schema: &Schema) -> bool {
- schema
- .fields()
- .iter()
- .map(|f| f.data_type())
- .all(Self::supported_type)
+ /// Collect vectorized context by checking hash values of `cols` in `map`
+ ///
+ /// 1. If bucket not found
+ /// - Build and insert the `new inlined group index view`
+ /// and its hash value to `map`
+ /// - Add row index to `vectorized_append_row_indices`
+ /// - Set group index to row in `groups`
+ ///
+ /// 2. bucket found
+ /// - Add row index to `vectorized_equal_to_row_indices`
+ /// - Check if the `group index view` is `inlined` or `non_inlined`:
+ /// If it is inlined, add to `vectorized_equal_to_group_indices` directly.
+ /// Otherwise get all group indices from `group_index_lists`, and add them.
+ ///
+ fn collect_vectorized_process_context(
+ &mut self,
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ self.vectorized_append_row_indices.clear();
+ self.vectorized_equal_to_row_indices.clear();
+ self.vectorized_equal_to_group_indices.clear();
+
+ let mut group_values_len = self.group_values[0].len();
+ for (row, &target_hash) in batch_hashes.iter().enumerate() {
+ let entry = self.map.get(target_hash, |(exist_hash, _)| {
+ // Somewhat surprisingly, this closure can be called even if the
+ // hash doesn't match, so check the hash first with an integer
+ // comparison first avoid the more expensive comparison with
+ // group value. https://github.com/apache/datafusion/pull/11718
+ target_hash == *exist_hash
+ });
+
+ let Some((_, group_index_view)) = entry else {
+ // 1. Bucket not found case
+ // Build `new inlined group index view`
+ let current_group_idx = group_values_len;
+ let group_index_view =
+ GroupIndexView::new_inlined(current_group_idx as u64);
+
+ // Insert the `group index view` and its hash into `map`
+ // for hasher function, use precomputed hash value
+ self.map.insert_accounted(
+ (target_hash, group_index_view),
+ |(hash, _)| *hash,
+ &mut self.map_size,
+ );
+
+ // Add row index to `vectorized_append_row_indices`
+ self.vectorized_append_row_indices.push(row);
+
+ // Set group index to row in `groups`
+ groups[row] = current_group_idx;
+
+ group_values_len += 1;
+ continue;
+ };
+
+ // 2. bucket found
+ // Check if the `group index view` is `inlined` or `non_inlined`
+ if group_index_view.is_non_inlined() {
+ // Non-inlined case, the value of view is offset in `group_index_lists`.
+ // We use it to get `group_index_list`, and add related `rows` and `group_indices`
+ // into `vectorized_equal_to_row_indices` and `vectorized_equal_to_group_indices`.
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+ for &group_index in group_index_list {
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ } else {
+ let group_index = group_index_view.value() as usize;
+ self.vectorized_equal_to_row_indices.push(row);
+ self.vectorized_equal_to_group_indices.push(group_index);
+ }
+ }
+ }
+
+ /// Perform `vectorized_append`` for `rows` in `vectorized_append_row_indices`
+ fn vectorized_append(&mut self, cols: &[ArrayRef]) {
+ if self.vectorized_append_row_indices.is_empty() {
+ return;
+ }
+
+ let iter = self.group_values.iter_mut().zip(cols.iter());
+ for (group_column, col) in iter {
+ group_column.vectorized_append(col, &self.vectorized_append_row_indices);
+ }
}
- /// Returns true if the specified data type is supported by [`GroupValuesColumn`]
+ /// Perform `vectorized_equal_to`
+ ///
+ /// 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ /// and `group_indices` in `vectorized_equal_to_group_indices`.
+ ///
+ /// 2. Check `equal_to_results`:
///
- /// In order to be supported, there must be a specialized implementation of
- /// [`GroupColumn`] for the data type, instantiated in [`Self::intern`]
- fn supported_type(data_type: &DataType) -> bool {
- matches!(
- *data_type,
- DataType::Int8
- | DataType::Int16
- | DataType::Int32
- | DataType::Int64
- | DataType::UInt8
- | DataType::UInt16
- | DataType::UInt32
- | DataType::UInt64
- | DataType::Float32
- | DataType::Float64
- | DataType::Utf8
- | DataType::LargeUtf8
- | DataType::Binary
- | DataType::LargeBinary
- | DataType::Date32
- | DataType::Date64
- | DataType::Utf8View
- | DataType::BinaryView
- )
+ /// If found equal to `rows`, set the `group_indices` to `rows` in `groups`.
+ ///
+ /// If found not equal to `row`s, just add them to `scalarized_indices`,
+ /// and perform `scalarized_intern` for them after.
+ /// Usually, such `rows` having same hash but different value with `exists rows`
+ /// are very few.
+ fn vectorized_equal_to(&mut self, cols: &[ArrayRef], groups: &mut Vec<usize>) {
+ assert_eq!(
+ self.vectorized_equal_to_group_indices.len(),
+ self.vectorized_equal_to_row_indices.len()
+ );
+
+ if self.vectorized_equal_to_group_indices.is_empty() {
+ return;
+ }
+
+ // 1. Perform `vectorized_equal_to` for `rows` in `vectorized_equal_to_group_indices`
+ // and `group_indices` in `vectorized_equal_to_group_indices`
+ let mut equal_to_results = mem::take(&mut self.vectorized_equal_to_results);
+ equal_to_results.clear();
+ equal_to_results.resize(self.vectorized_equal_to_group_indices.len(), true);
+
+ for (col_idx, group_col) in self.group_values.iter().enumerate() {
+ group_col.vectorized_equal_to(
+ &self.vectorized_equal_to_group_indices,
+ &cols[col_idx],
+ &self.vectorized_equal_to_row_indices,
+ &mut equal_to_results,
+ );
+ }
+
+ // 2. Check `equal_to_results`, if found not equal to `row`s, just add them
+ // to `scalarized_indices`, and perform `scalarized_intern` for them after.
+ let mut current_row_equal_to_result = false;
+ for (idx, &row) in self.vectorized_equal_to_row_indices.iter().enumerate() {
+ let equal_to_result = equal_to_results[idx];
+
+ // Equal to case, set the `group_indices` to `rows` in `groups`
+ if equal_to_result {
+ groups[row] = self.vectorized_equal_to_group_indices[idx];
+ }
+ current_row_equal_to_result |= equal_to_result;
+
+ // Look forward next one row to check if have checked all results
+ // of current row
+ let next_row = self
+ .vectorized_equal_to_row_indices
+ .get(idx + 1)
+ .unwrap_or(&usize::MAX);
+
+ // Have checked all results of current row, check the total result
+ if row != *next_row {
+ // Not equal to case, add `row` to `scalarized_indices`
+ if !current_row_equal_to_result {
+ self.scalarized_indices.push(row);
+ }
+
+ // Init the total result for checking next row
+ current_row_equal_to_result = false;
+ }
+ }
+
+ self.vectorized_equal_to_results = equal_to_results;
+ }
+
+ fn scalarized_equal_to(
+ &self,
+ group_index_view: &GroupIndexView,
+ cols: &[ArrayRef],
+ row: usize,
+ groups: &mut Vec<usize>,
+ ) -> bool {
+ // Check if this row exists in `group_values`
+ fn check_row_equal(
+ array_row: &dyn GroupColumn,
+ lhs_row: usize,
+ array: &ArrayRef,
+ rhs_row: usize,
+ ) -> bool {
+ array_row.equal_to(lhs_row, array, rhs_row)
+ }
+
+ if group_index_view.is_non_inlined() {
+ let list_offset = group_index_view.value() as usize;
+ let group_index_list = &self.group_index_lists[list_offset];
+
+ for &group_idx in group_index_list {
+ let mut check_result = true;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ check_result = false;
+ break;
+ }
+ }
+
+ if check_result {
+ groups[row] = group_idx;
+ return true;
+ }
+ }
+
+ // All groups unmatched, return false result
+ false
+ } else {
+ let group_idx = group_index_view.value() as usize;
+ for (i, group_val) in self.group_values.iter().enumerate() {
+ if !check_row_equal(group_val.as_ref(), group_idx, &cols[i], row) {
+ return false;
+ }
+ }
+
+ groups[row] = group_idx;
+ true
+ }
+ }
+
+ fn scalarized_intern(
+ &mut self,
+ cols: &[ArrayRef],
+ batch_hashes: &[u64],
+ groups: &mut Vec<usize>,
+ ) {
+ if self.scalarized_indices.is_empty() {
+ return;
+ }
+
+ let mut map = mem::take(&mut self.map);
+
+ for &row in &self.scalarized_indices {
+ let target_hash = batch_hashes[row];
Review Comment:
๐ฅ Yes, the alternative is to store the bucket as I tried in previous.
But we need to reserve to ensure the hashtable big enough at the beginning of `intern`.
Otherwise when reserve and rehash happens, the bucket(actually a pointer to hashtable slot) will not point to the expected `(hash, group_index_view)` anymore.
I think it actually a bit dangerous to do so, and give up the try in later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823765258
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
Review Comment:
Totally right! Seems the `placeholders` indeed not easy to understand, i will try to improvement comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823813010
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +570,324 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
+ batch_hashes.clear();
+ batch_hashes.resize(n_rows, 0);
+ create_hashes(cols, &self.random_state, &mut batch_hashes)?;
+
+ // General steps for one round `vectorized equal_to & append`:
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`,
+ // mainly fill `vectorized_append_row_indices`, `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`
+ //
+ // 2. Perform `vectorized_append` for `vectorized_append_row_indices`.
+ // `vectorized_append` must be performed before `vectorized_equal_to`,
+ // because some `group indices` in `vectorized_equal_to_group_indices`
+ // may be actually placeholders, and still point to no actual values in
+ // `group_values` before performing append.
+ //
+ // 3. Perform `vectorized_equal_to` for `vectorized_equal_to_row_indices`
+ // and `vectorized_equal_to_group_indices`. If found some rows in input `cols`
+ // not equal to `exist rows` in `group_values`, place them in `scalarized_indices`
+ // and perform `scalarized_intern` for them similar as what in [`GroupValuesColumn`]
+ // after.
+ //
+ // 4. Perform `scalarized_intern` for rows mentioned above, when we process like this
+ // can see the comments of `scalarized_intern`.
+ //
+
+ // 1. Collect vectorized context by checking hash values of `cols` in `map`
+ self.collect_vectorized_process_context(&batch_hashes, groups);
+
+ // 2. Perform `vectorized_append`
+ self.vectorized_append(cols);
+
+ // 3. Perform `vectorized_equal_to`
+ self.vectorized_equal_to(cols, groups);
+
+ // 4. Perform `scalarized_intern`
+ self.scalarized_intern(cols, &batch_hashes, groups);
+
+ self.hashes_buffer = batch_hashes;
+
+ Ok(())
+ }
+
+ fn size(&self) -> usize {
+ let group_values_size: usize = self.group_values.iter().map(|v| v.size()).sum();
+ group_values_size + self.map_size + self.hashes_buffer.allocated_size()
+ }
+
+ fn is_empty(&self) -> bool {
+ self.len() == 0
+ }
+
+ fn len(&self) -> usize {
+ if self.group_values.is_empty() {
+ return 0;
+ }
+
+ self.group_values[0].len()
+ }
+
+ fn emit(&mut self, emit_to: EmitTo) -> Result<Vec<ArrayRef>> {
+ let mut output = match emit_to {
+ EmitTo::All => {
+ let group_values = mem::take(&mut self.group_values);
+ debug_assert!(self.group_values.is_empty());
+
+ group_values
+ .into_iter()
+ .map(|v| v.build())
+ .collect::<Vec<_>>()
+ }
+ EmitTo::First(n) => {
+ let output = self
+ .group_values
+ .iter_mut()
+ .map(|v| v.take_n(n))
+ .collect::<Vec<_>>();
+ let new_group_index_lists =
+ Vec::with_capacity(self.group_index_lists.len());
+ let old_group_index_lists =
+ mem::replace(&mut self.group_index_lists, new_group_index_lists);
+
+ // SAFETY: self.map outlives iterator and is not modified concurrently
+ unsafe {
+ for bucket in self.map.iter() {
+ // Check if it is `inlined` or `non-inlined`
+ if bucket.as_ref().1.is_non_inlined() {
+ // Non-inlined case
+ // We take `group_index_list` from `old_group_index_lists`
+ let list_offset = bucket.as_ref().1.value() as usize;
+ let old_group_index_list =
+ &old_group_index_lists[list_offset];
+
+ let mut new_group_index_list = Vec::new();
+ for &group_index in old_group_index_list {
+ if let Some(remaining) = group_index.checked_sub(n) {
+ new_group_index_list.push(remaining);
+ }
+ }
+
+ // The possible results:
+ // - `new_group_index_list` is empty, we should erase this bucket
+ // - only one value in `new_group_index_list`, switch the `view` to `inlined`
+ // - still multiple values in `new_group_index_list`, build and set the new `unlined view`
+ if new_group_index_list.is_empty() {
+ self.map.erase(bucket);
+ } else if new_group_index_list.len() == 1 {
+ let group_index = new_group_index_list.first().unwrap();
+ bucket.as_mut().1 =
+ GroupIndexView::new_inlined(*group_index as u64);
+ } else {
+ let new_list_offset = self.group_index_lists.len();
+ self.group_index_lists.push(new_group_index_list);
Review Comment:
๐ค current when taking n happen, we need to create an `new group_index_lists`, and push the `group_index_list` still not empty from `old group_index_lists` to it.
The alternative is that, we don't create `new group_index_lists`, and just resue the `old group_index_lists`.
But I am afraid it will become something like following finally:
```text
[],
[1, 2],
[],
...
[],
[],
[3, 4],
```
I think we can actually reuse the `group_index_list` taken from `old group_index_lists`, rather than dropping it:
- We can define something like `taken_group_index_list_buffer`
- And we will place the `kept group indices` into them
- Finally we copy the `kept group indices` from taken_group_index_list_buffer` to the reused `group_index_list`,
and push it to the current `grou_index_lists`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1823151542
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +144,84 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ group_indices: &[usize],
+ array: &ArrayRef,
+ rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
Review Comment:
I plan to try it together with https://github.com/apache/datafusion/pull/12996#discussion_r1818601807 on follow up prs.
Seems code changes in this one are already a bit large...
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +144,84 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn vectorized_equal_to(
+ &self,
+ group_indices: &[usize],
+ array: &ArrayRef,
+ rows: &[usize],
+ equal_to_results: &mut [bool],
+ ) {
+ let array = array.as_primitive::<T>();
Review Comment:
I plan to try it together with https://github.com/apache/datafusion/pull/12996#discussion_r1818601807 on follow up prs.
Seems code changes in this one are already a bit large.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1807294769
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
Review Comment:
To eliminate this extra branch, I think we need to refactor the `intern` process metioned in https://github.com/apache/datafusion/issues/12821#issuecomment-2401542201
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] POC: Vectorize append value [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1807294769
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -128,6 +132,15 @@ impl<T: ArrowPrimitiveType, const NULLABLE: bool> GroupColumn
}
}
+ fn append_non_nullable_val(&mut self, array: &ArrayRef, row: usize) {
+ if NULLABLE {
+ self.nulls.append(false);
Review Comment:
To eliminate this extra branch, I think we need to refactor the `intern` process metioned in https://github.com/apache/datafusion/issues/12821#issuecomment-2401542201
I am trying it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#issuecomment-2458035232
As my admittedly sparse help for this PR I have filed some additional tickets for follow on work after this PR is merged:
- https://github.com/apache/datafusion/issues/13262
- https://github.com/apache/datafusion/issues/13263
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1830707982
##########
datafusion/physical-plan/src/aggregates/group_values/group_column.rs:
##########
@@ -56,14 +59,40 @@ pub trait GroupColumn: Send + Sync {
///
/// Note that this comparison returns true if both elements are NULL
fn equal_to(&self, lhs_row: usize, array: &ArrayRef, rhs_row: usize) -> bool;
+
/// Appends the row at `row` in `array` to this builder
fn append_val(&mut self, array: &ArrayRef, row: usize);
Review Comment:
I am a bit worried about if we merge them, some extra `if else` will be introduced.
It hurt much for performance for the row level operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826610308
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -35,29 +37,113 @@ use datafusion_common::{not_impl_err, DataFusionError, Result};
use datafusion_execution::memory_pool::proxy::{RawTableAllocExt, VecAllocExt};
use datafusion_expr::EmitTo;
use datafusion_physical_expr::binary_map::OutputType;
+
use hashbrown::raw::RawTable;
-use std::mem::size_of;
-/// A [`GroupValues`] that stores multiple columns of group values.
+const NON_INLINED_FLAG: u64 = 0x8000000000000000;
+const VALUE_MASK: u64 = 0x7FFFFFFFFFFFFFFF;
+
+/// The view of indices pointing to the actual values in `GroupValues`
///
+/// If only single `group index` represented by view,
+/// value of view is just the `group index`, and we call it a `inlined view`.
///
-pub struct GroupValuesColumn {
+/// If multiple `group indices` represented by view,
+/// value of view is the actually the index pointing to `group indices`,
+/// and we call it `non-inlined view`.
+///
+/// The view(a u64) format is like:
+/// +---------------------+---------------------------------------------+
+/// | inlined flag(1bit) | group index / index to group indices(63bit) |
+/// +---------------------+---------------------------------------------+
+///
+/// `inlined flag`: 1 represents `non-inlined`, and 0 represents `inlined`
+///
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+struct GroupIndexView(u64);
+
+impl GroupIndexView {
+ #[inline]
+ pub fn is_non_inlined(&self) -> bool {
+ (self.0 & NON_INLINED_FLAG) > 0
+ }
+
+ #[inline]
+ pub fn new_inlined(group_index: u64) -> Self {
+ Self(group_index)
+ }
+
+ #[inline]
+ pub fn new_non_inlined(list_offset: u64) -> Self {
+ let non_inlined_value = list_offset | NON_INLINED_FLAG;
+ Self(non_inlined_value)
+ }
+
+ #[inline]
+ pub fn value(&self) -> u64 {
+ self.0 & VALUE_MASK
+ }
+}
+
+/// A [`GroupValues`] that stores multiple columns of group values,
+/// and supports vectorized operators for them
+///
+pub struct VectorizedGroupValuesColumn {
/// The output schema
schema: SchemaRef,
/// Logically maps group values to a group_index in
/// [`Self::group_values`] and in each accumulator
///
- /// Uses the raw API of hashbrown to avoid actually storing the
- /// keys (group values) in the table
+ /// It is a `hashtable` based on `hashbrown`.
///
- /// keys: u64 hashes of the GroupValue
- /// values: (hash, group_index)
- map: RawTable<(u64, usize)>,
+ /// Key and value in the `hashtable`:
+ /// - The `key` is `hash value(u64)` of the `group value`
+ /// - The `value` is the `group values` with the same `hash value`
+ ///
+ /// We don't really store the actual `group values` in `hashtable`,
+ /// instead we store the `group indices` pointing to values in `GroupValues`.
+ /// And we use [`GroupIndexView`] to represent such `group indices` in table.
+ ///
+ ///
+ map: RawTable<(u64, GroupIndexView)>,
/// The size of `map` in bytes
map_size: usize,
+ /// The lists for group indices with the same hash value
+ ///
+ /// It is possible that hash value collision exists,
+ /// and we will chain the `group indices` with same hash value
+ ///
+ /// The chained indices is like:
+ /// `latest group index -> older group index -> even older group index -> ...`
+ ///
+ group_index_lists: Vec<Vec<usize>>,
+
+ /// When emitting first n, we need to decrease/erase group indices in
+ /// `map` and `group_index_lists`.
+ ///
+ /// This buffer is used to temporarily store the remaining group indices in
+ /// a specific list in `group_index_lists`.
+ emit_group_index_list_buffer: Vec<usize>,
+
+ /// Similar as `current_indices`, but `remaining_indices`
+ /// is used to store the rows will be processed in next round.
+ scalarized_indices: Vec<usize>,
+
+ /// The `vectorized_equal_tod` row indices buffer
Review Comment:
Good idea for readability, I defined `VectorizedOperationBuffers` to hold such buffers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1826322118
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -146,7 +550,7 @@ macro_rules! instantiate_primitive {
};
}
-impl GroupValues for GroupValuesColumn {
+impl GroupValues for VectorizedGroupValuesColumn {
Review Comment:
I wonder if there is any reason to rename this
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -196,6 +600,328 @@ impl GroupValues for GroupValuesColumn {
let b = ByteViewGroupValueBuilder::<BinaryViewType>::new();
v.push(Box::new(b) as _)
}
+ dt => {
+ return not_impl_err!(
+ "{dt} not supported in VectorizedGroupValuesColumn"
+ )
+ }
+ }
+ }
+ self.group_values = v;
+ }
+
+ // tracks to which group each of the input rows belongs
+ groups.clear();
+ groups.resize(n_rows, usize::MAX);
+
+ let mut batch_hashes = mem::take(&mut self.hashes_buffer);
Review Comment:
๐
##########
datafusion/physical-plan/src/aggregates/group_values/mod.rs:
##########
@@ -143,8 +148,12 @@ pub fn new_group_values(schema: SchemaRef) -> Result<Box<dyn GroupValues>> {
}
}
- if GroupValuesColumn::supported_schema(schema.as_ref()) {
- Ok(Box::new(GroupValuesColumn::try_new(schema)?))
+ if column::supported_schema(schema.as_ref()) {
Review Comment:
Can you explain here why GroupOrdering::None is required? Is it because the `VectorizedGroupValuesColumn` doesn't keep the groups in order?
If that is the case, it seems like maybe `emit_n` would never be called ๐ค
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -35,29 +37,113 @@ use datafusion_common::{not_impl_err, DataFusionError, Result};
use datafusion_execution::memory_pool::proxy::{RawTableAllocExt, VecAllocExt};
use datafusion_expr::EmitTo;
use datafusion_physical_expr::binary_map::OutputType;
+
use hashbrown::raw::RawTable;
-use std::mem::size_of;
-/// A [`GroupValues`] that stores multiple columns of group values.
+const NON_INLINED_FLAG: u64 = 0x8000000000000000;
+const VALUE_MASK: u64 = 0x7FFFFFFFFFFFFFFF;
+
+/// The view of indices pointing to the actual values in `GroupValues`
///
+/// If only single `group index` represented by view,
+/// value of view is just the `group index`, and we call it a `inlined view`.
///
-pub struct GroupValuesColumn {
+/// If multiple `group indices` represented by view,
+/// value of view is the actually the index pointing to `group indices`,
+/// and we call it `non-inlined view`.
+///
+/// The view(a u64) format is like:
+/// +---------------------+---------------------------------------------+
+/// | inlined flag(1bit) | group index / index to group indices(63bit) |
+/// +---------------------+---------------------------------------------+
+///
+/// `inlined flag`: 1 represents `non-inlined`, and 0 represents `inlined`
+///
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+struct GroupIndexView(u64);
+
+impl GroupIndexView {
+ #[inline]
+ pub fn is_non_inlined(&self) -> bool {
+ (self.0 & NON_INLINED_FLAG) > 0
+ }
+
+ #[inline]
+ pub fn new_inlined(group_index: u64) -> Self {
+ Self(group_index)
+ }
+
+ #[inline]
+ pub fn new_non_inlined(list_offset: u64) -> Self {
+ let non_inlined_value = list_offset | NON_INLINED_FLAG;
+ Self(non_inlined_value)
+ }
+
+ #[inline]
+ pub fn value(&self) -> u64 {
+ self.0 & VALUE_MASK
+ }
+}
+
+/// A [`GroupValues`] that stores multiple columns of group values,
+/// and supports vectorized operators for them
+///
+pub struct VectorizedGroupValuesColumn {
/// The output schema
schema: SchemaRef,
/// Logically maps group values to a group_index in
/// [`Self::group_values`] and in each accumulator
///
- /// Uses the raw API of hashbrown to avoid actually storing the
- /// keys (group values) in the table
+ /// It is a `hashtable` based on `hashbrown`.
///
- /// keys: u64 hashes of the GroupValue
- /// values: (hash, group_index)
- map: RawTable<(u64, usize)>,
+ /// Key and value in the `hashtable`:
+ /// - The `key` is `hash value(u64)` of the `group value`
+ /// - The `value` is the `group values` with the same `hash value`
+ ///
+ /// We don't really store the actual `group values` in `hashtable`,
+ /// instead we store the `group indices` pointing to values in `GroupValues`.
+ /// And we use [`GroupIndexView`] to represent such `group indices` in table.
+ ///
+ ///
+ map: RawTable<(u64, GroupIndexView)>,
/// The size of `map` in bytes
map_size: usize,
+ /// The lists for group indices with the same hash value
+ ///
+ /// It is possible that hash value collision exists,
+ /// and we will chain the `group indices` with same hash value
+ ///
+ /// The chained indices is like:
+ /// `latest group index -> older group index -> even older group index -> ...`
+ ///
+ group_index_lists: Vec<Vec<usize>>,
+
+ /// When emitting first n, we need to decrease/erase group indices in
+ /// `map` and `group_index_lists`.
+ ///
+ /// This buffer is used to temporarily store the remaining group indices in
+ /// a specific list in `group_index_lists`.
+ emit_group_index_list_buffer: Vec<usize>,
+
+ /// Similar as `current_indices`, but `remaining_indices`
+ /// is used to store the rows will be processed in next round.
+ scalarized_indices: Vec<usize>,
+
+ /// The `vectorized_equal_tod` row indices buffer
Review Comment:
Maybe we can rename these to "buffer" or something to make it clear they are temp processing space to avoid re-allocations rather than
Something like
```rust
buffer_equal_to_row_indices: Vec<usize>,
```
Or maybe we can even put all the scratch space into their own struct to make it clear
```rust
struct ScratchSpace {
vectorized_equal_to_row_indices: Vec<usize>,
/// The `vectorized_equal_tod` group indices buffer
vectorized_equal_to_group_indices: Vec<usize>,
/// The `vectorized_equal_tod` result buffer
vectorized_equal_to_results: Vec<bool>,
/// The `vectorized append` row indices buffer
vectorized_append_row_indices: Vec<usize>,
}
```
Or something
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1824330952
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -356,3 +1048,461 @@ impl GroupValues for GroupValuesColumn {
self.hashes_buffer.shrink_to(count);
}
}
+
+/// Returns true if [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+/// supported for the specified schema
+pub fn supported_schema(schema: &Schema) -> bool {
+ schema
+ .fields()
+ .iter()
+ .map(|f| f.data_type())
+ .all(supported_type)
+}
+
+/// Returns true if the specified data type is supported by
+/// [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+///
+/// In order to be supported, there must be a specialized implementation of
+/// [`GroupColumn`] for the data type, instantiated in [`GroupValuesColumn::intern`]
+/// or [`VectorizedGroupValuesColumn::intern`]
+fn supported_type(data_type: &DataType) -> bool {
+ matches!(
+ *data_type,
+ DataType::Int8
+ | DataType::Int16
+ | DataType::Int32
+ | DataType::Int64
+ | DataType::UInt8
+ | DataType::UInt16
+ | DataType::UInt32
+ | DataType::UInt64
+ | DataType::Float32
+ | DataType::Float64
+ | DataType::Utf8
+ | DataType::LargeUtf8
+ | DataType::Binary
+ | DataType::LargeBinary
+ | DataType::Date32
+ | DataType::Date64
+ | DataType::Utf8View
+ | DataType::BinaryView
+ )
+}
+
+#[cfg(test)]
+mod tests {
+ use std::sync::Arc;
+
+ use arrow::{compute::concat_batches, util::pretty::pretty_format_batches};
+ use arrow_array::{ArrayRef, Int64Array, RecordBatch, StringArray, StringViewArray};
+ use arrow_schema::{DataType, Field, Schema, SchemaRef};
+ use datafusion_expr::EmitTo;
+
+ use crate::aggregates::group_values::{
+ column::VectorizedGroupValuesColumn, GroupValues,
+ };
+
+ #[test]
+ fn test_intern_for_vectorized_group_values() {
+ let data_set = VectorizedTestDataSet::new();
+ let mut group_values =
+ VectorizedGroupValuesColumn::try_new(data_set.schema()).unwrap();
+
+ data_set.load_to_group_values(&mut group_values);
+ let actual_batch = group_values.emit(EmitTo::All).unwrap();
+ let actual_batch = RecordBatch::try_new(data_set.schema(), actual_batch).unwrap();
+
+ check_result(&actual_batch, &data_set.expected_batch);
+ }
+
+ #[test]
+ fn test_emit_first_n_for_vectorized_group_values() {
Review Comment:
I try to print somethings in first n code, but didn't see anything, it seems the test here doesn't cover the code in first n ๐ค
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Support vectorized append and compare for multi group by [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #12996:
URL: https://github.com/apache/datafusion/pull/12996#discussion_r1824330952
##########
datafusion/physical-plan/src/aggregates/group_values/column.rs:
##########
@@ -356,3 +1048,461 @@ impl GroupValues for GroupValuesColumn {
self.hashes_buffer.shrink_to(count);
}
}
+
+/// Returns true if [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+/// supported for the specified schema
+pub fn supported_schema(schema: &Schema) -> bool {
+ schema
+ .fields()
+ .iter()
+ .map(|f| f.data_type())
+ .all(supported_type)
+}
+
+/// Returns true if the specified data type is supported by
+/// [`GroupValuesColumn`] or [`VectorizedGroupValuesColumn`]
+///
+/// In order to be supported, there must be a specialized implementation of
+/// [`GroupColumn`] for the data type, instantiated in [`GroupValuesColumn::intern`]
+/// or [`VectorizedGroupValuesColumn::intern`]
+fn supported_type(data_type: &DataType) -> bool {
+ matches!(
+ *data_type,
+ DataType::Int8
+ | DataType::Int16
+ | DataType::Int32
+ | DataType::Int64
+ | DataType::UInt8
+ | DataType::UInt16
+ | DataType::UInt32
+ | DataType::UInt64
+ | DataType::Float32
+ | DataType::Float64
+ | DataType::Utf8
+ | DataType::LargeUtf8
+ | DataType::Binary
+ | DataType::LargeBinary
+ | DataType::Date32
+ | DataType::Date64
+ | DataType::Utf8View
+ | DataType::BinaryView
+ )
+}
+
+#[cfg(test)]
+mod tests {
+ use std::sync::Arc;
+
+ use arrow::{compute::concat_batches, util::pretty::pretty_format_batches};
+ use arrow_array::{ArrayRef, Int64Array, RecordBatch, StringArray, StringViewArray};
+ use arrow_schema::{DataType, Field, Schema, SchemaRef};
+ use datafusion_expr::EmitTo;
+
+ use crate::aggregates::group_values::{
+ column::VectorizedGroupValuesColumn, GroupValues,
+ };
+
+ #[test]
+ fn test_intern_for_vectorized_group_values() {
+ let data_set = VectorizedTestDataSet::new();
+ let mut group_values =
+ VectorizedGroupValuesColumn::try_new(data_set.schema()).unwrap();
+
+ data_set.load_to_group_values(&mut group_values);
+ let actual_batch = group_values.emit(EmitTo::All).unwrap();
+ let actual_batch = RecordBatch::try_new(data_set.schema(), actual_batch).unwrap();
+
+ check_result(&actual_batch, &data_set.expected_batch);
+ }
+
+ #[test]
+ fn test_emit_first_n_for_vectorized_group_values() {
Review Comment:
This test doesn't go to non inline case, I think we need another test that covers this part
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org