[GitHub] [doris] zhengshengjun commented on a diff in pull request #10392: [Enhancement][Vectorized] Use SIMD to skip batches of null data in nu…

[GitBox <gi...@apache.org>]

zhengshengjun commented on code in PR #10392:
URL: https://github.com/apache/doris/pull/10392#discussion_r916411169


##########
be/src/vec/aggregate_functions/aggregate_function_null.h:
##########
@@ -197,9 +197,50 @@ class AggregateFunctionNullUnary final
         }
     }
 
+    void add_batch(size_t batch_size, AggregateDataPtr* places, size_t place_offset,
+                   const IColumn** columns, Arena* arena) const override {
+        int processed_records_num = 0;
+
+        // we can use column->has_null() to judge whether whole batch of data is null and skip batch,
+        // but it's maybe too coarse-grained.
+#ifdef __AVX2__
+        const ColumnNullable* column = assert_cast<const ColumnNullable*>(columns[0]);
+        // The overhead introduced is negligible here, just an extra memory read from NullMap
+        const NullMap& null_map_data = column -> get_null_map_data();
+
+        // NullMap use uint8_t type to indicate values is null or not, 1 indicates null, 0 versus.
+        // It's important to keep consistent with element type size in NullMap
+        constexpr int simd_batch_size = 256 / (8 * sizeof(uint8_t));
+        __m256i all0 = _mm256_setzero_si256();
+        auto to_read_null_map_position = reinterpret_cast<const int8_t*>(null_map_data.data());
+
+        while (processed_records_num + simd_batch_size < batch_size) {
+            to_read_null_map_position = to_read_null_map_position + processed_records_num;
+            // load unaligned data from null_map, 1 means value is null, 0 versus
+            __m256i f = _mm256_loadu_si256(reinterpret_cast<const __m256i*>(to_read_null_map_position));
+            int mask = _mm256_movemask_epi8(_mm256_cmpgt_epi8(f, all0));
+            // all data is null
+            if (mask == 0xffffffff) {
+            } else {

Review Comment:
   It's good to invoke function without null check. And for the 2nd, we have discussed that since we don't know the sparseness,  introducing  __builtin_ctzll may incur performace downgrade.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org