[GitHub] [doris] morningman commented on a diff in pull request #13122: [feature-wip](parquet-reader) optimize the performance of column conversion

[GitBox <gi...@apache.org>]

morningman commented on code in PR #13122:
URL: https://github.com/apache/doris/pull/13122#discussion_r989263208


##########
be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:
##########
@@ -338,14 +343,23 @@ Status ArrayColumnReader::_load_nested_column(ColumnPtr& doris_column, DataTypeP
     level_t* definitions = _def_levels_buf.get();
     auto* nullable_column = reinterpret_cast<vectorized::ColumnNullable*>(
             (*std::move(doris_column)).mutate().get());
-    NullMap& map_data = nullable_column->get_null_map_data();
+    NullMap& map_data_column = nullable_column->get_null_map_data();
     MutableColumnPtr data_column = nullable_column->get_nested_column_ptr();
+    size_t null_map_size = 0;
     for (int i = _def_offset; i < _def_offset + read_values; ++i) {
         // should skip _EMPTY_ARRAY and _NULL_ARRAY
+        if (definitions[i] == _CONCRETE_ELEMENT || definitions[i] == _NULL_ELEMENT) {
+            null_map_size++;
+        }
+    }
+    auto origin_size = map_data_column.size();
+    map_data_column.resize(origin_size + null_map_size);
+    size_t null_map_idx = origin_size;
+    for (int i = _def_offset; i < _def_offset + read_values; ++i) {
         if (definitions[i] == _CONCRETE_ELEMENT) {
-            map_data.emplace_back(false);
+            map_data_column[null_map_idx++] = (UInt8) false;
         } else if (definitions[i] == _NULL_ELEMENT) {
-            map_data.emplace_back(true);
+            map_data_column[null_map_idx++] = (UInt8) true;

Review Comment:
   What if `definitions[i]` not equal to `_CONCRETE_ELEMENT` and `_NULL_ELEMENT`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org