You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@orc.apache.org by "Taiyang Li (Jira)" <ji...@apache.org> on 2025/07/08 09:16:00 UTC
[jira] [Updated] (ORC-1950) [C++] Replace std::unorder_map with google dense_hash_map as SortedStringDictionary and remove reorder to improve write performance of dict-encoding columns
[ https://issues.apache.org/jira/browse/ORC-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Taiyang Li updated ORC-1950:
----------------------------
Description:
Replace std::unorder_map with google dense_hash_map as SortedStringDictionary and remove reorder to improve write performance of dict-encoding columns
POC: [https://github.com/bigo-sg/ClickHouse/commit/b9fc51fd8ded21f84f31cfa169350906b9f14456]
baseline:
{code:java}
2025-07-08T16:22:54+08:00
Running ./build_gcc/src/Common/benchmarks/orc_string_dictionary
Run on (96 X 2900 MHz CPU s)
CPU Caches:
L1 Data 32 KiB (x48)
L1 Instruction 32 KiB (x48)
L2 Unified 1024 KiB (x48)
L3 Unified 36608 KiB (x2)
Load Average: 27.44, 62.03, 43.39
Benchmark Time CPU Iterations
BM_writeStringDictionary<NewSortedStringDictionary, 10> 49801815 ns 49800922 ns 11
BM_writeStringDictionary<NewSortedStringDictionary, 100> 60295648 ns 60294001 ns 12
BM_writeStringDictionary<NewSortedStringDictionary, 1000> 73385081 ns 73383192 ns 10
BM_writeStringDictionary<NewSortedStringDictionary, 10000> 121725939 ns 121642493 ns 6
BM_writeStringDictionary<NewSortedStringDictionary, 100000> 232034759 ns 232031059 ns 3 {code}
Opt1:
> [C++] Replace std::unorder_map with google dense_hash_map as SortedStringDictionary and remove reorder to improve write performance of dict-encoding columns
> ------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ORC-1950
> URL: https://issues.apache.org/jira/browse/ORC-1950
> Project: ORC
> Issue Type: Bug
> Reporter: Taiyang Li
> Priority: Major
>
> Replace std::unorder_map with google dense_hash_map as SortedStringDictionary and remove reorder to improve write performance of dict-encoding columns
>
> POC: [https://github.com/bigo-sg/ClickHouse/commit/b9fc51fd8ded21f84f31cfa169350906b9f14456]
>
> baseline:
> {code:java}
> 2025-07-08T16:22:54+08:00
> Running ./build_gcc/src/Common/benchmarks/orc_string_dictionary
> Run on (96 X 2900 MHz CPU s)
> CPU Caches:
> L1 Data 32 KiB (x48)
> L1 Instruction 32 KiB (x48)
> L2 Unified 1024 KiB (x48)
> L3 Unified 36608 KiB (x2)
> Load Average: 27.44, 62.03, 43.39
> Benchmark Time CPU Iterations
> BM_writeStringDictionary<NewSortedStringDictionary, 10> 49801815 ns 49800922 ns 11
> BM_writeStringDictionary<NewSortedStringDictionary, 100> 60295648 ns 60294001 ns 12
> BM_writeStringDictionary<NewSortedStringDictionary, 1000> 73385081 ns 73383192 ns 10
> BM_writeStringDictionary<NewSortedStringDictionary, 10000> 121725939 ns 121642493 ns 6
> BM_writeStringDictionary<NewSortedStringDictionary, 100000> 232034759 ns 232031059 ns 3 {code}
> Opt1:
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)