You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "dataroaring (via GitHub)" <gi...@apache.org> on 2026/03/14 12:11:08 UTC
[PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
dataroaring opened a new pull request, #61332:
URL: https://github.com/apache/doris/pull/61332
## Summary
Add two thread-safe primitive-key concurrent hash maps built on fastutil, designed as drop-in replacements for `ConcurrentHashMap<Long, V>` and `ConcurrentHashMap<Long, Long>` in memory-sensitive FE paths.
- **`ConcurrentLong2ObjectHashMap<V>`** — replaces `ConcurrentHashMap<Long, V>`
- **`ConcurrentLong2LongHashMap`** — replaces `ConcurrentHashMap<Long, Long>`
### Why
`ConcurrentHashMap<Long, V>` costs ~64 bytes per entry due to Long boxing, Node wrapper, and segment overhead. These fastutil-based maps reduce that to ~16 bytes per entry — a **4x memory reduction**.
In Doris FE, several critical data structures use `ConcurrentHashMap<Long, V>` at tablet/partition scale (millions of entries), making this a significant memory optimization opportunity.
### Design
- **Segment-based locking** (default 16 segments) for concurrent throughput, similar to Java 7's ConcurrentHashMap design
- Full `Map` interface compatibility for drop-in replacement
- Atomic operations: `putIfAbsent`, `computeIfAbsent`, `replace`, `remove(key, value)`
- Thread-safe iteration via snapshot-based `entrySet()`/`keySet()`/`values()`
### Memory comparison
| Collection | Per-entry overhead | 1M entries |
|------------|-------------------|------------|
| `ConcurrentHashMap<Long, V>` | ~64 bytes | ~61 MB |
| `ConcurrentLong2ObjectHashMap<V>` | ~16 bytes | ~15 MB |
| `ConcurrentHashMap<Long, Long>` | ~80 bytes | ~76 MB |
| `ConcurrentLong2LongHashMap` | ~16 bytes | ~15 MB |
## Test plan
- [x] `ConcurrentLong2ObjectHashMapTest` — 432 lines covering put/get/remove, putIfAbsent, computeIfAbsent, replace, concurrent writes from multiple threads, iteration consistency, empty map edge cases
- [x] `ConcurrentLong2LongHashMapTest` — 455 lines covering CRUD, default value semantics, concurrent operations, atomic operations, iteration, edge cases
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4102816341
<details>
<summary>TPC-H: <b>Total hot run time: 26683 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b78116f98468bbe42f98a9d861eb8811f0c11190, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17653 4445 4272 4272
q2 q3 10650 750 513 513
q4 4671 369 248 248
q5 7563 1210 1029 1029
q6 175 174 145 145
q7 787 841 668 668
q8 9697 1456 1302 1302
q9 5132 4735 4732 4732
q10 6327 1912 1659 1659
q11 454 243 238 238
q12 754 584 476 476
q13 18043 3003 2162 2162
q14 231 237 216 216
q15 q16 738 729 664 664
q17 736 861 433 433
q18 5926 5421 5141 5141
q19 1179 972 630 630
q20 528 505 371 371
q21 4554 1825 1490 1490
q22 467 368 294 294
Total cold run time: 96265 ms
Total hot run time: 26683 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4752 4556 4686 4556
q2 q3 3846 4344 3803 3803
q4 891 1202 786 786
q5 4054 4350 4348 4348
q6 195 178 138 138
q7 1787 1667 1541 1541
q8 2498 2743 2628 2628
q9 7545 7417 7435 7417
q10 3791 3973 3627 3627
q11 525 453 447 447
q12 497 589 453 453
q13 2717 3328 2435 2435
q14 272 285 267 267
q15 q16 729 764 718 718
q17 1182 1364 1692 1364
q18 7278 6840 6616 6616
q19 964 886 976 886
q20 2149 2113 1982 1982
q21 3951 3481 3356 3356
q22 473 427 382 382
Total cold run time: 50096 ms
Total hot run time: 47750 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4124533934
# FE UT Coverage Report
Increment line coverage `` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/61332_5470db4e88116113f105c152c3fe0bd84a5faa84/fe_increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/61332_5470db4e88116113f105c152c3fe0bd84a5faa84/fe_report/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4134241521
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4073080316
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4066928943
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4060450552
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4074029285
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4134490794
<details>
<summary>TPC-H: <b>Total hot run time: 26309 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f49d1e7629aa582c8998bb32cd849a382b1bba31, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17663 4459 4333 4333
q2 q3 10649 758 515 515
q4 4678 343 247 247
q5 7563 1191 1001 1001
q6 173 174 143 143
q7 787 844 671 671
q8 9694 1508 1310 1310
q9 5363 4645 4650 4645
q10 6334 1903 1622 1622
q11 441 241 236 236
q12 743 565 456 456
q13 18031 2715 1965 1965
q14 224 239 208 208
q15 q16 738 716 662 662
q17 724 836 449 449
q18 6173 5357 5162 5162
q19 1150 977 628 628
q20 532 478 376 376
q21 4495 1820 1400 1400
q22 502 385 280 280
Total cold run time: 96657 ms
Total hot run time: 26309 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4800 4685 4591 4591
q2 q3 3890 4357 3847 3847
q4 936 1237 783 783
q5 4050 4372 4345 4345
q6 189 178 145 145
q7 1758 1604 1560 1560
q8 2501 2777 2585 2585
q9 7524 7327 7455 7327
q10 3835 4022 3565 3565
q11 513 426 422 422
q12 484 573 450 450
q13 2476 3144 2322 2322
q14 287 311 286 286
q15 q16 732 776 724 724
q17 1186 1404 1377 1377
q18 7048 6770 6787 6770
q19 982 964 936 936
q20 2071 2150 2022 2022
q21 3901 3457 3289 3289
q22 434 415 379 379
Total cold run time: 49597 ms
Total hot run time: 47725 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #61332:
URL: https://github.com/apache/doris/pull/61332#discussion_r2935224695
##########
fe/fe-core/src/main/java/org/apache/doris/common/ConcurrentLong2LongHashMap.java:
##########
@@ -0,0 +1,486 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common;
+
+import it.unimi.dsi.fastutil.HashCommon;
+import it.unimi.dsi.fastutil.longs.AbstractLong2LongMap;
+import it.unimi.dsi.fastutil.longs.Long2LongFunction;
+import it.unimi.dsi.fastutil.longs.Long2LongMap;
+import it.unimi.dsi.fastutil.longs.Long2LongOpenHashMap;
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+import it.unimi.dsi.fastutil.longs.LongBinaryOperator;
+import it.unimi.dsi.fastutil.longs.LongOpenHashSet;
+import it.unimi.dsi.fastutil.longs.LongSet;
+import it.unimi.dsi.fastutil.objects.ObjectArrayList;
+import it.unimi.dsi.fastutil.objects.ObjectOpenHashSet;
+import it.unimi.dsi.fastutil.objects.ObjectSet;
+
+import java.util.Map;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+import java.util.function.LongUnaryOperator;
+
+/**
+ * A concurrent map with primitive long keys and primitive long values, backed by segmented
+ * {@link Long2LongOpenHashMap} instances with {@link ReentrantReadWriteLock} per segment.
+ *
+ * <p>This class saves ~48 bytes per entry compared to {@code ConcurrentHashMap<Long, Long>}
+ * by avoiding boxing of both keys and values. For fields like partition update row counts
+ * with millions of entries, this translates to hundreds of MB of heap savings.
+ *
+ * <p>The {@link #addTo(long, long)} method provides atomic increment semantics, useful for
+ * counter patterns.
+ *
+ * <p><b>Important:</b> All compound operations from both {@link Long2LongMap} and {@link Map}
+ * interfaces are overridden to ensure atomicity within a segment's write lock.
Review Comment:
Similar to the object map: the Javadoc claims all compound operations from both `Long2LongMap` and `Map` are overridden for atomicity, but boxed `Map`-level compound methods like `putIfAbsent(Long,Long)`, `remove(Object,Object)`, `computeIfPresent(Long,...)`, `merge(Long,...)`, etc. are not implemented in this class. Either add explicit overrides that lock once per segment, or narrow the documentation so it matches actual coverage.
##########
fe/fe-core/src/main/java/org/apache/doris/common/ConcurrentLong2LongHashMap.java:
##########
@@ -0,0 +1,486 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common;
+
+import it.unimi.dsi.fastutil.HashCommon;
+import it.unimi.dsi.fastutil.longs.AbstractLong2LongMap;
+import it.unimi.dsi.fastutil.longs.Long2LongFunction;
+import it.unimi.dsi.fastutil.longs.Long2LongMap;
+import it.unimi.dsi.fastutil.longs.Long2LongOpenHashMap;
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+import it.unimi.dsi.fastutil.longs.LongBinaryOperator;
+import it.unimi.dsi.fastutil.longs.LongOpenHashSet;
+import it.unimi.dsi.fastutil.longs.LongSet;
+import it.unimi.dsi.fastutil.objects.ObjectArrayList;
Review Comment:
Unused imports `it.unimi.dsi.fastutil.longs.LongBinaryOperator` and `it.unimi.dsi.fastutil.objects.ObjectArrayList` will fail the FE checkstyle `UnusedImports` rule; please remove them (the code uses `java.util.function.LongBinaryOperator` fully-qualified and never uses `ObjectArrayList`).
##########
fe/fe-core/src/main/java/org/apache/doris/common/ConcurrentLong2ObjectHashMap.java:
##########
@@ -0,0 +1,443 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common;
+
+import it.unimi.dsi.fastutil.HashCommon;
+import it.unimi.dsi.fastutil.longs.AbstractLong2ObjectMap;
+import it.unimi.dsi.fastutil.longs.Long2ObjectFunction;
+import it.unimi.dsi.fastutil.longs.Long2ObjectMap;
+import it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap;
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+import it.unimi.dsi.fastutil.longs.LongOpenHashSet;
+import it.unimi.dsi.fastutil.longs.LongSet;
+import it.unimi.dsi.fastutil.objects.ObjectArrayList;
+import it.unimi.dsi.fastutil.objects.ObjectCollection;
+import it.unimi.dsi.fastutil.objects.ObjectOpenHashSet;
+import it.unimi.dsi.fastutil.objects.ObjectSet;
+
+import java.util.Map;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+import java.util.function.LongFunction;
+
+/**
+ * A concurrent map with primitive long keys and object values, backed by segmented
+ * {@link Long2ObjectOpenHashMap} instances with {@link ReentrantReadWriteLock} per segment.
+ *
+ * <p>This class provides similar concurrency guarantees to {@link java.util.concurrent.ConcurrentHashMap}
+ * while avoiding the memory overhead of boxing long keys. For a cluster with millions of tablet entries,
+ * this saves ~32 bytes per entry compared to {@code ConcurrentHashMap<Long, V>}.
+ *
+ * <p>Iteration methods ({@link #long2ObjectEntrySet()}, {@link #keySet()}, {@link #values()})
+ * return snapshot copies and are weakly consistent.
+ *
+ * <p><b>Important:</b> All compound operations (computeIfAbsent, computeIfPresent, compute, merge)
+ * from both {@link Long2ObjectMap} and {@link Map} interfaces are overridden to ensure atomicity
+ * within a segment. The default interface implementations would call get/put as separate locked
+ * operations, breaking atomicity.
+ *
+ * @param <V> the type of mapped values
+ */
+public class ConcurrentLong2ObjectHashMap<V> extends AbstractLong2ObjectMap<V> {
+
+ private static final int DEFAULT_SEGMENT_COUNT = 16;
+ private static final int DEFAULT_INITIAL_CAPACITY_PER_SEGMENT = 16;
+
+ private final Segment<V>[] segments;
+ private final int segmentMask;
+ private final int segmentBits;
+
+ public ConcurrentLong2ObjectHashMap() {
+ this(DEFAULT_SEGMENT_COUNT);
+ }
+
+ @SuppressWarnings("unchecked")
+ public ConcurrentLong2ObjectHashMap(int segmentCount) {
+ if (segmentCount <= 0 || (segmentCount & (segmentCount - 1)) != 0) {
+ throw new IllegalArgumentException("segmentCount must be a positive power of 2: " + segmentCount);
+ }
+ this.segmentBits = Integer.numberOfTrailingZeros(segmentCount);
+ this.segmentMask = segmentCount - 1;
+ this.segments = new Segment[segmentCount];
+ for (int i = 0; i < segmentCount; i++) {
+ segments[i] = new Segment<>(DEFAULT_INITIAL_CAPACITY_PER_SEGMENT);
+ }
+ }
+
+ private Segment<V> segmentFor(long key) {
+ return segments[(int) (HashCommon.mix(key) >>> (64 - segmentBits)) & segmentMask];
+ }
+
+ // ---- Read operations (read-lock) ----
+
+ @Override
+ public V get(long key) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.readLock().lock();
+ try {
+ return seg.map.get(key);
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+
+ @Override
+ public V getOrDefault(long key, V defaultValue) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.readLock().lock();
+ try {
+ return seg.map.getOrDefault(key, defaultValue);
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+
+ @Override
+ public boolean containsKey(long key) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.readLock().lock();
+ try {
+ return seg.map.containsKey(key);
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+
+ @Override
+ public boolean containsValue(Object value) {
+ for (Segment<V> seg : segments) {
+ seg.lock.readLock().lock();
+ try {
+ if (seg.map.containsValue(value)) {
+ return true;
+ }
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public int size() {
+ long total = 0;
+ for (Segment<V> seg : segments) {
+ seg.lock.readLock().lock();
+ try {
+ total += seg.map.size();
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+ return (int) Math.min(total, Integer.MAX_VALUE);
+ }
+
+ @Override
+ public boolean isEmpty() {
+ for (Segment<V> seg : segments) {
+ seg.lock.readLock().lock();
+ try {
+ if (!seg.map.isEmpty()) {
+ return false;
+ }
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+ return true;
+ }
+
+ // ---- Write operations (write-lock) ----
+
+ @Override
+ public V put(long key, V value) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.put(key, value);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public V remove(long key) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.remove(key);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public V putIfAbsent(long key, V value) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.putIfAbsent(key, value);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public boolean replace(long key, V oldValue, V newValue) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.replace(key, oldValue, newValue);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public V replace(long key, V value) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.replace(key, value);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public void clear() {
+ for (Segment<V> seg : segments) {
+ seg.lock.writeLock().lock();
+ try {
+ seg.map.clear();
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+ }
+
+ @Override
+ public void putAll(Map<? extends Long, ? extends V> m) {
+ for (Map.Entry<? extends Long, ? extends V> entry : m.entrySet()) {
+ put(entry.getKey().longValue(), entry.getValue());
+ }
+ }
+
+ // ---- Atomic compound operations ----
+ // Override ALL compound methods from both Long2ObjectMap and Map interfaces
+ // to ensure the check-then-act is atomic within a segment's write lock.
+
+ @Override
+ public V computeIfAbsent(long key, LongFunction<? extends V> mappingFunction) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ V val = seg.map.get(key);
+ if (val != null || seg.map.containsKey(key)) {
+ return val;
+ }
+ V newValue = mappingFunction.apply(key);
+ seg.map.put(key, newValue);
+ return newValue;
Review Comment:
`computeIfAbsent` should not insert a mapping when the mapping function returns null (per `Map.computeIfAbsent` contract). Currently a null return is still `put` into the underlying map, which also makes absent-vs-null-mapped semantics ambiguous.
##########
fe/fe-core/src/test/java/org/apache/doris/common/ConcurrentLong2LongHashMapTest.java:
##########
@@ -0,0 +1,455 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common;
+
+import com.google.gson.Gson;
+import it.unimi.dsi.fastutil.longs.Long2LongMap;
+import it.unimi.dsi.fastutil.longs.LongSet;
+import it.unimi.dsi.fastutil.objects.ObjectSet;
+import org.apache.doris.persist.gson.GsonUtils;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicInteger;
+
+class ConcurrentLong2LongHashMapTest {
+
+ @Test
+ void testPutAndGet() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(1L, 100L);
+ Assertions.assertEquals(100L, map.get(1L));
+ map.put(1L, 200L);
+ Assertions.assertEquals(200L, map.get(1L));
+ }
+
+ @Test
+ void testGetMissingKeyReturnsDefaultReturnValue() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ // Default return value is 0L
+ Assertions.assertEquals(0L, map.get(999L));
+ }
+
+ @Test
+ void testGetOrDefault() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(1L, 100L);
+ Assertions.assertEquals(100L, map.getOrDefault(1L, -1L));
+ Assertions.assertEquals(-1L, map.getOrDefault(2L, -1L));
+ }
+
+ @Test
+ void testRemove() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(1L, 100L);
+ Assertions.assertEquals(100L, map.remove(1L));
+ Assertions.assertFalse(map.containsKey(1L));
+ // Remove non-existent key returns defaultReturnValue
+ Assertions.assertEquals(0L, map.remove(1L));
+ }
+
+ @Test
+ void testContainsKey() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ Assertions.assertFalse(map.containsKey(1L));
+ map.put(1L, 0L);
+ Assertions.assertTrue(map.containsKey(1L));
+ }
+
+ @Test
+ void testContainsValue() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(1L, 100L);
+ map.put(2L, 200L);
+ Assertions.assertTrue(map.containsValue(100L));
+ Assertions.assertFalse(map.containsValue(300L));
+ }
+
+ @Test
+ void testSizeAndIsEmpty() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ Assertions.assertTrue(map.isEmpty());
+ Assertions.assertEquals(0, map.size());
+ map.put(1L, 100L);
+ map.put(2L, 200L);
+ Assertions.assertFalse(map.isEmpty());
+ Assertions.assertEquals(2, map.size());
+ }
+
+ @Test
+ void testClear() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(1L, 100L);
+ map.put(2L, 200L);
+ map.clear();
+ Assertions.assertTrue(map.isEmpty());
+ Assertions.assertEquals(0, map.size());
+ }
+
+ @Test
+ void testPutAll() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ Map<Long, Long> source = new HashMap<>();
+ source.put(1L, 100L);
+ source.put(2L, 200L);
+ source.put(3L, 300L);
+ map.putAll(source);
+ Assertions.assertEquals(3, map.size());
+ Assertions.assertEquals(200L, map.get(2L));
+ }
+
+ @Test
+ void testPutIfAbsent() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ Assertions.assertEquals(0L, map.putIfAbsent(1L, 100L));
+ Assertions.assertEquals(100L, map.putIfAbsent(1L, 200L));
+ Assertions.assertEquals(100L, map.get(1L));
+ }
+
+ @Test
+ void testComputeIfAbsent() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ long val = map.computeIfAbsent(1L, k -> k * 10);
+ Assertions.assertEquals(10L, val);
+ // Should not recompute
+ long val2 = map.computeIfAbsent(1L, k -> k * 20);
+ Assertions.assertEquals(10L, val2);
+ }
+
+ // ---- addTo tests ----
+
+ @Test
+ void testAddToNewKey() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ long result = map.addTo(1L, 5L);
+ Assertions.assertEquals(5L, result);
+ Assertions.assertEquals(5L, map.get(1L));
+ }
+
+ @Test
+ void testAddToExistingKey() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(1L, 10L);
+ long result = map.addTo(1L, 5L);
+ Assertions.assertEquals(15L, result);
+ Assertions.assertEquals(15L, map.get(1L));
+ }
+
+ @Test
+ void testAddToNegative() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(1L, 10L);
+ long result = map.addTo(1L, -3L);
+ Assertions.assertEquals(7L, result);
+ }
+
+ // ---- Iteration tests ----
+
+ @Test
+ void testEntrySet() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(1L, 100L);
+ map.put(2L, 200L);
+
+ ObjectSet<Long2LongMap.Entry> entries = map.long2LongEntrySet();
+ Assertions.assertEquals(2, entries.size());
+
+ Set<Long> keys = new HashSet<>();
+ for (Long2LongMap.Entry entry : entries) {
+ keys.add(entry.getLongKey());
+ }
+ Assertions.assertTrue(keys.contains(1L));
+ Assertions.assertTrue(keys.contains(2L));
+ }
+
+ @Test
+ void testKeySet() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(10L, 100L);
+ map.put(20L, 200L);
+ LongSet keys = map.keySet();
+ Assertions.assertEquals(2, keys.size());
+ Assertions.assertTrue(keys.contains(10L));
+ Assertions.assertTrue(keys.contains(20L));
+ }
+
+ @Test
+ void testValues() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(1L, 100L);
+ map.put(2L, 200L);
+ it.unimi.dsi.fastutil.longs.LongCollection values = map.values();
+ Assertions.assertEquals(2, values.size());
+ Assertions.assertTrue(values.contains(100L));
+ Assertions.assertTrue(values.contains(200L));
+ }
+
+ @Test
+ void testForEach() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(1L, 100L);
+ map.put(2L, 200L);
+ Map<Long, Long> collected = new HashMap<>();
+ map.forEach(collected::put);
+ Assertions.assertEquals(2, collected.size());
+ Assertions.assertEquals(100L, (long) collected.get(1L));
+ }
+
+ // ---- Large map test ----
+
+ @Test
+ void testLargeMap() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ int count = 100_000;
+ for (long i = 0; i < count; i++) {
+ map.put(i, i * 3);
+ }
+ Assertions.assertEquals(count, map.size());
+ for (long i = 0; i < count; i++) {
+ Assertions.assertEquals(i * 3, map.get(i));
+ }
+ }
+
+ @Test
+ void testCustomSegmentCount() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap(4);
+ for (long i = 0; i < 1000; i++) {
+ map.put(i, i);
+ }
+ Assertions.assertEquals(1000, map.size());
+ }
+
+ @Test
+ void testInvalidSegmentCount() {
+ Assertions.assertThrows(IllegalArgumentException.class, () -> new ConcurrentLong2LongHashMap(3));
+ Assertions.assertThrows(IllegalArgumentException.class, () -> new ConcurrentLong2LongHashMap(0));
+ }
+
+ // ---- Concurrency tests ----
+
+ @Test
+ void testConcurrentPuts() throws Exception {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ int threads = 8;
+ int keysPerThread = 10_000;
+ ExecutorService executor = Executors.newFixedThreadPool(threads);
+ CountDownLatch latch = new CountDownLatch(threads);
+
+ for (int t = 0; t < threads; t++) {
+ final int threadId = t;
+ executor.submit(() -> {
+ for (int i = 0; i < keysPerThread; i++) {
+ long key = (long) threadId * keysPerThread + i;
+ map.put(key, key * 2);
+ }
+ latch.countDown();
+ });
+ }
+ latch.await();
+ executor.shutdown();
+ Assertions.assertEquals(threads * keysPerThread, map.size());
+ }
+
+ @Test
+ void testConcurrentAddTo() throws Exception {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ int threads = 16;
+ int incrementsPerThread = 10_000;
+ long key = 42L;
+ ExecutorService executor = Executors.newFixedThreadPool(threads);
+ CountDownLatch latch = new CountDownLatch(threads);
+
+ for (int t = 0; t < threads; t++) {
+ executor.submit(() -> {
+ for (int i = 0; i < incrementsPerThread; i++) {
+ map.addTo(key, 1L);
+ }
+ latch.countDown();
+ });
+ }
+ latch.await();
+ executor.shutdown();
+
+ Assertions.assertEquals((long) threads * incrementsPerThread, map.get(key));
+ }
+
+ @Test
+ void testConcurrentReadWrite() throws Exception {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ for (long i = 0; i < 1000; i++) {
+ map.put(i, i);
+ }
+
+ int threads = 8;
+ ExecutorService executor = Executors.newFixedThreadPool(threads);
+ CountDownLatch latch = new CountDownLatch(threads);
+ AtomicInteger errors = new AtomicInteger();
+
+ for (int t = 0; t < threads; t++) {
+ final int threadId = t;
+ executor.submit(() -> {
+ try {
+ for (int i = 0; i < 5000; i++) {
+ long key = i % 1000;
+ if (threadId % 2 == 0) {
+ map.get(key);
+ map.containsKey(key);
+ } else {
+ map.put(key + 1000L * threadId, (long) i);
+ }
+ }
+ } catch (Exception e) {
+ errors.incrementAndGet();
+ } finally {
+ latch.countDown();
+ }
+ });
+ }
+ latch.await();
+ executor.shutdown();
+ Assertions.assertEquals(0, errors.get());
+ }
+
+ @Test
+ void testConcurrentComputeIfAbsent() throws Exception {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ int threads = 16;
+ long sharedKey = 42L;
+ AtomicInteger computeCount = new AtomicInteger();
+ ExecutorService executor = Executors.newFixedThreadPool(threads);
+ List<Future<Long>> futures = new ArrayList<>();
+
+ for (int t = 0; t < threads; t++) {
+ futures.add(executor.submit(() ->
+ map.computeIfAbsent(sharedKey, k -> {
+ computeCount.incrementAndGet();
+ return k * 10;
+ })
+ ));
+ }
+ Set<Long> results = new HashSet<>();
+ for (Future<Long> f : futures) {
+ results.add(f.get());
+ }
+ executor.shutdown();
+
+ Assertions.assertEquals(1, results.size());
+ Assertions.assertTrue(results.contains(420L));
+ Assertions.assertEquals(1, computeCount.get());
+ }
+
+ @Test
+ void testConcurrentIterationDuringModification() throws Exception {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ for (long i = 0; i < 1000; i++) {
+ map.put(i, i);
+ }
+
+ int threads = 4;
+ ExecutorService executor = Executors.newFixedThreadPool(threads);
+ CountDownLatch latch = new CountDownLatch(threads);
+ AtomicInteger errors = new AtomicInteger();
+
+ for (int t = 0; t < threads; t++) {
+ final int threadId = t;
+ executor.submit(() -> {
+ try {
+ for (int i = 0; i < 100; i++) {
+ if (threadId % 2 == 0) {
+ map.keySet();
+ map.values();
+ map.long2LongEntrySet();
+ } else {
+ map.put(1000L + threadId * 100 + i, (long) i);
+ map.remove((long) (i % 500));
+ }
+ }
+ } catch (Exception e) {
+ errors.incrementAndGet();
+ } finally {
+ latch.countDown();
+ }
+ });
+ }
+ latch.await();
+ executor.shutdown();
+ Assertions.assertEquals(0, errors.get());
+ }
+
+ // ---- Gson serialization tests ----
+
+ @Test
+ void testGsonRoundTrip() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ map.put(100L, 1000L);
+ map.put(200L, 2000L);
+
+ String json = GsonUtils.GSON.toJson(map);
+
+ ConcurrentLong2LongHashMap deserialized = GsonUtils.GSON.fromJson(json, ConcurrentLong2LongHashMap.class);
+
+ Assertions.assertEquals(2, deserialized.size());
+ Assertions.assertEquals(1000L, deserialized.get(100L));
+ Assertions.assertEquals(2000L, deserialized.get(200L));
+ }
+
+ @Test
+ void testGsonFormatCompatibleWithConcurrentHashMap() {
+ ConcurrentHashMap<Long, Long> chm = new ConcurrentHashMap<>();
+ chm.put(1L, 100L);
+ chm.put(2L, 200L);
+ String chmJson = new Gson().toJson(chm);
+
+ ConcurrentLong2LongHashMap fastMap = new ConcurrentLong2LongHashMap();
+ fastMap.put(1L, 100L);
+ fastMap.put(2L, 200L);
+ String fastJson = GsonUtils.GSON.toJson(fastMap);
+
+ Gson gson = new Gson();
+ Map<?, ?> chmParsed = gson.fromJson(chmJson, Map.class);
+ Map<?, ?> fastParsed = gson.fromJson(fastJson, Map.class);
+ Assertions.assertEquals(chmParsed, fastParsed);
+ }
+
+ @Test
+ void testDefaultReturnValueBehavior() {
+ ConcurrentLong2LongHashMap map = new ConcurrentLong2LongHashMap();
+ // Primitive get returns 0L (defaultReturnValue) for missing keys
+ Assertions.assertEquals(0L, map.get(999L));
+
+ // Store 0L explicitly
+ map.put(1L, 0L);
+ Assertions.assertTrue(map.containsKey(1L));
+ Assertions.assertEquals(0L, map.get(1L));
+
+ // Boxed get via Map<Long,Long> interface returns null for missing keys
+ Long boxedResult = map.getOrDefault(999L, map.defaultReturnValue());
+ Assertions.assertEquals(0L, boxedResult);
Review Comment:
The comment says “Boxed get via Map<Long,Long> interface returns null for missing keys”, but the code uses `getOrDefault(...)` which never returns null (it returns the provided default). Either adjust the comment or add an assertion that actually exercises `((Map<Long, Long>) map).get(999L)` if that behavior is important.
##########
fe/fe-core/src/main/java/org/apache/doris/common/ConcurrentLong2LongHashMap.java:
##########
@@ -0,0 +1,486 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common;
+
+import it.unimi.dsi.fastutil.HashCommon;
+import it.unimi.dsi.fastutil.longs.AbstractLong2LongMap;
+import it.unimi.dsi.fastutil.longs.Long2LongFunction;
+import it.unimi.dsi.fastutil.longs.Long2LongMap;
+import it.unimi.dsi.fastutil.longs.Long2LongOpenHashMap;
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+import it.unimi.dsi.fastutil.longs.LongBinaryOperator;
+import it.unimi.dsi.fastutil.longs.LongOpenHashSet;
+import it.unimi.dsi.fastutil.longs.LongSet;
+import it.unimi.dsi.fastutil.objects.ObjectArrayList;
+import it.unimi.dsi.fastutil.objects.ObjectOpenHashSet;
+import it.unimi.dsi.fastutil.objects.ObjectSet;
+
+import java.util.Map;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+import java.util.function.LongUnaryOperator;
+
+/**
+ * A concurrent map with primitive long keys and primitive long values, backed by segmented
+ * {@link Long2LongOpenHashMap} instances with {@link ReentrantReadWriteLock} per segment.
+ *
+ * <p>This class saves ~48 bytes per entry compared to {@code ConcurrentHashMap<Long, Long>}
+ * by avoiding boxing of both keys and values. For fields like partition update row counts
+ * with millions of entries, this translates to hundreds of MB of heap savings.
+ *
+ * <p>The {@link #addTo(long, long)} method provides atomic increment semantics, useful for
+ * counter patterns.
+ *
+ * <p><b>Important:</b> All compound operations from both {@link Long2LongMap} and {@link Map}
+ * interfaces are overridden to ensure atomicity within a segment's write lock.
+ */
+public class ConcurrentLong2LongHashMap extends AbstractLong2LongMap {
+
+ private static final int DEFAULT_SEGMENT_COUNT = 16;
+ private static final int DEFAULT_INITIAL_CAPACITY_PER_SEGMENT = 16;
+
+ private final Segment[] segments;
+ private final int segmentMask;
+ private final int segmentBits;
+
+ public ConcurrentLong2LongHashMap() {
+ this(DEFAULT_SEGMENT_COUNT);
+ }
+
+ public ConcurrentLong2LongHashMap(int segmentCount) {
+ if (segmentCount <= 0 || (segmentCount & (segmentCount - 1)) != 0) {
+ throw new IllegalArgumentException("segmentCount must be a positive power of 2: " + segmentCount);
+ }
+ this.segmentBits = Integer.numberOfTrailingZeros(segmentCount);
+ this.segmentMask = segmentCount - 1;
+ this.segments = new Segment[segmentCount];
+ for (int i = 0; i < segmentCount; i++) {
+ segments[i] = new Segment(DEFAULT_INITIAL_CAPACITY_PER_SEGMENT);
+ }
+ }
+
+ private Segment segmentFor(long key) {
+ return segments[(int) (HashCommon.mix(key) >>> (64 - segmentBits)) & segmentMask];
+ }
+
+ // ---- Read operations (read-lock) ----
+
+ @Override
+ public long get(long key) {
+ Segment seg = segmentFor(key);
+ seg.lock.readLock().lock();
+ try {
+ return seg.map.get(key);
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+
+ @Override
+ public long getOrDefault(long key, long defaultValue) {
+ Segment seg = segmentFor(key);
+ seg.lock.readLock().lock();
+ try {
+ return seg.map.getOrDefault(key, defaultValue);
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+
+ @Override
+ public boolean containsKey(long key) {
+ Segment seg = segmentFor(key);
+ seg.lock.readLock().lock();
+ try {
+ return seg.map.containsKey(key);
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+
+ @Override
+ public boolean containsValue(long value) {
+ for (Segment seg : segments) {
+ seg.lock.readLock().lock();
+ try {
+ if (seg.map.containsValue(value)) {
+ return true;
+ }
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public int size() {
+ long total = 0;
+ for (Segment seg : segments) {
+ seg.lock.readLock().lock();
+ try {
+ total += seg.map.size();
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+ return (int) Math.min(total, Integer.MAX_VALUE);
+ }
+
+ @Override
+ public boolean isEmpty() {
+ for (Segment seg : segments) {
+ seg.lock.readLock().lock();
+ try {
+ if (!seg.map.isEmpty()) {
+ return false;
+ }
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+ return true;
+ }
+
+ // ---- Write operations (write-lock) ----
+
+ @Override
+ public long put(long key, long value) {
+ Segment seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.put(key, value);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public long remove(long key) {
+ Segment seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.remove(key);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public long putIfAbsent(long key, long value) {
+ Segment seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.putIfAbsent(key, value);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public boolean replace(long key, long oldValue, long newValue) {
+ Segment seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.replace(key, oldValue, newValue);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public long replace(long key, long value) {
+ Segment seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.replace(key, value);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public void clear() {
+ for (Segment seg : segments) {
+ seg.lock.writeLock().lock();
+ try {
+ seg.map.clear();
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+ }
+
+ @Override
+ public void putAll(Map<? extends Long, ? extends Long> m) {
+ for (Map.Entry<? extends Long, ? extends Long> entry : m.entrySet()) {
+ put(entry.getKey().longValue(), entry.getValue().longValue());
+ }
+ }
+
+ // ---- Atomic compound operations ----
+ // Override ALL compound methods from both Long2LongMap and Map interfaces.
+
+ /**
+ * Atomically adds the given increment to the value associated with the key.
+ * If the key is not present, the entry is created with the increment as value
+ * (starting from defaultReturnValue, which is 0L by default).
+ *
+ * @return the new value after the increment
+ */
+ public long addTo(long key, long increment) {
+ Segment seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ long newValue = seg.map.addTo(key, increment) + increment;
+ return newValue;
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public long computeIfAbsent(long key, LongUnaryOperator mappingFunction) {
+ Segment seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ if (seg.map.containsKey(key)) {
+ return seg.map.get(key);
+ }
+ long newValue = mappingFunction.applyAsLong(key);
+ seg.map.put(key, newValue);
+ return newValue;
+ } finally {
Review Comment:
`ConcurrentLong2LongHashMap` extends fastutil's function types, so callers can change `defaultReturnValue(...)`. The wrapper currently calls `seg.map.get/remove/...` directly, but each segment's `Long2LongOpenHashMap` defaultReturnValue remains its own (0 by default), so the wrapper's `defaultReturnValue()` can diverge from actual read/remove behavior. Consider overriding `defaultReturnValue(long)` to propagate into all segments (and ensure reads/removes reflect it), or explicitly document/guard that the default return value is fixed at 0.
##########
fe/fe-core/src/main/java/org/apache/doris/common/ConcurrentLong2ObjectHashMap.java:
##########
@@ -0,0 +1,443 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common;
+
+import it.unimi.dsi.fastutil.HashCommon;
+import it.unimi.dsi.fastutil.longs.AbstractLong2ObjectMap;
+import it.unimi.dsi.fastutil.longs.Long2ObjectFunction;
+import it.unimi.dsi.fastutil.longs.Long2ObjectMap;
+import it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap;
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+import it.unimi.dsi.fastutil.longs.LongOpenHashSet;
+import it.unimi.dsi.fastutil.longs.LongSet;
+import it.unimi.dsi.fastutil.objects.ObjectArrayList;
+import it.unimi.dsi.fastutil.objects.ObjectCollection;
+import it.unimi.dsi.fastutil.objects.ObjectOpenHashSet;
+import it.unimi.dsi.fastutil.objects.ObjectSet;
+
+import java.util.Map;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+import java.util.function.LongFunction;
+
+/**
+ * A concurrent map with primitive long keys and object values, backed by segmented
+ * {@link Long2ObjectOpenHashMap} instances with {@link ReentrantReadWriteLock} per segment.
+ *
+ * <p>This class provides similar concurrency guarantees to {@link java.util.concurrent.ConcurrentHashMap}
+ * while avoiding the memory overhead of boxing long keys. For a cluster with millions of tablet entries,
+ * this saves ~32 bytes per entry compared to {@code ConcurrentHashMap<Long, V>}.
+ *
+ * <p>Iteration methods ({@link #long2ObjectEntrySet()}, {@link #keySet()}, {@link #values()})
+ * return snapshot copies and are weakly consistent.
+ *
+ * <p><b>Important:</b> All compound operations (computeIfAbsent, computeIfPresent, compute, merge)
+ * from both {@link Long2ObjectMap} and {@link Map} interfaces are overridden to ensure atomicity
Review Comment:
The class Javadoc states that *all* compound operations from both `Long2ObjectMap` and `Map` are overridden for atomicity, but key `Map`-level compound methods (e.g. `remove(Object key, Object value)`, `putIfAbsent(Long, V)`, `computeIfPresent(Long, ...)`, `merge(Long, ...)`, etc.) are not implemented in this class. Either add explicit overrides that take the segment write-lock once, or narrow the documentation to avoid over-promising atomicity.
##########
fe/fe-core/src/main/java/org/apache/doris/common/ConcurrentLong2ObjectHashMap.java:
##########
@@ -0,0 +1,443 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common;
+
+import it.unimi.dsi.fastutil.HashCommon;
+import it.unimi.dsi.fastutil.longs.AbstractLong2ObjectMap;
+import it.unimi.dsi.fastutil.longs.Long2ObjectFunction;
+import it.unimi.dsi.fastutil.longs.Long2ObjectMap;
+import it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap;
+import it.unimi.dsi.fastutil.longs.LongArrayList;
+import it.unimi.dsi.fastutil.longs.LongOpenHashSet;
+import it.unimi.dsi.fastutil.longs.LongSet;
+import it.unimi.dsi.fastutil.objects.ObjectArrayList;
+import it.unimi.dsi.fastutil.objects.ObjectCollection;
+import it.unimi.dsi.fastutil.objects.ObjectOpenHashSet;
+import it.unimi.dsi.fastutil.objects.ObjectSet;
+
+import java.util.Map;
+import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.function.BiFunction;
+import java.util.function.Function;
+import java.util.function.LongFunction;
+
+/**
+ * A concurrent map with primitive long keys and object values, backed by segmented
+ * {@link Long2ObjectOpenHashMap} instances with {@link ReentrantReadWriteLock} per segment.
+ *
+ * <p>This class provides similar concurrency guarantees to {@link java.util.concurrent.ConcurrentHashMap}
+ * while avoiding the memory overhead of boxing long keys. For a cluster with millions of tablet entries,
+ * this saves ~32 bytes per entry compared to {@code ConcurrentHashMap<Long, V>}.
+ *
+ * <p>Iteration methods ({@link #long2ObjectEntrySet()}, {@link #keySet()}, {@link #values()})
+ * return snapshot copies and are weakly consistent.
+ *
+ * <p><b>Important:</b> All compound operations (computeIfAbsent, computeIfPresent, compute, merge)
+ * from both {@link Long2ObjectMap} and {@link Map} interfaces are overridden to ensure atomicity
+ * within a segment. The default interface implementations would call get/put as separate locked
+ * operations, breaking atomicity.
+ *
+ * @param <V> the type of mapped values
+ */
+public class ConcurrentLong2ObjectHashMap<V> extends AbstractLong2ObjectMap<V> {
+
+ private static final int DEFAULT_SEGMENT_COUNT = 16;
+ private static final int DEFAULT_INITIAL_CAPACITY_PER_SEGMENT = 16;
+
+ private final Segment<V>[] segments;
+ private final int segmentMask;
+ private final int segmentBits;
+
+ public ConcurrentLong2ObjectHashMap() {
+ this(DEFAULT_SEGMENT_COUNT);
+ }
+
+ @SuppressWarnings("unchecked")
+ public ConcurrentLong2ObjectHashMap(int segmentCount) {
+ if (segmentCount <= 0 || (segmentCount & (segmentCount - 1)) != 0) {
+ throw new IllegalArgumentException("segmentCount must be a positive power of 2: " + segmentCount);
+ }
+ this.segmentBits = Integer.numberOfTrailingZeros(segmentCount);
+ this.segmentMask = segmentCount - 1;
+ this.segments = new Segment[segmentCount];
+ for (int i = 0; i < segmentCount; i++) {
+ segments[i] = new Segment<>(DEFAULT_INITIAL_CAPACITY_PER_SEGMENT);
+ }
+ }
+
+ private Segment<V> segmentFor(long key) {
+ return segments[(int) (HashCommon.mix(key) >>> (64 - segmentBits)) & segmentMask];
+ }
+
+ // ---- Read operations (read-lock) ----
+
+ @Override
+ public V get(long key) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.readLock().lock();
+ try {
+ return seg.map.get(key);
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+
+ @Override
+ public V getOrDefault(long key, V defaultValue) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.readLock().lock();
+ try {
+ return seg.map.getOrDefault(key, defaultValue);
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+
+ @Override
+ public boolean containsKey(long key) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.readLock().lock();
+ try {
+ return seg.map.containsKey(key);
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+
+ @Override
+ public boolean containsValue(Object value) {
+ for (Segment<V> seg : segments) {
+ seg.lock.readLock().lock();
+ try {
+ if (seg.map.containsValue(value)) {
+ return true;
+ }
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+ return false;
+ }
+
+ @Override
+ public int size() {
+ long total = 0;
+ for (Segment<V> seg : segments) {
+ seg.lock.readLock().lock();
+ try {
+ total += seg.map.size();
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+ return (int) Math.min(total, Integer.MAX_VALUE);
+ }
+
+ @Override
+ public boolean isEmpty() {
+ for (Segment<V> seg : segments) {
+ seg.lock.readLock().lock();
+ try {
+ if (!seg.map.isEmpty()) {
+ return false;
+ }
+ } finally {
+ seg.lock.readLock().unlock();
+ }
+ }
+ return true;
+ }
+
+ // ---- Write operations (write-lock) ----
+
+ @Override
+ public V put(long key, V value) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.put(key, value);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public V remove(long key) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.remove(key);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public V putIfAbsent(long key, V value) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.putIfAbsent(key, value);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public boolean replace(long key, V oldValue, V newValue) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.replace(key, oldValue, newValue);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public V replace(long key, V value) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ return seg.map.replace(key, value);
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public void clear() {
+ for (Segment<V> seg : segments) {
+ seg.lock.writeLock().lock();
+ try {
+ seg.map.clear();
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+ }
+
+ @Override
+ public void putAll(Map<? extends Long, ? extends V> m) {
+ for (Map.Entry<? extends Long, ? extends V> entry : m.entrySet()) {
+ put(entry.getKey().longValue(), entry.getValue());
+ }
+ }
+
+ // ---- Atomic compound operations ----
+ // Override ALL compound methods from both Long2ObjectMap and Map interfaces
+ // to ensure the check-then-act is atomic within a segment's write lock.
+
+ @Override
+ public V computeIfAbsent(long key, LongFunction<? extends V> mappingFunction) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ V val = seg.map.get(key);
+ if (val != null || seg.map.containsKey(key)) {
+ return val;
+ }
+ V newValue = mappingFunction.apply(key);
+ seg.map.put(key, newValue);
+ return newValue;
+ } finally {
+ seg.lock.writeLock().unlock();
+ }
+ }
+
+ @Override
+ public V computeIfAbsent(long key, Long2ObjectFunction<? extends V> mappingFunction) {
+ Segment<V> seg = segmentFor(key);
+ seg.lock.writeLock().lock();
+ try {
+ V val = seg.map.get(key);
+ if (val != null || seg.map.containsKey(key)) {
+ return val;
+ }
+ V newValue = mappingFunction.get(key);
+ seg.map.put(key, newValue);
+ return newValue;
Review Comment:
Same issue as the other overload: if `mappingFunction.get(key)` returns null, `computeIfAbsent` must not store the key (it should return null and leave the map unchanged).
##########
fe/fe-core/src/test/java/org/apache/doris/common/ConcurrentLong2ObjectHashMapTest.java:
##########
@@ -0,0 +1,432 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common;
+
+import com.google.gson.Gson;
+import com.google.gson.reflect.TypeToken;
+import it.unimi.dsi.fastutil.longs.Long2ObjectMap;
+import it.unimi.dsi.fastutil.longs.LongSet;
+import it.unimi.dsi.fastutil.objects.ObjectCollection;
+import it.unimi.dsi.fastutil.objects.ObjectSet;
+import org.apache.doris.persist.gson.GsonUtils;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.lang.reflect.Type;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicInteger;
+
+class ConcurrentLong2ObjectHashMapTest {
+
+ @Test
+ void testPutAndGet() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ Assertions.assertNull(map.put(1L, "one"));
+ Assertions.assertEquals("one", map.get(1L));
+ Assertions.assertEquals("one", map.put(1L, "ONE"));
+ Assertions.assertEquals("ONE", map.get(1L));
+ }
+
+ @Test
+ void testGetMissingKey() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ Assertions.assertNull(map.get(999L));
+ }
+
+ @Test
+ void testGetOrDefault() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ map.put(1L, "one");
+ Assertions.assertEquals("one", map.getOrDefault(1L, "default"));
+ Assertions.assertEquals("default", map.getOrDefault(2L, "default"));
+ }
+
+ @Test
+ void testRemove() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ map.put(1L, "one");
+ Assertions.assertEquals("one", map.remove(1L));
+ Assertions.assertNull(map.get(1L));
+ Assertions.assertNull(map.remove(1L));
+ }
+
+ @Test
+ void testContainsKey() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ Assertions.assertFalse(map.containsKey(1L));
+ map.put(1L, "one");
+ Assertions.assertTrue(map.containsKey(1L));
+ }
+
+ @Test
+ void testContainsValue() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ map.put(1L, "one");
+ map.put(2L, "two");
+ Assertions.assertTrue(map.containsValue("one"));
+ Assertions.assertFalse(map.containsValue("three"));
+ }
+
+ @Test
+ void testSizeAndIsEmpty() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ Assertions.assertTrue(map.isEmpty());
+ Assertions.assertEquals(0, map.size());
+ map.put(1L, "one");
+ map.put(2L, "two");
+ Assertions.assertFalse(map.isEmpty());
+ Assertions.assertEquals(2, map.size());
+ }
+
+ @Test
+ void testClear() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ map.put(1L, "one");
+ map.put(2L, "two");
+ map.clear();
+ Assertions.assertTrue(map.isEmpty());
+ Assertions.assertEquals(0, map.size());
+ }
+
+ @Test
+ void testPutAll() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ Map<Long, String> source = new HashMap<>();
+ source.put(1L, "one");
+ source.put(2L, "two");
+ source.put(3L, "three");
+ map.putAll(source);
+ Assertions.assertEquals(3, map.size());
+ Assertions.assertEquals("two", map.get(2L));
+ }
+
+ @Test
+ void testPutIfAbsent() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ Assertions.assertNull(map.putIfAbsent(1L, "one"));
+ Assertions.assertEquals("one", map.putIfAbsent(1L, "ONE"));
+ Assertions.assertEquals("one", map.get(1L));
+ }
+
+ @Test
+ void testComputeIfAbsent() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ String val = map.computeIfAbsent(1L, k -> "computed-" + k);
+ Assertions.assertEquals("computed-1", val);
+ // Should not recompute
+ String val2 = map.computeIfAbsent(1L, k -> "recomputed-" + k);
+ Assertions.assertEquals("computed-1", val2);
+ }
+
+ @Test
+ void testComputeIfPresent() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ // Not present — should return null
+ Assertions.assertNull(map.computeIfPresent(1L, (k, v) -> v + "-updated"));
+
+ map.put(1L, "one");
+ String val = map.computeIfPresent(1L, (k, v) -> v + "-updated");
+ Assertions.assertEquals("one-updated", val);
+ Assertions.assertEquals("one-updated", map.get(1L));
+
+ // Return null to remove
+ Assertions.assertNull(map.computeIfPresent(1L, (k, v) -> null));
+ Assertions.assertFalse(map.containsKey(1L));
+ }
+
+ @Test
+ void testEntrySet() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ map.put(1L, "one");
+ map.put(2L, "two");
+
+ ObjectSet<Long2ObjectMap.Entry<String>> entries = map.long2ObjectEntrySet();
+ Assertions.assertEquals(2, entries.size());
+
+ Set<Long> keys = new HashSet<>();
+ for (Long2ObjectMap.Entry<String> entry : entries) {
+ keys.add(entry.getLongKey());
+ }
+ Assertions.assertTrue(keys.contains(1L));
+ Assertions.assertTrue(keys.contains(2L));
+ }
+
+ @Test
+ void testKeySet() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ map.put(10L, "ten");
+ map.put(20L, "twenty");
+ LongSet keys = map.keySet();
+ Assertions.assertEquals(2, keys.size());
+ Assertions.assertTrue(keys.contains(10L));
+ Assertions.assertTrue(keys.contains(20L));
+ }
+
+ @Test
+ void testValues() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ map.put(1L, "one");
+ map.put(2L, "two");
+ ObjectCollection<String> values = map.values();
+ Assertions.assertEquals(2, values.size());
+ Assertions.assertTrue(values.contains("one"));
+ Assertions.assertTrue(values.contains("two"));
+ }
+
+ @Test
+ void testStream() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ for (long i = 0; i < 100; i++) {
+ map.put(i, "val-" + i);
+ }
+ long count = map.values().stream().filter(v -> v.startsWith("val-")).count();
+ Assertions.assertEquals(100, count);
+ }
+
+ @Test
+ void testForEach() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ map.put(1L, "one");
+ map.put(2L, "two");
+ Map<Long, String> collected = new HashMap<>();
+ map.forEach(collected::put);
+ Assertions.assertEquals(2, collected.size());
+ Assertions.assertEquals("one", collected.get(1L));
+ }
+
+ @Test
+ void testNullValues() {
+ ConcurrentLong2ObjectHashMap<String> map = new ConcurrentLong2ObjectHashMap<>();
+ map.put(1L, null);
+ Assertions.assertTrue(map.containsKey(1L));
+ Assertions.assertNull(map.get(1L));
Review Comment:
This test (and the current implementation) allows storing null values, but `ConcurrentHashMap` forbids nulls. Since the PR describes these maps as drop-in replacements for `ConcurrentHashMap<Long, V>`, consider rejecting null values (throw NPE on `put`/`putIfAbsent`/`replace`/`compute*` results) and update/remove this test, or update the docs to explicitly state that null values are supported and semantics differ from `ConcurrentHashMap`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4061044642
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4152617358
<details>
<summary>TPC-H: <b>Total hot run time: 26400 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bae76abe915e1ef35b14daf6f665242add61ef91, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17595 4491 4269 4269
q2 q3 10653 777 518 518
q4 4684 361 248 248
q5 7567 1219 1048 1048
q6 179 175 147 147
q7 805 825 678 678
q8 9299 1446 1304 1304
q9 4880 4712 4706 4706
q10 6233 1889 1650 1650
q11 469 264 237 237
q12 716 580 476 476
q13 18033 2721 1976 1976
q14 230 236 209 209
q15 q16 717 742 661 661
q17 735 848 442 442
q18 6042 5371 5197 5197
q19 1127 980 613 613
q20 540 493 374 374
q21 4360 1849 1401 1401
q22 350 302 246 246
Total cold run time: 95214 ms
Total hot run time: 26400 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 5015 4651 4560 4560
q2 q3 3894 4393 3816 3816
q4 894 1217 763 763
q5 4055 4356 4373 4356
q6 184 179 140 140
q7 1749 1665 1529 1529
q8 2466 2651 2567 2567
q9 7691 7624 7416 7416
q10 3798 4022 3626 3626
q11 524 452 412 412
q12 486 598 444 444
q13 2397 2885 2033 2033
q14 295 296 273 273
q15 q16 731 784 703 703
q17 1178 1277 1373 1277
q18 7454 6854 6595 6595
q19 922 919 955 919
q20 2071 2164 1977 1977
q21 4012 3678 3353 3353
q22 437 436 376 376
Total cold run time: 50253 ms
Total hot run time: 47135 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4096969952
<details>
<summary>TPC-DS: <b>Total hot run time: 168447 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b78116f98468bbe42f98a9d861eb8811f0c11190, data reload: false
query5 4336 629 502 502
query6 325 239 204 204
query7 4201 464 260 260
query8 343 245 217 217
query9 8688 2705 2694 2694
query10 538 408 336 336
query11 6992 5075 4880 4880
query12 186 129 123 123
query13 1286 478 369 369
query14 5776 3701 3477 3477
query14_1 2871 2819 2842 2819
query15 216 192 173 173
query16 974 503 467 467
query17 890 736 668 668
query18 2454 465 373 373
query19 215 212 195 195
query20 139 129 127 127
query21 215 148 112 112
query22 13254 14109 14644 14109
query23 16181 15754 15726 15726
query23_1 15740 15948 15479 15479
query24 7222 1643 1223 1223
query24_1 1212 1216 1232 1216
query25 613 455 416 416
query26 1230 254 142 142
query27 2773 481 289 289
query28 4495 1828 1841 1828
query29 834 573 481 481
query30 312 227 194 194
query31 1016 963 885 885
query32 83 69 70 69
query33 521 341 281 281
query34 882 866 531 531
query35 630 715 610 610
query36 1086 1103 982 982
query37 130 95 86 86
query38 2958 2931 2888 2888
query39 846 838 817 817
query39_1 793 784 807 784
query40 234 152 134 134
query41 62 58 58 58
query42 259 256 253 253
query43 244 242 222 222
query44
query45 191 191 184 184
query46 878 991 597 597
query47 2083 2101 2077 2077
query48 307 313 228 228
query49 638 469 383 383
query50 703 280 212 212
query51 4115 4091 3957 3957
query52 269 263 257 257
query53 286 334 276 276
query54 308 273 275 273
query55 97 93 92 92
query56 335 328 321 321
query57 1945 1756 1574 1574
query58 279 278 273 273
query59 2780 2938 2772 2772
query60 348 346 326 326
query61 159 154 150 150
query62 631 590 541 541
query63 319 286 278 278
query64 4991 1263 1005 1005
query65
query66 1469 452 360 360
query67 24164 24230 24157 24157
query68
query69 399 303 284 284
query70 995 977 945 945
query71 347 311 303 303
query72 2730 2583 2454 2454
query73 547 540 317 317
query74 9539 9627 9428 9428
query75 2872 2762 2486 2486
query76 2278 1022 677 677
query77 363 393 324 324
query78 10915 11022 10430 10430
query79 3168 745 581 581
query80 1751 632 555 555
query81 581 262 223 223
query82 980 154 118 118
query83 333 257 243 243
query84 307 124 103 103
query85 921 519 461 461
query86 509 301 293 293
query87 3136 3120 3074 3074
query88 3560 2664 2670 2664
query89 430 369 349 349
query90 2227 183 192 183
query91 173 168 142 142
query92 96 75 68 68
query93 2075 866 500 500
query94 658 324 281 281
query95 598 410 331 331
query96 648 505 225 225
query97 2434 2502 2420 2420
query98 249 226 229 226
query99 1006 997 907 907
Total cold run time: 252635 ms
Total hot run time: 168447 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4102779091
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4124169642
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4134576596
<details>
<summary>TPC-DS: <b>Total hot run time: 168823 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f49d1e7629aa582c8998bb32cd849a382b1bba31, data reload: false
query5 4342 618 510 510
query6 333 226 209 209
query7 4222 470 277 277
query8 344 237 223 223
query9 8698 2680 2679 2679
query10 485 368 337 337
query11 6973 5087 4862 4862
query12 188 135 121 121
query13 1301 471 353 353
query14 5685 3704 3406 3406
query14_1 2881 2850 2804 2804
query15 209 200 177 177
query16 977 470 450 450
query17 1124 726 629 629
query18 2461 445 361 361
query19 227 212 188 188
query20 141 126 128 126
query21 221 135 111 111
query22 13420 13910 14392 13910
query23 16732 16179 16347 16179
query23_1 16120 15830 15649 15649
query24 7129 1600 1234 1234
query24_1 1226 1237 1227 1227
query25 601 521 397 397
query26 1228 262 153 153
query27 2786 495 293 293
query28 4978 1839 1881 1839
query29 868 572 478 478
query30 310 238 192 192
query31 1011 974 872 872
query32 80 71 70 70
query33 505 350 287 287
query34 907 908 534 534
query35 656 691 609 609
query36 1097 1160 990 990
query37 142 98 83 83
query38 2933 2889 2868 2868
query39 860 839 807 807
query39_1 792 798 790 790
query40 227 150 139 139
query41 62 59 58 58
query42 253 255 254 254
query43 237 249 216 216
query44
query45 198 186 183 183
query46 891 973 597 597
query47 2557 2503 2102 2102
query48 332 312 231 231
query49 632 459 388 388
query50 695 280 210 210
query51 4091 4070 3972 3972
query52 269 264 258 258
query53 290 328 280 280
query54 286 279 262 262
query55 88 86 86 86
query56 311 324 312 312
query57 1889 1833 1770 1770
query58 278 269 270 269
query59 2781 2944 2755 2755
query60 346 334 322 322
query61 166 157 157 157
query62 606 587 541 541
query63 311 274 275 274
query64 5145 1303 1019 1019
query65
query66 1461 452 353 353
query67 24187 24279 24283 24279
query68
query69 408 309 291 291
query70 944 940 956 940
query71 334 317 292 292
query72 2844 2709 2484 2484
query73 551 559 313 313
query74 9689 9614 9434 9434
query75 2874 2734 2478 2478
query76 2293 1023 664 664
query77 359 378 308 308
query78 10940 11130 10414 10414
query79 1109 756 577 577
query80 1375 624 529 529
query81 574 261 225 225
query82 1037 151 117 117
query83 368 262 238 238
query84 248 118 101 101
query85 914 515 453 453
query86 418 305 294 294
query87 3136 3139 2999 2999
query88 3545 2652 2651 2651
query89 429 366 351 351
query90 1974 171 174 171
query91 178 171 140 140
query92 79 76 73 73
query93 930 900 515 515
query94 639 327 306 306
query95 576 340 312 312
query96 636 507 233 233
query97 2471 2463 2400 2400
query98 231 225 221 221
query99 996 938 907 907
Total cold run time: 251233 ms
Total hot run time: 168823 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4061044586
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4096920740
<details>
<summary>TPC-H: <b>Total hot run time: 26773 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b78116f98468bbe42f98a9d861eb8811f0c11190, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17602 4456 4266 4266
q2 q3 10642 776 518 518
q4 4676 350 244 244
q5 7559 1218 1004 1004
q6 171 173 145 145
q7 772 839 678 678
q8 9308 1464 1339 1339
q9 4925 4750 4720 4720
q10 6298 1895 1666 1666
q11 483 255 246 246
q12 740 581 460 460
q13 18055 2952 2191 2191
q14 232 234 209 209
q15 q16 731 749 668 668
q17 736 837 434 434
q18 6099 5421 5224 5224
q19 1113 963 616 616
q20 535 481 372 372
q21 4533 1823 1440 1440
q22 407 446 333 333
Total cold run time: 95617 ms
Total hot run time: 26773 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4835 4546 4645 4546
q2 q3 3877 4417 3827 3827
q4 886 1200 833 833
q5 4065 4401 4350 4350
q6 189 170 141 141
q7 1736 1608 1532 1532
q8 2449 2743 2569 2569
q9 7667 7447 7430 7430
q10 3749 3959 3553 3553
q11 518 433 415 415
q12 490 598 433 433
q13 2682 3288 2341 2341
q14 274 291 303 291
q15 q16 752 789 768 768
q17 1201 1345 1363 1345
q18 7397 6935 6692 6692
q19 1010 985 1061 985
q20 2055 2136 2123 2123
q21 4019 3562 3277 3277
q22 480 429 386 386
Total cold run time: 50331 ms
Total hot run time: 47837 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4102779780
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4102830376
<details>
<summary>TPC-DS: <b>Total hot run time: 168565 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b78116f98468bbe42f98a9d861eb8811f0c11190, data reload: false
query5 4325 651 507 507
query6 335 224 200 200
query7 4216 466 262 262
query8 337 248 230 230
query9 8696 2749 2706 2706
query10 532 392 361 361
query11 6951 5070 4910 4910
query12 194 135 123 123
query13 1273 466 355 355
query14 5791 3695 3583 3583
query14_1 2853 2826 2803 2803
query15 200 197 178 178
query16 980 461 463 461
query17 903 729 627 627
query18 2460 471 343 343
query19 233 211 182 182
query20 136 125 124 124
query21 213 139 107 107
query22 13144 13980 14773 13980
query23 16644 15923 15502 15502
query23_1 15690 15762 15559 15559
query24 7111 1624 1241 1241
query24_1 1226 1208 1214 1208
query25 543 472 399 399
query26 1244 269 152 152
query27 2762 499 297 297
query28 4450 1851 1838 1838
query29 877 584 477 477
query30 296 226 192 192
query31 1049 951 886 886
query32 77 72 67 67
query33 507 348 284 284
query34 890 873 533 533
query35 668 701 613 613
query36 1079 1136 1003 1003
query37 139 99 83 83
query38 3003 3009 2905 2905
query39 867 844 823 823
query39_1 805 792 794 792
query40 233 160 134 134
query41 63 60 58 58
query42 260 255 258 255
query43 251 257 228 228
query44
query45 195 190 197 190
query46 891 973 605 605
query47 2577 2163 2074 2074
query48 332 321 234 234
query49 634 455 385 385
query50 695 281 214 214
query51 4044 4017 3990 3990
query52 262 261 262 261
query53 289 338 291 291
query54 305 265 270 265
query55 90 86 83 83
query56 341 331 317 317
query57 1913 1806 1741 1741
query58 292 278 276 276
query59 2760 2955 2724 2724
query60 344 341 328 328
query61 153 144 149 144
query62 632 593 551 551
query63 310 276 280 276
query64 5087 1288 1001 1001
query65
query66 1458 449 358 358
query67 24314 24246 24136 24136
query68
query69 405 317 283 283
query70 957 967 940 940
query71 335 311 308 308
query72 2776 2874 2561 2561
query73 553 548 332 332
query74 9596 9531 9410 9410
query75 2851 2747 2468 2468
query76 2299 1029 693 693
query77 382 405 319 319
query78 10933 11038 10428 10428
query79 2676 777 581 581
query80 1788 661 592 592
query81 545 267 236 236
query82 1002 161 123 123
query83 353 269 249 249
query84 309 160 106 106
query85 910 487 466 466
query86 423 340 282 282
query87 3176 3095 3050 3050
query88 3587 2672 2669 2669
query89 421 359 352 352
query90 2013 185 186 185
query91 165 165 145 145
query92 80 78 74 74
query93 1131 850 495 495
query94 646 318 283 283
query95 592 411 324 324
query96 656 543 231 231
query97 2484 2479 2402 2402
query98 235 225 242 225
query99 996 995 921 921
Total cold run time: 251494 ms
Total hot run time: 168565 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4126894273
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4153510693
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4096828533
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4152509221
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #61332:
URL: https://github.com/apache/doris/pull/61332#discussion_r3007234902
##########
fe/fe-core/src/test/java/org/apache/doris/foundation/util/ConcurrentLong2LongHashMapTest.java:
##########
Review Comment:
should move to fe-foundation too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #61332:
URL: https://github.com/apache/doris/pull/61332#issuecomment-4152663125
<details>
<summary>TPC-DS: <b>Total hot run time: 168750 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bae76abe915e1ef35b14daf6f665242add61ef91, data reload: false
query5 4325 643 493 493
query6 339 235 200 200
query7 4203 494 271 271
query8 351 261 225 225
query9 8700 2664 2729 2664
query10 553 419 335 335
query11 7062 5123 4907 4907
query12 182 126 123 123
query13 1266 454 330 330
query14 5769 3759 3481 3481
query14_1 2874 2796 2800 2796
query15 210 191 172 172
query16 973 473 458 458
query17 925 695 628 628
query18 2436 433 341 341
query19 232 214 199 199
query20 132 123 123 123
query21 210 131 114 114
query22 13279 14094 14786 14094
query23 16851 16208 15876 15876
query23_1 16213 15658 15727 15658
query24 7224 1596 1213 1213
query24_1 1235 1244 1228 1228
query25 588 505 447 447
query26 1248 264 154 154
query27 2786 480 302 302
query28 4500 1841 1838 1838
query29 846 598 503 503
query30 305 228 193 193
query31 1024 946 886 886
query32 84 72 76 72
query33 531 341 297 297
query34 905 867 520 520
query35 645 697 597 597
query36 1087 1162 999 999
query37 140 100 89 89
query38 3018 2961 2838 2838
query39 871 829 817 817
query39_1 800 798 789 789
query40 242 161 142 142
query41 76 112 60 60
query42 272 258 254 254
query43 243 246 220 220
query44
query45 200 189 184 184
query46 892 965 616 616
query47 2107 2134 2038 2038
query48 313 314 227 227
query49 628 443 369 369
query50 719 273 213 213
query51 4032 4055 3983 3983
query52 263 267 255 255
query53 291 335 283 283
query54 312 271 264 264
query55 94 88 81 81
query56 310 342 315 315
query57 1882 1736 1839 1736
query58 284 276 269 269
query59 2788 2949 2744 2744
query60 340 333 308 308
query61 159 154 150 150
query62 642 592 537 537
query63 301 274 275 274
query64 4943 1304 1005 1005
query65
query66 1480 454 356 356
query67 24234 24395 24298 24298
query68
query69 417 308 295 295
query70 944 968 883 883
query71 338 303 307 303
query72 2864 2701 2413 2413
query73 537 546 325 325
query74 9632 9566 9399 9399
query75 2852 2763 2469 2469
query76 2297 1053 670 670
query77 375 396 323 323
query78 10986 11152 10445 10445
query79 1072 817 568 568
query80 708 621 584 584
query81 500 259 226 226
query82 1356 152 123 123
query83 322 257 239 239
query84 287 121 100 100
query85 854 494 465 465
query86 376 302 305 302
query87 3138 3109 3039 3039
query88 3563 2655 2652 2652
query89 436 367 342 342
query90 1962 179 188 179
query91 170 164 140 140
query92 80 77 73 73
query93 929 880 505 505
query94 464 331 307 307
query95 597 415 309 309
query96 649 517 232 232
query97 2474 2476 2422 2422
query98 227 239 219 219
query99 995 992 842 842
Total cold run time: 249557 ms
Total hot run time: 168750 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Enhancement](memory) Add ConcurrentLong2ObjectHashMap and ConcurrentLong2LongHashMap [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #61332:
URL: https://github.com/apache/doris/pull/61332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org