You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "Caideyipi (via GitHub)" <gi...@apache.org> on 2026/10/13 10:27:57 UTC
[PR] [To dev/1.3] [Performance] Pre-size collections with known cardinality (#18851) [iotdb]
Caideyipi opened a new pull request, #18854:
URL: https://github.com/apache/iotdb/pull/18854
## Description
Backport #18851 (`a78f5a736c7e34a951fa1dda34a314fc8a47b7c9`) to `dev/1.3` by re-scanning this branch's production Java code and adapting the collection allocation optimization to its implementations.
Collections with a known number of entries currently grow their backing arrays or hash tables while being filled. Pre-size 152 collections in 80 production Java files across six modules:
- Session batch-write buffers and schema lists, and both JDBC and Session result column metadata.
- Schema/path, query and write plan, logical-view, template, and ConfigNode plan/procedure deserialization.
- Query projections, expression transformers, CASE evaluation, and spill readers.
- Memtable row/tablet schema lists and recovery, TsFile device sets, Pipe payloads and progress metadata, and UDF point/tangent materialization.
Use serialized counts or the number of outputs produced from the input. Guava expected-size factories account for the hash-table load factor. Optional empty lists use capacity zero; memtable schema-list sizing handles nullable measurement arrays.
The dev/1.3 scan includes its aggregation/device-view/templated plan deserialization and logical-view maps, and uses the branch's existing DataNode transformation, AuthorPlan, DeviceTimeIndex, and PipeConsensus implementations.
## Validation
- Spotless applied to all six affected modules; Checkstyle, Spotless, and forbidden API checks passed in the reactor build.
- English reactor `test-compile` passed for the affected modules and their dependencies (28 modules).
- 48 existing test classes passed: 378 tests, zero failures/errors/skips. Coverage includes Session, schema/path and plan/procedure serialization, memtable partial inserts and recovery, TsFile metadata, Pipe payloads/progress metadata, expression/operator behavior, and UDF pattern processing.
- Chinese locale reactor `test-compile` passed for the same 28 modules with `-P with-zh-locale`.
- `git diff --cached --check` passed.
This PR has:
- [x] been self-reviewed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [To dev/1.3] [Performance] Pre-size collections with known cardinality (#18851) [iotdb]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #18854:
URL: https://github.com/apache/iotdb/pull/18854#issuecomment-6144468636
## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/18854?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `69.23077%` with `52 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 45.41%. Comparing base ([`c056ce6`](https://app.codecov.io/gh/apache/iotdb/commit/c056ce6f8615905e3c54938e969a8074d4f1681f?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`65a6768`](https://app.codecov.io/gh/apache/iotdb/commit/65a6768f10fd20d445f199f3c1c69808c34ea2f2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/iotdb/pull/18854?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...n/expression/visitor/ColumnTransformerVisitor.java](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Fexpression%2Fvisitor%2FColumnTransformerVisitor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9leHByZXNzaW9uL3Zpc2l0b3IvQ29sdW1uVHJhbnNmb3JtZXJWaXNpdG9yLmphdmE=) | 0.00% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...quest/PipeConsensusTransferFileSealWithModReq.java](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&filepath=iotdb-core%2Fnode-commons%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fcommons%2Fpipe%2Fsink%2Fpayload%2Fpipeconsensus%2Frequest%2FPipeConsensusTransferFileSealWithModReq.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9ub2RlLWNvbW1vbnMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2lvdGRiL2NvbW1vbnMvcGlwZS9zaW5rL3BheWxvYWQvcGlwZWNvbnNlbnN1cy9yZXF1ZXN0L1BpcGVDb25zZW5zdXNUcmFuc2ZlckZpbGVTZWFsV2l0aE1vZFJlcS5qYXZh) | 0.00% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ion/operator/process/FilterAndProjectOperator.java](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fexecution%2Foperator%2Fprocess%2FFilterAndProjectOperator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvZXhlY3V0aW9uL29wZXJhdG9yL3Byb2Nlc3MvRmlsdGVyQW5kUHJvamVjdE9wZXJhdG9yLmphdmE=) | 0.00% | [3 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...tion/dag/column/CaseWhenThenColumnTransformer.java](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Ftransformation%2Fdag%2Fcolumn%2FCaseWhenThenColumnTransformer.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvdHJhbnNmb3JtYXRpb24vZGFnL2NvbHVtbi9DYXNlV2hlblRoZW5Db2x1bW5UcmFuc2Zvcm1lci5qYXZh) | 0.00% | [3 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...in/java/org/apache/iotdb/rpc/IoTDBJDBCDataSet.java](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&filepath=iotdb-client%2Fservice-rpc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Frpc%2FIoTDBJDBCDataSet.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY2xpZW50L3NlcnZpY2UtcnBjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9pb3RkYi9ycGMvSW9UREJKREJDRGF0YVNldC5qYXZh) | 50.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ain/java/org/apache/iotdb/rpc/IoTDBRpcDataSet.java](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&filepath=iotdb-client%2Fservice-rpc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Frpc%2FIoTDBRpcDataSet.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY2xpZW50L3NlcnZpY2UtcnBjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9pb3RkYi9ycGMvSW9UREJScGNEYXRhU2V0LmphdmE=) | 50.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...tdb/db/queryengine/plan/analyze/TemplatedInfo.java](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Fanalyze%2FTemplatedInfo.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9hbmFseXplL1RlbXBsYXRlZEluZm8uamF2YQ==) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...plan/parameter/model/ModelInferenceDescriptor.java](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Fplanner%2Fplan%2Fparameter%2Fmodel%2FModelInferenceDescriptor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9wbGFubmVyL3BsYW4vcGFyYW1ldGVyL21vZGVsL01vZGVsSW5mZXJlbmNlRGVzY3JpcHRvci5qYXZh) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../logfile/visitor/SchemaRegionPlanDeserializer.java](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fschemaengine%2Fschemaregion%2Flogfile%2Fvisitor%2FSchemaRegionPlanDeserializer.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc2NoZW1hZW5naW5lL3NjaGVtYXJlZ2lvbi9sb2dmaWxlL3Zpc2l0b3IvU2NoZW1hUmVnaW9uUGxhbkRlc2VyaWFsaXplci5qYXZh) | 81.81% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...geengine/dataregion/memtable/AbstractMemTable.java](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fstorageengine%2Fdataregion%2Fmemtable%2FAbstractMemTable.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvc3RvcmFnZWVuZ2luZS9kYXRhcmVnaW9uL21lbXRhYmxlL0Fic3RyYWN0TWVtVGFibGUuamF2YQ==) | 83.33% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| ... and [23 more](https://app.codecov.io/gh/apache/iotdb/pull/18854?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## dev/1.3 #18854 +/- ##
==========================================
Coverage 45.41% 45.41%
Complexity 198 198
==========================================
Files 3667 3667
Lines 251096 251115 +19
Branches 31089 31109 +20
==========================================
+ Hits 114038 114052 +14
- Misses 137058 137063 +5
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/iotdb/pull/18854?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [To dev/1.3] [Performance] Pre-size collections with known cardinality (#18851) [iotdb]
Posted by "jt2594838 (via GitHub)" <gi...@apache.org>.
jt2594838 merged PR #18854:
URL: https://github.com/apache/iotdb/pull/18854
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org