You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "eldenmoon (via GitHub)" <gi...@apache.org> on 2026/10/22 08:51:25 UTC
[PR] branch-4.1: [fix](be) Reject legacy Variant blocks before deserialization [doris]
eldenmoon opened a new pull request, #69454:
URL: https://github.com/apache/doris/pull/69454
### What problem does this PR solve?
Problem Summary: During a mixed-version upgrade, a 4.1 backend can receive a Variant V1 block from an older backend. 4.0, for example, reports `be_exec_version` 8, which this branch still accepts, and its V1 payload ends after the sparse Map. This branch's V1 deserializer reads a trailing doc-value Map, so a legacy block makes the decoder read past the Variant payload and can crash the receiving backend during a load or an exchange.
Pass the presence of `variant_enable_doc_mode` through the factory's Variant V1 branch and reject a V1 block at the Variant deserialization entry when the field is absent. Presence, not the field value, is the format marker: this branch sets the field even when doc mode is false, and it always writes the trailing Map. Variant V2 keeps its own marker and is not affected. Nested ARRAY/MAP/STRUCT variant columns use the existing recursive type construction and are covered as well. Block, the wire layout and `be_exec_version` are unchanged.
The check runs after Block decompression and may run after preceding columns or enclosing container headers have been decoded, using the existing exception-to-Status handling. It protects updated receivers against decoding a legacy Variant V1 payload as the current V1 layout; it does not add sender capability negotiation or bounded parsing, validate unrelated malformed headers, or protect unpatched old receivers.
### Release note
Return an explicit error for legacy Variant V1 blocks that do not carry doc-value data.
### Check List (For Author)
- Test: Focused ASAN `BlockVariantWireFormatTest.*:BlockTest.*` run on branch-4.1 with `run-be-ut.sh`: 33/33 passed after the fix. Without it, the legacy-payload test aborts with an ASAN heap-buffer-overflow reading 8 bytes past a 76-byte Variant V1 payload (`DataTypeVariant::deserialize` -> `DataTypeMap::deserialize`), and the marker-removal test fails because the block is accepted. Temporary focused CMake source selection was restored before committing.
- Validation limit: focused BE test build with a temporary CMake source selection (restored before committing); no 4.1 cluster runtime RPC validation was performed for this backport. The 4.1 V1 reader keeps accepting 4.1 blocks that carry the doc-value marker; only the older layout is rejected.
- Behavior changed: Yes; legacy Variant V1 blocks fail before Variant payload decoding.
- Does this need documentation: No
cherry-pick #69434. The 4.1 branch keeps a V1 reader, so only V1 blocks without the doc-value marker are rejected instead of all V1 blocks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] branch-4.1: [fix](be) Reject legacy Variant blocks before deserialization [doris]
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon closed pull request #69454: branch-4.1: [fix](be) Reject legacy Variant blocks before deserialization
URL: https://github.com/apache/doris/pull/69454
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] branch-4.1: [fix](be) Reject legacy Variant blocks before deserialization [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #69454:
URL: https://github.com/apache/doris/pull/69454#issuecomment-6336736753
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org