You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@doris.apache.org by "Yukang-Lian (via GitHub)" <gi...@apache.org> on 2026/10/15 11:30:31 UTC

[PR] [fix](cloud) Remove intermediate rowsets of sorting schema change from file cache path [doris]

Yukang-Lian opened a new pull request, #69134:
URL: https://github.com/apache/doris/pull/69134

### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary:

In cloud mode, a schema change that has to sort data (e.g. adding a rollup or a sync materialized view whose key columns are in a different order) writes the intermediate rowsets of internal sorting to a local path. These rowsets have no storage resource, so `CloudRowsetWriter::init` puts them in the root of a file cache path as `<rowset_id>_<segment_id>.dat`. Every input rowset with data produces at least one intermediate rowset, so each such schema change writes a full copy of the converted data there.

Only the local mode subclass `VLocalSchemaChangeWithSorting` removes the intermediate rowsets. Cloud mode uses `VBaseSchemaChangeWithSorting` directly, which never removes them, whether the schema change succeeds or fails. These files are not part of the file cache: cache eviction and `clear_file_cache` API do not see them, and the cache loader skips regular files in the cache root on restart. They pile up on the cache disk until it is full, the file cache then evicts its own data and cannot admit new blocks, and reads fall back to remote storage. We observed hundreds of GB of such files on a cluster which created sync materialized views repeatedly.

This PR:

- Adds `VCloudSchemaChangeWithSorting`, used by cloud schema change, which removes the intermediate rowsets after each input rowset is processed, whether it succeeds or fails (like `VLocalSchemaChangeWithSorting` does in local mode).
- Removes the local rowset files left in file cache paths by previous BE processes (e.g. BE crashed during schema change) when `CloudStorageEngine` opens. A rowset id embeds the backend uid of the process that generated it, which is regenerated on every start, so only files from previous processes are removed and files written by the current process are never touched. Only regular files whose names match the local rowset file format are considered.
- Adds a debug point `VBaseSchemaChangeWithSorting._inner_process.before_external_sorting` to inject a failure in tests.

### Release note

Fix the intermediate rowsets of schema change with sorting being left in the file cache path in cloud mode, which could fill up the cache disk.

### Check List (For Author)

- Test 
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason

Unit tests (`CloudSchemaChangeSortingTest`) run a real sorting schema change in cloud mode and check the file cache root after success and after an injected failure, and cover the startup cleanup of stale rowset files. Without the cleanup, the success and failure cases fail with the intermediate rowsets left in the cache root.

The docker regression test `test_cloud_sc_sorting_tmp_rowset_cleanup` covers a rollup that reorders keys, a failed rollup, and BE restart with stale rowset files in the cache root. On the code before this PR it fails at the first step (`expected: <0> but was: <5>`, one leftover intermediate rowset per loaded rowset).

- Behavior changed:
- [ ] No.
- [x] Yes. 
In cloud mode, intermediate rowsets of schema change with sorting are removed after use, and stale ones in file cache paths are removed when BE starts.

- Does this need documentation?
- [x] No.
- [ ] Yes.

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label

🤖 Generated with [Claude Code](https://claude.com/claude-code)

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org