You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2026/10/08 07:54:58 UTC
[PR] [fix](lance) Restore data-cache reuse across dataset opens (branch-4.1) [doris]
Gabriel39 opened a new pull request, #68778:
URL: https://github.com/apache/doris/pull/68778
### What problem does this PR solve?
Related PR: #68615; https://github.com/zhangstar333/lance-c/pull/3
Reopening a Lance dataset can recreate its underlying object store. The per-instance UUID namespace then prevents the new reader from finding cached data, so repeated RowIDFetcher queries read from the origin again even with a shared session.
Revert the per-instance namespace portion of the Foyer patch and restore the original v1 store-prefix/path keys. Keep the optimization that avoids rewriting unchanged size records. Update the repeated-scan regression to require an immediate cache hit and add a multi-fragment `take_rows` regression covering close/reopen cycles and returned row values.
This restores the original keying limitation: different endpoints with identical store prefixes and paths are not isolated within one cache owner. The dependency README documents using separate cache owners/directories for those backends.
Validation on the pinned dependency with the updated patch:
- Reproduced before the change: the second open/read/close cycle requested 35,816 origin bytes; after the revert, both subsequent cycles request zero origin bytes.
- `cargo test --locked`: 495 tests passed; the three opt-in native consumer tests were not rerun.
- `cargo clippy --locked --all-targets -- -D warnings` and `cargo fmt -- --check` passed.
- The Doris Lance patch lifecycle harness passed fresh extraction, idempotence, source refresh, and patch-failure checks; `git diff --check` passed.
- Full Doris compilation and BE integration remain for CI. No FE or C/C++ source files are changed.
### Release note
Restore Lance data-cache reuse when datasets are reopened between queries.
### Check List (For Author)
- [x] Unit Test
- [x] Behavior changed: reopened datasets reuse store-prefix/path cache entries.
- [x] Documentation updated in the dependency README carried by the patch.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](lance) Restore data-cache reuse across dataset opens (branch-4.1) [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #68778:
URL: https://github.com/apache/doris/pull/68778#issuecomment-6055394548
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](lance) Restore data-cache reuse across dataset opens (branch-4.1) [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #68778:
URL: https://github.com/apache/doris/pull/68778#issuecomment-6055395777
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](lance) Restore data-cache reuse across dataset opens (branch-4.1) [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #68778:
URL: https://github.com/apache/doris/pull/68778#issuecomment-6055698825
Codex automated review failed and did not complete.
Error: Review recovery stopped: No final review submission was declared
Workflow run: https://github.com/apache/doris/actions/runs/37746442832
Please inspect the workflow logs and rerun the review after the underlying issue is resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](lance) Restore data-cache reuse across dataset opens (branch-4.1) [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #68778:
URL: https://github.com/apache/doris/pull/68778#issuecomment-6055393057
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://doris.apache.org/community/how-to-contribute/review-and-ci).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](lance) Restore data-cache reuse across dataset opens (branch-4.1) [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 closed pull request #68778: [fix](lance) Restore data-cache reuse across dataset opens (branch-4.1)
URL: https://github.com/apache/doris/pull/68778
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org