You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "morningman (via GitHub)" <gi...@apache.org> on 2026/03/31 00:20:31 UTC
[PR] [refactor](fs-spi) Migrate cloud.storage Remote* hierarchy to fs.obj module [doris]
morningman opened a new pull request, #61911:
URL: https://github.com/apache/doris/pull/61911
### What problem does this PR solve?
Issue Number: N/A
Problem Summary: Eliminates duplicate object storage abstraction in the Cloud mode codebase. Previously, cloud.storage.RemoteBase and its subclasses (S3Remote, OssRemote, CosRemote, ObsRemote, BosRemote, AzureRemote, etc.) provided object storage operations independently from the fs.obj module. This change migrates all functionality into fs.obj and removes the entire Remote* hierarchy, so all Cloud Stage/Copy operations now go through ObjFileSystem.
Key changes:
- Add OssObjStorage, CosObjStorage, ObsObjStorage, BosObjStorage as new ObjStorage implementations with STS token, presigned URL, list, and head support
- Extend AzureObjStorage and S3ObjStorage with getStsToken, getPresignedUrl, listObjectsWithPrefix, headObjectWithMeta
- Add ObjFileSystem passthrough methods: getStsToken, getPresignedUrl, listObjectsWithPrefix, headObjectWithMeta, deleteObjectsByKeys
- Extract RemoteBase.ObjectInfo inner class to standalone cloud.storage.ObjectInfo
- Rewrite ObjectInfoAdapter: no RemoteBase dependency; uses ObjFileSystem for STS in ARN flow
- Migrate all callers (StageUtil, CopyLoadPendingTask, CleanCopyJobTask, CopyIntoAction, CreateStageCommand, CopyIntoInfo) to ObjFileSystem
- Delete RemoteBase, S3Remote, OssRemote, CosRemote, ObsRemote, BosRemote, TosRemote, AzureRemote, DefaultRemote, MockRemote
- Rewrite CopyLoadPendingTaskTest and StageUtilTest to use MockUp<ObjFileSystem>
- Fix Maven build cache causing test execution to be skipped (run-fe-ut.sh)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fs-spi) Migrate cloud.storage Remote* hierarchy to fs.obj module [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman merged PR #61911:
URL: https://github.com/apache/doris/pull/61911
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fs-spi) Migrate cloud.storage Remote* hierarchy to fs.obj module [doris]
Posted by "morningman (via GitHub)" <gi...@apache.org>.
morningman commented on PR #61911:
URL: https://github.com/apache/doris/pull/61911#issuecomment-4158972949
tested in #61910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [refactor](fs-spi) Migrate cloud.storage Remote* hierarchy to fs.obj module [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #61911:
URL: https://github.com/apache/doris/pull/61911#issuecomment-4158969582
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org