You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/07/20 03:39:51 UTC
[GitHub] [doris] platoneko opened a new pull request, #11026: [fix] Fix potential null client bug in S3FileSystem
platoneko opened a new pull request, #11026:
URL: https://github.com/apache/doris/pull/11026
# Proposed changes
Issue Number: close #xxx
## Problem Summary:
1. Fix potential null client bug in S3FileSystem.
2. Use S3Conf instead of map to create S3FileSystem.
3. Remove a useless ut.
## Checklist(Required)
1. Does it affect the original behavior: (Yes/No/I Don't know)
4. Has unit tests been added: (Yes/No/No Need)
5. Has document been added or modified: (Yes/No/No Need)
6. Does it need to update dependencies: (Yes/No)
7. Are there any changes that cannot be rolled back: (Yes/No)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11026: [fix] Move s3 fs connect outside the lock critical area
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #11026:
URL: https://github.com/apache/doris/pull/11026#issuecomment-1192251038
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #11026: [fix] Move s3 fs connect outside the lock critical area
Posted by GitBox <gi...@apache.org>.
yiguolei merged PR #11026:
URL: https://github.com/apache/doris/pull/11026
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] platoneko commented on a diff in pull request #11026: [fix] Move s3 fs connect outside the lock critical area
Posted by GitBox <gi...@apache.org>.
platoneko commented on code in PR #11026:
URL: https://github.com/apache/doris/pull/11026#discussion_r927217568
##########
be/src/olap/storage_policy_mgr.cpp:
##########
@@ -24,43 +24,60 @@
namespace doris {
-void StoragePolicyMgr::update(const std::string& name, StoragePolicyPtr policy) {
- std::lock_guard<std::mutex> l(_mutex);
- auto it = _policy_map.find(name);
- if (it != _policy_map.end()) {
- // just support change ak, sk, cooldown_ttl, cooldown_datetime
- LOG(INFO) << "update storage policy name: " << name;
- auto s3_fs = std::dynamic_pointer_cast<io::S3FileSystem>(
- io::FileSystemMap::instance()->get(name));
- DCHECK(s3_fs);
- s3_fs->set_ak(policy->s3_ak);
- s3_fs->set_sk(policy->s3_sk);
- s3_fs->connect();
- it->second = std::move(policy);
- } else {
- // can't find name's policy, so do nothing.
+void StoragePolicyMgr::update(const std::string& name, const StoragePolicyPtr& policy) {
Review Comment:
i will fix soon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] compiletheworld commented on pull request #11026: [fix] Fix potential null client bug in S3FileSystem
Posted by GitBox <gi...@apache.org>.
compiletheworld commented on PR #11026:
URL: https://github.com/apache/doris/pull/11026#issuecomment-1190940405
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morningman commented on a diff in pull request #11026: [fix] Move s3 fs connect outside the lock critical area
Posted by GitBox <gi...@apache.org>.
morningman commented on code in PR #11026:
URL: https://github.com/apache/doris/pull/11026#discussion_r926878863
##########
be/src/olap/storage_policy_mgr.cpp:
##########
@@ -24,43 +24,60 @@
namespace doris {
-void StoragePolicyMgr::update(const std::string& name, StoragePolicyPtr policy) {
- std::lock_guard<std::mutex> l(_mutex);
- auto it = _policy_map.find(name);
- if (it != _policy_map.end()) {
- // just support change ak, sk, cooldown_ttl, cooldown_datetime
- LOG(INFO) << "update storage policy name: " << name;
- auto s3_fs = std::dynamic_pointer_cast<io::S3FileSystem>(
- io::FileSystemMap::instance()->get(name));
- DCHECK(s3_fs);
- s3_fs->set_ak(policy->s3_ak);
- s3_fs->set_sk(policy->s3_sk);
- s3_fs->connect();
- it->second = std::move(policy);
- } else {
- // can't find name's policy, so do nothing.
+void StoragePolicyMgr::update(const std::string& name, const StoragePolicyPtr& policy) {
Review Comment:
Looks like the `update` and `refresh` policy process may be called concurrently?
If so, it is not safe to update `s3_fs` outside the lock
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #11026: [fix] Move s3 fs connect outside the lock critical area
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #11026:
URL: https://github.com/apache/doris/pull/11026#issuecomment-1192251063
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org