You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by GitBox <gi...@apache.org> on 2022/08/31 02:04:15 UTC
[GitHub] [incubator-pegasus] hycdong opened a new pull request, #1146: feat(backup): 8. meta handle backup_response during uploading and succeed
hycdong opened a new pull request, #1146:
URL: https://github.com/apache/incubator-pegasus/pull/1146
https://github.com/apache/incubator-pegasus/issues/1081
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] empiredan merged pull request #1146: feat(backup): 8. meta handle backup_response during uploading and succeed
Posted by GitBox <gi...@apache.org>.
empiredan merged PR #1146:
URL: https://github.com/apache/incubator-pegasus/pull/1146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] foreverneverer commented on a diff in pull request #1146: feat(backup): 8. meta handle backup_response during uploading and succeed
Posted by GitBox <gi...@apache.org>.
foreverneverer commented on code in PR #1146:
URL: https://github.com/apache/incubator-pegasus/pull/1146#discussion_r966585448
##########
src/rdsn/src/meta/meta_backup_engine.cpp:
##########
@@ -246,7 +246,31 @@ void meta_backup_engine::on_backup_reply(const error_code err,
}
}
- // TODO(heyuchen): handle other status
+ if (response.status == backup_status::UPLOADING) {
+ ddebug_f(
+ "backup_id({}): backup for partition {} from server {} is uploading, progress = {}",
+ _cur_backup.backup_id,
+ pid.to_string(),
+ primary.to_string(),
+ response.upload_progress);
+ }
+
+ if (response.status == backup_status::SUCCEED) {
+ ddebug_f("backup_id({}): backup for partition {} from server {} succeed",
Review Comment:
log seem can be combine one code for `uploading` and `succeed`:
```c++
dedebug(.... status=xxx...)
if (success) {
.....
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] hycdong commented on a diff in pull request #1146: feat(backup): 8. meta handle backup_response during uploading and succeed
Posted by GitBox <gi...@apache.org>.
hycdong commented on code in PR #1146:
URL: https://github.com/apache/incubator-pegasus/pull/1146#discussion_r965427601
##########
src/rdsn/src/meta/meta_backup_engine.cpp:
##########
@@ -246,7 +246,31 @@ void meta_backup_engine::on_backup_reply(const error_code err,
}
}
- // TODO(heyuchen): handle other status
+ if (response.status == backup_status::UPLOADING) {
Review Comment:
The related logic of `on_backup_reply` function is like:
```
if(response.status == backup_status::CHECKPOINTED){
// do something
}
if(response.status == backup_status::UPLOADING){
// do something
}
if(response.status == backup_status::SUCCEED){
// do something
return;
}
// status: CHECKPOINTING, CHECKPOINTED, UPLOADING
print_log
retry_backup
```
The purpose that I don't use `if...else` is to print log and call retry_backup function, you can see the whole `on_backup_reply` function.
##########
src/rdsn/src/meta/meta_backup_engine.cpp:
##########
@@ -246,7 +246,31 @@ void meta_backup_engine::on_backup_reply(const error_code err,
}
}
- // TODO(heyuchen): handle other status
+ if (response.status == backup_status::UPLOADING) {
+ ddebug_f(
+ "backup_id({}): backup for partition {} from server {} is uploading, progress = {}",
+ _cur_backup.backup_id,
+ pid.to_string(),
+ primary.to_string(),
+ response.upload_progress);
+ }
+
+ if (response.status == backup_status::SUCCEED) {
Review Comment:
Same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] empiredan commented on a diff in pull request #1146: feat(backup): 8. meta handle backup_response during uploading and succeed
Posted by GitBox <gi...@apache.org>.
empiredan commented on code in PR #1146:
URL: https://github.com/apache/incubator-pegasus/pull/1146#discussion_r964759538
##########
src/rdsn/src/meta/meta_backup_engine.cpp:
##########
@@ -246,7 +246,31 @@ void meta_backup_engine::on_backup_reply(const error_code err,
}
}
- // TODO(heyuchen): handle other status
+ if (response.status == backup_status::UPLOADING) {
Review Comment:
Should we use `else if` instead ?
##########
src/rdsn/src/meta/meta_backup_engine.cpp:
##########
@@ -259,6 +283,48 @@ void meta_backup_engine::on_backup_reply(const error_code err,
retry_backup(pid);
}
+// ThreadPool: THREAD_POOL_DEFAULT
+void meta_backup_engine::complete_backup()
+{
+ std::string remote_backup_dir =
+ get_backup_path(get_backup_root(FLAGS_cold_backup_root, _cur_backup.backup_path),
+ _cur_backup.app_name,
+ _cur_backup.app_id,
+ _cur_backup.backup_id);
+ app_backup_info backup_info;
+ backup_info.app_id = _cur_backup.app_id;
+ backup_info.app_name = _cur_backup.app_name;
+ backup_info.backup_id = _cur_backup.backup_id;
+ backup_info.start_time_ms = _cur_backup.start_time_ms;
+ const auto end_time_ms = dsn_now_ms();
+ backup_info.end_time_ms = end_time_ms;
+ blob buf = json::json_forwarder<app_backup_info>::encode(backup_info);
+ error_code err = write_backup_file(remote_backup_dir, backup_constant::BACKUP_INFO, buf);
Review Comment:
```suggestion
const auto err = write_backup_file(remote_backup_dir, backup_constant::BACKUP_INFO, buf);
```
##########
src/rdsn/src/meta/meta_backup_engine.cpp:
##########
@@ -259,6 +283,48 @@ void meta_backup_engine::on_backup_reply(const error_code err,
retry_backup(pid);
}
+// ThreadPool: THREAD_POOL_DEFAULT
+void meta_backup_engine::complete_backup()
+{
+ std::string remote_backup_dir =
+ get_backup_path(get_backup_root(FLAGS_cold_backup_root, _cur_backup.backup_path),
+ _cur_backup.app_name,
+ _cur_backup.app_id,
+ _cur_backup.backup_id);
+ app_backup_info backup_info;
+ backup_info.app_id = _cur_backup.app_id;
+ backup_info.app_name = _cur_backup.app_name;
+ backup_info.backup_id = _cur_backup.backup_id;
+ backup_info.start_time_ms = _cur_backup.start_time_ms;
+ const auto end_time_ms = dsn_now_ms();
Review Comment:
```suggestion
const auto end_time_ms = static_cast<int64_t>(dsn_now_ms());
```
##########
src/rdsn/src/meta/meta_backup_engine.cpp:
##########
@@ -246,7 +246,31 @@ void meta_backup_engine::on_backup_reply(const error_code err,
}
}
- // TODO(heyuchen): handle other status
+ if (response.status == backup_status::UPLOADING) {
+ ddebug_f(
+ "backup_id({}): backup for partition {} from server {} is uploading, progress = {}",
+ _cur_backup.backup_id,
+ pid.to_string(),
+ primary.to_string(),
+ response.upload_progress);
+ }
+
+ if (response.status == backup_status::SUCCEED) {
Review Comment:
The same
##########
src/rdsn/src/meta/meta_backup_engine.cpp:
##########
@@ -259,6 +283,48 @@ void meta_backup_engine::on_backup_reply(const error_code err,
retry_backup(pid);
}
+// ThreadPool: THREAD_POOL_DEFAULT
+void meta_backup_engine::complete_backup()
+{
+ std::string remote_backup_dir =
+ get_backup_path(get_backup_root(FLAGS_cold_backup_root, _cur_backup.backup_path),
+ _cur_backup.app_name,
+ _cur_backup.app_id,
+ _cur_backup.backup_id);
+ app_backup_info backup_info;
+ backup_info.app_id = _cur_backup.app_id;
+ backup_info.app_name = _cur_backup.app_name;
+ backup_info.backup_id = _cur_backup.backup_id;
+ backup_info.start_time_ms = _cur_backup.start_time_ms;
+ const auto end_time_ms = dsn_now_ms();
+ backup_info.end_time_ms = end_time_ms;
+ blob buf = json::json_forwarder<app_backup_info>::encode(backup_info);
Review Comment:
```suggestion
const auto buf = json::json_forwarder<app_backup_info>::encode(backup_info);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] hycdong commented on a diff in pull request #1146: feat(backup): 8. meta handle backup_response during uploading and succeed
Posted by GitBox <gi...@apache.org>.
hycdong commented on code in PR #1146:
URL: https://github.com/apache/incubator-pegasus/pull/1146#discussion_r966655678
##########
src/rdsn/src/meta/meta_backup_engine.cpp:
##########
@@ -246,7 +246,31 @@ void meta_backup_engine::on_backup_reply(const error_code err,
}
}
- // TODO(heyuchen): handle other status
+ if (response.status == backup_status::UPLOADING) {
+ ddebug_f(
+ "backup_id({}): backup for partition {} from server {} is uploading, progress = {}",
+ _cur_backup.backup_id,
+ pid.to_string(),
+ primary.to_string(),
+ response.upload_progress);
+ }
+
+ if (response.status == backup_status::SUCCEED) {
+ ddebug_f("backup_id({}): backup for partition {} from server {} succeed",
Review Comment:
I think it is better to seperate them, you can reference the whole function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org