You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/11/26 08:54:35 UTC
[GitHub] [incubator-doris] morningman opened a new pull request #4964: [Bug][Compaction] Fix bug that output rowset is not deleted after compaction failure.
morningman opened a new pull request #4964:
URL: https://github.com/apache/incubator-doris/pull/4964
## Proposed changes
When the compaction fails, we must explicitly delete the output rowset,
otherwise the GC logic cannot process these rows.
## Types of changes
- [x] Bugfix (non-breaking change which fixes an issue)
## Checklist
- [x] I have create an issue on (Fix #4963 ), and have described the bug/feature there in detail
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #4964: [Bug][Compaction] Fix bug that output rowset is not deleted after compaction failure.
Posted by GitBox <gi...@apache.org>.
morningman merged pull request #4964:
URL: https://github.com/apache/incubator-doris/pull/4964
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] xy720 commented on a change in pull request #4964: [Bug][Compaction] Fix bug that output rowset is not deleted after compaction failure.
Posted by GitBox <gi...@apache.org>.
xy720 commented on a change in pull request #4964:
URL: https://github.com/apache/incubator-doris/pull/4964#discussion_r531036649
##########
File path: be/src/olap/reader.h
##########
@@ -128,7 +128,7 @@ class Reader {
}
uint64_t filtered_rows() const {
- return _stats.rows_del_filtered;
Review comment:
Hi, is it compatible with segment v1?
In alpha_rowset_reader.h, filtered_rows() only calculates rows_del_filtered.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] chaoyli commented on a change in pull request #4964: [Bug][Compaction] Fix bug that output rowset is not deleted after compaction failure.
Posted by GitBox <gi...@apache.org>.
chaoyli commented on a change in pull request #4964:
URL: https://github.com/apache/incubator-doris/pull/4964#discussion_r531494539
##########
File path: be/src/olap/compaction.cpp
##########
@@ -164,17 +173,11 @@ void Compaction::modify_rowsets() {
_tablet->save_meta();
}
-OLAPStatus Compaction::gc_unused_rowsets() {
+void Compaction::gc_output_rowset() {
Review comment:
why not call gc_output_rowset in every succeed? It seems a new imported bug ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #4964: [Bug][Compaction] Fix bug that output rowset is not deleted after compaction failure.
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #4964:
URL: https://github.com/apache/incubator-doris/pull/4964#discussion_r531352353
##########
File path: be/src/olap/reader.h
##########
@@ -128,7 +128,7 @@ class Reader {
}
uint64_t filtered_rows() const {
- return _stats.rows_del_filtered;
Review comment:
Yes, it is compatible. Because rows_conditions_filtered is only used for SegmentV2, so it should always to 0 in Segment V1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] xy720 commented on a change in pull request #4964: [Bug][Compaction] Fix bug that output rowset is not deleted after compaction failure.
Posted by GitBox <gi...@apache.org>.
xy720 commented on a change in pull request #4964:
URL: https://github.com/apache/incubator-doris/pull/4964#discussion_r531036649
##########
File path: be/src/olap/reader.h
##########
@@ -128,7 +128,7 @@ class Reader {
}
uint64_t filtered_rows() const {
- return _stats.rows_del_filtered;
Review comment:
Hi, is it compatible with segment v1?
In alpha_rowset_reader.h, filtered_rows() is only calculate rows_del_filtered.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #4964: [Bug][Compaction] Fix bug that output rowset is not deleted after compaction failure.
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #4964:
URL: https://github.com/apache/incubator-doris/pull/4964#discussion_r531827194
##########
File path: be/src/olap/compaction.cpp
##########
@@ -164,17 +173,11 @@ void Compaction::modify_rowsets() {
_tablet->save_meta();
}
-OLAPStatus Compaction::gc_unused_rowsets() {
+void Compaction::gc_output_rowset() {
Review comment:
`gc_output_rowset()` should only be called when compaction succeed.
In #4212, we have implemented a new compaction logic, so that the input rowset will be deleted later. And we only need to delete the output rowset when failed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org