You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/03/13 03:30:07 UTC
[GitHub] [incubator-doris] morningman opened a new pull request #3098: [Bug]
tablet meta is not updated correctly after compaction
morningman opened a new pull request #3098: [Bug] tablet meta is not updated correctly after compaction
URL: https://github.com/apache/incubator-doris/pull/3098
This CL try to fix a potential bug describe in ISSUE: #3097. But I'm not sure this is the root cause.
Also remove lots of verbose log, and fix a memory leak.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#3098: [Bug] tablet meta is not updated correctly after compaction
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #3098: [Bug] tablet meta is not updated correctly after compaction
URL: https://github.com/apache/incubator-doris/pull/3098#discussion_r392133557
##########
File path: be/src/olap/task/engine_publish_version_task.cpp
##########
@@ -60,7 +60,7 @@ OLAPStatus EnginePublishVersionTask::finish() {
OLAPStatus publish_status = OLAP_SUCCESS;
TabletInfo tablet_info = tablet_rs.first;
RowsetSharedPtr rowset = tablet_rs.second;
- LOG(INFO) << "begin to publish version on tablet. "
+ VLOG(3) << "begin to publish version on tablet. "
Review comment:
There are too many this kind of logs in our env. Why you need a "begin" log?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#3098: [Bug] tablet meta is not updated correctly after compaction
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #3098: [Bug] tablet meta is not updated correctly after compaction
URL: https://github.com/apache/incubator-doris/pull/3098#discussion_r392197304
##########
File path: be/src/olap/task/engine_publish_version_task.cpp
##########
@@ -60,7 +60,7 @@ OLAPStatus EnginePublishVersionTask::finish() {
OLAPStatus publish_status = OLAP_SUCCESS;
TabletInfo tablet_info = tablet_rs.first;
RowsetSharedPtr rowset = tablet_rs.second;
- LOG(INFO) << "begin to publish version on tablet. "
+ VLOG(3) << "begin to publish version on tablet. "
Review comment:
Would better keep all publish version and transaction log. Our publish version and transaction performance is still poor. We need these logs to track issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#3098: [Bug] tablet meta is not updated correctly after compaction
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #3098: [Bug] tablet meta is not updated correctly after compaction
URL: https://github.com/apache/incubator-doris/pull/3098#discussion_r392239593
##########
File path: be/src/olap/task/engine_publish_version_task.cpp
##########
@@ -60,7 +60,7 @@ OLAPStatus EnginePublishVersionTask::finish() {
OLAPStatus publish_status = OLAP_SUCCESS;
TabletInfo tablet_info = tablet_rs.first;
RowsetSharedPtr rowset = tablet_rs.second;
- LOG(INFO) << "begin to publish version on tablet. "
+ VLOG(3) << "begin to publish version on tablet. "
Review comment:
OK, I will leave the begin and finish log
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request
#3098: [Bug] tablet meta is not updated correctly after compaction
Posted by GitBox <gi...@apache.org>.
morningman commented on a change in pull request #3098: [Bug] tablet meta is not updated correctly after compaction
URL: https://github.com/apache/incubator-doris/pull/3098#discussion_r392132474
##########
File path: be/src/olap/rowset/alpha_rowset.cpp
##########
@@ -166,7 +166,9 @@ OLAPStatus AlphaRowset::split_range(
RowBlockPosition step_pos;
std::shared_ptr<SegmentGroup> largest_segment_group = _segment_group_with_largest_size();
- if (largest_segment_group == nullptr) {
+ if (largest_segment_group == nullptr || largest_segment_group->current_num_rows_per_row_block() == 0) {
Review comment:
Yes, so I and a check here to avoid coredump in our product env, but leave a warning log to observe. This problem still needs to be analyzed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#3098: [Bug] tablet meta is not updated correctly after compaction
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #3098: [Bug] tablet meta is not updated correctly after compaction
URL: https://github.com/apache/incubator-doris/pull/3098#discussion_r392016663
##########
File path: be/src/olap/task/engine_publish_version_task.cpp
##########
@@ -60,7 +60,7 @@ OLAPStatus EnginePublishVersionTask::finish() {
OLAPStatus publish_status = OLAP_SUCCESS;
TabletInfo tablet_info = tablet_rs.first;
RowsetSharedPtr rowset = tablet_rs.second;
- LOG(INFO) << "begin to publish version on tablet. "
+ VLOG(3) << "begin to publish version on tablet. "
Review comment:
I think we shouldn't remove these logs.
These logs are very important for track transaction issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#3098: [Bug] tablet meta is not updated correctly after compaction
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #3098: [Bug] tablet meta is not updated correctly after compaction
URL: https://github.com/apache/incubator-doris/pull/3098#discussion_r392195850
##########
File path: be/src/olap/rowset/alpha_rowset.cpp
##########
@@ -166,7 +166,9 @@ OLAPStatus AlphaRowset::split_range(
RowBlockPosition step_pos;
std::shared_ptr<SegmentGroup> largest_segment_group = _segment_group_with_largest_size();
- if (largest_segment_group == nullptr) {
+ if (largest_segment_group == nullptr || largest_segment_group->current_num_rows_per_row_block() == 0) {
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #3098: [Bug]
tablet meta is not updated correctly after compaction
Posted by GitBox <gi...@apache.org>.
morningman merged pull request #3098: [Bug] tablet meta is not updated correctly after compaction
URL: https://github.com/apache/incubator-doris/pull/3098
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on a change in pull request
#3098: [Bug] tablet meta is not updated correctly after compaction
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on a change in pull request #3098: [Bug] tablet meta is not updated correctly after compaction
URL: https://github.com/apache/incubator-doris/pull/3098#discussion_r392016339
##########
File path: be/src/olap/rowset/alpha_rowset.cpp
##########
@@ -166,7 +166,9 @@ OLAPStatus AlphaRowset::split_range(
RowBlockPosition step_pos;
std::shared_ptr<SegmentGroup> largest_segment_group = _segment_group_with_largest_size();
- if (largest_segment_group == nullptr) {
+ if (largest_segment_group == nullptr || largest_segment_group->current_num_rows_per_row_block() == 0) {
Review comment:
Why add a check for `largest_segment_group->current_num_rows_per_row_block() == 0`?
You encounter this issue https://github.com/apache/incubator-doris/issues/2750?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org