You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/06/19 02:05:06 UTC
[GitHub] [incubator-doris] morningman commented on issue #3893: [Proposal] remove strict report version check of TabletReport
morningman commented on issue #3893:
URL: https://github.com/apache/incubator-doris/issues/3893#issuecomment-646393889
The `report version` is mainly used to avoid errors in the following scenarios:
```
Time Line
+
+-------------------------+ | +--------------------------+
| FE: Create Table Thread | | | BE: Tablet Report Thread |
+-------------------------+ | +--------------------------+
|
|
| 1. Fetch all tablets currently exist
| on BE(which is empty), and ready to
| report.
|
2. FE send Create Tablet Task to BE. |
BE receives the task and create Tablet X. |
|
|
3. BE finished the Create Tablet Task |
and send a "FinishTaskReport" to FE. |
|
|
4. FE receive the "FinishTaskReport", and |
finally, Tablet X takes effect on FE's |
meta. |
|
| 5. The tablet report arrives, which
| contains no tablets.
| Therefore, the FE will think that
| the Tablet X does not exist on the BE,
| and will delete the Tablet X informationn
| in the metadata, thereby causing information loss.
v
```
But with the report version. In step 1, The report thread when take report version 0.
And in step 3 and 4, the report version of the BE will be updated to 1.
So finally in step 5, FE will find that the report version is stale, and ignore that tablet report.
The reason why there are so many `out-of-date` reports in the production environment is because we update the report version in some unnecessary places. For example, when the BE processes the publish task, we will also increase the report version of the BE. If the load is very frequent, it will result in a large number of `out-of-date` reports.
It is not necessary to update the report version after the publish task. Because this is actually a problem left over by history. In the reporting logic of the current version, we will no longer decrease the version information of the replica in the FE metadata according to the report. So even if we receive a stale version of the report, it does not matter.
In our test environment (8 BEs), the average `out-of-date` report will occur 60-80 times an hour before, and after the upgrade, it will be reduced to 1-2 times per hour. This will no longer affect the cluster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org