You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Kikyou1997 (via GitHub)" <gi...@apache.org> on 2023/04/07 07:14:23 UTC
[GitHub] [doris] Kikyou1997 opened a new pull request, #18460: [feat](stats) Support preheating for stats
Kikyou1997 opened a new pull request, #18460:
URL: https://github.com/apache/doris/pull/18460
# Proposed changes
Load the most recently used stats as FE booted.
StatsCacheLoader will record the column(tblid-idxid-colname) it loaded to a map. This PR add a shutdown hook to FE so that we could writes the records in map to the metadata.
When booting, Stats of columns that recorded previously would be loaded by a single thread.
NOTE
This PR add new operation type of log, so the metadata is not backforward compatitable, once user or dev run their FE on the code of this version they cannot rollback to previous FE version.
This PR modified the
## Problem summary
Describe your changes.
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [ ] Has document been added or modified
* [ ] Does it need to update dependencies
* [ ] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #18460: [feat](stats) Support preheating for stats
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #18460:
URL: https://github.com/apache/doris/pull/18460#issuecomment-1500158205
and we should load the statistics info after analyze job success immediately
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #18460: [feat](stats) Support preheating for stats
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on PR #18460:
URL: https://github.com/apache/doris/pull/18460#issuecomment-1500170603
> and we should load the statistics info after analyze job success immediately
This logic already exists currently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 closed pull request #18460: [feat](stats) Support preheating for stats
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 closed pull request #18460: [feat](stats) Support preheating for stats
URL: https://github.com/apache/doris/pull/18460
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] englefly commented on pull request #18460: [feat](stats) Support preheating for stats
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #18460:
URL: https://github.com/apache/doris/pull/18460#issuecomment-1500087182
if there is no history information, can we load some stats on FE startup?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #18460: [feat](stats) Support preheating for stats
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on PR #18460:
URL: https://github.com/apache/doris/pull/18460#issuecomment-1500013568
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #18460: [feat](stats) Support preheating for stats
Posted by "Kikyou1997 (via GitHub)" <gi...@apache.org>.
Kikyou1997 commented on PR #18460:
URL: https://github.com/apache/doris/pull/18460#issuecomment-1500156372
> if there is no history information, can we load some stats on FE startup?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #18460: [feat](stats) Support preheating for stats
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on PR #18460:
URL: https://github.com/apache/doris/pull/18460#issuecomment-1500156832
i think just load top k newest column statistics is good enough, log the cache status is not necessary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] englefly commented on pull request #18460: [feat](stats) Support preheating for stats
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #18460:
URL: https://github.com/apache/doris/pull/18460#issuecomment-1500184561
>
if there is no history information, could you just load `k` stats?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org