You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/05/22 08:17:16 UTC
[GitHub] [incubator-doris] SaintBacchus opened a new issue, #9724: [Enhancement] Support record all the query profile on a persist storage
SaintBacchus opened a new issue, #9724:
URL: https://github.com/apache/incubator-doris/issues/9724
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Description
Now the FE ui only can show the last 100 query in the web page(with the request `/rest/v1/query_profile`).
100 may be not enough for the real bussiness scenario. But increase this hard code number will cause more memory use in the fe java process.
### Solution
So a nature idea is to store it in the persist storage system and browser the fe ui by reading the query profile data from storage.
The storage system will choose the InMemory to keep the same behaviour with now and DorisNative which is suitable for **frequent insert** and **not frequent point loop up** and **timed range scan**.
In my solution, I will dive thie impl into 4 Step:
- [ ] Abstract an interface named `ProfileStorage` with `write` and `read` method and impl the `InMemoryProfileStorage` which will keep the same behaviour with now.
- [ ] Impl the `DorisNativeProfileStorage` to storage the records in a doris olap table.
- [ ] Improve the insert performance in `DorisNativeProfileStorage` by use wal to submit all request in batch
- [ ] Modify the Web UI and separate the running and finished job to avoid update semantic for performance.
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] SaintBacchus closed issue #9724: [Enhancement] Support record all the query profile on a persist storage
Posted by GitBox <gi...@apache.org>.
SaintBacchus closed issue #9724: [Enhancement] Support record all the query profile on a persist storage
URL: https://github.com/apache/doris/issues/9724
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] SaintBacchus commented on issue #9724: [Enhancement] Support record all the query profile on a persist storage
Posted by GitBox <gi...@apache.org>.
SaintBacchus commented on issue #9724:
URL: https://github.com/apache/doris/issues/9724#issuecomment-1171978906
close this since this enhancement will be covered by https://github.com/apache/doris/pull/10533
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org