You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "DA1OOO (via GitHub)" <gi...@apache.org> on 2023/12/26 03:30:59 UTC
[I] [Bug] enable pipeline resulting in longer query time In situations where there are too many tablets (0.06s -> 19s) [doris]
DA1OOO opened a new issue, #29041:
URL: https://github.com/apache/doris/issues/29041
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
Doris 2.3
### What's Wrong?
**table size**: `8MB, 9k+ rows, partitioned by hour, buckets num = 9`
**tablets number**: 15957 / 3 replica
**difference when enable pipeline**: `0.06s -> 19s`
<img width="884" alt="image" src="https://github.com/apache/doris/assets/56857867/8809f7a5-c692-44fd-9fe1-1e7e694b5642">
btw, in query profile, this query time as follow, different to mysql client. https://github.com/apache/doris/discussions/29024
Finally, explain the reason **why we have so many tablets**:
we have a table. **size = 25GB, rowCount = 302965685**, due to business continuity, we **partition by hour**.
then, we create a simple materialized view `create materialized view as select geo_country from xxx group by geo_country`.
after created, found that tablets has doubled. but materialized view size is small. when we select on materialized view, the query is very slow.
so we create a test table (as above) that only have 1 column, partitions by hour to test query time.
Then we discovered the problem mentioned at the beginning of the issue.
### What You Expected?
fix, we want enable pipeline to use workload group function.
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Bug] enable pipeline resulting in longer query time In situations where there are too many tablets (0.06s -> 19s) [doris]
Posted by "DA1OOO (via GitHub)" <gi...@apache.org>.
DA1OOO commented on issue #29041:
URL: https://github.com/apache/doris/issues/29041#issuecomment-1869311659
Perhaps due to the Query Profile, after turning off the profile, the query speed is basically the same when opening and closing the pipeline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Bug] enable pipeline resulting in longer query time In situations where there are too many tablets (0.06s -> 19s) [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on issue #29041:
URL: https://github.com/apache/doris/issues/29041#issuecomment-1872552949
@DA1OOO Yes, it is the profile. Currently, profile will too long time to generate. If you set enable_profile=true, then the query time maybe very long.
In doris 2.2. version, we will construct async profile to solve the problem,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Bug] enable pipeline resulting in longer query time In situations where there are too many tablets (0.06s -> 19s) [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei closed issue #29041: [Bug] enable pipeline resulting in longer query time In situations where there are too many tablets (0.06s -> 19s)
URL: https://github.com/apache/doris/issues/29041
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org