You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/07/21 02:39:34 UTC
[GitHub] [doris] jacktengg opened a new pull request, #11060: [improvement] Simplify scan node logic: removing _materialized_blocks…
jacktengg opened a new pull request, #11060:
URL: https://github.com/apache/doris/pull/11060
… and transfer thread
# Proposed changes
Issue Number: close #xxx
## Problem Summary:
Describe the overview of changes.
## Checklist(Required)
1. Does it affect the original behavior: (Yes/No/I Don't know)
2. Has unit tests been added: (Yes/No/No Need)
3. Has document been added or modified: (Yes/No/No Need)
4. Does it need to update dependencies: (Yes/No)
5. Are there any changes that cannot be rolled back: (Yes/No)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morningman commented on a diff in pull request #11060: [improvement] Simplify scan node logic: removing _materialized_blocks…
Posted by GitBox <gi...@apache.org>.
morningman commented on code in PR #11060:
URL: https://github.com/apache/doris/pull/11060#discussion_r935712255
##########
be/src/vec/exec/volap_scan_node.cpp:
##########
@@ -502,17 +377,18 @@ void VOlapScanNode::scanner_thread(VOlapScanner* scanner) {
int64_t raw_bytes_read = 0;
int64_t raw_bytes_threshold = config::doris_scanner_row_bytes;
bool get_free_block = true;
+ bool reached_limit = false;
int num_rows_in_block = 0;
// Has to wait at least one full block, or it will cause a lot of schedule task in priority
// queue, it will affect query latency and query concurrency for example ssb 3.3.
while (!eos && raw_bytes_read < raw_bytes_threshold &&
((raw_rows_read < raw_rows_threshold && get_free_block) ||
num_rows_in_block < _runtime_state->batch_size())) {
- if (UNLIKELY(_transfer_done)) {
+ if (UNLIKELY(state->is_cancelled())) {
eos = true;
status = Status::Cancelled("Cancelled");
- LOG(INFO) << "Scan thread cancelled, cause query done, maybe reach limit.";
+ LOG(INFO) << "Scan thread cancelled.";
Review Comment:
I think this log can be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morningman closed pull request #11060: [improvement] Simplify scan node logic: removing _materialized_blocks…
Posted by GitBox <gi...@apache.org>.
morningman closed pull request #11060: [improvement] Simplify scan node logic: removing _materialized_blocks…
URL: https://github.com/apache/doris/pull/11060
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morningman commented on pull request #11060: [improvement] Simplify scan node logic: removing _materialized_blocks…
Posted by GitBox <gi...@apache.org>.
morningman commented on PR #11060:
URL: https://github.com/apache/doris/pull/11060#issuecomment-1202849217
Could you please explain more about the threading model of this olap scan node.
For example:
1. How to determine the concurrency.
2. How to submit the scanner thread.
3. How to transfer the block from scanner to scan node.
So that other developer can easily understand its logic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org