You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/04/16 09:35:44 UTC
[GitHub] [incubator-doris] HappenLee opened a new issue #3337: 为什么在单机执行的聚合查询需要多个FRAGMENT
HappenLee opened a new issue #3337: 为什么在单机执行的聚合查询需要多个FRAGMENT
URL: https://github.com/apache/incubator-doris/issues/3337
在进行单机doris(单个be,fe) 进行性能测试时, 发现下面的查询语句生成了3个fragment：
explain select max(C_PHONE) from customer group by C_MKTSEGMENT;
+-----------------------------------------------------------------------------+
| Explain String |
+-----------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS:<slot 3> | |
| PARTITION: UNPARTITIONED |
| |
| RESULT SINK |
| |
| 4:EXCHANGE |
| tuple ids: 1 |
| |
| PLAN FRAGMENT 1 |
| OUTPUT EXPRS: |
| PARTITION: HASH_PARTITIONED: <slot 2> |
| |
| STREAM DATA SINK |
| EXCHANGE ID: 04 |
| UNPARTITIONED |
| |
| 3:AGGREGATE (merge finalize) |
| | output: max(<slot 3>) |
| | group by: <slot 2> |
| | tuple ids: 1 |
| | |
| 2:EXCHANGE |
| tuple ids: 1 |
| |
| PLAN FRAGMENT 2 |
| OUTPUT EXPRS: |
| PARTITION: RANDOM |
| |
| STREAM DATA SINK |
| EXCHANGE ID: 02 |
| HASH_PARTITIONED: <slot 2> |
| |
| 1:AGGREGATE (update serialize) |
| | STREAMING |
| | output: max(`C_PHONE`) |
| | group by: `C_MKTSEGMENT` |
| | tuple ids: 1 |
| | |
| 0:OlapScanNode |
| TABLE: customer |
| PREAGGREGATION: ON |
| partitions=1/1 |
| rollup: customer |
| tabletRatio=10/10 |
| tabletList=11039,11041,11043,11045,11047,11049,11051,11053,11055,11057 |
| cardinality=29999999 |
| avgRowSize=46.694683 |
| numNodes=1 |
| tuple ids: 0 |
+-----------------------------------------------------------------------------+
是否可以在单机执行时将多个FRAGMENT简化为单个FRAGMENT，来节省对应的资源消耗。@imay
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] wutiangan commented on issue #3337: 为什么在单机执行的聚合查询需要多个FRAGMENT
Posted by GitBox <gi...@apache.org>.
wutiangan commented on issue #3337: 为什么在单机执行的聚合查询需要多个FRAGMENT
URL: https://github.com/apache/incubator-doris/issues/3337#issuecomment-614594783
@imay
“机器数只有在coordinate阶段才能感知”
在analysis时为啥不能感知机器的数目呢？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] imay commented on issue #3337: 为什么在单机执行的聚合查询需要多个FRAGMENT
Posted by GitBox <gi...@apache.org>.
imay commented on issue #3337: 为什么在单机执行的聚合查询需要多个FRAGMENT
URL: https://github.com/apache/incubator-doris/issues/3337#issuecomment-614542372
@HappenLee
这个是比较难以做到的，当前在规划分片的时候只能通过tablet数目来做，而机器数只有在coordinate阶段才能感知。
如果想达到你的目的，那么你可以把tablet数目改成1就可以了。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] kangkaisen commented on issue #3337: 为什么在单机执行的聚合查询需要多个FRAGMENT
Posted by GitBox <gi...@apache.org>.
kangkaisen commented on issue #3337:
URL: https://github.com/apache/incubator-doris/issues/3337#issuecomment-753956723
> 可以做到的。
在DistributedPlanner的createAggregationFragment 方法里，发现只有一个BE 节点，不拆两阶段就行。 改动很简单。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] HappenLee closed issue #3337: 为什么在单机执行的聚合查询需要多个FRAGMENT
Posted by GitBox <gi...@apache.org>.
HappenLee closed issue #3337:
URL: https://github.com/apache/incubator-doris/issues/3337
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org