You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "gojumprope (via GitHub)" <gi...@apache.org> on 2023/02/27 07:45:50 UTC
[GitHub] [doris] gojumprope opened a new issue, #17184: [Bug] Cross join performance very slow
gojumprope opened a new issue, #17184:
URL: https://github.com/apache/doris/issues/17184
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
1.2.1
### What's Wrong?
I have table A with 323 rows data, table B with 96 rows data.
select * from table A (278 ms)
select * from table B (275 ms)
I want to make each rows in table A to have all of the table B data, the final outcome will be 323 * 96 = 31008 rows
select * from table A
cross join table B
but the query takes more than 30 seconds to load all the data
### What You Expected?
cross join data should load faster, query only 30k data should load within 1 second.
I have try using left join on 1=1, its speed even faster than cross join
select * from area_test
left join table_15_minute_test on 1 = 1
### How to Reproduce?
CREATE TABLE `area_test` (
`id` bigint,
`childid` bigint
)
DISTRIBUTED BY HASH(id) BUCKETS 32
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);
INSERT INTO area_test
VALUES
(645,637),
(645,640),
(646,637),
(646,640),
(647,637),
(647,640),
(648,637),
(648,640),
(649,637),
(649,640),
(650,637),
(650,640),
(651,637),
(651,640),
(652,637),
(652,640),
(653,637),
(653,640),
(654,637),
(654,640),
(656,637),
(656,640),
(657,637),
(657,640),
(658,637),
(658,640),
(659,637),
(659,640),
(660,637),
(660,640),
(661,637),
(661,640),
(662,637),
(662,640),
(663,637),
(663,640),
(664,637),
(664,640),
(665,637),
(665,640),
(666,637),
(666,640),
(667,637),
(667,640),
(670,639),
(670,640),
(670,641),
(671,639),
(671,640),
(671,641),
(672,639),
(672,640),
(672,641),
(673,639),
(673,640),
(673,641),
(675,639),
(675,640),
(675,641),
(676,638),
(676,640),
(676,642),
(677,638),
(677,640),
(677,642),
(678,638),
(678,640),
(678,642),
(679,638),
(679,640),
(679,642),
(680,638),
(680,640),
(680,642),
(681,638),
(681,640),
(681,642),
(682,638),
(682,640),
(682,642),
(683,638),
(683,640),
(683,642),
(684,638),
(684,640),
(684,642),
(685,638),
(685,640),
(685,642),
(686,638),
(686,640),
(686,642),
(687,638),
(687,640),
(687,642),
(690,638),
(690,640),
(690,642),
(692,638),
(692,640),
(692,642),
(693,638),
(693,640),
(693,642),
(694,638),
(694,640),
(694,642),
(695,638),
(695,640),
(695,642),
(696,638),
(696,640),
(696,642),
(697,638),
(697,640),
(697,642),
(698,638),
(698,640),
(698,642),
(699,638),
(699,640),
(699,642),
(700,638),
(700,640),
(700,642),
(701,638),
(701,640),
(701,642),
(702,638),
(702,640),
(702,642),
(703,638),
(703,640),
(703,642),
(704,638),
(704,640),
(704,642),
(705,638),
(705,640),
(705,642),
(706,638),
(706,640),
(706,642),
(707,638),
(707,640),
(707,642),
(708,638),
(708,640),
(708,642),
(709,638),
(709,640),
(709,642),
(710,638),
(710,640),
(710,642),
(711,638),
(711,640),
(711,642),
(712,638),
(712,640),
(712,643),
(713,638),
(713,640),
(713,643),
(714,638),
(714,640),
(714,643),
(716,638),
(716,640),
(716,643),
(718,638),
(718,640),
(718,643),
(719,638),
(719,640),
(719,643),
(720,638),
(720,640),
(720,643),
(721,638),
(721,640),
(721,643),
(722,638),
(722,640),
(722,643),
(725,638),
(725,640),
(725,643),
(726,638),
(726,640),
(726,643),
(727,638),
(727,640),
(727,643),
(728,638),
(728,640),
(728,643),
(729,638),
(729,640),
(729,643),
(730,638),
(730,640),
(730,643),
(731,638),
(731,640),
(731,643),
(733,638),
(733,640),
(733,643),
(734,638),
(734,640),
(734,643),
(735,638),
(735,640),
(735,643),
(737,638),
(737,640),
(737,643),
(738,638),
(738,640),
(738,643),
(739,638),
(739,640),
(739,643),
(740,638),
(740,640),
(740,643),
(741,638),
(741,640),
(741,644),
(744,638),
(744,640),
(744,644),
(745,638),
(745,640),
(745,644),
(746,638),
(746,640),
(746,644),
(747,638),
(747,640),
(747,644),
(748,638),
(748,640),
(748,644),
(749,638),
(749,640),
(749,644),
(750,638),
(750,640),
(750,644),
(751,638),
(751,640),
(751,644),
(752,638),
(752,640),
(752,644),
(753,638),
(753,640),
(753,644),
(754,638),
(754,640),
(754,644),
(755,638),
(755,640),
(755,644),
(756,638),
(756,640),
(756,644),
(757,638),
(757,640),
(757,644),
(758,638),
(758,640),
(758,644),
(759,638),
(759,640),
(759,644),
(761,638),
(761,640),
(761,644),
(762,638),
(762,640),
(762,644),
(763,638),
(763,640),
(763,644),
(764,638),
(764,640),
(764,644),
(766,639),
(766,640),
(768,639),
(768,640),
(776,812),
(779,638),
(779,640),
(779,643),
(780,638),
(780,640),
(780,643),
(781,638),
(781,640),
(781,643),
(782,639),
(782,640),
(782,641),
(783,638),
(783,640),
(783,644),
(785,638),
(785,640),
(785,644),
(787,639),
(787,640),
(788,638),
(788,640),
(788,643),
(811,639),
(811,640),
(813,638),
(813,640),
(813,643)
CREATE TABLE `table_15_minute_test` (
`__time` datetime
)
DISTRIBUTED BY HASH(__time) BUCKETS 32
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);
INSERT INTO table_15_minute_test
Values
('2023-02-27 00:00:00'),
('2023-02-27 00:15:00'),
('2023-02-27 00:30:00'),
('2023-02-27 00:45:00'),
('2023-02-27 01:00:00'),
('2023-02-27 01:15:00'),
('2023-02-27 01:30:00'),
('2023-02-27 01:45:00'),
('2023-02-27 02:00:00'),
('2023-02-27 02:15:00'),
('2023-02-27 02:30:00'),
('2023-02-27 02:45:00'),
('2023-02-27 03:00:00'),
('2023-02-27 03:15:00'),
('2023-02-27 03:30:00'),
('2023-02-27 03:45:00'),
('2023-02-27 04:00:00'),
('2023-02-27 04:15:00'),
('2023-02-27 04:30:00'),
('2023-02-27 04:45:00'),
('2023-02-27 05:00:00'),
('2023-02-27 05:15:00'),
('2023-02-27 05:30:00'),
('2023-02-27 05:45:00'),
('2023-02-27 06:00:00'),
('2023-02-27 06:15:00'),
('2023-02-27 06:30:00'),
('2023-02-27 06:45:00'),
('2023-02-27 07:00:00'),
('2023-02-27 07:15:00'),
('2023-02-27 07:30:00'),
('2023-02-27 07:45:00'),
('2023-02-27 08:00:00'),
('2023-02-27 08:15:00'),
('2023-02-27 08:30:00'),
('2023-02-27 08:45:00'),
('2023-02-27 09:00:00'),
('2023-02-27 09:15:00'),
('2023-02-27 09:30:00'),
('2023-02-27 09:45:00'),
('2023-02-27 10:00:00'),
('2023-02-27 10:15:00'),
('2023-02-27 10:30:00'),
('2023-02-27 10:45:00'),
('2023-02-27 11:00:00'),
('2023-02-27 11:15:00'),
('2023-02-27 11:30:00'),
('2023-02-27 11:45:00'),
('2023-02-27 12:00:00'),
('2023-02-27 12:15:00'),
('2023-02-27 12:30:00'),
('2023-02-27 12:45:00'),
('2023-02-27 13:00:00'),
('2023-02-27 13:15:00'),
('2023-02-27 13:30:00'),
('2023-02-27 13:45:00'),
('2023-02-27 14:00:00'),
('2023-02-27 14:15:00'),
('2023-02-27 14:30:00'),
('2023-02-27 14:45:00'),
('2023-02-27 15:00:00'),
('2023-02-27 15:15:00'),
('2023-02-27 15:30:00'),
('2023-02-27 15:45:00'),
('2023-02-27 16:00:00'),
('2023-02-27 16:15:00'),
('2023-02-27 16:30:00'),
('2023-02-27 16:45:00'),
('2023-02-27 17:00:00'),
('2023-02-27 17:15:00'),
('2023-02-27 17:30:00'),
('2023-02-27 17:45:00'),
('2023-02-27 18:00:00'),
('2023-02-27 18:15:00'),
('2023-02-27 18:30:00'),
('2023-02-27 18:45:00'),
('2023-02-27 19:00:00'),
('2023-02-27 19:15:00'),
('2023-02-27 19:30:00'),
('2023-02-27 19:45:00'),
('2023-02-27 20:00:00'),
('2023-02-27 20:15:00'),
('2023-02-27 20:30:00'),
('2023-02-27 20:45:00'),
('2023-02-27 21:00:00'),
('2023-02-27 21:15:00'),
('2023-02-27 21:30:00'),
('2023-02-27 21:45:00'),
('2023-02-27 22:00:00'),
('2023-02-27 22:15:00'),
('2023-02-27 22:30:00'),
('2023-02-27 22:45:00'),
('2023-02-27 23:00:00'),
('2023-02-27 23:15:00'),
('2023-02-27 23:30:00'),
('2023-02-27 23:45:00')
select * from area_test
cross join table_15_minute_test
-- takes 35 seconds to finish query
### Anything Else?
explain select * from area_test
cross join table_15_minute_test
PLAN FRAGMENT 0
OUTPUT EXPRS:
<slot 3>
<slot 4>
<slot 5>
PARTITION: UNPARTITIONED
VRESULT SINK
4:VEXCHANGE
offset: 0
PLAN FRAGMENT 1
PARTITION: HASH_PARTITIONED: `default_cluster:dbo`.`area_test`.`id`
STREAM DATA SINK
EXCHANGE ID: 04
UNPARTITIONED
2:VNESTED LOOP JOIN
| join op: CROSS JOIN()
| cardinality=31,008
| vec output tuple id: 2
| vIntermediate tuple ids: 3 4
| output slot ids: 3 4 5
|
|----3:VEXCHANGE
| offset: 0
|
0:VOlapScanNode
TABLE: default_cluster:dbo.area_test(area_test), PREAGGREGATION: ON
partitions=1/1, tablets=32/32, tabletList=4124484,4124486,4124488 ...
cardinality=323, avgRowSize=214.87616, numNodes=3
PLAN FRAGMENT 2
PARTITION: HASH_PARTITIONED: `default_cluster:dbo`.`table_15_minute_test`.`__time`
STREAM DATA SINK
EXCHANGE ID: 03
UNPARTITIONED
1:VOlapScanNode
TABLE: default_cluster:dbo.table_15_minute_test(table_15_minute_test), PREAGGREGATION: ON
partitions=1/1, tablets=32/32, tabletList=4124416,4124418,4124420 ...
cardinality=96, avgRowSize=489.0625, numNodes=3
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] gojumprope commented on issue #17184: [Bug] Cross join performance very slow
Posted by "gojumprope (via GitHub)" <gi...@apache.org>.
gojumprope commented on issue #17184:
URL: https://github.com/apache/doris/issues/17184#issuecomment-1447343694
> select * from area_test
> cross join table_15_minute_test
I've tried in other server and no issue found (cost 0.19 sec)
Then I tried to restart all of the doris backend and the sql execute cost from 35 secs to 1 sec. I think that is something cache in the server, but I will close this issue first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] gojumprope closed issue #17184: [Bug] Cross join performance very slow
Posted by "gojumprope (via GitHub)" <gi...@apache.org>.
gojumprope closed issue #17184: [Bug] Cross join performance very slow
URL: https://github.com/apache/doris/issues/17184
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yagagagaga commented on issue #17184: [Bug] Cross join performance very slow
Posted by "yagagagaga (via GitHub)" <gi...@apache.org>.
yagagagaga commented on issue #17184:
URL: https://github.com/apache/doris/issues/17184#issuecomment-1445900567
I tried it in my own computer environment according to your process. And only cost 0.11 sec to execute this sql.
<img width="371" alt="image" src="https://user-images.githubusercontent.com/21151126/221511367-5f888a07-4060-4ae1-a4a9-54357d1c7872.png">
Maybe you can try it again in another server?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org