You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2023/01/07 03:08:35 UTC
[GitHub] [doris] luozenglin opened a new pull request, #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
luozenglin opened a new pull request, #15693:
URL: https://github.com/apache/doris/pull/15693
# Proposed changes
Issue Number: close #15678
## Problem summary
Describe your changes.
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [ ] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [ ] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [ ] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] luozenglin commented on pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
luozenglin commented on PR #15693:
URL: https://github.com/apache/doris/pull/15693#issuecomment-1374528763
> The root cause is that there is no implementation for float/doube in `get_creator` function in `be/src/olap/predicate_creator.h`. `get_creator` will return default value nullptr and cause create_predicate core in the following code.
>
> > template <PredicateType PT, typename ConditionType>
> > inline ColumnPredicate* create_predicate(const TabletColumn& column, int index,
> > const ConditionType& conditions, bool opposite,
> > MemPool* pool) {
> > return get_creator<PT, ConditionType>(column.type())
> > ->create(column, index, conditions, opposite, pool);
> > }
Yes, but what confuses me is why the create_predicate of type double was not implemented before? In other words, the double type was not pushed down to the storage layer before because of any pitfalls?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xiaokang commented on pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
xiaokang commented on PR #15693:
URL: https://github.com/apache/doris/pull/15693#issuecomment-1374520821
This PR can be merged for temporary fix. I'll fix get_creator and add testcases for every data type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xiaokang commented on pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
xiaokang commented on PR #15693:
URL: https://github.com/apache/doris/pull/15693#issuecomment-1374693541
> Yes, but what confuses me is why the create_predicate of type double was not implemented before? In other words, the double type was not pushed down to the storage layer before because of any pitfalls?
I have confirmed float/double type was not pushed down to the storage layer by checking the code in _normalize_conjuncts() and discussing with @moonming . The pitfalls may be the precision problem. I will do some research on other databases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
Gabriel39 merged PR #15693:
URL: https://github.com/apache/doris/pull/15693
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
hello-stephen commented on PR #15693:
URL: https://github.com/apache/doris/pull/15693#issuecomment-1374365306
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 35.07 seconds
load time: 475 seconds
storage size: 17121111998 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230107032959_clickbench_pr_75215.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
Gabriel39 commented on PR #15693:
URL: https://github.com/apache/doris/pull/15693#issuecomment-1374395182
PTAL @xiaokang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
yiguolei commented on code in PR #15693:
URL: https://github.com/apache/doris/pull/15693#discussion_r1064229242
##########
regression-test/suites/query_p0/sql_functions/string_functions/test_split_part.groovy:
##########
@@ -27,4 +27,6 @@ suite("test_split_part") {
"""
exception "[RUNTIME_ERROR]Argument at index 3 for function split_part must be constant"
}
+
+ qt_1 "select split_part(k8, '1', 1), k8, split_part(concat(k8, '12'), '1', 1) from test_query_db.test order by k8 limit 2;"
Review Comment:
Does this regression test related with the issue? It does not have any pushdown predicates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] luozenglin commented on a diff in pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
luozenglin commented on code in PR #15693:
URL: https://github.com/apache/doris/pull/15693#discussion_r1064261303
##########
regression-test/suites/query_p0/sql_functions/string_functions/test_split_part.groovy:
##########
@@ -27,4 +27,6 @@ suite("test_split_part") {
"""
exception "[RUNTIME_ERROR]Argument at index 3 for function split_part must be constant"
}
+
+ qt_1 "select split_part(k8, '1', 1), k8, split_part(concat(k8, '12'), '1', 1) from test_query_db.test order by k8 limit 2;"
Review Comment:
topn generated a runtime predicate, a newly supported feature in pr #15558
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xiaokang commented on pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
xiaokang commented on PR #15693:
URL: https://github.com/apache/doris/pull/15693#issuecomment-1374492130
> PTAL @xiaokang
I will check it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] luozenglin commented on pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
luozenglin commented on PR #15693:
URL: https://github.com/apache/doris/pull/15693#issuecomment-1374528489
> create_predicate
Yes, but what confuses me is why the `create_predicate` of type double was not implemented before? In other words, the double type was not pushed down to the storage layer before because of any pitfalls?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xiaokang commented on pull request #15693: [fix](predicate) fix be core dump caused by pushing down the double column predicate
Posted by GitBox <gi...@apache.org>.
xiaokang commented on PR #15693:
URL: https://github.com/apache/doris/pull/15693#issuecomment-1374520544
The root cause is that there is no implementation for float/doube in `get_creator` function in `be/src/olap/predicate_creator.h`. `get_creator` will return default value nullptr and cause create_predicate core in the following code.
> template <PredicateType PT, typename ConditionType>
inline ColumnPredicate* create_predicate(const TabletColumn& column, int index,
const ConditionType& conditions, bool opposite,
MemPool* pool) {
return get_creator<PT, ConditionType>(column.type())
->create(column, index, conditions, opposite, pool);
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org