You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "amorynan (via GitHub)" <gi...@apache.org> on 2024/04/02 09:59:23 UTC
[PR] [FIX](inverted_index) fix inverted index write array with _doc is empty [doris]
amorynan opened a new pull request, #33170:
URL: https://github.com/apache/doris/pull/33170
## Proposed changes
when a array in single row is null or array has all null elements , the _doc param may has empty field, in this situation if we also call add_document, inverted index will call process_document() make sure field with inverted , but field is cleared, and _doc is empty which will make heap use after free, so we should avoid this situation happen!
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [FIX](inverted_index) fix inverted index write array with _doc is empty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #33170:
URL: https://github.com/apache/doris/pull/33170#issuecomment-2031650098
TeamCity be ut coverage result:
Function Coverage: 35.64% (8879/24911)
Line Coverage: 27.37% (72895/266304)
Region Coverage: 26.55% (37703/141987)
Branch Coverage: 23.34% (19211/82312)
Coverage Report: http://coverage.selectdb-in.cc/coverage/02a4c3a0f0faa2f8d02cb7d165825eefc82862b2_02a4c3a0f0faa2f8d02cb7d165825eefc82862b2/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [FIX](inverted_index) fix inverted index write array with _doc is empty [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #33170:
URL: https://github.com/apache/doris/pull/33170#issuecomment-2031589247
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [FIX](inverted_index) fix inverted index write array with _doc is empty [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on PR #33170:
URL: https://github.com/apache/doris/pull/33170#issuecomment-2033472794
run cloud_p1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [FIX](inverted_index) fix inverted index write array with _doc is empty [doris]
Posted by "amorynan (via GitHub)" <gi...@apache.org>.
amorynan commented on PR #33170:
URL: https://github.com/apache/doris/pull/33170#issuecomment-2031582988
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [FIX](inverted_index) fix inverted index write array with _doc is empty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #33170:
URL: https://github.com/apache/doris/pull/33170#issuecomment-2031649872
<details>
<summary>TPC-H: <b>Total hot run time: 38783 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 02a4c3a0f0faa2f8d02cb7d165825eefc82862b2, data reload: false
------ Round 1 ----------------------------------
q1 17647 4243 4081 4081
q2 2010 180 172 172
q3 10504 1196 1359 1196
q4 10193 886 1004 886
q5 7460 2979 2890 2890
q6 213 129 135 129
q7 1106 630 590 590
q8 9388 2016 2002 2002
q9 6689 6145 6155 6145
q10 8705 3570 3566 3566
q11 420 244 257 244
q12 387 237 222 222
q13 17776 2919 2906 2906
q14 281 251 241 241
q15 520 473 483 473
q16 527 388 375 375
q17 953 913 916 913
q18 7185 6544 6499 6499
q19 1614 1549 1533 1533
q20 605 311 328 311
q21 3606 3099 3107 3099
q22 373 310 316 310
Total cold run time: 108162 ms
Total hot run time: 38783 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4072 4052 4098 4052
q2 328 222 219 219
q3 2970 2973 2958 2958
q4 1872 1842 1813 1813
q5 5276 5253 5247 5247
q6 212 131 134 131
q7 2352 1917 1934 1917
q8 3246 3340 3329 3329
q9 8694 8639 8814 8639
q10 3902 3924 3818 3818
q11 536 455 456 455
q12 757 547 576 547
q13 18434 2943 2968 2943
q14 292 276 267 267
q15 512 469 471 469
q16 451 403 414 403
q17 1727 1698 1672 1672
q18 8083 7831 7928 7831
q19 4804 1684 1663 1663
q20 2014 1842 1830 1830
q21 5073 4895 4790 4790
q22 499 456 440 440
Total cold run time: 76106 ms
Total hot run time: 55433 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [FIX](inverted_index) fix inverted index write array with _doc is empty [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #33170:
URL: https://github.com/apache/doris/pull/33170#issuecomment-2033447128
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [FIX](inverted_index) fix inverted index write array with _doc is empty [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #33170:
URL: https://github.com/apache/doris/pull/33170#issuecomment-2033447096
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [FIX](inverted_index) fix inverted index write array with _doc is empty [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #33170:
URL: https://github.com/apache/doris/pull/33170#issuecomment-2031577541
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [FIX](inverted_index) fix inverted index write array with _doc is empty [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang merged PR #33170:
URL: https://github.com/apache/doris/pull/33170
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org