You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "zzcclp (via GitHub)" <gi...@apache.org> on 2025/03/03 07:00:31 UTC
[PR] [GLUTEN-8872][CH] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
zzcclp opened a new pull request, #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873
## What changes were proposed in this pull request?
Support Delta Deletion Vectors read for CH backend.
(Fixes: \#ISSUE-ID)
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8872][CH] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693452311
https://github.com/apache/incubator-gluten/issues/8872
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
Posted by "zzcclp (via GitHub)" <gi...@apache.org>.
zzcclp commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693636751
@CodiumAI-Agent /improve
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8872][CH] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
Posted by "zzcclp (via GitHub)" <gi...@apache.org>.
zzcclp commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693632684
@CodiumAI-Agent /review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693666929
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
Posted by "CodiumAI-Agent (via GitHub)" <gi...@apache.org>.
CodiumAI-Agent commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2695982710
## PR Code Suggestions โจ
No code suggestions found for the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8872][CH] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693599165
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
Posted by "CodiumAI-Agent (via GitHub)" <gi...@apache.org>.
CodiumAI-Agent commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693638605
## PR Code Suggestions โจ
<!-- df8f606 -->
<table><thead><tr><td><strong>Category</strong></td><td align=left><strong>Suggestion </strong></td><td align=center><strong>Impact</strong></td></tr><tbody><tr><td rowspan=1>Possible issue</td>
<td>
<details><summary>Correct checksum error message</summary>
___
**Update the error message in the checksum validation to indicate a checksum mismatch <br>instead of a magic number mismatch.**
[cpp-ch/local-engine/Storages/SubstraitSource/Delta/Bitmap/DeltaDVRoaringBitmapArray.cpp [89-90]](https://github.com/apache/incubator-gluten/pull/8873/files#diff-f1625a6ac2a39d4b79842f262935a102a793681c99382f3439566aa10c12f776R89-R90)
```diff
if (expected_checksum != checksum_value)
- throw DB::Exception(DB::ErrorCodes::BAD_ARGUMENTS, "The magic num is mismatch.");
+ throw DB::Exception(DB::ErrorCodes::BAD_ARGUMENTS, "Checksum mismatch.");
```
<details><summary>Suggestion importance[1-10]: 7</summary>
__
Why: The suggestion improves the error message by accurately reflecting a checksum mismatch rather than a magic number mismatch, which enhances clarity and debugging; however, it is a minor improvement addressing a non-critical issue.
</details></details></td><td align=center>Medium
</td></tr></tr></tbody></table>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
Posted by "zzcclp (via GitHub)" <gi...@apache.org>.
zzcclp merged PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
Posted by "CodiumAI-Agent (via GitHub)" <gi...@apache.org>.
CodiumAI-Agent commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693635008
## PR Reviewer Guide ๐
Here are some key observations to aid the review process:
<table>
<tr><td>โฑ๏ธ <strong>Estimated effort to review</strong>: 3 ๐ต๐ต๐ตโชโช</td></tr>
<tr><td>๐งช <strong>PR contains tests</strong></td></tr>
<tr><td>๐ <strong>No security concerns identified</strong></td></tr>
<tr><td>โก <strong>Recommended focus areas for review</strong><br><br>
<details><summary><a href='https://github.com/apache/incubator-gluten/pull/8873/files#diff-f1625a6ac2a39d4b79842f262935a102a793681c99382f3439566aa10c12f776R64-R90'><strong>Error Message Consistency</strong></a>
The same error message ("The magic num is mismatch.") is reused for both the magic number and checksum validations. It is recommended to provide distinct and descriptive error messages for each failure scenario.
</summary>
```c++
if (data_size != size)
throw DB::Exception(DB::ErrorCodes::BAD_ARGUMENTS, "The size of the deletion vector is mismatch.");
int checksum_value = static_cast<Int32>(crc32_z(0L, reinterpret_cast<unsigned char*>(in.position()), size));
int magic_num;
readBinaryLittleEndian(magic_num, in);
if (magic_num != 1681511377)
throw DB::Exception(DB::ErrorCodes::BAD_ARGUMENTS, "The magic num is mismatch.");
int64_t bitmap_array_size;
readBinaryLittleEndian(bitmap_array_size, in);
roaring_bitmap_array.reserve(bitmap_array_size);
for (size_t i = 0; i < bitmap_array_size; ++i)
{
int bitmap_index;
readBinaryLittleEndian(bitmap_index, in);
roaring::Roaring r = roaring::Roaring::read(in.position());
size_t current_bitmap_size = r.getSizeInBytes();
in.ignore(current_bitmap_size);
roaring_bitmap_array.push_back(r);
}
int expected_checksum;
readBinaryBigEndian(expected_checksum, in);
if (expected_checksum != checksum_value)
throw DB::Exception(DB::ErrorCodes::BAD_ARGUMENTS, "The magic num is mismatch.");
```
</details>
</td></tr>
</table>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8872][CH] Support Delta Deletion Vectors read for CH backend [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693452780
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org