You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "zzcclp (via GitHub)" <gi...@apache.org> on 2025/03/03 07:00:31 UTC

[PR] [GLUTEN-8872][CH] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

zzcclp opened a new pull request, #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873

   ## What changes were proposed in this pull request?
   
   Support Delta Deletion Vectors read for CH backend.
   
   (Fixes: \#ISSUE-ID)
   
   ## How was this patch tested?
   
   (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
   
   
   (If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-8872][CH] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693452311

   https://github.com/apache/incubator-gluten/issues/8872


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

Posted by "zzcclp (via GitHub)" <gi...@apache.org>.
zzcclp commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693636751

   @CodiumAI-Agent /improve


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-8872][CH] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

Posted by "zzcclp (via GitHub)" <gi...@apache.org>.
zzcclp commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693632684

   @CodiumAI-Agent /review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693666929

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

Posted by "CodiumAI-Agent (via GitHub)" <gi...@apache.org>.
CodiumAI-Agent commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2695982710

   ## PR Code Suggestions โœจ
   
   No code suggestions found for the PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-8872][CH] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693599165

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

Posted by "CodiumAI-Agent (via GitHub)" <gi...@apache.org>.
CodiumAI-Agent commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693638605

   ## PR Code Suggestions โœจ
   
   <!-- df8f606 -->
   
   <table><thead><tr><td><strong>Category</strong></td><td align=left><strong>Suggestion&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </strong></td><td align=center><strong>Impact</strong></td></tr><tbody><tr><td rowspan=1>Possible issue</td>
   <td>
   
   
   
   <details><summary>Correct checksum error message</summary>
   
   ___
   
   
   **Update the error message in the checksum validation to indicate a checksum mismatch <br>instead of a magic number mismatch.**
   
   [cpp-ch/local-engine/Storages/SubstraitSource/Delta/Bitmap/DeltaDVRoaringBitmapArray.cpp [89-90]](https://github.com/apache/incubator-gluten/pull/8873/files#diff-f1625a6ac2a39d4b79842f262935a102a793681c99382f3439566aa10c12f776R89-R90)
   
   ```diff
    if (expected_checksum != checksum_value)
   -    throw DB::Exception(DB::ErrorCodes::BAD_ARGUMENTS, "The magic num is mismatch.");
   +    throw DB::Exception(DB::ErrorCodes::BAD_ARGUMENTS, "Checksum mismatch.");
   ```
   <details><summary>Suggestion importance[1-10]: 7</summary>
   
   __
   
   Why: The suggestion improves the error message by accurately reflecting a checksum mismatch rather than a magic number mismatch, which enhances clarity and debugging; however, it is a minor improvement addressing a non-critical issue.
   
   
   </details></details></td><td align=center>Medium
   
   </td></tr></tr></tbody></table>
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

Posted by "zzcclp (via GitHub)" <gi...@apache.org>.
zzcclp merged PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-8872][CH][Part-1] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

Posted by "CodiumAI-Agent (via GitHub)" <gi...@apache.org>.
CodiumAI-Agent commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693635008

   ## PR Reviewer Guide ๐Ÿ”
   
   Here are some key observations to aid the review process:
   
   <table>
   <tr><td>โฑ๏ธ&nbsp;<strong>Estimated effort to review</strong>: 3 ๐Ÿ”ต๐Ÿ”ต๐Ÿ”ตโšชโšช</td></tr>
   <tr><td>๐Ÿงช&nbsp;<strong>PR contains tests</strong></td></tr>
   <tr><td>๐Ÿ”’&nbsp;<strong>No security concerns identified</strong></td></tr>
   <tr><td>โšก&nbsp;<strong>Recommended focus areas for review</strong><br><br>
   
   <details><summary><a href='https://github.com/apache/incubator-gluten/pull/8873/files#diff-f1625a6ac2a39d4b79842f262935a102a793681c99382f3439566aa10c12f776R64-R90'><strong>Error Message Consistency</strong></a>
   
   The same error message ("The magic num is mismatch.") is reused for both the magic number and checksum validations. It is recommended to provide distinct and descriptive error messages for each failure scenario.
   </summary>
   
   ```c++
   if (data_size != size)
       throw DB::Exception(DB::ErrorCodes::BAD_ARGUMENTS, "The size of the deletion vector is mismatch.");
   
   int checksum_value = static_cast<Int32>(crc32_z(0L, reinterpret_cast<unsigned char*>(in.position()), size));
   
   int magic_num;
   readBinaryLittleEndian(magic_num, in);
   if (magic_num != 1681511377)
       throw DB::Exception(DB::ErrorCodes::BAD_ARGUMENTS, "The magic num is mismatch.");
   
   int64_t bitmap_array_size;
   readBinaryLittleEndian(bitmap_array_size, in);
   
   roaring_bitmap_array.reserve(bitmap_array_size);
   for (size_t i = 0; i < bitmap_array_size; ++i)
   {
       int bitmap_index;
       readBinaryLittleEndian(bitmap_index, in);
       roaring::Roaring r = roaring::Roaring::read(in.position());
       size_t current_bitmap_size = r.getSizeInBytes();
       in.ignore(current_bitmap_size);
       roaring_bitmap_array.push_back(r);
   }
   int expected_checksum;
   readBinaryBigEndian(expected_checksum, in);
   if (expected_checksum != checksum_value)
       throw DB::Exception(DB::ErrorCodes::BAD_ARGUMENTS, "The magic num is mismatch.");
   ```
   
   </details>
   
   </td></tr>
   </table>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-8872][CH] Support Delta Deletion Vectors read for CH backend [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8873:
URL: https://github.com/apache/incubator-gluten/pull/8873#issuecomment-2693452780

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org