[GitHub] [incubator-doris] gaodayue commented on issue #1753: Use crc32c checksum for segment v2

[GitBox <gi...@apache.org>]

gaodayue commented on issue #1753: Use crc32c checksum for segment v2
URL: https://github.com/apache/incubator-doris/pull/1753#issuecomment-528738158
 
 
   @imay comments inline
   
   > Even if HashUtils::crc_hash doesn't generate standard CRC-32 checksum, the checksums should be equal. Because all the checksums are compute by this function.
   
   The real problem is that the result of crc_hash can't be used as the input of the next crc_hash call. That is, `crc_hash({"hello ", "world"}, 0) != crc_hash({"hello"}, 0)`.
   
   > we have already had a CRC-32 implementation in be/src/olap/utils.cpp. Can you unify this and your new implementation?
   
   `olap_adler32` and `olap_crc32` are both used in V1 segment, which we can't change because of compatibility issue. I think they can be removed when we're removing V1 related code.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@doris.apache.org
For additional commands, e-mail: dev-help@doris.apache.org