[jira] [Commented] (HUDI-8902) Investigate different read behavior on a field after promotion from float to double

["Danny Chen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-8902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17951307#comment-17951307 ] 

Danny Chen commented on HUDI-8902:
----------------------------------

A 1.1.0 fix via master branch: cac1bea2c5bde002f78b2203f9295832c6831716

> Investigate different read behavior on a field after promotion from float to double
> -----------------------------------------------------------------------------------
>
>                 Key: HUDI-8902
>                 URL: https://issues.apache.org/jira/browse/HUDI-8902
>             Project: Apache Hudi
>          Issue Type: Sub-task
>            Reporter: Y Ethan Guo
>            Assignee: Davis Zhang
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 1.1.0
>
>   Original Estimate: 8h
>  Remaining Estimate: 8h
>
> After adding stricter data validation in the test `TestSpark3DDL`.`Test alter table properties and add rename drop column with table services`, we see that reading the field value field after promotion from float to double could be different between COW, MOR, before and after compaction and clustering.  The value has precision difference, e.g., 102.0199966430664 vs 102.02. Right now the test has to differentiate this by using "getDouble".  We should look into this and see if this is expected, and if the behavior can be made consistent.
> Note that running the same test without enabling the fg reader in compaction or clustering for schema on read also pass.  So this is related to existing discrepancy on how we evolve the field from float to double type.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)