You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "jonvex (via GitHub)" <gi...@apache.org> on 2025/05/12 17:42:18 UTC

[PR] [HUDI-8902] Fix schema evolution from float to double for avro log blocks and for vectorized reader [hudi]

jonvex opened a new pull request, #13289:
URL: https://github.com/apache/hudi/pull/13289

   ### Change Logs
   
   The precision is incorrect for evolution from float to double and has been fixed for most of spark by https://github.com/apache/hudi/pull/13188. However, there are some edge cases that still need to be fixed.
   
   This pr fixes:
   
   1. avro log blocks the evolution is fixed by adding the float -> double conversion into the method that determines if we need to manually convert the record (and the manual conversion already has the float->double conversion correct)
   2. For vectorized reader, we only use the typechangeinfos with vectorized reader if there is an internal schema. This requirement is too strict and if typechangeinfos is nonempty we should use it.
   
   ### Impact
   
   Fix float -> double precision issue for 2 edge cases
   
   ### Risk level (write none, low medium or high below)
   
   low
   
   ### Documentation Update
   
   N/A
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Change Logs and Impact were stated clearly
   - [ ] Adequate tests were added if applicable
   - [ ] CI passed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [HUDI-8902] Fix schema evolution from float to double for avro log blocks and for vectorized reader [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13289:
URL: https://github.com/apache/hudi/pull/13289#issuecomment-2873426565

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "0e4bc0194accff3e3024bc68ef44700a93b07041",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "0e4bc0194accff3e3024bc68ef44700a93b07041",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 0e4bc0194accff3e3024bc68ef44700a93b07041 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [HUDI-8902] Fix schema evolution from float to double for avro log blocks and for vectorized reader [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13289:
URL: https://github.com/apache/hudi/pull/13289#issuecomment-2873674349

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "0e4bc0194accff3e3024bc68ef44700a93b07041",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5417",
       "triggerID" : "0e4bc0194accff3e3024bc68ef44700a93b07041",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 0e4bc0194accff3e3024bc68ef44700a93b07041 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5417) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [HUDI-8902] Fix schema evolution from float to double for avro log blocks and for vectorized reader [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13289:
URL: https://github.com/apache/hudi/pull/13289#issuecomment-2873432294

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "0e4bc0194accff3e3024bc68ef44700a93b07041",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5417",
       "triggerID" : "0e4bc0194accff3e3024bc68ef44700a93b07041",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 0e4bc0194accff3e3024bc68ef44700a93b07041 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5417) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org