You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2024/11/26 23:42:00 UTC

[jira] [Created] (HUDI-8585) Fix unsupported data types for SPARK record type w/ col stats

sivabalan narayanan created HUDI-8585:
-----------------------------------------

             Summary: Fix unsupported data types for SPARK record type w/ col stats
                 Key: HUDI-8585
                 URL: https://issues.apache.org/jira/browse/HUDI-8585
             Project: Apache Hudi
          Issue Type: Improvement
          Components: metadata
            Reporter: sivabalan narayanan


When we tried enabling col stats by default, we realized that BYTES and FIXED data type is not supported in SPARK record type. 

 

Ref patch: 

[https://github.com/apache/hudi/pull/12331] 

 

Ref test: 

TestMORDataSource.testPrunedFiltered 

 

lets fix the user-exp by adding these as unsupported types upfront. rather than throwing random errors. 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)