[I] [BUG] ProtoConversionUtil UINT64 default fails Avro 1.12 FIXED validator [hudi]

["tiennguyen-onehouse (via GitHub)" <gi...@apache.org>]

tiennguyen-onehouse opened a new issue, #18573:
URL: https://github.com/apache/hudi/issues/18573

   ### Bug Description
   
   **What happened:**
   `ProtoConversionUtil$AvroSupport.getDefault` returns a `byte[9]` default for proto `uint64` fields (modelled as a `FIXED(size=9)` with `decimal` logicalType). Avro 1.11 coerced that into its canonical wire form (an ISO-8859-1 string of length == fixed size). Avro 1.12 tightened `Schema.validateDefault` for FIXED to require a `String` directly and reject `byte[]` — the byte array gets base64-encoded by Jackson to `"AAAAAAAAAAAA"` (12 chars, not 9), and the length check fails:
   
   ```
   org.apache.avro.AvroTypeException: Invalid default for field primitive_unsigned_long:
     "AAAAAAAAAAAA" not a {"type":"fixed","name":"unsigned_long",...,"size":9,
                          "logicalType":"decimal","precision":20,"scale":0}
       at org.apache.hudi.utilities.sources.helpers.ProtoConversionUtil$AvroSupport.lambda$getSchema$0(ProtoConversionUtil.java:…)
   ```
   
   Like #18569 (`ProtoConversionUtil$AvroSupport` static init under Avro 1.12), this surfaces on JVMs that have Avro 1.12+ on the classpath. Any proto schema containing a `uint64` field triggers the failure at schema-materialization time.
   
   **What you expected:**
   Schema generation for `uint64` fields should succeed under Avro 1.12.
   
   **Steps to reproduce:**
   1. Set Avro 1.12.0 on the classpath.
   2. Call `ProtoConversionUtil.getSchemaForMessageClass(...)` (or `getSchemaForMessageDescriptor`) with a proto class/descriptor that has any `uint64` field.
   3. `AvroTypeException` at schema construction.
   
   ### Environment
   
   **Hudi version:** master (Avro still pinned 1.11.4 in Hudi's pom.xml, so default builds don't hit it; downstream consumers bumping Avro to 1.12+ do)
   **Query engine:** N/A — fails at schema generation
   **Relevant configs:** none — triggered purely by classpath + a `uint64` field in the proto
   
   ### Logs and Stack Trace
   
   ```
   org.apache.avro.AvroTypeException: Invalid default for field primitive_unsigned_long:
     "AAAAAAAAAAAA" not a {"type":"fixed","name":"unsigned_long",...,"size":9,...}
       at org.apache.avro.Schema.validateDefault(Schema.java:1719)
       at org.apache.avro.Schema$Field.<init>(Schema.java:578)
       ...
   ```
   
   ### Root Cause
   
   In `hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/ProtoConversionUtil.java`, the `UINT64` branch of `getDefault` returns `.bytes()` directly:
   
   ```java
   case UINT64:
     return DECIMAL_CONVERSION.toFixed(new BigDecimal(BigInteger.ZERO), fieldSchema.toAvroSchema(), fieldSchema.toAvroSchema().getLogicalType()).bytes();
   ```
   
   `.bytes()` is a `byte[9]` (9 zero bytes). Avro 1.11.x accepted this and produced the canonical ISO-8859-1 string form on `Schema.toString`; Avro 1.12's stricter validator rejects it.
   
   ### Fix
   
   Pre-convert the `byte[]` to a `String` via ISO-8859-1 so it matches the form Avro 1.12's validator expects. Bit-identical wire representation to what 1.11 was emitting; forward-compatible with both Avro 1.11 and 1.12.
   
   ### Backward compatibility
   
   Empirically verified with a standalone probe against both Avro 1.11.4 and 1.12.0:
   - Avro 1.11.4: `byte[]` form and `String` form produce byte-identical `Schema.toString(true)` output.
   - Avro 1.12.0: `byte[]` form throws; `String` form works.
   
   The default value is only consulted when a field is missing on read (decodes to `BigInteger.ZERO` / zero GenericFixed either way). Avro Parsing Canonical Form strips `"default"` so fingerprints are unchanged. Avro compatibility resolution operates on the decoded value, not its JSON encoding.
   
   Corroborating evidence: Hudi's own test fixtures at `hudi-utilities/src/test/resources/schema-provider/proto/sample_schema_defaults.avsc` and `sample_schema_wrapped_and_timestamp_as_record.avsc` already contain this 9-NUL form — they were generated against Avro 1.11's JSON output, which matches both the old byte[] path and the new String path.
   
   PR incoming.
   
   ### Related
   
   - Sibling Avro 1.12 fix for the proto overflow BYTES default: #18569


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [BUG] ProtoConversionUtil UINT64 default fails Avro 1.12 FIXED validator [hudi]

["danny0405 (via GitHub)" <gi...@apache.org>]

danny0405 closed issue #18573: [BUG] ProtoConversionUtil UINT64 default fails Avro 1.12 FIXED validator
URL: https://github.com/apache/hudi/issues/18573


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org