You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 05:49:05 UTC
[I] Add validation as optinal param for generic rocord rewrite [hudi]
hudi-bot opened a new issue, #15409:
URL: https://github.com/apache/hudi/issues/15409
HoodieAvroUtils.rewriteRecordDeep does not do any datatype/null check validations .
So we need capability to do validations when target schema is not same as inferred schema
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-4791
- Type: Improvement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add validation as optinal param for generic rocord rewrite [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #15409: Add validation as optinal param for generic rocord rewrite
URL: https://github.com/apache/hudi/issues/15409
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add validation as optinal param for generic rocord rewrite [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #15409:
URL: https://github.com/apache/hudi/issues/15409#issuecomment-4795362892
This appears to be already implemented on `master`. `HoodieAvroUtils` exposes an optional-validation overload:
```java
// hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java
public static GenericRecord rewriteRecordDeep(GenericRecord oldRecord, Schema newSchema, boolean validate) {
return rewriteRecordWithNewSchema(oldRecord, newSchema, Collections.EMPTY_MAP, validate);
}
```
When `validate=true`, the rewritten record is checked with `ConvertingGenericData.INSTANCE.validate(newSchema, newRecord)` and a `SchemaCompatibilityException` is thrown if it fails (see the private `rewriteRecordWithNewSchema(...)` method around line 816). Equivalent overloads exist for the `IndexedRecord` and `Iterator<GenericRecord>` variants.
This was added in commit b6490c160bf7 (HUDI-5813, PR #7982). Closing seems appropriate unless a different validation semantic is needed — please reopen with specifics if so.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org