You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/22 00:10:43 UTC
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7021: [Minor] fix multi deser avro payload
alexeykudinkin commented on code in PR #7021:
URL: https://github.com/apache/hudi/pull/7021#discussion_r1002268684
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieAvroRecord.java:
##########
@@ -189,14 +191,21 @@ public Option<Map<String, String>> getMetadata() {
@Override
public Option<HoodieAvroIndexedRecord> toIndexedRecord(Schema recordSchema, Properties props) throws IOException {
- Option<IndexedRecord> avroData = getData().getInsertValue(recordSchema, props);
+ Option<IndexedRecord> avroData = getDeserData(recordSchema, props);
if (avroData.isPresent()) {
return Option.of(new HoodieAvroIndexedRecord(avroData.get()));
} else {
return Option.empty();
}
}
+ private Option<IndexedRecord> getDeserData(Schema recordSchema, Properties props) throws IOException {
+ if (deserRecord == null) {
+ this.deserRecord = this.data.getInsertValue(recordSchema, props);
+ }
+ return this.deserRecord;
Review Comment:
We will also have to do following
- Keep the schema this record has been deserialized with, and
- Make sure schemas are identical (comparing schemas w/ equals will kill perf-gain so we should compare them as strings)
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieAvroRecord.java:
##########
@@ -39,6 +39,8 @@
public class HoodieAvroRecord<T extends HoodieRecordPayload> extends HoodieRecord<T> {
+ private transient Option<IndexedRecord> deserRecord = null;
Review Comment:
nit: `cachedDeserializedRecord`
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieAvroRecord.java:
##########
@@ -189,14 +191,21 @@ public Option<Map<String, String>> getMetadata() {
@Override
public Option<HoodieAvroIndexedRecord> toIndexedRecord(Schema recordSchema, Properties props) throws IOException {
- Option<IndexedRecord> avroData = getData().getInsertValue(recordSchema, props);
+ Option<IndexedRecord> avroData = getDeserData(recordSchema, props);
if (avroData.isPresent()) {
return Option.of(new HoodieAvroIndexedRecord(avroData.get()));
} else {
return Option.empty();
}
}
+ private Option<IndexedRecord> getDeserData(Schema recordSchema, Properties props) throws IOException {
Review Comment:
nit: `getCachedDeserializedRecord`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org