[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6805: [MINOR] copy to avoid the problem of reusing buffer underlying the Row

[GitBox <gi...@apache.org>]

alexeykudinkin commented on code in PR #6805:
URL: https://github.com/apache/hudi/pull/6805#discussion_r980715777


##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/cdc/HoodieCDCRDD.scala:
##########
@@ -318,7 +318,7 @@ class HoodieCDCRDD(
               val after = record.get(3).asInstanceOf[GenericRecord]
               recordToLoad.update(3, convertToUTF8String(HoodieCDCUtils.recordToJson(after)))
             case HoodieCDCSupplementalLoggingMode.WITH_BEFORE =>
-              val row = cdcRecordDeserializer.deserialize(record).get.asInstanceOf[InternalRow]
+              val row = cdcRecordDeserialize(record)

Review Comment:
   We actually don't need to copy it here right away: if someone just iterates over these records and simply writes them out into a file there would be no issue (and there's no need to make additional copies).
   
   However if we have a place where we do retain reference after iteration, like:
   
   ```
   val rows = iter.collect()
   ```
   
   Then we will need to make copies before we do that, like below:
   
   ```
   val copiedRows = iter.map(_.copy()).collect()
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org