Re: [PR] feat: eliminate GenericDatum in Avro reader for performance [iceberg-cpp]

["wgtmac (via GitHub)" <gi...@apache.org>]

wgtmac commented on code in PR #374:
URL: https://github.com/apache/iceberg-cpp/pull/374#discussion_r2575408341


##########
src/iceberg/avro/avro_reader.cc:
##########
@@ -140,12 +141,15 @@ class AvroReader::Impl {
       if (split_end_ && reader_->pastSync(split_end_.value())) {
         break;
       }
-      if (!reader_->read(*context_->datum_)) {
+      if (!reader_->hasMore()) {
         break;
       }
+      reader_->decr();
+
+      // Use direct decoder instead of GenericDatum
       ICEBERG_RETURN_UNEXPECTED(
-          AppendDatumToBuilder(reader_->readerSchema().root(), *context_->datum_,
-                               projection_, *read_schema_, context_->builder_.get()));
+          DecodeAvroToBuilder(reader_->readerSchema().root(), reader_->decoder(),

Review Comment:
   I think it would be better to use a feature flag to enable users to use the old reader just in case there is any bug.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org