[I] Avro decoder fails to decode Iceberg manifest when data file has empty partition info [datafusion]

["pepijnve (via GitHub)" <gi...@apache.org>]

pepijnve opened a new issue, #25022:
URL: https://github.com/apache/datafusion/issues/25022

   ### Describe the bug
   
   When an Iceberg table has no partitioning (probably rare, but valid per the spec) the data file record's partition filed will be encoded as an Avro record with no fields. This results in Arrow structs with no fields which cause two issues:
   
   - https://github.com/apache/arrow-rs/issues/10770 in the Avro decoder
   - An error in `nested_struct::validate_struct_compatibility` with message `Cannot cast struct with 0 fields to 0 fields because there is no field name overlap` because the zero source and target fields case is not special cased.
   
   ### To Reproduce
   
   - Create an Iceberg table with no partitioning
   - Open one of the manifest Avro files as an external table
   - Perform a `select *` query on the table
   
   ### Expected behavior
   
   The query executes successfully
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org

Re: [I] Avro decoder fails to decode Iceberg manifest when data file has empty partition info [datafusion]

["kumarUjjawal (via GitHub)" <gi...@apache.org>]

kumarUjjawal closed issue #25022: Avro decoder fails to decode Iceberg manifest when data file has empty partition info
URL: https://github.com/apache/datafusion/issues/25022


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org