[GitHub] [iceberg] Fokko commented on a diff in pull request #5796: Python: Handle optional fields in Avro conversion

[GitBox <gi...@apache.org>]

Fokko commented on code in PR #5796:
URL: https://github.com/apache/iceberg/pull/5796#discussion_r975130887


##########
python/pyiceberg/manifest.py:
##########
@@ -179,16 +179,20 @@ def _(struct_type: StructType, values: AvroStruct) -> Dict[str, Any]:
 @_convert_pos_to_dict.register
 def _(list_type: ListType, values: List[Any]) -> Any:
     """In the case of a list, we'll go over the elements in the list to handle complex types"""
-    return [_convert_pos_to_dict(list_type.element_type, value) for value in values]
+    return [_convert_pos_to_dict(list_type.element_type, value) for value in values] if values is not None else None

Review Comment:
   Great catch! Looks like we need another guard there as well:
   
   ```python
   def test_null_struct_convert_pos_to_dict():
       data = _convert_pos_to_dict(
           Schema(
               NestedField(
                   name="field",
                   field_id=1,
                   field_type=StructType(
                       NestedField(2, "required_field", StringType(), True),
                       NestedField(3, "optional_field", IntegerType())
                   ),
                   required=False
               )
           ),
           AvroStruct([None]),
       )
       assert data["field"] is None
   ```
   
   Raises `AttributeError`:
   ```
   >   return {field.name: _convert_pos_to_dict(field.field_type, values.get(pos)) for pos, field in enumerate(struct_type.fields)}
   E   AttributeError: 'NoneType' object has no attribute 'get'
   
   pyiceberg/manifest.py:176: AttributeError
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org