You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/01/12 18:52:36 UTC

[GitHub] [arrow-rs] tustvold opened a new issue #1167: StructArray Nullability

tustvold opened a new issue #1167:
URL: https://github.com/apache/arrow-rs/issues/1167


   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   
   I couldn't work out how to produce a parquet file with the following schema using the arrow APIs
   
   ```
   "message nested {
     OPTIONAL Group group {
       REQUIRED INT32 leaf;
     }
   }";
   ```
   
   The closest I got was
   
   ```
   let group1 = Arc::new(StructArray::from(vec![(
       Field::new("leaf", ArrowDataType::Int32, false),
       Arc::new(PrimitiveArray::<arrow::datatypes::Int32Type>::from(vec![
           1, 43, 6,
       ])) as ArrayRef,
   )])) as ArrayRef;
   
   let batch = RecordBatch::try_from_iter(vec![("group", group1)]).unwrap();
   ```
   
   But this has the schema
   
   ```
   Schema { fields: [Field { name: "group", data_type: Struct([Field { name: "leaf", data_type: Int32, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: None }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: None }], metadata: {} }
   ```
   
   By comparison
   
   ```
   let group1 = Arc::new(StructArray::from(vec![(
       Field::new("leaf", ArrowDataType::Int32, false),
       Arc::new(PrimitiveArray::<arrow::datatypes::Int32Type>::from_iter(
           vec![Some(1), None, Some(6)],
       )) as ArrayRef,
   )])) as ArrayRef;
   
   let batch = RecordBatch::try_from_iter(vec![("group", group1)]).unwrap();
   ```
   
   Has the schema
   
   ```
   Schema { fields: [Field { name: "group", data_type: Struct([Field { name: "leaf", data_type: Int32, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: None }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: None }], metadata: {} }
   ```
   
   We've somehow lost nullability? :exploding_head: 
   
   **Describe the solution you'd like**
   
   I don't really know what I expect to happen here, or what the API should look like, but I wanted to flag that there might be something funky going on here.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org