[GitHub] [arrow-rs] spebern opened a new issue, #3900: Nullable field with nested not nullable map in json

["spebern (via GitHub)" <gi...@apache.org>]

spebern opened a new issue, #3900:
URL: https://github.com/apache/arrow-rs/issues/3900

   While trying to use the new json decoder in delta I stumbled over the following issue:
   
   ```rust
       #[test]
       fn test_delta_checkpoint() {
           let json = "{\"protocol\":{\"minReaderVersion\":1,\"minWriterVersion\":2}}";
           let schema = Arc::new(Schema::new(vec![
               Field::new(
                   "protocol",
                   DataType::Struct(vec![
                       Field::new("minReaderVersion", DataType::Int32, true),
                       Field::new("minWriterVersion", DataType::Int32, true),
                   ]),
                   true,
               ),
               Field::new(
                   "add",
                   DataType::Struct(vec![Field::new(
                       "partitionValues",
                       DataType::Map(
                           Box::new(Field::new(
                               "key_value",
                               DataType::Struct(vec![
                                   Field::new("key", DataType::Utf8, false),
                                   Field::new("value", DataType::Utf8, true),
                               ]),
                               false,
                           )),
                           false,
                       ),
                       false, // <-- when this is true the test passes
                   )]),
                   true,
               ),
           ]));
   
           let batches = do_read(json, 1024, true, schema);
           assert_eq!(batches.len(), 1);
       }
   ```
   
   This fails with:
   
   ```
   thread 'raw::tests::test_delta_checkpoint' panicked at 'called `Result::unwrap()` on an `Err` value: JsonError("expected { got null")', arrow-json/src/raw/mod.rs:389:18
   ```
   
   It is related to nested map being not nullable (see comment in the code snippet). I'd expect that if a field (in the example "add") is null, the inner "not nullables" should
   not get checked.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold closed issue #3900: Nullable field with nested not nullable map in json

["tustvold (via GitHub)" <gi...@apache.org>]

tustvold closed issue #3900: Nullable field with nested not nullable map in json
URL: https://github.com/apache/arrow-rs/issues/3900


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold commented on issue #3900: Nullable field with nested not nullable map in json

["tustvold (via GitHub)" <gi...@apache.org>]

tustvold commented on issue #3900:
URL: https://github.com/apache/arrow-rs/issues/3900#issuecomment-1479560979

   This is a bug, and I understand what causes it, thank you for the report. Working on a fix


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org