You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "LuciferYang (via GitHub)" <gi...@apache.org> on 2026/09/03 10:04:29 UTC

[I] [Bug] JSON format accepts unsupported types nested inside ARRAY/MAP/ROW at create time [paimon]

LuciferYang opened a new issue, #9584:
URL: https://github.com/apache/paimon/issues/9584

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
   
   ### Paimon version
   
   master, `2788fe596` (2.1-SNAPSHOT).
   
   ### Compute Engine
   
   Flink and Spark, creating a managed table with `'file.format' = 'json'`.
   
   ### Minimal reproduce step
   
   Create a JSON table with an unsupported type nested inside a container:
   
   ```sql
   CREATE TABLE t (v ARRAY<VARIANT>) WITH ('file.format' = 'json');
   ```
   
   `CREATE TABLE` succeeds. The failure comes later: on write, `JsonFormatWriter` reaches `CastExecutors.resolveToString` and throws `Cast VARIANT to StringType is not supported.`, and on read `JsonFileReader.convertPrimitiveStringToType` gets a null cast executor and dereferences it, so the query fails with a `NullPointerException` carrying no message.
   
   `JsonFileFormat.validateDataType` only ever looks at the outermost type root, because the four container cases fall through to a single `break`:
   
   ```java
   case ARRAY:
   case VECTOR:
   case MAP:
   case ROW:
       // All types are supported in JSON
       break;
   default:
       throw new UnsupportedOperationException("Unsupported data type for JSON format: " + dataType);
   ```
   
   An unsupported type at the top level is rejected at create time with that message; the same type one level down is not.
   
   ### What doesn't meet your expectations?
   
   The other formats check nesting: ORC through `OrcTypeUtil.convertToOrcType`, Parquet through `ParquetSchemaConverter.convertToParquetType` and Avro through `AvroSchemaConverter.convertToSchema` all walk into element, key, value and field types and reject what they cannot represent. JSON was written like CSV, whose whitelist has no container types at all, but with the container types added to it, so the recursion was never there.
   
   The comment saying all types are supported is not accurate either: the whitelist has always rejected `MULTISET`, `VARIANT`, `BLOB`, `GEOMETRY` and `GEOGRAPHY` at the top level.
   
   ### Anything else?
   
   This is about when the error arrives, not about data being lost: none of those five types has a cast rule to or from string, so they already fail at runtime today. Note also that `validateDataFields` is only reached for managed tables; a format table is created through `CatalogUtils.validateCreateTable`, which does not call it.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] JSON format accepts unsupported types nested inside ARRAY/MAP/ROW at create time [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi closed issue #9584: [Bug] JSON format accepts unsupported types nested inside ARRAY/MAP/ROW at create time
URL: https://github.com/apache/paimon/issues/9584


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org