[jira] [Created] (SPARK-25718) Detect recursive reference in Avro schema and throw exception

["Gengliang Wang (JIRA)" <ji...@apache.org>]

Gengliang Wang created SPARK-25718:
--------------------------------------

             Summary: Detect recursive reference in Avro schema and throw exception
                 Key: SPARK-25718
                 URL: https://issues.apache.org/jira/browse/SPARK-25718
             Project: Spark
          Issue Type: Sub-task
          Components: SQL
    Affects Versions: 2.4.0
            Reporter: Gengliang Wang


Avro schema allows recursive reference, e.g. the example schema in https://avro.apache.org/docs/1.8.2/spec.html#schema_record
```
{
  "type": "record",
  "name": "LongList",
  "aliases": ["LinkedLongs"],                      // old name for this
  "fields" : [
    {"name": "value", "type": "long"},             // each element has a long
    {"name": "next", "type": ["null", "LongList"]} // optional next element
  ]
}
```

However, it is impossible to convert the schema as `StructType` in current Spark SQL. Running `SchemaConverters.toSqlType(avroSchema: Schema)` and we will get stack overflow exception.

We should detect the recursive reference and throw exception for it.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org