[jira] [Created] (NIFI-6654) Stack overflow with self referencing avro schema

["Ron Chittaro (Jira)" <ji...@apache.org>]

Ron Chittaro created NIFI-6654:
----------------------------------

             Summary: Stack overflow with self referencing avro schema
                 Key: NIFI-6654
                 URL: https://issues.apache.org/jira/browse/NIFI-6654
             Project: Apache NiFi
          Issue Type: Bug
          Components: Core Framework
    Affects Versions: 1.9.2
            Reporter: Ron Chittaro
             Fix For: 1.10.0


We (and our customers, thus why I marked this a blocker as we cannot proceed with 1.9.2 and in some cases product depolyment at all) have run into a stack overflow exception with the 'UpdateRecord' processor when upgrading from Nifi 1.7 to 1.9.2. I did some digging and it happens when:
 * A schema comparison operation is performed, AND
 * the schema in question is self referential

I was a bit confused initially as I found that a unit test case for this already existed and was passing when I checked out and built 1.9.2. The test case is:
{quote}testHashCodeAndEqualsWithSelfReferencingSchema() from TestSimpleRecordSchema.java located in: nifi\nifi-commons\nifi-record\src\test\java\org\apache\nifi\serialization
{quote}
However, if you dig a bit into this test case it is passing because the two schemas being compared contain the same memory references for the fields within them, thus comparisons do not exercise completely the underlying equals() operators of the objects contained in the schema. See bold below:
{quote}final SimpleRecordSchema schema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);

final List<RecordField> personFields = new ArrayList<>();
personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
personFields.add(new RecordField("sibling", RecordFieldType.RECORD.getRecordDataType(schema)));

*schema.setFields(personFields);*

schema.hashCode();
assertTrue(schema.equals(schema));

final SimpleRecordSchema secondSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
*secondSchema.setFields(personFields);*
assertTrue(schema.equals(secondSchema));
assertTrue(secondSchema.equals(schema));
{quote}
 

To reproduce the stack overflow I wrote the following test cases which I believe behaves more like it would in the 'UpdateProcessor' (or anywhere else a schema comparison is happening) where the schema object and all objects within are completely different memory references.

 
{quote}@Test
public void testSchemaCompareSelfRef() {

 // Set up first schema
 final SimpleRecordSchema personSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
 final List<RecordField> personFields = new ArrayList<>();
 personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
 personFields.add(new RecordField("address", RecordFieldType.STRING.getDataType()));
 personFields.add(new RecordField("SIN", RecordFieldType.STRING.getDataType()));
 personFields.add(new RecordField("PersonalData", RecordFieldType.RECORD.getRecordDataType(personSchema)));
 personSchema.setFields(personFields);

 // Set up second schema. Must be completely separate set of objects otherwise overloaded comparison
 // operators (particularly that from the SimpleRecordSchema class) will not be called.
 SimpleRecordSchema secondSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
 personFields.clear();
 personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
 personFields.add(new RecordField("address", RecordFieldType.STRING.getDataType()));
 personFields.add(new RecordField("SIN", RecordFieldType.STRING.getDataType()));
 personFields.add(new RecordField("PersonalData", RecordFieldType.RECORD.getRecordDataType(secondSchema)));

 secondSchema.setFields(personFields);

 assertTrue(personSchema.equals(secondSchema));
}
{quote}
 

 

 



--
This message was sent by Atlassian Jira
(v8.3.2#803003)