[jira] [Issue Comment Deleted] (NIFI-6654) Stack overflow with self referencing avro schema

["Otto Fowler (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Otto Fowler updated NIFI-6654:
------------------------------
    Comment: was deleted

(was: If you use a second instance of the fields array, instead of re-using the same array this test passes.


{code:java}
 @Test
    public void testSchemaCompareSelfRef() {
        // Set up first schema
        final SimpleRecordSchema personSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
        final List<RecordField> personFields = new ArrayList<>();
        personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
        personFields.add(new RecordField("address", RecordFieldType.STRING.getDataType()));
        personFields.add(new RecordField("SIN", RecordFieldType.STRING.getDataType()));
        personFields.add(new RecordField("PersonalData", RecordFieldType.RECORD.getRecordDataType(personSchema)));
        personSchema.setFields(personFields);
        // Set up second schema. Must be completely separate set of objects otherwise overloaded comparison
        // operators (particularly that from the SimpleRecordSchema class) will not be called.
        SimpleRecordSchema secondSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
        //personFields.clear();
        final List<RecordField> personFields2 = new ArrayList<>();
        personFields2.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
        personFields2.add(new RecordField("address", RecordFieldType.STRING.getDataType()));
        personFields2.add(new RecordField("SIN", RecordFieldType.STRING.getDataType()));
        personFields2.add(new RecordField("PersonalData", RecordFieldType.RECORD.getRecordDataType(secondSchema)));
        secondSchema.setFields(personFields);
        assertTrue(personSchema.equals(secondSchema));
    }
{code})

> Stack overflow with self referencing avro schema
> ------------------------------------------------
>
>                 Key: NIFI-6654
>                 URL: https://issues.apache.org/jira/browse/NIFI-6654
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 1.9.2
>            Reporter: Ron Chittaro
>            Priority: Major
>
> We (and our customers, thus why I marked this a blocker as we cannot proceed with 1.9.2 and in some cases product depolyment at all) have run into a stack overflow exception with the 'UpdateRecord' processor when upgrading from Nifi 1.7 to 1.9.2. I did some digging and it happens when:
>  * A schema comparison operation is performed, AND
>  * the schema in question is self referential
> I was a bit confused initially as I found that a unit test case for this already existed and was passing when I checked out and built 1.9.2. The test case is:
> {quote}testHashCodeAndEqualsWithSelfReferencingSchema() from TestSimpleRecordSchema.java located in: nifi\nifi-commons\nifi-record\src\test\java\org\apache\nifi\serialization
> {quote}
> However, if you dig a bit into this test case it is passing because the two schemas being compared contain the same memory references for the fields within them, thus comparisons do not exercise completely the underlying equals() operators of the objects contained in the schema. See bold below:
> {quote}final SimpleRecordSchema schema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
> final List<RecordField> personFields = new ArrayList<>();
>  personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("sibling", RecordFieldType.RECORD.getRecordDataType(schema)));
> *schema.setFields(personFields);*
> schema.hashCode();
>  assertTrue(schema.equals(schema));
> final SimpleRecordSchema secondSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
>  *secondSchema.setFields(personFields);*
>  assertTrue(schema.equals(secondSchema));
>  assertTrue(secondSchema.equals(schema));
> {quote}
>  
> To reproduce the stack overflow I wrote the following test cases which I believe behaves more like it would in the 'UpdateProcessor' (or anywhere else a schema comparison is happening) where the schema object and all objects within are completely different memory references.
>  
> @Test
> public void testSchemaCompareSelfRef() {
>  // Set up first schema
>  final SimpleRecordSchema personSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
>  final List<RecordField> personFields = new ArrayList<>();
>  personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("address", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("SIN", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("PersonalData", RecordFieldType.RECORD.getRecordDataType(personSchema)));
>  personSchema.setFields(personFields);
>  // Set up second schema. Must be completely separate set of objects otherwise overloaded comparison
>  // operators (particularly that from the SimpleRecordSchema class) will not be called.
>  SimpleRecordSchema secondSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
>  personFields.clear();
>  personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("address", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("SIN", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("PersonalData", RecordFieldType.RECORD.getRecordDataType(secondSchema)));
>  secondSchema.setFields(personFields);
>  assertTrue(personSchema.equals(secondSchema));
> }
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)