[jira] [Commented] (NIFI-6654) Stack overflow with self referencing avro schema

["Ron Chittaro (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928600#comment-16928600 ] 

Ron Chittaro commented on NIFI-6654:
------------------------------------

Meant to add, the problem is down in the RecordData.equals() where ultimately 2 schemas are compared, when they are self referential this is endless recursion:

 
{quote}@Override
public boolean equals(final Object obj) {
 if (obj == this) {
 return true;
 }
 if (obj == null) {
 return false;
 }
 if (!(obj instanceof RecordDataType)) {
 return false;
 }

 final RecordDataType other = (RecordDataType) obj;
 *return Objects.equals(childSchema, other.childSchema);*
}
{quote}
 

> Stack overflow with self referencing avro schema
> ------------------------------------------------
>
>                 Key: NIFI-6654
>                 URL: https://issues.apache.org/jira/browse/NIFI-6654
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 1.9.2
>            Reporter: Ron Chittaro
>            Priority: Major
>             Fix For: 1.10.0
>
>
> We (and our customers, thus why I marked this a blocker as we cannot proceed with 1.9.2 and in some cases product depolyment at all) have run into a stack overflow exception with the 'UpdateRecord' processor when upgrading from Nifi 1.7 to 1.9.2. I did some digging and it happens when:
>  * A schema comparison operation is performed, AND
>  * the schema in question is self referential
> I was a bit confused initially as I found that a unit test case for this already existed and was passing when I checked out and built 1.9.2. The test case is:
> {quote}testHashCodeAndEqualsWithSelfReferencingSchema() from TestSimpleRecordSchema.java located in: nifi\nifi-commons\nifi-record\src\test\java\org\apache\nifi\serialization
> {quote}
> However, if you dig a bit into this test case it is passing because the two schemas being compared contain the same memory references for the fields within them, thus comparisons do not exercise completely the underlying equals() operators of the objects contained in the schema. See bold below:
> {quote}final SimpleRecordSchema schema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
> final List<RecordField> personFields = new ArrayList<>();
> personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
> personFields.add(new RecordField("sibling", RecordFieldType.RECORD.getRecordDataType(schema)));
> *schema.setFields(personFields);*
> schema.hashCode();
> assertTrue(schema.equals(schema));
> final SimpleRecordSchema secondSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
> *secondSchema.setFields(personFields);*
> assertTrue(schema.equals(secondSchema));
> assertTrue(secondSchema.equals(schema));
> {quote}
>  
> To reproduce the stack overflow I wrote the following test cases which I believe behaves more like it would in the 'UpdateProcessor' (or anywhere else a schema comparison is happening) where the schema object and all objects within are completely different memory references.
>  
> {quote}@Test
> public void testSchemaCompareSelfRef() {
>  // Set up first schema
>  final SimpleRecordSchema personSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
>  final List<RecordField> personFields = new ArrayList<>();
>  personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("address", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("SIN", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("PersonalData", RecordFieldType.RECORD.getRecordDataType(personSchema)));
>  personSchema.setFields(personFields);
>  // Set up second schema. Must be completely separate set of objects otherwise overloaded comparison
>  // operators (particularly that from the SimpleRecordSchema class) will not be called.
>  SimpleRecordSchema secondSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
>  personFields.clear();
>  personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("address", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("SIN", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("PersonalData", RecordFieldType.RECORD.getRecordDataType(secondSchema)));
>  secondSchema.setFields(personFields);
>  assertTrue(personSchema.equals(secondSchema));
> }
> {quote}
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)