[jira] [Created] (CALCITE-5690) ConcurrentModificationException during validation of table with DynamicRecordType

["Abhishek Singh Chouhan (Jira)" <ji...@apache.org>]

Abhishek Singh Chouhan created CALCITE-5690:
-----------------------------------------------

             Summary: ConcurrentModificationException during validation of table with DynamicRecordType
                 Key: CALCITE-5690
                 URL: https://issues.apache.org/jira/browse/CALCITE-5690
             Project: Calcite
          Issue Type: Bug
    Affects Versions: 1.32.0
            Reporter: Abhishek Singh Chouhan


When multiple threads are doing validation on a Table with DynamicRecordTypeImpl, we run into a ConcurrentModificationException. One of the instances where this happens is when two validations are happening in parallel 

Thread1 - 

SELECT DS_GET_QUANTILE(DS_QUANTILES_SKETCH(COL1), 0.9) FROM TABLE1 GROUP BY COL1, COL2, COL3

Thread2 - SELECT DS_GET_QUANTILE(DS_QUANTILES_SKETCH(COL1), 0.9) FROM TABLE1 GROUP BY COL1, COL2

While expanding the groupby in thread1 we intern the type at multiple places, eg. DelegatingScope#fullyQualify -> SqlValidatorScope#rowType -> RelDataTypeFactoryImpl#createTypeWithNullability -> canonize

During the validation in thread1 we cache DynamicRecordTypeImpl with Col1 and Col2 as the fields.

The race condition happens when thread2 gets a hold of this(abovementioned) cached DynamicRecordTypeImpl during its validation and is iterating over the fields eg during SqlValidatorImpl.validateGroupItem and thread 1 then goes and inserts Col3 field in the holder.
{code:java}
java.util.ConcurrentModificationException
    at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
    at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
    at org.apache.calcite.rel.type.RelDataTypeHolder.getFieldOrInsert(RelDataTypeHolder.java:56)
    at org.apache.calcite.rel.type.DynamicRecordTypeImpl.getField(DynamicRecordTypeImpl.java:59)
    at org.apache.calcite.sql.validate.SqlNameMatchers$BaseMatcher.field(SqlNameMatchers.java:126)
    at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6455)
    at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6360)
    at org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:324)
    at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1869)
    at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1854)
    at org.apache.calcite.sql.validate.SqlValidatorImpl.validateGroupItem(SqlValidatorImpl.java:4393)
    at org.apache.calcite.sql.validate.SqlValidatorImpl.validateGroupClause(SqlValidatorImpl.java:4366)
    at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3701)
    at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)
    at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)
    at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1107)
    at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1078)
    at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:248)
    at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1053)
    at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:759) {code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)