Query compile exception after upgrade to 1.16

[François Méthot <fm...@gmail.com>]

Hi all,

 Drill 1.12 has been serving us well for far, we are now interested in
Kafka adhoc query support and decided to upgrade to Drill 1.16.

I have a query that involve multiple embedded select statement that now
fails after the update.
It uses our custom function regexExtract that parses each full row of text
into array of string using some regex. The function has been rebuilt using
Drill Jars from 1.16.
Here is the relevant part of the query:

create table FinalResult as(
   select NAME,GROUP,STATUS from
        (select ... from
            (select ... from
               (select regexExtract(rowTextData) from some.files)
            )
        )
   where STATUS IS NOT NULL and NAME is NOT NULL and GROUP is not NULL.
)


The error I get is:

Error: SYSTEM ERROR: Compile Exception: Line 167, Column 74 "value" is
neither a method, a field nor a member of class
"org.apache.drill.exec.vector.UntypedNullHolder".


I could make the query work by
  1)  generating an intermediate table without the "IS NOT NULL" filters,
but I don't know how reliable this workaround is:

  alter session `store.format`='parquet';
  create table IntermediateResult as(
   select NAME,GROUP,STATUS from
        (select ... from
            (select ... from
               (select regexExtract(rowTextData) from some.files)
            )
        )
  ) -- where filter is omitted;



  2) Then create my FinalResult table using the filter on the intermediate
table.

  create table FinalResult as(
     select NAME,GROUP,STATUS from IntermediateResult
     where STATUS IS NOT NULL, and NAME is NOT NULL and GROUP is not NULL.
 -- where filter applied on the intermediate result
 )


So what seems to be triggering the issue is the filter "where STATUS IS NOT
NULL, and NAME is NOT NULL and GROUP is not NULL".

The error happens in code generated by drill for that query and is
difficult to debug, so far I dedicated my time at finding a work around.

From the drillbit log I get:

ClassTransformationException: Failure generating transformation classes for
value:

...Java code..
Class FilterGen58166
{
...
}



I am aware that the description of this issue is broad, but if it rings a
bell to anyone  regarding a known bug, please let me know. I am trying to
replicate the problem with a simple data set.

Thanks in advance for reading and for any advises.

Francois