Dynamic star (DynamicRecordType) vs _extra vs PEEK_FIELDS_NO_EXPAND

[Vladimir Sitnikov <vl...@apache.org>]

Hi,

I was looking for "optional fields that are accessible yet invisible via
regular select *".
The fields might be costly to compute in my case, so I would like to make
them available on demand.

It looks like PEEK_FIELDS_NO_EXPAND matches my needs (I'll play with it),
however I see the sources have traces for "_extra" column support.

As far as I understand, the design was as follows:
1) _extra field should be the very last field in the table type
2) In case the field is present, new fields are generated on the fly (the
type is taken from the type of _extra)
3) The index of "newly generated field" is set to -1 (<--!!)

if (lastField.getName().equals("_extra")) {
  return new RelDataTypeFieldImpl(
      fieldName, -1, lastField.getType());
}

4) Due to #3, validator bails out as it thinks "-1" means "field does not
exist".

Does it mean it's time to delete traces of "_extra"?

Vladimir

Re: Dynamic star (DynamicRecordType) vs _extra vs PEEK_FIELDS_NO_EXPAND

[Julian Hyde <jh...@gmail.com>]

“_extra” is implemented in a hacky way, but it is still being used (by the Splunk adapter, for instance). So I’d prefer that we replace it with something less hacky and better documented and tested.

The purpose of “_extra", by the way, is to allow you to write “select empno, x, y from Emp” when “Emp” has declared columns (empno, deptno) but also allows references to undeclared columns.

Julian


> On Sep 8, 2017, at 12:52 AM, Vladimir Sitnikov <vl...@apache.org> wrote:
> 
> Hi,
> 
> I was looking for "optional fields that are accessible yet invisible via
> regular select *".
> The fields might be costly to compute in my case, so I would like to make
> them available on demand.
> 
> It looks like PEEK_FIELDS_NO_EXPAND matches my needs (I'll play with it),
> however I see the sources have traces for "_extra" column support.
> 
> As far as I understand, the design was as follows:
> 1) _extra field should be the very last field in the table type
> 2) In case the field is present, new fields are generated on the fly (the
> type is taken from the type of _extra)
> 3) The index of "newly generated field" is set to -1 (<--!!)
> 
> if (lastField.getName().equals("_extra")) {
>  return new RelDataTypeFieldImpl(
>      fieldName, -1, lastField.getType());
> }
> 
> 4) Due to #3, validator bails out as it thinks "-1" means "field does not
> exist".
> 
> Does it mean it's time to delete traces of "_extra"?
> 
> Vladimir