You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Julian Hyde <jh...@apache.org> on 2021/05/04 18:29:06 UTC
Re: RelFieldTrimmer throws an exception in certain cases
Regardless of which direction we go (allowing zero-field record types, or disallowing them), Konstantin has found a bug. Konstantin, can you log it, please.
On 2021/04/29 14:25:27, Konstantin Orlov <ko...@gridgain.com> wrote:
> Hi all.
>
> I faced a problem preventing certain queries being planned because RelFieldTrimmer throws
> an ArrayIndexOutOfBoundsException with message "Index -1 out of bounds for length 0”.
>
> The problem is here [1]:
>
> // If they are asking for no fields, we can't give them what they want,
> // because zero-column records are illegal. Give them the last field,
> // which is unlikely to be a system field.
> if (fieldsUsed.isEmpty()) {
> fieldsUsed = ImmutableBitSet.range(fieldCount - 1, fieldCount);
> }
>
> In case fieldsUsed.isEmpty we returns last field, but it is currently possible that fieldCount=0 as well.
>
> After some investigation I find out that the reason is empty record derived as row type for Aggregate.
> It is possible when an aggregate has an empty group key and no aggregate calls.
>
> So the question is whether an empty record is a legal row type for an aggregation node?
>
> Below is a reproducer for this problem, just put it at RelFieldTrimmerTest:
>
> @Test void test() {
> class ContextImpl implements Context {
> final Object target;
>
> ContextImpl(Object target) {
> this.target = Objects.requireNonNull(target, "target");
> }
>
> @Override public <T extends Object> @Nullable T unwrap(Class<T> clazz) {
> if (clazz.isInstance(target)) {
> return clazz.cast(target);
> }
> return null;
> }
> }
>
> // RelBuilder hides problem when simplifyValues=true, hence we need to disable it
> final RelBuilder builder = RelBuilder.create(config()
> .context(new ContextImpl(RelBuilder.Config.DEFAULT.withSimplifyValues(false))).build());
>
> final RelNode root =
> builder.scan("EMP")
> .aggregate(builder.groupKey())
> .filter(builder.literal(false))
> .project(builder.literal(42))
> .build();
>
> final RelFieldTrimmer fieldTrimmer = new RelFieldTrimmer(null, builder);
> fieldTrimmer.trim(root); // fails with ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 0
> }
>
>
> [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java#L1197
>
> --
> Regards,
> Konstantin Orlov
>
>
>
>
>
Re: RelFieldTrimmer throws an exception in certain cases
Posted by Julian Hyde <jh...@gmail.com>.
Thanks, Konstantin.
I have logged https://issues.apache.org/jira/browse/CALCITE-4597 to make the policy configurable. Eventually I would like to allow empty row types throughout the system, but until then, rules and RelFieldTrimmer should follow Postel’s law [1] and accept empty row types but try not to produce them.
Julian
[1] https://en.wikipedia.org/wiki/Robustness_principle
> On May 5, 2021, at 12:51 AM, Konstantin Orlov <ko...@gridgain.com> wrote:
>
>> Konstantin, can you log it, please
>
> Yes, sure. Here it is [1]
>
> [1] https://issues.apache.org/jira/browse/CALCITE-4596 <https://issues.apache.org/jira/browse/CALCITE-4596>
>
> --
> Regards,
> Konstantin Orlov
>
>
>
>> On 4 May 2021, at 21:29, Julian Hyde <jh...@apache.org> wrote:
>>
>> Regardless of which direction we go (allowing zero-field record types, or disallowing them), Konstantin has found a bug. Konstantin, can you log it, please.
>>
>> On 2021/04/29 14:25:27, Konstantin Orlov <ko...@gridgain.com> wrote:
>>> Hi all.
>>>
>>> I faced a problem preventing certain queries being planned because RelFieldTrimmer throws
>>> an ArrayIndexOutOfBoundsException with message "Index -1 out of bounds for length 0”.
>>>
>>> The problem is here [1]:
>>>
>>> // If they are asking for no fields, we can't give them what they want,
>>> // because zero-column records are illegal. Give them the last field,
>>> // which is unlikely to be a system field.
>>> if (fieldsUsed.isEmpty()) {
>>> fieldsUsed = ImmutableBitSet.range(fieldCount - 1, fieldCount);
>>> }
>>>
>>> In case fieldsUsed.isEmpty we returns last field, but it is currently possible that fieldCount=0 as well.
>>>
>>> After some investigation I find out that the reason is empty record derived as row type for Aggregate.
>>> It is possible when an aggregate has an empty group key and no aggregate calls.
>>>
>>> So the question is whether an empty record is a legal row type for an aggregation node?
>>>
>>> Below is a reproducer for this problem, just put it at RelFieldTrimmerTest:
>>>
>>> @Test void test() {
>>> class ContextImpl implements Context {
>>> final Object target;
>>>
>>> ContextImpl(Object target) {
>>> this.target = Objects.requireNonNull(target, "target");
>>> }
>>>
>>> @Override public <T extends Object> @Nullable T unwrap(Class<T> clazz) {
>>> if (clazz.isInstance(target)) {
>>> return clazz.cast(target);
>>> }
>>> return null;
>>> }
>>> }
>>>
>>> // RelBuilder hides problem when simplifyValues=true, hence we need to disable it
>>> final RelBuilder builder = RelBuilder.create(config()
>>> .context(new ContextImpl(RelBuilder.Config.DEFAULT.withSimplifyValues(false))).build());
>>>
>>> final RelNode root =
>>> builder.scan("EMP")
>>> .aggregate(builder.groupKey())
>>> .filter(builder.literal(false))
>>> .project(builder.literal(42))
>>> .build();
>>>
>>> final RelFieldTrimmer fieldTrimmer = new RelFieldTrimmer(null, builder);
>>> fieldTrimmer.trim(root); // fails with ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 0
>>> }
>>>
>>>
>>> [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java#L1197
>>>
>>> --
>>> Regards,
>>> Konstantin Orlov
>>>
>>>
>>>
>>>
>>>
>
Re: RelFieldTrimmer throws an exception in certain cases
Posted by Konstantin Orlov <ko...@gridgain.com>.
> Konstantin, can you log it, please
Yes, sure. Here it is [1]
[1] https://issues.apache.org/jira/browse/CALCITE-4596 <https://issues.apache.org/jira/browse/CALCITE-4596>
--
Regards,
Konstantin Orlov
> On 4 May 2021, at 21:29, Julian Hyde <jh...@apache.org> wrote:
>
> Regardless of which direction we go (allowing zero-field record types, or disallowing them), Konstantin has found a bug. Konstantin, can you log it, please.
>
> On 2021/04/29 14:25:27, Konstantin Orlov <ko...@gridgain.com> wrote:
>> Hi all.
>>
>> I faced a problem preventing certain queries being planned because RelFieldTrimmer throws
>> an ArrayIndexOutOfBoundsException with message "Index -1 out of bounds for length 0”.
>>
>> The problem is here [1]:
>>
>> // If they are asking for no fields, we can't give them what they want,
>> // because zero-column records are illegal. Give them the last field,
>> // which is unlikely to be a system field.
>> if (fieldsUsed.isEmpty()) {
>> fieldsUsed = ImmutableBitSet.range(fieldCount - 1, fieldCount);
>> }
>>
>> In case fieldsUsed.isEmpty we returns last field, but it is currently possible that fieldCount=0 as well.
>>
>> After some investigation I find out that the reason is empty record derived as row type for Aggregate.
>> It is possible when an aggregate has an empty group key and no aggregate calls.
>>
>> So the question is whether an empty record is a legal row type for an aggregation node?
>>
>> Below is a reproducer for this problem, just put it at RelFieldTrimmerTest:
>>
>> @Test void test() {
>> class ContextImpl implements Context {
>> final Object target;
>>
>> ContextImpl(Object target) {
>> this.target = Objects.requireNonNull(target, "target");
>> }
>>
>> @Override public <T extends Object> @Nullable T unwrap(Class<T> clazz) {
>> if (clazz.isInstance(target)) {
>> return clazz.cast(target);
>> }
>> return null;
>> }
>> }
>>
>> // RelBuilder hides problem when simplifyValues=true, hence we need to disable it
>> final RelBuilder builder = RelBuilder.create(config()
>> .context(new ContextImpl(RelBuilder.Config.DEFAULT.withSimplifyValues(false))).build());
>>
>> final RelNode root =
>> builder.scan("EMP")
>> .aggregate(builder.groupKey())
>> .filter(builder.literal(false))
>> .project(builder.literal(42))
>> .build();
>>
>> final RelFieldTrimmer fieldTrimmer = new RelFieldTrimmer(null, builder);
>> fieldTrimmer.trim(root); // fails with ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 0
>> }
>>
>>
>> [1] https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/RelFieldTrimmer.java#L1197
>>
>> --
>> Regards,
>> Konstantin Orlov
>>
>>
>>
>>
>>