You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@calcite.apache.org by "Vladimir Sitnikov (JIRA)" <ji...@apache.org> on 2018/08/27 11:49:00 UTC
[jira] [Commented] (CALCITE-2283) aggregate over empty set , throw
java.lang.NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593551#comment-16593551 ]
Vladimir Sitnikov commented on CALCITE-2283:
--------------------------------------------
[~julianhyde], your patch does reproduce the problem, however could you please suggest a way to debug a statement in the middle of {{*.iq}} file?
Relevant testcase:
{code:java}select count(*) from "hr"."emps" where false group by grouping sets (("deptno"),()){code}
Produces 0 results, however it should produce exactly one row (grand total).
If I patch {{Aggregate}} as follows
{code:java}
public static boolean isNotGrandTotal(Aggregate aggregate) {
- return aggregate.getGroupCount() > 0;
+ return !aggregate.getGroupSets().contains(ImmutableBitSet.of());
}
{code}
Then plan becomes
{noformat}
EnumerableCalc(expr#0..1=[{inputs}], EXPR$0=[$t1])
EnumerableAggregate(group=[{0}], groups=[[{0}, {}]], EXPR$0=[COUNT()])
EnumerableValues(tuples=[[]]){noformat}
however the results are still empty.
I suspect:
1) {{Aggregate.isNotGrandTotal}} is a bug. It should check something like {{getGroupSets().contains(ImmutableBitSet.of());}} (see CALCITE-2490 as well)
2) All the usages of {{Aggregate#getGroupCount}} are prone to bugs like {{isNotGrandTotal}}.
For instance:
{code:java}
org.apache.calcite.rel.rules.SemiJoinRule#perform
if (!joinInfo.rightSet().equals(
ImmutableBitSet.range(aggregate.getGroupCount()))) {
// Rule requires that aggregate key to be the same as the join key.
// By the way, neither a super-set nor a sub-set would work.
return;
}
{code}
Will that blow for {{grouping sets ((a), ())}} case?
In the same way
{code}select a, count(*)
from (
select * from (values(null)) as t(a)
) x
group by grouping sets ((a),());{code}
would defeat {{RelMdColumnUniqueness}}
{code:java}
public Boolean areColumnsUnique(Aggregate rel, RelMetadataQuery mq,
ImmutableBitSet columns, boolean ignoreNulls) {
// group by keys form a unique key
ImmutableBitSet groupKey = ImmutableBitSet.range(rel.getGroupCount());
return columns.contains(groupKey);
}
{code}
It would assume that column {{a}} is a part of group by, then the column would be unique in the results.
However the query above produces two rows like (a=null, count=1).
So the metadata is screwed, then we could produce invalid output.
2) There's a bug in {{EnumerableAggregate}} (or in Linq4j) for {{groups}} case. The plan above looks fine, yet the result is empty.
The generated code is
{code:java}/* 1 */ public static class Record1_0 implements java.io.Serializable {
/* 2 */ public long f0;
/* 3 */ public Record1_0() {}
/* 4 */ public boolean equals(Object o) {
/* 5 */ if (this == o) {
/* 6 */ return true;
/* 7 */ }
/* 8 */ if (!(o instanceof Record1_0)) {
/* 9 */ return false;
/* 10 */ }
/* 11 */ return this.f0 == ((Record1_0) o).f0;
/* 12 */ }
/* 13 */
/* 14 */ public int hashCode() {
/* 15 */ int h = 0;
/* 16 */ h = org.apache.calcite.runtime.Utilities.hash(h, this.f0);
/* 17 */ return h;
/* 18 */ }
/* 19 */
/* 20 */ public int compareTo(Record1_0 that) {
/* 21 */ final int c;
/* 22 */ c = org.apache.calcite.runtime.Utilities.compare(this.f0, that.f0);
/* 23 */ if (c != 0) {
/* 24 */ return c;
/* 25 */ }
/* 26 */ return 0;
/* 27 */ }
/* 28 */
/* 29 */ public String toString() {
/* 30 */ return "{f0=" + this.f0 + "}";
/* 31 */ }
/* 32 */
/* 33 */ }
/* 34 */
/* 35 */ org.apache.calcite.DataContext root;
/* 36 */
/* 37 */ public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root0) {
/* 38 */ root = root0;
/* 39 */ final org.apache.calcite.linq4j.Enumerable _inputEnumerable = org.apache.calcite.linq4j.EnumerableDefaults.groupByMultiple(org.apache.calcite.linq4j.Linq4j.asEnumerable(new Integer[] {}), java.util.Arrays.asList(new org.apache.calcite.linq4j.function.Function1() {
/* 40 */ public java.util.List apply(int a0) {
/* 41 */ return org.apache.calcite.runtime.FlatLists.of(a0, false);
/* 42 */ }
/* 43 */ public Object apply(Integer a0) {
/* 44 */ return apply(
/* 45 */ a0.intValue());
/* 46 */ }
/* 47 */ public Object apply(Object a0) {
/* 48 */ return apply(
/* 49 */ (Integer) a0);
/* 50 */ }
/* 51 */ }
/* 52 */ , new org.apache.calcite.linq4j.function.Function1() {
/* 53 */ public java.util.List apply(int a0) {
/* 54 */ return org.apache.calcite.runtime.FlatLists.of(0, true);
/* 55 */ }
/* 56 */ public Object apply(Integer a0) {
/* 57 */ return apply(
/* 58 */ a0.intValue());
/* 59 */ }
/* 60 */ public Object apply(Object a0) {
/* 61 */ return apply(
/* 62 */ (Integer) a0);
/* 63 */ }
/* 64 */ }
/* 65 */ ), new org.apache.calcite.linq4j.function.Function0() {
/* 66 */ public Object apply() {
/* 67 */ long COUNTa0s0;
/* 68 */ COUNTa0s0 = 0L;
/* 69 */ Record1_0 record0;
/* 70 */ record0 = new Record1_0();
/* 71 */ record0.f0 = COUNTa0s0;
/* 72 */ return record0;
/* 73 */ }
/* 74 */ }
/* 75 */ , new org.apache.calcite.linq4j.function.Function2() {
/* 76 */ public Record1_0 apply(Record1_0 acc, int in) {
/* 77 */ acc.f0++;
/* 78 */ return acc;
/* 79 */ }
/* 80 */ public Record1_0 apply(Record1_0 acc, Integer in) {
/* 81 */ return apply(
/* 82 */ acc,
/* 83 */ in.intValue());
/* 84 */ }
/* 85 */ public Record1_0 apply(Object acc, Object in) {
/* 86 */ return apply(
/* 87 */ (Record1_0) acc,
/* 88 */ (Integer) in);
/* 89 */ }
/* 90 */ }
/* 91 */ , new org.apache.calcite.linq4j.function.Function2() {
/* 92 */ public Object[] apply(org.apache.calcite.runtime.FlatLists.ComparableList key, Record1_0 acc) {
/* 93 */ return new Object[] {
/* 94 */ org.apache.calcite.runtime.SqlFunctions.toBoolean(key.get(1)) ? (Integer) null : Integer.valueOf(org.apache.calcite.runtime.SqlFunctions.toInt(key.get(0))),
/* 95 */ acc.f0};
/* 96 */ }
/* 97 */ public Object[] apply(Object key, Object acc) {
/* 98 */ return apply(
/* 99 */ (org.apache.calcite.runtime.FlatLists.ComparableList) key,
/* 100 */ (Record1_0) acc);
/* 101 */ }
/* 102 */ }
/* 103 */ );
/* 104 */ return new org.apache.calcite.linq4j.AbstractEnumerable(){
/* 105 */ public org.apache.calcite.linq4j.Enumerator enumerator() {
/* 106 */ return new org.apache.calcite.linq4j.Enumerator(){
/* 107 */ public final org.apache.calcite.linq4j.Enumerator inputEnumerator = _inputEnumerable.enumerator();
/* 108 */ public void reset() {
/* 109 */ inputEnumerator.reset();
/* 110 */ }
/* 111 */
/* 112 */ public boolean moveNext() {
/* 113 */ return inputEnumerator.moveNext();
/* 114 */ }
/* 115 */
/* 116 */ public void close() {
/* 117 */ inputEnumerator.close();
/* 118 */ }
/* 119 */
/* 120 */ public Object current() {
/* 121 */ return org.apache.calcite.runtime.SqlFunctions.toLong(((Object[]) inputEnumerator.current())[1]);
/* 122 */ }
/* 123 */
/* 124 */ };
/* 125 */ }
/* 126 */
/* 127 */ };
/* 128 */ }
/* 129 */
/* 130 */
/* 131 */ public Class getElementType() {
/* 132 */ return long.class;
/* 133 */ }
/* 134 */
/* 135 */ {code}
[~julianhyde], it looks like we might want to drop {{Aggregate#getGroupCount}} in Calcite 2.0
> aggregate over empty set , throw java.lang.NullPointerException
> ---------------------------------------------------------------
>
> Key: CALCITE-2283
> URL: https://issues.apache.org/jira/browse/CALCITE-2283
> Project: Calcite
> Issue Type: Bug
> Affects Versions: 1.16.0
> Reporter: foxes
> Assignee: Julian Hyde
> Priority: Major
>
> {code:java}
> //代码
> String sql =
> "select " +
> " count(distinct(bill_id)) as FULLY_REPAY_BILLS2_LST6 ," +
> " count(last_month) . as FULLY_REPAY_MONTHS_LST6 " +
> "from " +
> " cardBill " +
> "WHERE " +
> " 1=2 ";
> ResultSet result = st.executeQuery(sql.toUpperCase());
> result.next();// this throw NullPointerException
> {code}
> {code:java}
>
> Exception in thread "main" java.lang.NullPointerException
> at Baz$4$1.current(Unknown Source)
> at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:688)
> at org.apache.calcite.avatica.util.IteratorCursor.next(IteratorCursor.java:46)
> at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:207)
> at com.terry.netease.calcite.test.test.TestMemoryQuery.main(TestMemoryQuery.java:84)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)