You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@calcite.apache.org by "Laurent Goujon (JIRA)" <ji...@apache.org> on 2019/07/26 16:33:00 UTC
[jira] [Comment Edited] (CALCITE-3216) ClassCastException when
running window function over union
[ https://issues.apache.org/jira/browse/CALCITE-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16893969#comment-16893969 ]
Laurent Goujon edited comment on CALCITE-3216 at 7/26/19 4:32 PM:
------------------------------------------------------------------
The problem is likely to be common to any set operator: if the children of the operator are compatible but not exactly the same, the operator row type is the least restrictive of all the input types. But as far as I can tell, during execution, Calcite execution engine doesn't coerce the the records generated by the operator to match the row type, causing issues for parent operators as their physical representation do not match the expected type.
It's easy to spot in the generated source code:
{code:java}
/* 1 */ public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root) {
/* 2 */ final org.apache.calcite.linq4j.Enumerable _inputEnumerable = org.apache.calcite.linq4j.Linq4j.asEnumerable(new Integer[] {
/* 3 */ 0});
/* 4 */ final org.apache.calcite.linq4j.AbstractEnumerable child0 = new org.apache.calcite.linq4j.AbstractEnumerable(){
/* 5 */ public org.apache.calcite.linq4j.Enumerator enumerator() {
/* 6 */ return new org.apache.calcite.linq4j.Enumerator(){
/* 7 */ public final org.apache.calcite.linq4j.Enumerator inputEnumerator = _inputEnumerable.enumerator();
/* 8 */ public void reset() {
/* 9 */ inputEnumerator.reset();
/* 10 */ }
/* 11 */
/* 12 */ public boolean moveNext() {
/* 13 */ return inputEnumerator.moveNext();
/* 14 */ }
/* 15 */
/* 16 */ public void close() {
/* 17 */ inputEnumerator.close();
/* 18 */ }
/* 19 */
/* 20 */ public Object current() {
/* 21 */ return (byte)1;
/* 22 */ }
/* 23 */
/* 24 */ };
/* 25 */ }
/* 26 */
/* 27 */ };
/* 28 */ int prevStart;
/* 29 */ int prevEnd;
/* 30 */ final java.util.Comparator comparator = new java.util.Comparator(){
/* 31 */ public int compare(Integer v0, Integer v1) {
/* 32 */ int c;
/* 33 */ return 0;
/* 34 */ }
/* 35 */
/* 36 */ public int compare(Object o0, Object o1) {
/* 37 */ return this.compare((Integer) o0, (Integer) o1);
/* 38 */ }
/* 39 */
/* 40 */ };
/* 41 */ final org.apache.calcite.runtime.SortedMultiMap multiMap = new org.apache.calcite.runtime.SortedMultiMap();
/* 42 */ child0.union(org.apache.calcite.linq4j.Linq4j.asEnumerable(new Integer[] {
/* 43 */ 2})).foreach(new org.apache.calcite.linq4j.function.Function1() {
/* 44 */ public Object apply(int v) {
/* 45 */ int key = v;
/* 46 */ multiMap.putMulti(key, v);
/* 47 */ return null;
/* 48 */ }
/* 49 */ public Object apply(Integer v) {
/* 50 */ return apply(
/* 51 */ v.intValue());
/* 52 */ }
/* 53 */ public Object apply(Object v) {
/* 54 */ return apply(
/* 55 */ (Integer) v);
/* 56 */ }
/* 57 */ }
/* 58 */ );
/* 59 */ final java.util.Iterator iterator = multiMap.arrays(comparator);
/* 60 */ final java.util.ArrayList _list = new java.util.ArrayList(
/* 61 */ multiMap.size());
/* 62 */ Long COUNTa0w0 = (Long) null;
/* 63 */ while (iterator.hasNext()) {
/* 64 */ final Object[] _rows = (Object[]) iterator.next();
/* 65 */ prevStart = -1;
/* 66 */ prevEnd = 2147483647;
/* 67 */ final int maxX = _rows.length - 1;
/* 68 */ for (int i = 0; i < _rows.length; ++i) {
/* 69 */ if (maxX != prevEnd) {
/* 70 */ int actualStart = maxX < prevEnd ? 0 : prevEnd + 1;
/* 71 */ prevEnd = maxX;
/* 72 */ COUNTa0w0 = Long.valueOf(maxX + 1);
/* 73 */ }
/* 74 */ _list.add(new Object[] {
/* 75 */ org.apache.calcite.runtime.SqlFunctions.toInt(_rows[i]),
/* 76 */ COUNTa0w0});
/* 77 */ }
/* 78 */ }
/* 79 */ multiMap.clear();
/* 80 */ return org.apache.calcite.linq4j.Linq4j.asEnumerable(_list);
/* 81 */ }
/* 82 */
/* 83 */
/* 84 */ public Class getElementType() {
/* 85 */ return java.lang.Object[].class;
/* 86 */ }
{code}
Alas, I'm not familiar at all with the execution engine, and have no idea what the fix should look like.
was (Author: laurentgo):
The problem is likely to be common to any set operator: if the children of the operator are compatible but not exactly the same, the operator row type is the least restrictive of all the input types. But as far as I can tell, during execution, Calcite execution engine doesn't coerce the the records generated by the operator to match the row type, causing issues for parent operators as their physical representation do not match the expected type.
It's easy to spot in the generated source code:
{code:java}
public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root) {
final org.apache.calcite.linq4j.Enumerable _inputEnumerable = org.apache.calcite.linq4j.Linq4j.asEnumerable(new Integer[] {
0});
final org.apache.calcite.linq4j.AbstractEnumerable child0 = new org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
return new org.apache.calcite.linq4j.Enumerator(){
public final org.apache.calcite.linq4j.Enumerator inputEnumerator = _inputEnumerable.enumerator();
public void reset() {
inputEnumerator.reset();
}
public boolean moveNext() {
return inputEnumerator.moveNext();
}
public void close() {
inputEnumerator.close();
}
public Object current() {
return (byte)1;
}
};
}
};
final org.apache.calcite.linq4j.Enumerable _inputEnumerable0 = child0.union(org.apache.calcite.linq4j.Linq4j.asEnumerable(new Integer[] {
2}));
return new org.apache.calcite.linq4j.AbstractEnumerable(){
public org.apache.calcite.linq4j.Enumerator enumerator() {
return new org.apache.calcite.linq4j.Enumerator(){
public final org.apache.calcite.linq4j.Enumerator inputEnumerator = _inputEnumerable0.enumerator();
public void reset() {
inputEnumerator.reset();
}
public boolean moveNext() {
return inputEnumerator.moveNext();
}
public void close() {
inputEnumerator.close();
}
public Object current() {
return org.apache.calcite.runtime.SqlFunctions.toInt(inputEnumerator.current()) * 2;
}
};
}
};
}
public Class getElementType() {
return int.class;
}
{code}
Alas, I'm not familiar at all with the execution engine, and have no idea what the fix should look like.
> ClassCastException when running window function over union
> ----------------------------------------------------------
>
> Key: CALCITE-3216
> URL: https://issues.apache.org/jira/browse/CALCITE-3216
> Project: Calcite
> Issue Type: Bug
> Components: core
> Reporter: Laurent Goujon
> Priority: Major
>
> I discovered an issue in Calcite execution engine which can be captured by this simple query:
> {code:sql}
> select *, count(*) over (partition by "id") from (
> select "id" from (VALUES(CAST(1 AS TINYINT))) "foo"("id")
> union
> select "id" from (VALUES(2)) "foo"("id"))
> {code}
> When running this query using JdbcTest for example, I got the following stacktrace:
> {noformat}
> Caused by: java.lang.ClassCastException: java.lang.Byte cannot be cast to java.lang.Integer
> at Baz$3.apply(ANONYMOUS.java:55)
> at org.apache.calcite.linq4j.DefaultEnumerable.foreach(DefaultEnumerable.java:77)
> at Baz.bind(Baz.java:43)
> at org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:355)
> at org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:316)
> at org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:506)
> at org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:497)
> at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:182)
> at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64)
> at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:1)
> at org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:667)
> at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:566)
> at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
> at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)