You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@calcite.apache.org by Rui Wang <ru...@google.com.INVALID> on 2018/08/31 18:03:42 UTC

Had some exceptions when testing Row type

Hi community,

I am testing ROW type on top of Apache Beam. I am tying to make these cases
work:

INSERT INTO table_with_row_column
SELECT row from another_table_with_row_column

or

INSERT INTO table_with_row_column
SELECT row FROM (
  SELECT ... FROM (
     SELECT ...
  )
)

I tested a bunch of queries and had some exceptions that stopped queries to
finish. Here are some JIRAs I created with what query I tested, what Java
stack was and the source code link where the code execution failed:

https://issues.apache.org/jira/browse/CALCITE-2515
https://issues.apache.org/jira/browse/CALCITE-2516
https://issues.apache.org/jira/browse/CALCITE-2517


I am wondering whether these JIRAs are valid?

Thanks,
Rui

Re: Had some exceptions when testing Row type

Posted by Rui Wang <ru...@google.com.INVALID>.

Thanks Julian.

I am aware of the "flattener"  who flattens RexNode if the
RexNode.isStruct() == true in Projections.

My observation is that the three JIRAs above happen before the flattening
logic. Precisely, they are at SqlToRelConverter.convertQuery (and
sqlToRelConverter.flattenTypes
is the next step).

If these JIRAs are valid corner cases, and if the use case in Apache Beam
makes sense, will community support it? Or could I fix those JIRAs and
submit to Calcite?

-Rui

On Fri, Aug 31, 2018 at 11:43 AM Julian Hyde <jh...@apache.org> wrote:

> They all look plausible.
>
> Calcite uses a “flattener” to convert structs into regular columns and if
> it doesn’t do it right, you tend to get off-by-one errors.
>
> > On Aug 31, 2018, at 11:03 AM, Rui Wang <ru...@google.com.INVALID>
> wrote:
> >
> > Hi community,
> >
> > I am testing ROW type on top of Apache Beam. I am tying to make these
> cases
> > work:
> >
> > INSERT INTO table_with_row_column
> > SELECT row from another_table_with_row_column
> >
> > or
> >
> > INSERT INTO table_with_row_column
> > SELECT row FROM (
> >  SELECT ... FROM (
> >     SELECT ...
> >  )
> > )
> >
> > I tested a bunch of queries and had some exceptions that stopped queries
> to
> > finish. Here are some JIRAs I created with what query I tested, what Java
> > stack was and the source code link where the code execution failed:
> >
> > https://issues.apache.org/jira/browse/CALCITE-2515
> > https://issues.apache.org/jira/browse/CALCITE-2516
> > https://issues.apache.org/jira/browse/CALCITE-2517
> >
> >
> > I am wondering whether these JIRAs are valid?
> >
> > Thanks,
> > Rui
>
>

Re: Had some exceptions when testing Row type

Posted by Julian Hyde <jh...@apache.org>.

They all look plausible.

Calcite uses a “flattener” to convert structs into regular columns and if it doesn’t do it right, you tend to get off-by-one errors.

> On Aug 31, 2018, at 11:03 AM, Rui Wang <ru...@google.com.INVALID> wrote:
> 
> Hi community,
> 
> I am testing ROW type on top of Apache Beam. I am tying to make these cases
> work:
> 
> INSERT INTO table_with_row_column
> SELECT row from another_table_with_row_column
> 
> or
> 
> INSERT INTO table_with_row_column
> SELECT row FROM (
>  SELECT ... FROM (
>     SELECT ...
>  )
> )
> 
> I tested a bunch of queries and had some exceptions that stopped queries to
> finish. Here are some JIRAs I created with what query I tested, what Java
> stack was and the source code link where the code execution failed:
> 
> https://issues.apache.org/jira/browse/CALCITE-2515
> https://issues.apache.org/jira/browse/CALCITE-2516
> https://issues.apache.org/jira/browse/CALCITE-2517
> 
> 
> I am wondering whether these JIRAs are valid?
> 
> Thanks,
> Rui