You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Shuyi Chen <su...@gmail.com> on 2017/06/17 01:07:10 UTC
Question regarding "select * from" behavior with RelRecordType(PEEK_FIELDS,
...)
Hi all,
Our team are looking into integration of Flink with Calcite. We found that
when we set the RelRecordType to use the PEEK_FIELDS/PEEK_FIELDS_DEFAULT
name resolution policy. A "select * from Table" will yield a recursively
flatten result. For example,
Table:
a: Int
b: Record(f1: Int, f2: Record(f3: String, f4:Int))
"select * from Table" will return tuples of (a, b.f1, b.f2.f3, b.f2.f4)
instead of (a, b) when we use PEEK_FIELDS/PEEK_FIELDS_DEFAULT for the
RelRecordType.
Using FULLY_QUALIFIED will yield the expected result (a, b)
We found that this recursive flattening behavior was controlled by this
*kind* field in RelRecordType, which states in the documentation that it's
used for resolving fields in record types.
I am wondering if this is an expected behavior, or should we fix it? Thanks
a lot.
Shuyi
--
"So you have to trust that the dots will somehow connect in your future."
Re: Question regarding "select * from" behavior with
RelRecordType(PEEK_FIELDS, ...)
Posted by Julian Hyde <jh...@apache.org>.
Looks good; see my comments on the JIRA case.
On Thu, Jun 22, 2017 at 12:00 AM, Shuyi Chen <su...@gmail.com> wrote:
> Can someone help take a look at this JIRA
> <https://issues.apache.org/jira/browse/CALCITE-1856> (
> https://issues.apache.org/jira/browse/CALCITE-1856) and PR
> <https://github.com/apache/calcite/pull/479> (
> https://github.com/apache/calcite/pull/479)? Thanks a lot.
>
> On Mon, Jun 19, 2017 at 6:01 PM, Shuyi Chen <su...@gmail.com> wrote:
>
>> Can we add an option in StructKind, say, PEEK_FIELDS_NO_STAR_EXPANSION?
>> And modify SqlValidatorImpl.java to disable star expansion optionally?
>> Thanks a lot.
>>
>> Shuyi
>>
>> On Sun, Jun 18, 2017 at 11:59 PM, Shuyi Chen <su...@gmail.com> wrote:
>>
>>> We do want to use the PEEK_FIELDS behavior as Flink does support
>>> RecordType. But I believe Avatica is not used in Flink SQL API.
>>>
>>> Is it possible to add an option in Calcite to disable the flattening
>>> behavior when using PEEK_FIELDS? Or what do you suggest?
>>>
>>> Thanks.
>>>
>>> On Fri, Jun 16, 2017 at 9:03 PM, Julian Hyde <jh...@apache.org> wrote:
>>>
>>>> PEEK_FIELDS was added as part of https://issues.apache.org/jira
>>>> /browse/CALCITE-1208 <https://issues.apache.org/jira/browse/CALCITE-1208>,
>>>> basically to allow us to implement Phoenix’s “column family” feature, where
>>>> it’s optional whether you qualify a column with its column family.
>>>>
>>>> If you don’t have that requirement, you probably shouldn’t use
>>>> PEEK_FIELDS or PEEK_FIELDS_DEFAULT.
>>>>
>>>> Julian
>>>>
>>>> > On Jun 16, 2017, at 6:07 PM, Shuyi Chen <su...@gmail.com> wrote:
>>>> >
>>>> > Hi all,
>>>> >
>>>> > Our team are looking into integration of Flink with Calcite. We found
>>>> that
>>>> > when we set the RelRecordType to use the PEEK_FIELDS/PEEK_FIELDS_DEFAUL
>>>> T
>>>> > name resolution policy. A "select * from Table" will yield a
>>>> recursively
>>>> > flatten result. For example,
>>>> >
>>>> > Table:
>>>> > a: Int
>>>> > b: Record(f1: Int, f2: Record(f3: String, f4:Int))
>>>> >
>>>> > "select * from Table" will return tuples of (a, b.f1, b.f2.f3, b.f2.f4)
>>>> > instead of (a, b) when we use PEEK_FIELDS/PEEK_FIELDS_DEFAULT for the
>>>> > RelRecordType.
>>>> >
>>>> > Using FULLY_QUALIFIED will yield the expected result (a, b)
>>>> >
>>>> > We found that this recursive flattening behavior was controlled by this
>>>> > *kind* field in RelRecordType, which states in the documentation that
>>>> it's
>>>> > used for resolving fields in record types.
>>>> >
>>>> > I am wondering if this is an expected behavior, or should we fix it?
>>>> Thanks
>>>> > a lot.
>>>> >
>>>> > Shuyi
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > "So you have to trust that the dots will somehow connect in your
>>>> future."
>>>>
>>>>
>>>
>>>
>>> --
>>> "So you have to trust that the dots will somehow connect in your future."
>>>
>>
>>
>>
>> --
>> "So you have to trust that the dots will somehow connect in your future."
>>
>
>
>
> --
> "So you have to trust that the dots will somehow connect in your future."
Re: Question regarding "select * from" behavior with
RelRecordType(PEEK_FIELDS, ...)
Posted by Shuyi Chen <su...@gmail.com>.
Can someone help take a look at this JIRA
<https://issues.apache.org/jira/browse/CALCITE-1856> (
https://issues.apache.org/jira/browse/CALCITE-1856) and PR
<https://github.com/apache/calcite/pull/479> (
https://github.com/apache/calcite/pull/479)? Thanks a lot.
On Mon, Jun 19, 2017 at 6:01 PM, Shuyi Chen <su...@gmail.com> wrote:
> Can we add an option in StructKind, say, PEEK_FIELDS_NO_STAR_EXPANSION?
> And modify SqlValidatorImpl.java to disable star expansion optionally?
> Thanks a lot.
>
> Shuyi
>
> On Sun, Jun 18, 2017 at 11:59 PM, Shuyi Chen <su...@gmail.com> wrote:
>
>> We do want to use the PEEK_FIELDS behavior as Flink does support
>> RecordType. But I believe Avatica is not used in Flink SQL API.
>>
>> Is it possible to add an option in Calcite to disable the flattening
>> behavior when using PEEK_FIELDS? Or what do you suggest?
>>
>> Thanks.
>>
>> On Fri, Jun 16, 2017 at 9:03 PM, Julian Hyde <jh...@apache.org> wrote:
>>
>>> PEEK_FIELDS was added as part of https://issues.apache.org/jira
>>> /browse/CALCITE-1208 <https://issues.apache.org/jira/browse/CALCITE-1208>,
>>> basically to allow us to implement Phoenix’s “column family” feature, where
>>> it’s optional whether you qualify a column with its column family.
>>>
>>> If you don’t have that requirement, you probably shouldn’t use
>>> PEEK_FIELDS or PEEK_FIELDS_DEFAULT.
>>>
>>> Julian
>>>
>>> > On Jun 16, 2017, at 6:07 PM, Shuyi Chen <su...@gmail.com> wrote:
>>> >
>>> > Hi all,
>>> >
>>> > Our team are looking into integration of Flink with Calcite. We found
>>> that
>>> > when we set the RelRecordType to use the PEEK_FIELDS/PEEK_FIELDS_DEFAUL
>>> T
>>> > name resolution policy. A "select * from Table" will yield a
>>> recursively
>>> > flatten result. For example,
>>> >
>>> > Table:
>>> > a: Int
>>> > b: Record(f1: Int, f2: Record(f3: String, f4:Int))
>>> >
>>> > "select * from Table" will return tuples of (a, b.f1, b.f2.f3, b.f2.f4)
>>> > instead of (a, b) when we use PEEK_FIELDS/PEEK_FIELDS_DEFAULT for the
>>> > RelRecordType.
>>> >
>>> > Using FULLY_QUALIFIED will yield the expected result (a, b)
>>> >
>>> > We found that this recursive flattening behavior was controlled by this
>>> > *kind* field in RelRecordType, which states in the documentation that
>>> it's
>>> > used for resolving fields in record types.
>>> >
>>> > I am wondering if this is an expected behavior, or should we fix it?
>>> Thanks
>>> > a lot.
>>> >
>>> > Shuyi
>>> >
>>> >
>>> >
>>> > --
>>> > "So you have to trust that the dots will somehow connect in your
>>> future."
>>>
>>>
>>
>>
>> --
>> "So you have to trust that the dots will somehow connect in your future."
>>
>
>
>
> --
> "So you have to trust that the dots will somehow connect in your future."
>
--
"So you have to trust that the dots will somehow connect in your future."
Re: Question regarding "select * from" behavior with
RelRecordType(PEEK_FIELDS, ...)
Posted by Shuyi Chen <su...@gmail.com>.
Can we add an option in StructKind, say, PEEK_FIELDS_NO_STAR_EXPANSION? And
modify SqlValidatorImpl.java to disable star expansion optionally? Thanks a
lot.
Shuyi
On Sun, Jun 18, 2017 at 11:59 PM, Shuyi Chen <su...@gmail.com> wrote:
> We do want to use the PEEK_FIELDS behavior as Flink does support
> RecordType. But I believe Avatica is not used in Flink SQL API.
>
> Is it possible to add an option in Calcite to disable the flattening
> behavior when using PEEK_FIELDS? Or what do you suggest?
>
> Thanks.
>
> On Fri, Jun 16, 2017 at 9:03 PM, Julian Hyde <jh...@apache.org> wrote:
>
>> PEEK_FIELDS was added as part of https://issues.apache.org/jira
>> /browse/CALCITE-1208 <https://issues.apache.org/jira/browse/CALCITE-1208>,
>> basically to allow us to implement Phoenix’s “column family” feature, where
>> it’s optional whether you qualify a column with its column family.
>>
>> If you don’t have that requirement, you probably shouldn’t use
>> PEEK_FIELDS or PEEK_FIELDS_DEFAULT.
>>
>> Julian
>>
>> > On Jun 16, 2017, at 6:07 PM, Shuyi Chen <su...@gmail.com> wrote:
>> >
>> > Hi all,
>> >
>> > Our team are looking into integration of Flink with Calcite. We found
>> that
>> > when we set the RelRecordType to use the PEEK_FIELDS/PEEK_FIELDS_DEFAUL
>> T
>> > name resolution policy. A "select * from Table" will yield a recursively
>> > flatten result. For example,
>> >
>> > Table:
>> > a: Int
>> > b: Record(f1: Int, f2: Record(f3: String, f4:Int))
>> >
>> > "select * from Table" will return tuples of (a, b.f1, b.f2.f3, b.f2.f4)
>> > instead of (a, b) when we use PEEK_FIELDS/PEEK_FIELDS_DEFAULT for the
>> > RelRecordType.
>> >
>> > Using FULLY_QUALIFIED will yield the expected result (a, b)
>> >
>> > We found that this recursive flattening behavior was controlled by this
>> > *kind* field in RelRecordType, which states in the documentation that
>> it's
>> > used for resolving fields in record types.
>> >
>> > I am wondering if this is an expected behavior, or should we fix it?
>> Thanks
>> > a lot.
>> >
>> > Shuyi
>> >
>> >
>> >
>> > --
>> > "So you have to trust that the dots will somehow connect in your
>> future."
>>
>>
>
>
> --
> "So you have to trust that the dots will somehow connect in your future."
>
--
"So you have to trust that the dots will somehow connect in your future."
Re: Question regarding "select * from" behavior with
RelRecordType(PEEK_FIELDS, ...)
Posted by Shuyi Chen <su...@gmail.com>.
We do want to use the PEEK_FIELDS behavior as Flink does support
RecordType. But I believe Avatica is not used in Flink SQL API.
Is it possible to add an option in Calcite to disable the flattening
behavior when using PEEK_FIELDS? Or what do you suggest?
Thanks.
On Fri, Jun 16, 2017 at 9:03 PM, Julian Hyde <jh...@apache.org> wrote:
> PEEK_FIELDS was added as part of https://issues.apache.org/
> jira/browse/CALCITE-1208 <https://issues.apache.org/
> jira/browse/CALCITE-1208>, basically to allow us to implement Phoenix’s
> “column family” feature, where it’s optional whether you qualify a column
> with its column family.
>
> If you don’t have that requirement, you probably shouldn’t use PEEK_FIELDS
> or PEEK_FIELDS_DEFAULT.
>
> Julian
>
> > On Jun 16, 2017, at 6:07 PM, Shuyi Chen <su...@gmail.com> wrote:
> >
> > Hi all,
> >
> > Our team are looking into integration of Flink with Calcite. We found
> that
> > when we set the RelRecordType to use the PEEK_FIELDS/PEEK_FIELDS_DEFAULT
> > name resolution policy. A "select * from Table" will yield a recursively
> > flatten result. For example,
> >
> > Table:
> > a: Int
> > b: Record(f1: Int, f2: Record(f3: String, f4:Int))
> >
> > "select * from Table" will return tuples of (a, b.f1, b.f2.f3, b.f2.f4)
> > instead of (a, b) when we use PEEK_FIELDS/PEEK_FIELDS_DEFAULT for the
> > RelRecordType.
> >
> > Using FULLY_QUALIFIED will yield the expected result (a, b)
> >
> > We found that this recursive flattening behavior was controlled by this
> > *kind* field in RelRecordType, which states in the documentation that
> it's
> > used for resolving fields in record types.
> >
> > I am wondering if this is an expected behavior, or should we fix it?
> Thanks
> > a lot.
> >
> > Shuyi
> >
> >
> >
> > --
> > "So you have to trust that the dots will somehow connect in your future."
>
>
--
"So you have to trust that the dots will somehow connect in your future."
Re: Question regarding "select * from" behavior with
RelRecordType(PEEK_FIELDS, ...)
Posted by Julian Hyde <jh...@apache.org>.
PEEK_FIELDS was added as part of https://issues.apache.org/jira/browse/CALCITE-1208 <https://issues.apache.org/jira/browse/CALCITE-1208>, basically to allow us to implement Phoenix’s “column family” feature, where it’s optional whether you qualify a column with its column family.
If you don’t have that requirement, you probably shouldn’t use PEEK_FIELDS or PEEK_FIELDS_DEFAULT.
Julian
> On Jun 16, 2017, at 6:07 PM, Shuyi Chen <su...@gmail.com> wrote:
>
> Hi all,
>
> Our team are looking into integration of Flink with Calcite. We found that
> when we set the RelRecordType to use the PEEK_FIELDS/PEEK_FIELDS_DEFAULT
> name resolution policy. A "select * from Table" will yield a recursively
> flatten result. For example,
>
> Table:
> a: Int
> b: Record(f1: Int, f2: Record(f3: String, f4:Int))
>
> "select * from Table" will return tuples of (a, b.f1, b.f2.f3, b.f2.f4)
> instead of (a, b) when we use PEEK_FIELDS/PEEK_FIELDS_DEFAULT for the
> RelRecordType.
>
> Using FULLY_QUALIFIED will yield the expected result (a, b)
>
> We found that this recursive flattening behavior was controlled by this
> *kind* field in RelRecordType, which states in the documentation that it's
> used for resolving fields in record types.
>
> I am wondering if this is an expected behavior, or should we fix it? Thanks
> a lot.
>
> Shuyi
>
>
>
> --
> "So you have to trust that the dots will somehow connect in your future."