You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Julian Hyde <jh...@apache.org> on 2015/01/27 18:38:06 UTC
Towards RC2
On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran
<jp...@hortonworks.com> wrote:
> On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran <jp...@hortonworks.com> wrote:
>
>> We seems to have issues with union, subquery, and constants.
>>
>> Currently we are debugging these to find more details.
>>
>> Would update EOB today.
>
> So far 2 issues been discovered with Calcite RC; CALCITE-570, CALCITE-571.
John,
Is that your final list? I have a fix for
https://issues.apache.org/jira/browse/CALCITE-570, am working on
https://issues.apache.org/jira/browse/CALCITE-571. I intend to produce
a new RC including these and
https://issues.apache.org/jira/browse/CALCITE-567 and
https://issues.apache.org/jira/browse/CALCITE-568 (both issues that
Vladimir found in RC1). If your list is final RC2 could be as early as
end of today.
When there is an RC2, would it help to have a few days to review it
before I start a vote?
Julian
Re: Towards RC2
Posted by John Pullokkaran <jp...@hortonworks.com>.
Yes that would work.
Just for union, I could modify AST converter to make names match.
We are working on translating Calcite Ops to Hive Ops at which point this
would be a non issue.
On 1/28/15, 2:54 PM, "Jacques Nadeau" <ja...@apache.org> wrote:
>John,
>
>I think the reason we haven't been bit as much by this is that Drill
>rewrites the entire tree right before going back to names. With most
>operators, this will regenerate and propagate the names in the row types
>to
>be consistent. This would be guaranteed if you enforce that requirement
>it
>in your own rels (which I assume you're at when you're about to translate
>back to your internal representation). I don't know the conversion to AST
>code well enough to know if this would help you but it might get you
>around
>this issue short term.
>
>On Wed, Jan 28, 2015 at 11:27 AM, Julian Hyde <ju...@gmail.com>
>wrote:
>
>> John,
>>
>> First of all, let me say I feel your pain.
>>
>> But a client should not make assumptions about the names that Calcite
>> assigns. (It¹s like using a Java HashMap and making assumptions about
>> iteration order. If the iteration order changes from one version of the
>>JDK
>> to the next, it is not a bug in the JDK; it is a bug in your app.) There
>> have been discussions about this issue on this list recently.
>>
>> Hive needs to use Calcite in the way it was designed ‹ where you
>>identify
>> fields by position, not by name. How long would you need to fix Hive to
>>do
>> this? If you can commit to fixing Hive by a particular date, then I will
>> accept the patch short-term. The modifications would go in deprecated
>>(so
>> no one else starts using them) and I would log an issue to remove them.
>>
>> By the way, I have fixed the other issues. As soon as this is resolved,
>>we
>> can make another RC.
>>
>> Julian
>>
>>
>> On Jan 28, 2015, at 10:17 AM, John Pullokkaran <
>> jpullokkaran@hortonworks.com> wrote:
>>
>> > I filed CALCITE-575 to keep track of this.
>> > I have attached a patch to it.
>> >
>> > Patch fixes many of the issues for Hive (~ 25).
>> >
>> >
>> > On Tue, Jan 27, 2015 at 3:12 PM, John Pullokkaran <
>> > jpullokkaran@hortonworks.com> wrote:
>> >
>> >> Unfortunately this requires a patch to Calcite.
>> >> I am testing the patch right now.
>> >>
>> >> So far changes are to RelFieldTrimmer, RexUtil, ProjectRemoveRule.
>> >>
>> >> On Tue, Jan 27, 2015 at 10:55 AM, John Pullokkaran <
>> >> jpullokkaran@hortonworks.com> wrote:
>> >>
>> >>> Many of Hive issues seems to be the result of CALCITE-92.
>> >>>
>> >>> I don¹t think there is anything wrong with the patch itself, but it
>> >>> breaks Hive¹s usage of Calcite (due to Calcite OP tree getting
>> converted
>> >>> back to Hive AST).
>> >>>
>> >>> I am going to try disabling it in Hive¹s usage of Calcite; will
>>update
>> >>> soon.
>> >>>
>> >>> John
>> >>>
>> >>> On Jan 27, 2015, at 9:38 AM, Julian Hyde <jhyde@apache.org<mailto:
>> >>> jhyde@apache.org>> wrote:
>> >>>
>> >>> On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran
>> >>> <jp...@hortonworks.com>>
>> >>> wrote:
>> >>>
>> >>> On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran <
>> >>> jpullokkaran@hortonworks.com<ma...@hortonworks.com>>
>> wrote:
>> >>>
>> >>> We seems to have issues with union, subquery, and constants.
>> >>>
>> >>> Currently we are debugging these to find more details.
>> >>>
>> >>> Would update EOB today.
>> >>>
>> >>> So far 2 issues been discovered with Calcite RC; CALCITE-570,
>> CALCITE-571.
>> >>>
>> >>> John,
>> >>>
>> >>> Is that your final list? I have a fix for
>> >>> https://issues.apache.org/jira/browse/CALCITE-570, am working on
>> >>> https://issues.apache.org/jira/browse/CALCITE-571. I intend to
>>produce
>> >>> a new RC including these and
>> >>> https://issues.apache.org/jira/browse/CALCITE-567 and
>> >>> https://issues.apache.org/jira/browse/CALCITE-568 (both issues that
>> >>> Vladimir found in RC1). If your list is final RC2 could be as early
>>as
>> >>> end of today.
>> >>>
>> >>> When there is an RC2, would it help to have a few days to review it
>> >>> before I start a vote?
>> >>>
>> >>> Julian
>> >>>
>> >>>
>> >>
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>>entity
>> to
>> > which it is addressed and may contain information that is
>>confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>>reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>>
Re: Towards RC2
Posted by Jacques Nadeau <ja...@apache.org>.
John,
I think the reason we haven't been bit as much by this is that Drill
rewrites the entire tree right before going back to names. With most
operators, this will regenerate and propagate the names in the row types to
be consistent. This would be guaranteed if you enforce that requirement it
in your own rels (which I assume you're at when you're about to translate
back to your internal representation). I don't know the conversion to AST
code well enough to know if this would help you but it might get you around
this issue short term.
On Wed, Jan 28, 2015 at 11:27 AM, Julian Hyde <ju...@gmail.com> wrote:
> John,
>
> First of all, let me say I feel your pain.
>
> But a client should not make assumptions about the names that Calcite
> assigns. (It’s like using a Java HashMap and making assumptions about
> iteration order. If the iteration order changes from one version of the JDK
> to the next, it is not a bug in the JDK; it is a bug in your app.) There
> have been discussions about this issue on this list recently.
>
> Hive needs to use Calcite in the way it was designed — where you identify
> fields by position, not by name. How long would you need to fix Hive to do
> this? If you can commit to fixing Hive by a particular date, then I will
> accept the patch short-term. The modifications would go in deprecated (so
> no one else starts using them) and I would log an issue to remove them.
>
> By the way, I have fixed the other issues. As soon as this is resolved, we
> can make another RC.
>
> Julian
>
>
> On Jan 28, 2015, at 10:17 AM, John Pullokkaran <
> jpullokkaran@hortonworks.com> wrote:
>
> > I filed CALCITE-575 to keep track of this.
> > I have attached a patch to it.
> >
> > Patch fixes many of the issues for Hive (~ 25).
> >
> >
> > On Tue, Jan 27, 2015 at 3:12 PM, John Pullokkaran <
> > jpullokkaran@hortonworks.com> wrote:
> >
> >> Unfortunately this requires a patch to Calcite.
> >> I am testing the patch right now.
> >>
> >> So far changes are to RelFieldTrimmer, RexUtil, ProjectRemoveRule.
> >>
> >> On Tue, Jan 27, 2015 at 10:55 AM, John Pullokkaran <
> >> jpullokkaran@hortonworks.com> wrote:
> >>
> >>> Many of Hive issues seems to be the result of CALCITE-92.
> >>>
> >>> I don’t think there is anything wrong with the patch itself, but it
> >>> breaks Hive’s usage of Calcite (due to Calcite OP tree getting
> converted
> >>> back to Hive AST).
> >>>
> >>> I am going to try disabling it in Hive’s usage of Calcite; will update
> >>> soon.
> >>>
> >>> John
> >>>
> >>> On Jan 27, 2015, at 9:38 AM, Julian Hyde <jhyde@apache.org<mailto:
> >>> jhyde@apache.org>> wrote:
> >>>
> >>> On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran
> >>> <jp...@hortonworks.com>>
> >>> wrote:
> >>>
> >>> On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran <
> >>> jpullokkaran@hortonworks.com<ma...@hortonworks.com>>
> wrote:
> >>>
> >>> We seems to have issues with union, subquery, and constants.
> >>>
> >>> Currently we are debugging these to find more details.
> >>>
> >>> Would update EOB today.
> >>>
> >>> So far 2 issues been discovered with Calcite RC; CALCITE-570,
> CALCITE-571.
> >>>
> >>> John,
> >>>
> >>> Is that your final list? I have a fix for
> >>> https://issues.apache.org/jira/browse/CALCITE-570, am working on
> >>> https://issues.apache.org/jira/browse/CALCITE-571. I intend to produce
> >>> a new RC including these and
> >>> https://issues.apache.org/jira/browse/CALCITE-567 and
> >>> https://issues.apache.org/jira/browse/CALCITE-568 (both issues that
> >>> Vladimir found in RC1). If your list is final RC2 could be as early as
> >>> end of today.
> >>>
> >>> When there is an RC2, would it help to have a few days to review it
> >>> before I start a vote?
> >>>
> >>> Julian
> >>>
> >>>
> >>
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
>
Re: Towards RC2
Posted by Julian Hyde <ju...@gmail.com>.
We hit feature freeze before RC1. Only high-priority bug-fixes now. (Those two pull requests are nice but I can see how they could each cause instability…)
In fact I’m just about to start RC2, based on https://github.com/julianhyde/incubator-calcite/commit/4755cae86c36002b26072c62ef2d4b942c935e1a. I’m going to try making the release on a branch, for the first time. If that works, we should be able to open up master for commits even before the 2 votes have completed.
Julian
On Jan 28, 2015, at 2:12 PM, Vladimir Sitnikov <si...@gmail.com> wrote:
> By the way: when we hit feature freeze for 1.0?
>
> I believe "keep more user-given column names"
> (https://github.com/apache/incubator-calcite/pull/46) and "remove
> Project.flags" (https://github.com/apache/incubator-calcite/pull/45)
> are nice to have in 1.0
>
> Vladimir
Re: Towards RC2
Posted by Vladimir Sitnikov <si...@gmail.com>.
By the way: when we hit feature freeze for 1.0?
I believe "keep more user-given column names"
(https://github.com/apache/incubator-calcite/pull/46) and "remove
Project.flags" (https://github.com/apache/incubator-calcite/pull/45)
are nice to have in 1.0
Vladimir
Re: Towards RC2
Posted by Julian Hyde <ju...@gmail.com>.
“3 months or so” is far too long. My best offer is that I can put it in for 1.0, and remove it in 1.1 (which will be at least 4 weeks, probably early March). You’ll need to fix the issue in Hive in order to upgrade to 1.1.
Julian
On Jan 28, 2015, at 12:12 PM, John Pullokkaran <jp...@hortonworks.com> wrote:
> As I said before and in the bug the reason for Hive¹s dependency on names
> is due to the fact that Calcite plan gets translated to Hive AST. Hive Gen
> Plan enforces constraints on certain ops (like in the case of union, the
> field names should match).
>
> Once Hive translates Calcite Ops to Hive Ops directly, this problem would
> go away.
> The work is scheduled to be completed in another 3 months or so.
>
>
> On 1/28/15, 11:27 AM, "Julian Hyde" <ju...@gmail.com> wrote:
>
>> John,
>>
>> First of all, let me say I feel your pain.
>>
>> But a client should not make assumptions about the names that Calcite
>> assigns. (It¹s like using a Java HashMap and making assumptions about
>> iteration order. If the iteration order changes from one version of the
>> JDK to the next, it is not a bug in the JDK; it is a bug in your app.)
>> There have been discussions about this issue on this list recently.
>>
>> Hive needs to use Calcite in the way it was designed ‹ where you identify
>> fields by position, not by name. How long would you need to fix Hive to
>> do this? If you can commit to fixing Hive by a particular date, then I
>> will accept the patch short-term. The modifications would go in
>> deprecated (so no one else starts using them) and I would log an issue to
>> remove them.
>>
>> By the way, I have fixed the other issues. As soon as this is resolved,
>> we can make another RC.
>>
>> Julian
>>
>>
>> On Jan 28, 2015, at 10:17 AM, John Pullokkaran
>> <jp...@hortonworks.com> wrote:
>>
>>> I filed CALCITE-575 to keep track of this.
>>> I have attached a patch to it.
>>>
>>> Patch fixes many of the issues for Hive (~ 25).
>>>
>>>
>>> On Tue, Jan 27, 2015 at 3:12 PM, John Pullokkaran <
>>> jpullokkaran@hortonworks.com> wrote:
>>>
>>>> Unfortunately this requires a patch to Calcite.
>>>> I am testing the patch right now.
>>>>
>>>> So far changes are to RelFieldTrimmer, RexUtil, ProjectRemoveRule.
>>>>
>>>> On Tue, Jan 27, 2015 at 10:55 AM, John Pullokkaran <
>>>> jpullokkaran@hortonworks.com> wrote:
>>>>
>>>>> Many of Hive issues seems to be the result of CALCITE-92.
>>>>>
>>>>> I don¹t think there is anything wrong with the patch itself, but it
>>>>> breaks Hive¹s usage of Calcite (due to Calcite OP tree getting
>>>>> converted
>>>>> back to Hive AST).
>>>>>
>>>>> I am going to try disabling it in Hive¹s usage of Calcite; will update
>>>>> soon.
>>>>>
>>>>> John
>>>>>
>>>>> On Jan 27, 2015, at 9:38 AM, Julian Hyde <jhyde@apache.org<mailto:
>>>>> jhyde@apache.org>> wrote:
>>>>>
>>>>> On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran
>>>>> <jp...@hortonworks.com>>
>>>>> wrote:
>>>>>
>>>>> On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran <
>>>>> jpullokkaran@hortonworks.com<ma...@hortonworks.com>>
>>>>> wrote:
>>>>>
>>>>> We seems to have issues with union, subquery, and constants.
>>>>>
>>>>> Currently we are debugging these to find more details.
>>>>>
>>>>> Would update EOB today.
>>>>>
>>>>> So far 2 issues been discovered with Calcite RC; CALCITE-570,
>>>>> CALCITE-571.
>>>>>
>>>>> John,
>>>>>
>>>>> Is that your final list? I have a fix for
>>>>> https://issues.apache.org/jira/browse/CALCITE-570, am working on
>>>>> https://issues.apache.org/jira/browse/CALCITE-571. I intend to produce
>>>>> a new RC including these and
>>>>> https://issues.apache.org/jira/browse/CALCITE-567 and
>>>>> https://issues.apache.org/jira/browse/CALCITE-568 (both issues that
>>>>> Vladimir found in RC1). If your list is final RC2 could be as early as
>>>>> end of today.
>>>>>
>>>>> When there is an RC2, would it help to have a few days to review it
>>>>> before I start a vote?
>>>>>
>>>>> Julian
>>>>>
>>>>>
>>>>
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or
>>> entity to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the
>>> reader
>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>>> immediately
>>> and delete it from your system. Thank You.
>>
>
Re: Towards RC2
Posted by John Pullokkaran <jp...@hortonworks.com>.
As I said before and in the bug the reason for Hive¹s dependency on names
is due to the fact that Calcite plan gets translated to Hive AST. Hive Gen
Plan enforces constraints on certain ops (like in the case of union, the
field names should match).
Once Hive translates Calcite Ops to Hive Ops directly, this problem would
go away.
The work is scheduled to be completed in another 3 months or so.
On 1/28/15, 11:27 AM, "Julian Hyde" <ju...@gmail.com> wrote:
>John,
>
>First of all, let me say I feel your pain.
>
>But a client should not make assumptions about the names that Calcite
>assigns. (It¹s like using a Java HashMap and making assumptions about
>iteration order. If the iteration order changes from one version of the
>JDK to the next, it is not a bug in the JDK; it is a bug in your app.)
>There have been discussions about this issue on this list recently.
>
>Hive needs to use Calcite in the way it was designed ‹ where you identify
>fields by position, not by name. How long would you need to fix Hive to
>do this? If you can commit to fixing Hive by a particular date, then I
>will accept the patch short-term. The modifications would go in
>deprecated (so no one else starts using them) and I would log an issue to
>remove them.
>
>By the way, I have fixed the other issues. As soon as this is resolved,
>we can make another RC.
>
>Julian
>
>
>On Jan 28, 2015, at 10:17 AM, John Pullokkaran
><jp...@hortonworks.com> wrote:
>
>> I filed CALCITE-575 to keep track of this.
>> I have attached a patch to it.
>>
>> Patch fixes many of the issues for Hive (~ 25).
>>
>>
>> On Tue, Jan 27, 2015 at 3:12 PM, John Pullokkaran <
>> jpullokkaran@hortonworks.com> wrote:
>>
>>> Unfortunately this requires a patch to Calcite.
>>> I am testing the patch right now.
>>>
>>> So far changes are to RelFieldTrimmer, RexUtil, ProjectRemoveRule.
>>>
>>> On Tue, Jan 27, 2015 at 10:55 AM, John Pullokkaran <
>>> jpullokkaran@hortonworks.com> wrote:
>>>
>>>> Many of Hive issues seems to be the result of CALCITE-92.
>>>>
>>>> I don¹t think there is anything wrong with the patch itself, but it
>>>> breaks Hive¹s usage of Calcite (due to Calcite OP tree getting
>>>>converted
>>>> back to Hive AST).
>>>>
>>>> I am going to try disabling it in Hive¹s usage of Calcite; will update
>>>> soon.
>>>>
>>>> John
>>>>
>>>> On Jan 27, 2015, at 9:38 AM, Julian Hyde <jhyde@apache.org<mailto:
>>>> jhyde@apache.org>> wrote:
>>>>
>>>> On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran
>>>> <jp...@hortonworks.com>>
>>>> wrote:
>>>>
>>>> On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran <
>>>> jpullokkaran@hortonworks.com<ma...@hortonworks.com>>
>>>>wrote:
>>>>
>>>> We seems to have issues with union, subquery, and constants.
>>>>
>>>> Currently we are debugging these to find more details.
>>>>
>>>> Would update EOB today.
>>>>
>>>> So far 2 issues been discovered with Calcite RC; CALCITE-570,
>>>>CALCITE-571.
>>>>
>>>> John,
>>>>
>>>> Is that your final list? I have a fix for
>>>> https://issues.apache.org/jira/browse/CALCITE-570, am working on
>>>> https://issues.apache.org/jira/browse/CALCITE-571. I intend to produce
>>>> a new RC including these and
>>>> https://issues.apache.org/jira/browse/CALCITE-567 and
>>>> https://issues.apache.org/jira/browse/CALCITE-568 (both issues that
>>>> Vladimir found in RC1). If your list is final RC2 could be as early as
>>>> end of today.
>>>>
>>>> When there is an RC2, would it help to have a few days to review it
>>>> before I start a vote?
>>>>
>>>> Julian
>>>>
>>>>
>>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or
>>entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the
>>reader
>> of this message is not the intended recipient, you are hereby notified
>>that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>>immediately
>> and delete it from your system. Thank You.
>
Re: Towards RC2
Posted by Julian Hyde <ju...@gmail.com>.
John,
First of all, let me say I feel your pain.
But a client should not make assumptions about the names that Calcite assigns. (It’s like using a Java HashMap and making assumptions about iteration order. If the iteration order changes from one version of the JDK to the next, it is not a bug in the JDK; it is a bug in your app.) There have been discussions about this issue on this list recently.
Hive needs to use Calcite in the way it was designed — where you identify fields by position, not by name. How long would you need to fix Hive to do this? If you can commit to fixing Hive by a particular date, then I will accept the patch short-term. The modifications would go in deprecated (so no one else starts using them) and I would log an issue to remove them.
By the way, I have fixed the other issues. As soon as this is resolved, we can make another RC.
Julian
On Jan 28, 2015, at 10:17 AM, John Pullokkaran <jp...@hortonworks.com> wrote:
> I filed CALCITE-575 to keep track of this.
> I have attached a patch to it.
>
> Patch fixes many of the issues for Hive (~ 25).
>
>
> On Tue, Jan 27, 2015 at 3:12 PM, John Pullokkaran <
> jpullokkaran@hortonworks.com> wrote:
>
>> Unfortunately this requires a patch to Calcite.
>> I am testing the patch right now.
>>
>> So far changes are to RelFieldTrimmer, RexUtil, ProjectRemoveRule.
>>
>> On Tue, Jan 27, 2015 at 10:55 AM, John Pullokkaran <
>> jpullokkaran@hortonworks.com> wrote:
>>
>>> Many of Hive issues seems to be the result of CALCITE-92.
>>>
>>> I don’t think there is anything wrong with the patch itself, but it
>>> breaks Hive’s usage of Calcite (due to Calcite OP tree getting converted
>>> back to Hive AST).
>>>
>>> I am going to try disabling it in Hive’s usage of Calcite; will update
>>> soon.
>>>
>>> John
>>>
>>> On Jan 27, 2015, at 9:38 AM, Julian Hyde <jhyde@apache.org<mailto:
>>> jhyde@apache.org>> wrote:
>>>
>>> On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran
>>> <jp...@hortonworks.com>>
>>> wrote:
>>>
>>> On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran <
>>> jpullokkaran@hortonworks.com<ma...@hortonworks.com>> wrote:
>>>
>>> We seems to have issues with union, subquery, and constants.
>>>
>>> Currently we are debugging these to find more details.
>>>
>>> Would update EOB today.
>>>
>>> So far 2 issues been discovered with Calcite RC; CALCITE-570, CALCITE-571.
>>>
>>> John,
>>>
>>> Is that your final list? I have a fix for
>>> https://issues.apache.org/jira/browse/CALCITE-570, am working on
>>> https://issues.apache.org/jira/browse/CALCITE-571. I intend to produce
>>> a new RC including these and
>>> https://issues.apache.org/jira/browse/CALCITE-567 and
>>> https://issues.apache.org/jira/browse/CALCITE-568 (both issues that
>>> Vladimir found in RC1). If your list is final RC2 could be as early as
>>> end of today.
>>>
>>> When there is an RC2, would it help to have a few days to review it
>>> before I start a vote?
>>>
>>> Julian
>>>
>>>
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Towards RC2
Posted by John Pullokkaran <jp...@hortonworks.com>.
I filed CALCITE-575 to keep track of this.
I have attached a patch to it.
Patch fixes many of the issues for Hive (~ 25).
On Tue, Jan 27, 2015 at 3:12 PM, John Pullokkaran <
jpullokkaran@hortonworks.com> wrote:
> Unfortunately this requires a patch to Calcite.
> I am testing the patch right now.
>
> So far changes are to RelFieldTrimmer, RexUtil, ProjectRemoveRule.
>
> On Tue, Jan 27, 2015 at 10:55 AM, John Pullokkaran <
> jpullokkaran@hortonworks.com> wrote:
>
>> Many of Hive issues seems to be the result of CALCITE-92.
>>
>> I don’t think there is anything wrong with the patch itself, but it
>> breaks Hive’s usage of Calcite (due to Calcite OP tree getting converted
>> back to Hive AST).
>>
>> I am going to try disabling it in Hive’s usage of Calcite; will update
>> soon.
>>
>> John
>>
>> On Jan 27, 2015, at 9:38 AM, Julian Hyde <jhyde@apache.org<mailto:
>> jhyde@apache.org>> wrote:
>>
>> On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran
>> <jp...@hortonworks.com>>
>> wrote:
>>
>> On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran <
>> jpullokkaran@hortonworks.com<ma...@hortonworks.com>> wrote:
>>
>> We seems to have issues with union, subquery, and constants.
>>
>> Currently we are debugging these to find more details.
>>
>> Would update EOB today.
>>
>> So far 2 issues been discovered with Calcite RC; CALCITE-570, CALCITE-571.
>>
>> John,
>>
>> Is that your final list? I have a fix for
>> https://issues.apache.org/jira/browse/CALCITE-570, am working on
>> https://issues.apache.org/jira/browse/CALCITE-571. I intend to produce
>> a new RC including these and
>> https://issues.apache.org/jira/browse/CALCITE-567 and
>> https://issues.apache.org/jira/browse/CALCITE-568 (both issues that
>> Vladimir found in RC1). If your list is final RC2 could be as early as
>> end of today.
>>
>> When there is an RC2, would it help to have a few days to review it
>> before I start a vote?
>>
>> Julian
>>
>>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Towards RC2
Posted by John Pullokkaran <jp...@hortonworks.com>.
Unfortunately this requires a patch to Calcite.
I am testing the patch right now.
So far changes are to RelFieldTrimmer, RexUtil, ProjectRemoveRule.
On Tue, Jan 27, 2015 at 10:55 AM, John Pullokkaran <
jpullokkaran@hortonworks.com> wrote:
> Many of Hive issues seems to be the result of CALCITE-92.
>
> I don’t think there is anything wrong with the patch itself, but it breaks
> Hive’s usage of Calcite (due to Calcite OP tree getting converted back to
> Hive AST).
>
> I am going to try disabling it in Hive’s usage of Calcite; will update
> soon.
>
> John
>
> On Jan 27, 2015, at 9:38 AM, Julian Hyde <jhyde@apache.org<mailto:
> jhyde@apache.org>> wrote:
>
> On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran
> <jp...@hortonworks.com>> wrote:
>
> On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran <
> jpullokkaran@hortonworks.com<ma...@hortonworks.com>> wrote:
>
> We seems to have issues with union, subquery, and constants.
>
> Currently we are debugging these to find more details.
>
> Would update EOB today.
>
> So far 2 issues been discovered with Calcite RC; CALCITE-570, CALCITE-571.
>
> John,
>
> Is that your final list? I have a fix for
> https://issues.apache.org/jira/browse/CALCITE-570, am working on
> https://issues.apache.org/jira/browse/CALCITE-571. I intend to produce
> a new RC including these and
> https://issues.apache.org/jira/browse/CALCITE-567 and
> https://issues.apache.org/jira/browse/CALCITE-568 (both issues that
> Vladimir found in RC1). If your list is final RC2 could be as early as
> end of today.
>
> When there is an RC2, would it help to have a few days to review it
> before I start a vote?
>
> Julian
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Towards RC2
Posted by John Pullokkaran <jp...@hortonworks.com>.
Many of Hive issues seems to be the result of CALCITE-92.
I don’t think there is anything wrong with the patch itself, but it breaks Hive’s usage of Calcite (due to Calcite OP tree getting converted back to Hive AST).
I am going to try disabling it in Hive’s usage of Calcite; will update soon.
John
On Jan 27, 2015, at 9:38 AM, Julian Hyde <jh...@apache.org>> wrote:
On Mon, Jan 26, 2015 at 10:11 PM, John Pullokkaran
<jp...@hortonworks.com>> wrote:
On Mon, Jan 26, 2015 at 1:42 PM, John Pullokkaran <jp...@hortonworks.com>> wrote:
We seems to have issues with union, subquery, and constants.
Currently we are debugging these to find more details.
Would update EOB today.
So far 2 issues been discovered with Calcite RC; CALCITE-570, CALCITE-571.
John,
Is that your final list? I have a fix for
https://issues.apache.org/jira/browse/CALCITE-570, am working on
https://issues.apache.org/jira/browse/CALCITE-571. I intend to produce
a new RC including these and
https://issues.apache.org/jira/browse/CALCITE-567 and
https://issues.apache.org/jira/browse/CALCITE-568 (both issues that
Vladimir found in RC1). If your list is final RC2 could be as early as
end of today.
When there is an RC2, would it help to have a few days to review it
before I start a vote?
Julian