You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by "rdblue@gmail.com" <rd...@gmail.com> on 2024/09/27 21:36:18 UTC
[VOTE] Table v3 spec: Add unknown and new type promotion
Hi everyone,
I'd like to vote on PR #10955
<https://github.com/apache/iceberg/pull/10955> that
has been open for a while with the changes to add new type promotion cases.
After discussion, the PR has been scoped down to keep complexity low. It
now adds:
* An `unknown` type for cases when only `null` values have been observed
* Type promotion from `unknown` to any other type
* Type promotion from `date` to `timestamp` or `timestamp_ns`
* Clarification that promotion is not allowed if it breaks transform results
The set of changes is quite a bit smaller than originally proposed because
of the issue already discussed about lower and upper bounds values, and it
no longer includes variant. I think that we can add more type promotion
cases after we improve bounds metadata. This adds what we can now to keep
v3 moving forward.
Please vote in the next 72 hours:
[ ] +1, commit the proposed spec changes
[ ] -0
[ ] -1, do not make these changes because . . .
Thanks,
Ryan
Re: [VOTE] Table v3 spec: Add unknown and new type promotion
Posted by "rdblue@gmail.com" <rd...@gmail.com>.
With 7 +1 votes and 1 -0, this passes. I'll work with Micah to clarify some
of the sections that he pointed out, but I don't think that we need to
block moving forward on the overall set of things that are proposed.
Thanks, everyone!
On Tue, Oct 1, 2024 at 1:49 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> +1 (non binding)
>
> Regards
> JB
>
> On Fri, Sep 27, 2024 at 11:36 PM rdblue@gmail.com <rd...@gmail.com>
> wrote:
> >
> > Hi everyone,
> >
> > I'd like to vote on PR #10955 that has been open for a while with the
> changes to add new type promotion cases. After discussion, the PR has been
> scoped down to keep complexity low. It now adds:
> >
> > * An `unknown` type for cases when only `null` values have been observed
> > * Type promotion from `unknown` to any other type
> > * Type promotion from `date` to `timestamp` or `timestamp_ns`
> > * Clarification that promotion is not allowed if it breaks transform
> results
> >
> > The set of changes is quite a bit smaller than originally proposed
> because of the issue already discussed about lower and upper bounds values,
> and it no longer includes variant. I think that we can add more type
> promotion cases after we improve bounds metadata. This adds what we can now
> to keep v3 moving forward.
> >
> > Please vote in the next 72 hours:
> >
> > [ ] +1, commit the proposed spec changes
> > [ ] -0
> > [ ] -1, do not make these changes because . . .
> >
> > Thanks,
> >
> > Ryan
>
Re: [VOTE] Table v3 spec: Add unknown and new type promotion
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
+1 (non binding)
Regards
JB
On Fri, Sep 27, 2024 at 11:36 PM rdblue@gmail.com <rd...@gmail.com> wrote:
>
> Hi everyone,
>
> I'd like to vote on PR #10955 that has been open for a while with the changes to add new type promotion cases. After discussion, the PR has been scoped down to keep complexity low. It now adds:
>
> * An `unknown` type for cases when only `null` values have been observed
> * Type promotion from `unknown` to any other type
> * Type promotion from `date` to `timestamp` or `timestamp_ns`
> * Clarification that promotion is not allowed if it breaks transform results
>
> The set of changes is quite a bit smaller than originally proposed because of the issue already discussed about lower and upper bounds values, and it no longer includes variant. I think that we can add more type promotion cases after we improve bounds metadata. This adds what we can now to keep v3 moving forward.
>
> Please vote in the next 72 hours:
>
> [ ] +1, commit the proposed spec changes
> [ ] -0
> [ ] -1, do not make these changes because . . .
>
> Thanks,
>
> Ryan
Re: [VOTE] Table v3 spec: Add unknown and new type promotion
Posted by Eduard Tudenhöfner <et...@apache.org>.
+1 (binding)
On Tue, Oct 1, 2024 at 1:45 AM Micah Kornfield <em...@gmail.com>
wrote:
> I'm -0.0 as worded currently. I think there are some more aspects that
> should be defined for date->timestamp/timestamp_ns promotion (left comments
> on the PR). The addition of an Unknown type seems like a good addition.
>
> Thanks,
> Micah
>
> On Mon, Sep 30, 2024 at 2:32 PM Yufei Gu <fl...@gmail.com> wrote:
>
>> +1(binding)
>>
>> Yufei
>>
>>
>> On Mon, Sep 30, 2024 at 12:42 PM Amogh Jahagirdar <2a...@gmail.com>
>> wrote:
>>
>>> +1 (binding)
>>>
>>> Thanks,
>>> Amogh Jahagirdar
>>>
>>> On Mon, Sep 30, 2024 at 1:39 PM rdblue@gmail.com <rd...@gmail.com>
>>> wrote:
>>>
>>>> +1 (binding)
>>>>
>>>> On Mon, Sep 30, 2024 at 12:32 PM Daniel Weeks <dw...@apache.org>
>>>> wrote:
>>>>
>>>>> +1 (binding)
>>>>>
>>>>> On Fri, Sep 27, 2024 at 2:41 PM Russell Spitzer <
>>>>> russell.spitzer@gmail.com> wrote:
>>>>>
>>>>>> +1 (binding)
>>>>>>
>>>>>> On Fri, Sep 27, 2024 at 4:37 PM rdblue@gmail.com <rd...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi everyone,
>>>>>>>
>>>>>>> I'd like to vote on PR #10955
>>>>>>> <https://github.com/apache/iceberg/pull/10955> that has been open
>>>>>>> for a while with the changes to add new type promotion cases. After
>>>>>>> discussion, the PR has been scoped down to keep complexity low. It now adds:
>>>>>>>
>>>>>>> * An `unknown` type for cases when only `null` values have been
>>>>>>> observed
>>>>>>> * Type promotion from `unknown` to any other type
>>>>>>> * Type promotion from `date` to `timestamp` or `timestamp_ns`
>>>>>>> * Clarification that promotion is not allowed if it breaks transform
>>>>>>> results
>>>>>>>
>>>>>>> The set of changes is quite a bit smaller than originally proposed
>>>>>>> because of the issue already discussed about lower and upper bounds values,
>>>>>>> and it no longer includes variant. I think that we can add more type
>>>>>>> promotion cases after we improve bounds metadata. This adds what we can now
>>>>>>> to keep v3 moving forward.
>>>>>>>
>>>>>>> Please vote in the next 72 hours:
>>>>>>>
>>>>>>> [ ] +1, commit the proposed spec changes
>>>>>>> [ ] -0
>>>>>>> [ ] -1, do not make these changes because . . .
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Ryan
>>>>>>>
>>>>>>
Re: [VOTE] Table v3 spec: Add unknown and new type promotion
Posted by Micah Kornfield <em...@gmail.com>.
I'm -0.0 as worded currently. I think there are some more aspects that
should be defined for date->timestamp/timestamp_ns promotion (left comments
on the PR). The addition of an Unknown type seems like a good addition.
Thanks,
Micah
On Mon, Sep 30, 2024 at 2:32 PM Yufei Gu <fl...@gmail.com> wrote:
> +1(binding)
>
> Yufei
>
>
> On Mon, Sep 30, 2024 at 12:42 PM Amogh Jahagirdar <2a...@gmail.com>
> wrote:
>
>> +1 (binding)
>>
>> Thanks,
>> Amogh Jahagirdar
>>
>> On Mon, Sep 30, 2024 at 1:39 PM rdblue@gmail.com <rd...@gmail.com>
>> wrote:
>>
>>> +1 (binding)
>>>
>>> On Mon, Sep 30, 2024 at 12:32 PM Daniel Weeks <dw...@apache.org> wrote:
>>>
>>>> +1 (binding)
>>>>
>>>> On Fri, Sep 27, 2024 at 2:41 PM Russell Spitzer <
>>>> russell.spitzer@gmail.com> wrote:
>>>>
>>>>> +1 (binding)
>>>>>
>>>>> On Fri, Sep 27, 2024 at 4:37 PM rdblue@gmail.com <rd...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> I'd like to vote on PR #10955
>>>>>> <https://github.com/apache/iceberg/pull/10955> that has been open
>>>>>> for a while with the changes to add new type promotion cases. After
>>>>>> discussion, the PR has been scoped down to keep complexity low. It now adds:
>>>>>>
>>>>>> * An `unknown` type for cases when only `null` values have been
>>>>>> observed
>>>>>> * Type promotion from `unknown` to any other type
>>>>>> * Type promotion from `date` to `timestamp` or `timestamp_ns`
>>>>>> * Clarification that promotion is not allowed if it breaks transform
>>>>>> results
>>>>>>
>>>>>> The set of changes is quite a bit smaller than originally proposed
>>>>>> because of the issue already discussed about lower and upper bounds values,
>>>>>> and it no longer includes variant. I think that we can add more type
>>>>>> promotion cases after we improve bounds metadata. This adds what we can now
>>>>>> to keep v3 moving forward.
>>>>>>
>>>>>> Please vote in the next 72 hours:
>>>>>>
>>>>>> [ ] +1, commit the proposed spec changes
>>>>>> [ ] -0
>>>>>> [ ] -1, do not make these changes because . . .
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Ryan
>>>>>>
>>>>>
Re: [VOTE] Table v3 spec: Add unknown and new type promotion
Posted by Yufei Gu <fl...@gmail.com>.
+1(binding)
Yufei
On Mon, Sep 30, 2024 at 12:42 PM Amogh Jahagirdar <2a...@gmail.com> wrote:
> +1 (binding)
>
> Thanks,
> Amogh Jahagirdar
>
> On Mon, Sep 30, 2024 at 1:39 PM rdblue@gmail.com <rd...@gmail.com> wrote:
>
>> +1 (binding)
>>
>> On Mon, Sep 30, 2024 at 12:32 PM Daniel Weeks <dw...@apache.org> wrote:
>>
>>> +1 (binding)
>>>
>>> On Fri, Sep 27, 2024 at 2:41 PM Russell Spitzer <
>>> russell.spitzer@gmail.com> wrote:
>>>
>>>> +1 (binding)
>>>>
>>>> On Fri, Sep 27, 2024 at 4:37 PM rdblue@gmail.com <rd...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> I'd like to vote on PR #10955
>>>>> <https://github.com/apache/iceberg/pull/10955> that has been open for
>>>>> a while with the changes to add new type promotion cases. After discussion,
>>>>> the PR has been scoped down to keep complexity low. It now adds:
>>>>>
>>>>> * An `unknown` type for cases when only `null` values have been
>>>>> observed
>>>>> * Type promotion from `unknown` to any other type
>>>>> * Type promotion from `date` to `timestamp` or `timestamp_ns`
>>>>> * Clarification that promotion is not allowed if it breaks transform
>>>>> results
>>>>>
>>>>> The set of changes is quite a bit smaller than originally proposed
>>>>> because of the issue already discussed about lower and upper bounds values,
>>>>> and it no longer includes variant. I think that we can add more type
>>>>> promotion cases after we improve bounds metadata. This adds what we can now
>>>>> to keep v3 moving forward.
>>>>>
>>>>> Please vote in the next 72 hours:
>>>>>
>>>>> [ ] +1, commit the proposed spec changes
>>>>> [ ] -0
>>>>> [ ] -1, do not make these changes because . . .
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Ryan
>>>>>
>>>>
Re: [VOTE] Table v3 spec: Add unknown and new type promotion
Posted by Amogh Jahagirdar <2a...@gmail.com>.
+1 (binding)
Thanks,
Amogh Jahagirdar
On Mon, Sep 30, 2024 at 1:39 PM rdblue@gmail.com <rd...@gmail.com> wrote:
> +1 (binding)
>
> On Mon, Sep 30, 2024 at 12:32 PM Daniel Weeks <dw...@apache.org> wrote:
>
>> +1 (binding)
>>
>> On Fri, Sep 27, 2024 at 2:41 PM Russell Spitzer <
>> russell.spitzer@gmail.com> wrote:
>>
>>> +1 (binding)
>>>
>>> On Fri, Sep 27, 2024 at 4:37 PM rdblue@gmail.com <rd...@gmail.com>
>>> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> I'd like to vote on PR #10955
>>>> <https://github.com/apache/iceberg/pull/10955> that has been open for
>>>> a while with the changes to add new type promotion cases. After discussion,
>>>> the PR has been scoped down to keep complexity low. It now adds:
>>>>
>>>> * An `unknown` type for cases when only `null` values have been observed
>>>> * Type promotion from `unknown` to any other type
>>>> * Type promotion from `date` to `timestamp` or `timestamp_ns`
>>>> * Clarification that promotion is not allowed if it breaks transform
>>>> results
>>>>
>>>> The set of changes is quite a bit smaller than originally proposed
>>>> because of the issue already discussed about lower and upper bounds values,
>>>> and it no longer includes variant. I think that we can add more type
>>>> promotion cases after we improve bounds metadata. This adds what we can now
>>>> to keep v3 moving forward.
>>>>
>>>> Please vote in the next 72 hours:
>>>>
>>>> [ ] +1, commit the proposed spec changes
>>>> [ ] -0
>>>> [ ] -1, do not make these changes because . . .
>>>>
>>>> Thanks,
>>>>
>>>> Ryan
>>>>
>>>
Re: [VOTE] Table v3 spec: Add unknown and new type promotion
Posted by "rdblue@gmail.com" <rd...@gmail.com>.
+1 (binding)
On Mon, Sep 30, 2024 at 12:32 PM Daniel Weeks <dw...@apache.org> wrote:
> +1 (binding)
>
> On Fri, Sep 27, 2024 at 2:41 PM Russell Spitzer <ru...@gmail.com>
> wrote:
>
>> +1 (binding)
>>
>> On Fri, Sep 27, 2024 at 4:37 PM rdblue@gmail.com <rd...@gmail.com>
>> wrote:
>>
>>> Hi everyone,
>>>
>>> I'd like to vote on PR #10955
>>> <https://github.com/apache/iceberg/pull/10955> that has been open for a
>>> while with the changes to add new type promotion cases. After discussion,
>>> the PR has been scoped down to keep complexity low. It now adds:
>>>
>>> * An `unknown` type for cases when only `null` values have been observed
>>> * Type promotion from `unknown` to any other type
>>> * Type promotion from `date` to `timestamp` or `timestamp_ns`
>>> * Clarification that promotion is not allowed if it breaks transform
>>> results
>>>
>>> The set of changes is quite a bit smaller than originally proposed
>>> because of the issue already discussed about lower and upper bounds values,
>>> and it no longer includes variant. I think that we can add more type
>>> promotion cases after we improve bounds metadata. This adds what we can now
>>> to keep v3 moving forward.
>>>
>>> Please vote in the next 72 hours:
>>>
>>> [ ] +1, commit the proposed spec changes
>>> [ ] -0
>>> [ ] -1, do not make these changes because . . .
>>>
>>> Thanks,
>>>
>>> Ryan
>>>
>>
Re: [VOTE] Table v3 spec: Add unknown and new type promotion
Posted by Daniel Weeks <dw...@apache.org>.
+1 (binding)
On Fri, Sep 27, 2024 at 2:41 PM Russell Spitzer <ru...@gmail.com>
wrote:
> +1 (binding)
>
> On Fri, Sep 27, 2024 at 4:37 PM rdblue@gmail.com <rd...@gmail.com> wrote:
>
>> Hi everyone,
>>
>> I'd like to vote on PR #10955
>> <https://github.com/apache/iceberg/pull/10955> that has been open for a
>> while with the changes to add new type promotion cases. After discussion,
>> the PR has been scoped down to keep complexity low. It now adds:
>>
>> * An `unknown` type for cases when only `null` values have been observed
>> * Type promotion from `unknown` to any other type
>> * Type promotion from `date` to `timestamp` or `timestamp_ns`
>> * Clarification that promotion is not allowed if it breaks transform
>> results
>>
>> The set of changes is quite a bit smaller than originally proposed
>> because of the issue already discussed about lower and upper bounds values,
>> and it no longer includes variant. I think that we can add more type
>> promotion cases after we improve bounds metadata. This adds what we can now
>> to keep v3 moving forward.
>>
>> Please vote in the next 72 hours:
>>
>> [ ] +1, commit the proposed spec changes
>> [ ] -0
>> [ ] -1, do not make these changes because . . .
>>
>> Thanks,
>>
>> Ryan
>>
>
Re: [VOTE] Table v3 spec: Add unknown and new type promotion
Posted by Russell Spitzer <ru...@gmail.com>.
+1 (binding)
On Fri, Sep 27, 2024 at 4:37 PM rdblue@gmail.com <rd...@gmail.com> wrote:
> Hi everyone,
>
> I'd like to vote on PR #10955
> <https://github.com/apache/iceberg/pull/10955> that has been open for a
> while with the changes to add new type promotion cases. After discussion,
> the PR has been scoped down to keep complexity low. It now adds:
>
> * An `unknown` type for cases when only `null` values have been observed
> * Type promotion from `unknown` to any other type
> * Type promotion from `date` to `timestamp` or `timestamp_ns`
> * Clarification that promotion is not allowed if it breaks transform
> results
>
> The set of changes is quite a bit smaller than originally proposed because
> of the issue already discussed about lower and upper bounds values, and it
> no longer includes variant. I think that we can add more type promotion
> cases after we improve bounds metadata. This adds what we can now to keep
> v3 moving forward.
>
> Please vote in the next 72 hours:
>
> [ ] +1, commit the proposed spec changes
> [ ] -0
> [ ] -1, do not make these changes because . . .
>
> Thanks,
>
> Ryan
>