You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Jean-Baptiste Onofré <jb...@nanthrax.net> on 2025/01/09 06:51:28 UTC
[DISCUSS] Apache Iceberg (java) 1.8.0 release
Hi folks,
We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
our release "pace", 1.8.0 should be released around mid February.
I think we already have a good "train" of merged PRs (or should be
merged soon): default values, REST auth improvements, dependencies
updates, etc.
WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
and PRs we would like to "target" to 1.8.0.
Thoughts ?
Regards
JB
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by huaxin gao <hu...@gmail.com>.
Can we also include the Comet and Iceberg integration? Here is the PR
<https://github.com/apache/iceberg/pull/9841>; most of the comments have
been addressed, and I am currently working with Anton to finalize this.
Thanks,
Huaxin
On Mon, Jan 13, 2025 at 11:55 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
> Hey all,
>
> Sorry I should've been more explicit in my previous email, but I was also
> intending to volunteer for release as part of my statement, and was
> waiting to formalize that at the community sync.
>
> Thanks,
> Amogh Jahagirdar
>
> On Thu, Jan 9, 2025 at 9:47 PM Ajantha Bhat <aj...@gmail.com> wrote:
>
>> Hi,
>> I would like to have partition stats feature included in 1.8.0 release,
>> many users have been asking for it by commenting on the PR.
>> We missed it on the 1.7.0 release train as we wanted to have internal
>> writers/readers for Parquet/Avro and use it for partition stats to avoid
>> extra conversion. Internal writers PR has made significant progress and
>> mostly will get merged by this week. So, the end of Jan or mid Feb seems
>> like a good timeline.
>>
>> I am also happy to volunteer as release manager for this, I have released
>> 1.5.0 before.
>>
>> - Ajantha
>>
>> On Fri, Jan 10, 2025 at 12:49 AM Amogh Jahagirdar <2a...@gmail.com>
>> wrote:
>>
>>> Hey all,
>>>
>>> Thanks JB for kicking this discussion off! I agree that we should
>>> probably do a release soon, I was thinking even sooner than mid February.
>>> It would be great to have a 1.8 release before the end of January. I was
>>> going to bring this up in the community sync next week.
>>>
>>> There are some additional backport PRs for Spark DV integration and
>>> default values that should get in, but those are pretty straightforward
>>> backports. There are additional memory-consumption optimizations for DVs
>>> that ideally would get in for 1.8 but I'd say they are not a must have.
>>>
>>> I'd propose that folks add what they think should get into 1.8.0 to the
>>> milestone
>>> <https://github.com/apache/iceberg/pulls?q=is%3Aopen+is%3Apr+milestone%3A%22Iceberg+1.8.0%22> .
>>> In next week's community sync we can get to a consensus on any lingering
>>> specific items, and follow up on this thread if needed. In the meantime, we
>>> can review any of the PRs in the milestone and if there's not a consensus
>>> on the approach or if the change requires significantly more work, we may
>>> just want to punt those to a follow on 1.9 release.
>>>
>>> Let me know what you think!
>>>
>>> Thanks,
>>> Amogh Jahagirdar
>>>
>>> On Thu, Jan 9, 2025 at 9:52 AM Dmitri Bourlatchkov <di...@apache.org>
>>> wrote:
>>>
>>>> I think it would be nice to include the Auth Manager improvements into
>>>> 1.8.0.
>>>>
>>>> Here's the most recent PR on that subject, but I think there will be a
>>>> couple more (per previous discussions):
>>>>
>>>> https://github.com/apache/iceberg/pull/11844
>>>>
>>>> Thanks,
>>>> Dmitri.
>>>>
>>>> On Thu, Jan 9, 2025 at 1:52 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>> wrote:
>>>>
>>>>> Hi folks,
>>>>>
>>>>> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
>>>>> our release "pace", 1.8.0 should be released around mid February.
>>>>>
>>>>> I think we already have a good "train" of merged PRs (or should be
>>>>> merged soon): default values, REST auth improvements, dependencies
>>>>> updates, etc.
>>>>>
>>>>> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>>>>> and PRs we would like to "target" to 1.8.0.
>>>>>
>>>>> Thoughts ?
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Amogh Jahagirdar <2a...@gmail.com>.
Hey all,
Sorry I should've been more explicit in my previous email, but I was also
intending to volunteer for release as part of my statement, and was
waiting to formalize that at the community sync.
Thanks,
Amogh Jahagirdar
On Thu, Jan 9, 2025 at 9:47 PM Ajantha Bhat <aj...@gmail.com> wrote:
> Hi,
> I would like to have partition stats feature included in 1.8.0 release,
> many users have been asking for it by commenting on the PR.
> We missed it on the 1.7.0 release train as we wanted to have internal
> writers/readers for Parquet/Avro and use it for partition stats to avoid
> extra conversion. Internal writers PR has made significant progress and
> mostly will get merged by this week. So, the end of Jan or mid Feb seems
> like a good timeline.
>
> I am also happy to volunteer as release manager for this, I have released
> 1.5.0 before.
>
> - Ajantha
>
> On Fri, Jan 10, 2025 at 12:49 AM Amogh Jahagirdar <2a...@gmail.com>
> wrote:
>
>> Hey all,
>>
>> Thanks JB for kicking this discussion off! I agree that we should
>> probably do a release soon, I was thinking even sooner than mid February.
>> It would be great to have a 1.8 release before the end of January. I was
>> going to bring this up in the community sync next week.
>>
>> There are some additional backport PRs for Spark DV integration and
>> default values that should get in, but those are pretty straightforward
>> backports. There are additional memory-consumption optimizations for DVs
>> that ideally would get in for 1.8 but I'd say they are not a must have.
>>
>> I'd propose that folks add what they think should get into 1.8.0 to the
>> milestone
>> <https://github.com/apache/iceberg/pulls?q=is%3Aopen+is%3Apr+milestone%3A%22Iceberg+1.8.0%22> .
>> In next week's community sync we can get to a consensus on any lingering
>> specific items, and follow up on this thread if needed. In the meantime, we
>> can review any of the PRs in the milestone and if there's not a consensus
>> on the approach or if the change requires significantly more work, we may
>> just want to punt those to a follow on 1.9 release.
>>
>> Let me know what you think!
>>
>> Thanks,
>> Amogh Jahagirdar
>>
>> On Thu, Jan 9, 2025 at 9:52 AM Dmitri Bourlatchkov <di...@apache.org>
>> wrote:
>>
>>> I think it would be nice to include the Auth Manager improvements into
>>> 1.8.0.
>>>
>>> Here's the most recent PR on that subject, but I think there will be a
>>> couple more (per previous discussions):
>>>
>>> https://github.com/apache/iceberg/pull/11844
>>>
>>> Thanks,
>>> Dmitri.
>>>
>>> On Thu, Jan 9, 2025 at 1:52 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Hi folks,
>>>>
>>>> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
>>>> our release "pace", 1.8.0 should be released around mid February.
>>>>
>>>> I think we already have a good "train" of merged PRs (or should be
>>>> merged soon): default values, REST auth improvements, dependencies
>>>> updates, etc.
>>>>
>>>> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>>>> and PRs we would like to "target" to 1.8.0.
>>>>
>>>> Thoughts ?
>>>>
>>>> Regards
>>>> JB
>>>>
>>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Ajantha Bhat <aj...@gmail.com>.
Hi,
I would like to have partition stats feature included in 1.8.0 release,
many users have been asking for it by commenting on the PR.
We missed it on the 1.7.0 release train as we wanted to have internal
writers/readers for Parquet/Avro and use it for partition stats to avoid
extra conversion. Internal writers PR has made significant progress and
mostly will get merged by this week. So, the end of Jan or mid Feb seems
like a good timeline.
I am also happy to volunteer as release manager for this, I have released
1.5.0 before.
- Ajantha
On Fri, Jan 10, 2025 at 12:49 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
> Hey all,
>
> Thanks JB for kicking this discussion off! I agree that we should probably
> do a release soon, I was thinking even sooner than mid February. It would
> be great to have a 1.8 release before the end of January. I was going to
> bring this up in the community sync next week.
>
> There are some additional backport PRs for Spark DV integration and
> default values that should get in, but those are pretty straightforward
> backports. There are additional memory-consumption optimizations for DVs
> that ideally would get in for 1.8 but I'd say they are not a must have.
>
> I'd propose that folks add what they think should get into 1.8.0 to the
> milestone
> <https://github.com/apache/iceberg/pulls?q=is%3Aopen+is%3Apr+milestone%3A%22Iceberg+1.8.0%22> .
> In next week's community sync we can get to a consensus on any lingering
> specific items, and follow up on this thread if needed. In the meantime, we
> can review any of the PRs in the milestone and if there's not a consensus
> on the approach or if the change requires significantly more work, we may
> just want to punt those to a follow on 1.9 release.
>
> Let me know what you think!
>
> Thanks,
> Amogh Jahagirdar
>
> On Thu, Jan 9, 2025 at 9:52 AM Dmitri Bourlatchkov <di...@apache.org>
> wrote:
>
>> I think it would be nice to include the Auth Manager improvements into
>> 1.8.0.
>>
>> Here's the most recent PR on that subject, but I think there will be a
>> couple more (per previous discussions):
>>
>> https://github.com/apache/iceberg/pull/11844
>>
>> Thanks,
>> Dmitri.
>>
>> On Thu, Jan 9, 2025 at 1:52 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Hi folks,
>>>
>>> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
>>> our release "pace", 1.8.0 should be released around mid February.
>>>
>>> I think we already have a good "train" of merged PRs (or should be
>>> merged soon): default values, REST auth improvements, dependencies
>>> updates, etc.
>>>
>>> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>>> and PRs we would like to "target" to 1.8.0.
>>>
>>> Thoughts ?
>>>
>>> Regards
>>> JB
>>>
>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Amogh Jahagirdar <2a...@gmail.com>.
Hey all,
Thanks JB for kicking this discussion off! I agree that we should probably
do a release soon, I was thinking even sooner than mid February. It would
be great to have a 1.8 release before the end of January. I was going to
bring this up in the community sync next week.
There are some additional backport PRs for Spark DV integration and default
values that should get in, but those are pretty straightforward backports.
There are additional memory-consumption optimizations for DVs that ideally
would get in for 1.8 but I'd say they are not a must have.
I'd propose that folks add what they think should get into 1.8.0 to the
milestone
<https://github.com/apache/iceberg/pulls?q=is%3Aopen+is%3Apr+milestone%3A%22Iceberg+1.8.0%22>
.
In next week's community sync we can get to a consensus on any lingering
specific items, and follow up on this thread if needed. In the meantime, we
can review any of the PRs in the milestone and if there's not a consensus
on the approach or if the change requires significantly more work, we may
just want to punt those to a follow on 1.9 release.
Let me know what you think!
Thanks,
Amogh Jahagirdar
On Thu, Jan 9, 2025 at 9:52 AM Dmitri Bourlatchkov <di...@apache.org> wrote:
> I think it would be nice to include the Auth Manager improvements into
> 1.8.0.
>
> Here's the most recent PR on that subject, but I think there will be a
> couple more (per previous discussions):
>
> https://github.com/apache/iceberg/pull/11844
>
> Thanks,
> Dmitri.
>
> On Thu, Jan 9, 2025 at 1:52 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi folks,
>>
>> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
>> our release "pace", 1.8.0 should be released around mid February.
>>
>> I think we already have a good "train" of merged PRs (or should be
>> merged soon): default values, REST auth improvements, dependencies
>> updates, etc.
>>
>> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>> and PRs we would like to "target" to 1.8.0.
>>
>> Thoughts ?
>>
>> Regards
>> JB
>>
>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Dmitri Bourlatchkov <di...@apache.org>.
I think it would be nice to include the Auth Manager improvements into
1.8.0.
Here's the most recent PR on that subject, but I think there will be a
couple more (per previous discussions):
https://github.com/apache/iceberg/pull/11844
Thanks,
Dmitri.
On Thu, Jan 9, 2025 at 1:52 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Hi folks,
>
> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
> our release "pace", 1.8.0 should be released around mid February.
>
> I think we already have a good "train" of merged PRs (or should be
> merged soon): default values, REST auth improvements, dependencies
> updates, etc.
>
> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
> and PRs we would like to "target" to 1.8.0.
>
> Thoughts ?
>
> Regards
> JB
>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Fokko Driesprong <fo...@apache.org>.
No, I don't think that's bound to a version anyway, as Amogh pointed out on
the issue. I've removed it from the milestone. Thanks
Kind regards,
Fokko
Op ma 10 feb 2025 om 11:20 schreef Manu Zhang <ow...@gmail.com>:
> https://github.com/apache/iceberg/issues/10745 should not be bound to
> 1.8.0 either?
>
> Regards,
> Manu
>
> On Mon, Feb 10, 2025 at 4:36 PM Fokko Driesprong <fo...@apache.org> wrote:
>
>> That shouldn't be included. There is still active discussion on the PR,
>> and it needs more work. The author also removed the milestone. The VOTE is
>> out, so we can start verifying.
>>
>> Kind regards,
>> Fokko
>>
>> Op ma 10 feb 2025 om 05:41 schreef Manu Zhang <ow...@gmail.com>:
>>
>>> There's still https://github.com/apache/iceberg/pull/11216 under 1.8.0
>>> milestone.
>>> Do we want to include it?
>>>
>>> On Sun, Feb 9, 2025 at 3:01 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Thanks Amogh
>>>>
>>>> I updated the PR with some cleanups.
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On Sun, Feb 9, 2025 at 4:04 AM Amogh Jahagirdar <2a...@gmail.com>
>>>> wrote:
>>>> >
>>>> > Thanks JB I left a review, it'll be good to get another set of eyes
>>>> on it! Thank you for surfacing and fixing these issues, it's very
>>>> appreciated.
>>>> >
>>>> > Amogh Jahagirdar
>>>> >
>>>> > On Sat, Feb 8, 2025 at 12:50 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>> wrote:
>>>> >>
>>>> >> Here's the PR: https://github.com/apache/iceberg/pull/12195 for ref.
>>>> >>
>>>> >> Regards
>>>> >> JB
>>>> >>
>>>> >> On Fri, Feb 7, 2025 at 5:48 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>> wrote:
>>>> >> >
>>>> >> > Hi Amogh,
>>>> >> >
>>>> >> > I found issues in the LICENSE/NOTICE from kafka-connect-runtime
>>>> >> > distribution (what's in the distribution zip). AFAIR, we plan to
>>>> >> > distribute this distribution, so it should be fixed.
>>>> >> > I will open a PR about that today.
>>>> >> >
>>>> >> > Sorry about that.
>>>> >> >
>>>> >> > Regards
>>>> >> > JB
>>>> >> >
>>>> >> > On Fri, Feb 7, 2025 at 11:35 AM Amogh Jahagirdar <2a...@gmail.com>
>>>> wrote:
>>>> >> > >
>>>> >> > > Hey all,
>>>> >> > >
>>>> >> > > An update, the final License/Notice release blockers are merged
>>>> (big thanks to JB, and Ryan/Fokko for helping review)! I'm in transit at
>>>> the moment, but once I get to a place with stable wifi I will cut a release
>>>> candidate.
>>>> >> > >
>>>> >> > > Thanks,
>>>> >> > > Amogh Jahagirdar
>>>> >> > >
>>>> >> > > On Wed, Jan 29, 2025 at 2:23 AM Amogh Jahagirdar <
>>>> 2amogh@gmail.com> wrote:
>>>> >> > >>
>>>> >> > >> Agreed, I wouldn't be opposed to looking into approaches to
>>>> make release times more predictable. At the same time, I'd advocate that in
>>>> the community, that anyone can propose a release at any point in time. Of
>>>> course, we can discuss as a community and make sure there's a reasonable
>>>> changeset, as well as focus review time on PRs which are close to being
>>>> ready for that release.
>>>> >> > >> To some degree this contradicts having a predictable release
>>>> schedule, but I feel like we can really just have a hybrid "Periodic
>>>> release + arbitrary off-cycle release" approach and things won't get too
>>>> crazy. It's a way to get the best of both frequency of release and user
>>>> expectations on release times.
>>>> >> > >>
>>>> >> > >> An update on 1.8 to the community, we're working on updating
>>>> LICENSE/NOTICE files in the AWS/GCP/Azure bundles, thank you JB for driving
>>>> that. It's something we need to get in for the release. Once that's in, I
>>>> will cut the RC.
>>>> >> > >>
>>>> >> > >> Thanks,
>>>> >> > >>
>>>> >> > >> Amogh Jahagirdar
>>>> >> > >>
>>>> >> > >> On Sun, Jan 26, 2025 at 1:16 PM Jean-Baptiste Onofré <
>>>> jb@nanthrax.net> wrote:
>>>> >> > >>>
>>>> >> > >>> Hi Amogh,
>>>> >> > >>>
>>>> >> > >>> Thanks !
>>>> >> > >>>
>>>> >> > >>> I agree we should have more frequent releases, but also more
>>>> >> > >>> "predictable" release time and give visibility to the community
>>>> >> > >>> (especially users).
>>>> >> > >>> Some ASF projects are providing "tables" with release plans:
>>>> >> > >>> - https://camel.apache.org/download/
>>>> >> > >>> - https://karaf.apache.org/download.html
>>>> >> > >>> - https://activemq.apache.org/components/classic/download/
>>>> >> > >>> - ...
>>>> >> > >>>
>>>> >> > >>> Maybe we can provide something similar ?
>>>> >> > >>>
>>>> >> > >>> Thanks !
>>>> >> > >>> Regards
>>>> >> > >>> JB
>>>> >> > >>>
>>>> >> > >>> On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <
>>>> 2amogh@gmail.com> wrote:
>>>> >> > >>> >
>>>> >> > >>> > Hey all,
>>>> >> > >>> >
>>>> >> > >>> > Just following up here with a bit of a status update, so in
>>>> the past week or so, items in the 1.8 release milestone have been closing
>>>> out.
>>>> >> > >>> > I'm aiming to cut a release next Tuesday, Jan 28.
>>>> >> > >>> >
>>>> >> > >>> > I'd like to reiterate that for any changes that don't make
>>>> the 1.8 release, we can do a fast follow 1.9 release, and from the last
>>>> community sync that seems to be the direction.
>>>> >> > >>> > In this particular case, the 1.8 release is a bit earlier
>>>> than our typical release cadence and with the 1.9 being a fast follow on, I
>>>> think we're well on track.
>>>> >> > >>> > Please add the proposed changes to the 1.9 milestone so
>>>> folks can review ahead of time!
>>>> >> > >>> >
>>>> >> > >>> > In general, I'd encourage more frequent releases, changes
>>>> which are ready can just go out and with the smaller diff it reduces the
>>>> risks that exist with larger updates.
>>>> >> > >>> >
>>>> >> > >>> > Thanks,
>>>> >> > >>> > Amogh Jahagirdar
>>>> >> > >>> >
>>>> >> > >>> > On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <
>>>> dweeks@apache.org> wrote:
>>>> >> > >>> >>
>>>> >> > >>> >> Robert,
>>>> >> > >>> >>
>>>> >> > >>> >> I hear your frustration with the progress on the Auth
>>>> Manager work, but I believe everyone recognizes that this was a large
>>>> refactor further complicated by the need to preserve backward compatibility
>>>> and handling deprecations appropriately. This work has gone through many
>>>> iterations as we explored how to make the changes cleanly. Eventually the
>>>> scale of the change led to breaking up the PR for closer review, which I
>>>> believe was the right decision because we identified multiple issues after
>>>> taking that step. That may have slowed down progress, but a lot of hours
>>>> have gone into discussing, reviewing, and validating the work in this area.
>>>> >> > >>> >>
>>>> >> > >>> >> As a project, we have leaned away from gating releases on
>>>> specific features because it leads to slower release cycles and prevents
>>>> other features that are ready from going out. We also don't want to rush
>>>> features just to hit a release target, but rather release more frequently
>>>> to make changes available as they are ready.
>>>> >> > >>> >>
>>>> >> > >>> >> At this point, I believe the plan is to follow up soon with
>>>> a 1.9 release.
>>>> >> > >>> >>
>>>> >> > >>> >> -Dan
>>>> >> > >>> >>
>>>> >> > >>> >> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <
>>>> snazy@snazy.de> wrote:
>>>> >> > >>> >>>
>>>> >> > >>> >>> Hey,
>>>> >> > >>> >>>
>>>> >> > >>> >>> IMHO 1.8 should definitely include the Auth-Manager work,
>>>> which tackles
>>>> >> > >>> >>> actual bugs in the Iceberg code base wrt OAuth
>>>> implementation. That work
>>>> >> > >>> >>> was originally intended to go into 1.7 and now it shall be
>>>> delayed again
>>>> >> > >>> >>> for 1.9. The PR was originally opened in July 2024, about
>>>> half a year
>>>> >> > >>> >>> ago and is still getting reviewed. In the meantime there
>>>> were more than
>>>> >> > >>> >>> 600 other PRs that got reviewed and merged.
>>>> >> > >>> >>>
>>>> >> > >>> >>> The overall agreement around spring 2024, please correct
>>>> me if I am
>>>> >> > >>> >>> wrong, was the whole REST/OAuth area needs to be improved,
>>>> and the oauth
>>>> >> > >>> >>> endpoint removed entirely.
>>>> >> > >>> >>>
>>>> >> > >>> >>> Generally speaking, and I know I'm not alone, getting
>>>> reviews from
>>>> >> > >>> >>> Iceberg committers is extremely hard. A lot of issues and
>>>> PRs just get
>>>> >> > >>> >>> closed (by that stale bot) without having gotten _any_
>>>> attention from an
>>>> >> > >>> >>> Iceberg committer. This is not a new situation but going
>>>> on for a long
>>>> >> > >>> >>> time. I have been talking to two Iceberg PMC members in
>>>> person many
>>>> >> > >>> >>> months ago that this is a very disappointing situation
>>>> that needs to be
>>>> >> > >>> >>> fixed. The reply was always "we are already working on it"
>>>> - but at
>>>> >> > >>> >>> least from my personal POV the situation did not improve.
>>>> >> > >>> >>>
>>>> >> > >>> >>> Robert
>>>> >> > >>> >>>
>>>> >> > >>> >>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
>>>> >> > >>> >>> > Hi folks,
>>>> >> > >>> >>> >
>>>> >> > >>> >>> > Following the Community Meeting yesterday, I would like
>>>> to propose the
>>>> >> > >>> >>> > following plan regarding releases:
>>>> >> > >>> >>> >
>>>> >> > >>> >>> > 0. As a prerequisite to any release (1.7.2, 1.8.0,
>>>> 1.9.0), as said by
>>>> >> > >>> >>> > Ryan, we have to double check the NOTICE/LICENSE.
>>>> Interestingly, I
>>>> >> > >>> >>> > discussed this point with Fokko at the beginning of this
>>>> week, because
>>>> >> > >>> >>> > I have some doubts about LICENSE/NOTICE content in the
>>>> "uber" jar
>>>> >> > >>> >>> > artifacts where we shade dependencies. I'm doing a
>>>> complete pass on
>>>> >> > >>> >>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I
>>>> should have a
>>>> >> > >>> >>> > complete analysis by tomorrow. This is potentially a
>>>> blocker for
>>>> >> > >>> >>> > release votes.
>>>> >> > >>> >>> >
>>>> >> > >>> >>> > 1. As soon as (0) is done, 1.7.2 can be submitted to
>>>> vote. I will work
>>>> >> > >>> >>> > with Fokko on this one.
>>>> >> > >>> >>> >
>>>> >> > >>> >>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is
>>>> the release
>>>> >> > >>> >>> > manager). Due to still some WIP, we "revisited" the
>>>> 1.8.0 release
>>>> >> > >>> >>> > content: for instance, as best effort, we wanted to
>>>> include REST Auth
>>>> >> > >>> >>> > Manager improvement (OAuth2 Manager) but we preferred to
>>>> postpone to
>>>> >> > >>> >>> > 1.9.0. That's totally fine to me, however, I would
>>>> propose to strongly
>>>> >> > >>> >>> > focus on pending PRs for 1.9.0. Imho, we should "target"
>>>> (again as
>>>> >> > >>> >>> > clear best effort) on variant, partition stats and Auth
>>>> Manager.
>>>> >> > >>> >>> >
>>>> >> > >>> >>> > 3. Assuming 1.8.0 will be released at the end of
>>>> Jan/beginning of Feb,
>>>> >> > >>> >>> > according to our "release cadence", what do you think
>>>> about planning
>>>> >> > >>> >>> > 1.9.0 in April ? Again with the main targets listed in
>>>> (2).
>>>> >> > >>> >>> >
>>>> >> > >>> >>> > I tried to sum up what we discussed yesterday :)
>>>> >> > >>> >>> > Thoughts ?
>>>> >> > >>> >>> >
>>>> >> > >>> >>> > Regards
>>>> >> > >>> >>> > JB
>>>> >> > >>> >>> >
>>>> >> > >>> >>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <
>>>> jb@nanthrax.net> wrote:
>>>> >> > >>> >>> >> Hi folks,
>>>> >> > >>> >>> >>
>>>> >> > >>> >>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If
>>>> we want to keep
>>>> >> > >>> >>> >> our release "pace", 1.8.0 should be released around mid
>>>> February.
>>>> >> > >>> >>> >>
>>>> >> > >>> >>> >> I think we already have a good "train" of merged PRs
>>>> (or should be
>>>> >> > >>> >>> >> merged soon): default values, REST auth improvements,
>>>> dependencies
>>>> >> > >>> >>> >> updates, etc.
>>>> >> > >>> >>> >>
>>>> >> > >>> >>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update
>>>> GitHub Issues
>>>> >> > >>> >>> >> and PRs we would like to "target" to 1.8.0.
>>>> >> > >>> >>> >>
>>>> >> > >>> >>> >> Thoughts ?
>>>> >> > >>> >>> >>
>>>> >> > >>> >>> >> Regards
>>>> >> > >>> >>> >> JB
>>>> >> > >>> >>>
>>>> >> > >>> >>> --
>>>> >> > >>> >>> Robert Stupp
>>>> >> > >>> >>> @snazy
>>>> >> > >>> >>>
>>>>
>>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Manu Zhang <ow...@gmail.com>.
https://github.com/apache/iceberg/issues/10745 should not be bound to 1.8.0
either?
Regards,
Manu
On Mon, Feb 10, 2025 at 4:36 PM Fokko Driesprong <fo...@apache.org> wrote:
> That shouldn't be included. There is still active discussion on the PR,
> and it needs more work. The author also removed the milestone. The VOTE is
> out, so we can start verifying.
>
> Kind regards,
> Fokko
>
> Op ma 10 feb 2025 om 05:41 schreef Manu Zhang <ow...@gmail.com>:
>
>> There's still https://github.com/apache/iceberg/pull/11216 under 1.8.0
>> milestone.
>> Do we want to include it?
>>
>> On Sun, Feb 9, 2025 at 3:01 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Thanks Amogh
>>>
>>> I updated the PR with some cleanups.
>>>
>>> Regards
>>> JB
>>>
>>> On Sun, Feb 9, 2025 at 4:04 AM Amogh Jahagirdar <2a...@gmail.com>
>>> wrote:
>>> >
>>> > Thanks JB I left a review, it'll be good to get another set of eyes on
>>> it! Thank you for surfacing and fixing these issues, it's very appreciated.
>>> >
>>> > Amogh Jahagirdar
>>> >
>>> > On Sat, Feb 8, 2025 at 12:50 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>> >>
>>> >> Here's the PR: https://github.com/apache/iceberg/pull/12195 for ref.
>>> >>
>>> >> Regards
>>> >> JB
>>> >>
>>> >> On Fri, Feb 7, 2025 at 5:48 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>> >> >
>>> >> > Hi Amogh,
>>> >> >
>>> >> > I found issues in the LICENSE/NOTICE from kafka-connect-runtime
>>> >> > distribution (what's in the distribution zip). AFAIR, we plan to
>>> >> > distribute this distribution, so it should be fixed.
>>> >> > I will open a PR about that today.
>>> >> >
>>> >> > Sorry about that.
>>> >> >
>>> >> > Regards
>>> >> > JB
>>> >> >
>>> >> > On Fri, Feb 7, 2025 at 11:35 AM Amogh Jahagirdar <2a...@gmail.com>
>>> wrote:
>>> >> > >
>>> >> > > Hey all,
>>> >> > >
>>> >> > > An update, the final License/Notice release blockers are merged
>>> (big thanks to JB, and Ryan/Fokko for helping review)! I'm in transit at
>>> the moment, but once I get to a place with stable wifi I will cut a release
>>> candidate.
>>> >> > >
>>> >> > > Thanks,
>>> >> > > Amogh Jahagirdar
>>> >> > >
>>> >> > > On Wed, Jan 29, 2025 at 2:23 AM Amogh Jahagirdar <
>>> 2amogh@gmail.com> wrote:
>>> >> > >>
>>> >> > >> Agreed, I wouldn't be opposed to looking into approaches to make
>>> release times more predictable. At the same time, I'd advocate that in the
>>> community, that anyone can propose a release at any point in time. Of
>>> course, we can discuss as a community and make sure there's a reasonable
>>> changeset, as well as focus review time on PRs which are close to being
>>> ready for that release.
>>> >> > >> To some degree this contradicts having a predictable release
>>> schedule, but I feel like we can really just have a hybrid "Periodic
>>> release + arbitrary off-cycle release" approach and things won't get too
>>> crazy. It's a way to get the best of both frequency of release and user
>>> expectations on release times.
>>> >> > >>
>>> >> > >> An update on 1.8 to the community, we're working on updating
>>> LICENSE/NOTICE files in the AWS/GCP/Azure bundles, thank you JB for driving
>>> that. It's something we need to get in for the release. Once that's in, I
>>> will cut the RC.
>>> >> > >>
>>> >> > >> Thanks,
>>> >> > >>
>>> >> > >> Amogh Jahagirdar
>>> >> > >>
>>> >> > >> On Sun, Jan 26, 2025 at 1:16 PM Jean-Baptiste Onofré <
>>> jb@nanthrax.net> wrote:
>>> >> > >>>
>>> >> > >>> Hi Amogh,
>>> >> > >>>
>>> >> > >>> Thanks !
>>> >> > >>>
>>> >> > >>> I agree we should have more frequent releases, but also more
>>> >> > >>> "predictable" release time and give visibility to the community
>>> >> > >>> (especially users).
>>> >> > >>> Some ASF projects are providing "tables" with release plans:
>>> >> > >>> - https://camel.apache.org/download/
>>> >> > >>> - https://karaf.apache.org/download.html
>>> >> > >>> - https://activemq.apache.org/components/classic/download/
>>> >> > >>> - ...
>>> >> > >>>
>>> >> > >>> Maybe we can provide something similar ?
>>> >> > >>>
>>> >> > >>> Thanks !
>>> >> > >>> Regards
>>> >> > >>> JB
>>> >> > >>>
>>> >> > >>> On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <
>>> 2amogh@gmail.com> wrote:
>>> >> > >>> >
>>> >> > >>> > Hey all,
>>> >> > >>> >
>>> >> > >>> > Just following up here with a bit of a status update, so in
>>> the past week or so, items in the 1.8 release milestone have been closing
>>> out.
>>> >> > >>> > I'm aiming to cut a release next Tuesday, Jan 28.
>>> >> > >>> >
>>> >> > >>> > I'd like to reiterate that for any changes that don't make
>>> the 1.8 release, we can do a fast follow 1.9 release, and from the last
>>> community sync that seems to be the direction.
>>> >> > >>> > In this particular case, the 1.8 release is a bit earlier
>>> than our typical release cadence and with the 1.9 being a fast follow on, I
>>> think we're well on track.
>>> >> > >>> > Please add the proposed changes to the 1.9 milestone so folks
>>> can review ahead of time!
>>> >> > >>> >
>>> >> > >>> > In general, I'd encourage more frequent releases, changes
>>> which are ready can just go out and with the smaller diff it reduces the
>>> risks that exist with larger updates.
>>> >> > >>> >
>>> >> > >>> > Thanks,
>>> >> > >>> > Amogh Jahagirdar
>>> >> > >>> >
>>> >> > >>> > On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <
>>> dweeks@apache.org> wrote:
>>> >> > >>> >>
>>> >> > >>> >> Robert,
>>> >> > >>> >>
>>> >> > >>> >> I hear your frustration with the progress on the Auth
>>> Manager work, but I believe everyone recognizes that this was a large
>>> refactor further complicated by the need to preserve backward compatibility
>>> and handling deprecations appropriately. This work has gone through many
>>> iterations as we explored how to make the changes cleanly. Eventually the
>>> scale of the change led to breaking up the PR for closer review, which I
>>> believe was the right decision because we identified multiple issues after
>>> taking that step. That may have slowed down progress, but a lot of hours
>>> have gone into discussing, reviewing, and validating the work in this area.
>>> >> > >>> >>
>>> >> > >>> >> As a project, we have leaned away from gating releases on
>>> specific features because it leads to slower release cycles and prevents
>>> other features that are ready from going out. We also don't want to rush
>>> features just to hit a release target, but rather release more frequently
>>> to make changes available as they are ready.
>>> >> > >>> >>
>>> >> > >>> >> At this point, I believe the plan is to follow up soon with
>>> a 1.9 release.
>>> >> > >>> >>
>>> >> > >>> >> -Dan
>>> >> > >>> >>
>>> >> > >>> >> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de>
>>> wrote:
>>> >> > >>> >>>
>>> >> > >>> >>> Hey,
>>> >> > >>> >>>
>>> >> > >>> >>> IMHO 1.8 should definitely include the Auth-Manager work,
>>> which tackles
>>> >> > >>> >>> actual bugs in the Iceberg code base wrt OAuth
>>> implementation. That work
>>> >> > >>> >>> was originally intended to go into 1.7 and now it shall be
>>> delayed again
>>> >> > >>> >>> for 1.9. The PR was originally opened in July 2024, about
>>> half a year
>>> >> > >>> >>> ago and is still getting reviewed. In the meantime there
>>> were more than
>>> >> > >>> >>> 600 other PRs that got reviewed and merged.
>>> >> > >>> >>>
>>> >> > >>> >>> The overall agreement around spring 2024, please correct me
>>> if I am
>>> >> > >>> >>> wrong, was the whole REST/OAuth area needs to be improved,
>>> and the oauth
>>> >> > >>> >>> endpoint removed entirely.
>>> >> > >>> >>>
>>> >> > >>> >>> Generally speaking, and I know I'm not alone, getting
>>> reviews from
>>> >> > >>> >>> Iceberg committers is extremely hard. A lot of issues and
>>> PRs just get
>>> >> > >>> >>> closed (by that stale bot) without having gotten _any_
>>> attention from an
>>> >> > >>> >>> Iceberg committer. This is not a new situation but going on
>>> for a long
>>> >> > >>> >>> time. I have been talking to two Iceberg PMC members in
>>> person many
>>> >> > >>> >>> months ago that this is a very disappointing situation that
>>> needs to be
>>> >> > >>> >>> fixed. The reply was always "we are already working on it"
>>> - but at
>>> >> > >>> >>> least from my personal POV the situation did not improve.
>>> >> > >>> >>>
>>> >> > >>> >>> Robert
>>> >> > >>> >>>
>>> >> > >>> >>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
>>> >> > >>> >>> > Hi folks,
>>> >> > >>> >>> >
>>> >> > >>> >>> > Following the Community Meeting yesterday, I would like
>>> to propose the
>>> >> > >>> >>> > following plan regarding releases:
>>> >> > >>> >>> >
>>> >> > >>> >>> > 0. As a prerequisite to any release (1.7.2, 1.8.0,
>>> 1.9.0), as said by
>>> >> > >>> >>> > Ryan, we have to double check the NOTICE/LICENSE.
>>> Interestingly, I
>>> >> > >>> >>> > discussed this point with Fokko at the beginning of this
>>> week, because
>>> >> > >>> >>> > I have some doubts about LICENSE/NOTICE content in the
>>> "uber" jar
>>> >> > >>> >>> > artifacts where we shade dependencies. I'm doing a
>>> complete pass on
>>> >> > >>> >>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I
>>> should have a
>>> >> > >>> >>> > complete analysis by tomorrow. This is potentially a
>>> blocker for
>>> >> > >>> >>> > release votes.
>>> >> > >>> >>> >
>>> >> > >>> >>> > 1. As soon as (0) is done, 1.7.2 can be submitted to
>>> vote. I will work
>>> >> > >>> >>> > with Fokko on this one.
>>> >> > >>> >>> >
>>> >> > >>> >>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the
>>> release
>>> >> > >>> >>> > manager). Due to still some WIP, we "revisited" the 1.8.0
>>> release
>>> >> > >>> >>> > content: for instance, as best effort, we wanted to
>>> include REST Auth
>>> >> > >>> >>> > Manager improvement (OAuth2 Manager) but we preferred to
>>> postpone to
>>> >> > >>> >>> > 1.9.0. That's totally fine to me, however, I would
>>> propose to strongly
>>> >> > >>> >>> > focus on pending PRs for 1.9.0. Imho, we should "target"
>>> (again as
>>> >> > >>> >>> > clear best effort) on variant, partition stats and Auth
>>> Manager.
>>> >> > >>> >>> >
>>> >> > >>> >>> > 3. Assuming 1.8.0 will be released at the end of
>>> Jan/beginning of Feb,
>>> >> > >>> >>> > according to our "release cadence", what do you think
>>> about planning
>>> >> > >>> >>> > 1.9.0 in April ? Again with the main targets listed in
>>> (2).
>>> >> > >>> >>> >
>>> >> > >>> >>> > I tried to sum up what we discussed yesterday :)
>>> >> > >>> >>> > Thoughts ?
>>> >> > >>> >>> >
>>> >> > >>> >>> > Regards
>>> >> > >>> >>> > JB
>>> >> > >>> >>> >
>>> >> > >>> >>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <
>>> jb@nanthrax.net> wrote:
>>> >> > >>> >>> >> Hi folks,
>>> >> > >>> >>> >>
>>> >> > >>> >>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If
>>> we want to keep
>>> >> > >>> >>> >> our release "pace", 1.8.0 should be released around mid
>>> February.
>>> >> > >>> >>> >>
>>> >> > >>> >>> >> I think we already have a good "train" of merged PRs (or
>>> should be
>>> >> > >>> >>> >> merged soon): default values, REST auth improvements,
>>> dependencies
>>> >> > >>> >>> >> updates, etc.
>>> >> > >>> >>> >>
>>> >> > >>> >>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update
>>> GitHub Issues
>>> >> > >>> >>> >> and PRs we would like to "target" to 1.8.0.
>>> >> > >>> >>> >>
>>> >> > >>> >>> >> Thoughts ?
>>> >> > >>> >>> >>
>>> >> > >>> >>> >> Regards
>>> >> > >>> >>> >> JB
>>> >> > >>> >>>
>>> >> > >>> >>> --
>>> >> > >>> >>> Robert Stupp
>>> >> > >>> >>> @snazy
>>> >> > >>> >>>
>>>
>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Fokko Driesprong <fo...@apache.org>.
That shouldn't be included. There is still active discussion on the PR, and
it needs more work. The author also removed the milestone. The VOTE is out,
so we can start verifying.
Kind regards,
Fokko
Op ma 10 feb 2025 om 05:41 schreef Manu Zhang <ow...@gmail.com>:
> There's still https://github.com/apache/iceberg/pull/11216 under 1.8.0
> milestone.
> Do we want to include it?
>
> On Sun, Feb 9, 2025 at 3:01 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Thanks Amogh
>>
>> I updated the PR with some cleanups.
>>
>> Regards
>> JB
>>
>> On Sun, Feb 9, 2025 at 4:04 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
>> >
>> > Thanks JB I left a review, it'll be good to get another set of eyes on
>> it! Thank you for surfacing and fixing these issues, it's very appreciated.
>> >
>> > Amogh Jahagirdar
>> >
>> > On Sat, Feb 8, 2025 at 12:50 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>> >>
>> >> Here's the PR: https://github.com/apache/iceberg/pull/12195 for ref.
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On Fri, Feb 7, 2025 at 5:48 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>> >> >
>> >> > Hi Amogh,
>> >> >
>> >> > I found issues in the LICENSE/NOTICE from kafka-connect-runtime
>> >> > distribution (what's in the distribution zip). AFAIR, we plan to
>> >> > distribute this distribution, so it should be fixed.
>> >> > I will open a PR about that today.
>> >> >
>> >> > Sorry about that.
>> >> >
>> >> > Regards
>> >> > JB
>> >> >
>> >> > On Fri, Feb 7, 2025 at 11:35 AM Amogh Jahagirdar <2a...@gmail.com>
>> wrote:
>> >> > >
>> >> > > Hey all,
>> >> > >
>> >> > > An update, the final License/Notice release blockers are merged
>> (big thanks to JB, and Ryan/Fokko for helping review)! I'm in transit at
>> the moment, but once I get to a place with stable wifi I will cut a release
>> candidate.
>> >> > >
>> >> > > Thanks,
>> >> > > Amogh Jahagirdar
>> >> > >
>> >> > > On Wed, Jan 29, 2025 at 2:23 AM Amogh Jahagirdar <2a...@gmail.com>
>> wrote:
>> >> > >>
>> >> > >> Agreed, I wouldn't be opposed to looking into approaches to make
>> release times more predictable. At the same time, I'd advocate that in the
>> community, that anyone can propose a release at any point in time. Of
>> course, we can discuss as a community and make sure there's a reasonable
>> changeset, as well as focus review time on PRs which are close to being
>> ready for that release.
>> >> > >> To some degree this contradicts having a predictable release
>> schedule, but I feel like we can really just have a hybrid "Periodic
>> release + arbitrary off-cycle release" approach and things won't get too
>> crazy. It's a way to get the best of both frequency of release and user
>> expectations on release times.
>> >> > >>
>> >> > >> An update on 1.8 to the community, we're working on updating
>> LICENSE/NOTICE files in the AWS/GCP/Azure bundles, thank you JB for driving
>> that. It's something we need to get in for the release. Once that's in, I
>> will cut the RC.
>> >> > >>
>> >> > >> Thanks,
>> >> > >>
>> >> > >> Amogh Jahagirdar
>> >> > >>
>> >> > >> On Sun, Jan 26, 2025 at 1:16 PM Jean-Baptiste Onofré <
>> jb@nanthrax.net> wrote:
>> >> > >>>
>> >> > >>> Hi Amogh,
>> >> > >>>
>> >> > >>> Thanks !
>> >> > >>>
>> >> > >>> I agree we should have more frequent releases, but also more
>> >> > >>> "predictable" release time and give visibility to the community
>> >> > >>> (especially users).
>> >> > >>> Some ASF projects are providing "tables" with release plans:
>> >> > >>> - https://camel.apache.org/download/
>> >> > >>> - https://karaf.apache.org/download.html
>> >> > >>> - https://activemq.apache.org/components/classic/download/
>> >> > >>> - ...
>> >> > >>>
>> >> > >>> Maybe we can provide something similar ?
>> >> > >>>
>> >> > >>> Thanks !
>> >> > >>> Regards
>> >> > >>> JB
>> >> > >>>
>> >> > >>> On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <
>> 2amogh@gmail.com> wrote:
>> >> > >>> >
>> >> > >>> > Hey all,
>> >> > >>> >
>> >> > >>> > Just following up here with a bit of a status update, so in
>> the past week or so, items in the 1.8 release milestone have been closing
>> out.
>> >> > >>> > I'm aiming to cut a release next Tuesday, Jan 28.
>> >> > >>> >
>> >> > >>> > I'd like to reiterate that for any changes that don't make the
>> 1.8 release, we can do a fast follow 1.9 release, and from the last
>> community sync that seems to be the direction.
>> >> > >>> > In this particular case, the 1.8 release is a bit earlier than
>> our typical release cadence and with the 1.9 being a fast follow on, I
>> think we're well on track.
>> >> > >>> > Please add the proposed changes to the 1.9 milestone so folks
>> can review ahead of time!
>> >> > >>> >
>> >> > >>> > In general, I'd encourage more frequent releases, changes
>> which are ready can just go out and with the smaller diff it reduces the
>> risks that exist with larger updates.
>> >> > >>> >
>> >> > >>> > Thanks,
>> >> > >>> > Amogh Jahagirdar
>> >> > >>> >
>> >> > >>> > On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <
>> dweeks@apache.org> wrote:
>> >> > >>> >>
>> >> > >>> >> Robert,
>> >> > >>> >>
>> >> > >>> >> I hear your frustration with the progress on the Auth Manager
>> work, but I believe everyone recognizes that this was a large refactor
>> further complicated by the need to preserve backward compatibility and
>> handling deprecations appropriately. This work has gone through many
>> iterations as we explored how to make the changes cleanly. Eventually the
>> scale of the change led to breaking up the PR for closer review, which I
>> believe was the right decision because we identified multiple issues after
>> taking that step. That may have slowed down progress, but a lot of hours
>> have gone into discussing, reviewing, and validating the work in this area.
>> >> > >>> >>
>> >> > >>> >> As a project, we have leaned away from gating releases on
>> specific features because it leads to slower release cycles and prevents
>> other features that are ready from going out. We also don't want to rush
>> features just to hit a release target, but rather release more frequently
>> to make changes available as they are ready.
>> >> > >>> >>
>> >> > >>> >> At this point, I believe the plan is to follow up soon with a
>> 1.9 release.
>> >> > >>> >>
>> >> > >>> >> -Dan
>> >> > >>> >>
>> >> > >>> >> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de>
>> wrote:
>> >> > >>> >>>
>> >> > >>> >>> Hey,
>> >> > >>> >>>
>> >> > >>> >>> IMHO 1.8 should definitely include the Auth-Manager work,
>> which tackles
>> >> > >>> >>> actual bugs in the Iceberg code base wrt OAuth
>> implementation. That work
>> >> > >>> >>> was originally intended to go into 1.7 and now it shall be
>> delayed again
>> >> > >>> >>> for 1.9. The PR was originally opened in July 2024, about
>> half a year
>> >> > >>> >>> ago and is still getting reviewed. In the meantime there
>> were more than
>> >> > >>> >>> 600 other PRs that got reviewed and merged.
>> >> > >>> >>>
>> >> > >>> >>> The overall agreement around spring 2024, please correct me
>> if I am
>> >> > >>> >>> wrong, was the whole REST/OAuth area needs to be improved,
>> and the oauth
>> >> > >>> >>> endpoint removed entirely.
>> >> > >>> >>>
>> >> > >>> >>> Generally speaking, and I know I'm not alone, getting
>> reviews from
>> >> > >>> >>> Iceberg committers is extremely hard. A lot of issues and
>> PRs just get
>> >> > >>> >>> closed (by that stale bot) without having gotten _any_
>> attention from an
>> >> > >>> >>> Iceberg committer. This is not a new situation but going on
>> for a long
>> >> > >>> >>> time. I have been talking to two Iceberg PMC members in
>> person many
>> >> > >>> >>> months ago that this is a very disappointing situation that
>> needs to be
>> >> > >>> >>> fixed. The reply was always "we are already working on it" -
>> but at
>> >> > >>> >>> least from my personal POV the situation did not improve.
>> >> > >>> >>>
>> >> > >>> >>> Robert
>> >> > >>> >>>
>> >> > >>> >>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
>> >> > >>> >>> > Hi folks,
>> >> > >>> >>> >
>> >> > >>> >>> > Following the Community Meeting yesterday, I would like to
>> propose the
>> >> > >>> >>> > following plan regarding releases:
>> >> > >>> >>> >
>> >> > >>> >>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0),
>> as said by
>> >> > >>> >>> > Ryan, we have to double check the NOTICE/LICENSE.
>> Interestingly, I
>> >> > >>> >>> > discussed this point with Fokko at the beginning of this
>> week, because
>> >> > >>> >>> > I have some doubts about LICENSE/NOTICE content in the
>> "uber" jar
>> >> > >>> >>> > artifacts where we shade dependencies. I'm doing a
>> complete pass on
>> >> > >>> >>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I
>> should have a
>> >> > >>> >>> > complete analysis by tomorrow. This is potentially a
>> blocker for
>> >> > >>> >>> > release votes.
>> >> > >>> >>> >
>> >> > >>> >>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote.
>> I will work
>> >> > >>> >>> > with Fokko on this one.
>> >> > >>> >>> >
>> >> > >>> >>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the
>> release
>> >> > >>> >>> > manager). Due to still some WIP, we "revisited" the 1.8.0
>> release
>> >> > >>> >>> > content: for instance, as best effort, we wanted to
>> include REST Auth
>> >> > >>> >>> > Manager improvement (OAuth2 Manager) but we preferred to
>> postpone to
>> >> > >>> >>> > 1.9.0. That's totally fine to me, however, I would propose
>> to strongly
>> >> > >>> >>> > focus on pending PRs for 1.9.0. Imho, we should "target"
>> (again as
>> >> > >>> >>> > clear best effort) on variant, partition stats and Auth
>> Manager.
>> >> > >>> >>> >
>> >> > >>> >>> > 3. Assuming 1.8.0 will be released at the end of
>> Jan/beginning of Feb,
>> >> > >>> >>> > according to our "release cadence", what do you think
>> about planning
>> >> > >>> >>> > 1.9.0 in April ? Again with the main targets listed in (2).
>> >> > >>> >>> >
>> >> > >>> >>> > I tried to sum up what we discussed yesterday :)
>> >> > >>> >>> > Thoughts ?
>> >> > >>> >>> >
>> >> > >>> >>> > Regards
>> >> > >>> >>> > JB
>> >> > >>> >>> >
>> >> > >>> >>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <
>> jb@nanthrax.net> wrote:
>> >> > >>> >>> >> Hi folks,
>> >> > >>> >>> >>
>> >> > >>> >>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we
>> want to keep
>> >> > >>> >>> >> our release "pace", 1.8.0 should be released around mid
>> February.
>> >> > >>> >>> >>
>> >> > >>> >>> >> I think we already have a good "train" of merged PRs (or
>> should be
>> >> > >>> >>> >> merged soon): default values, REST auth improvements,
>> dependencies
>> >> > >>> >>> >> updates, etc.
>> >> > >>> >>> >>
>> >> > >>> >>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update
>> GitHub Issues
>> >> > >>> >>> >> and PRs we would like to "target" to 1.8.0.
>> >> > >>> >>> >>
>> >> > >>> >>> >> Thoughts ?
>> >> > >>> >>> >>
>> >> > >>> >>> >> Regards
>> >> > >>> >>> >> JB
>> >> > >>> >>>
>> >> > >>> >>> --
>> >> > >>> >>> Robert Stupp
>> >> > >>> >>> @snazy
>> >> > >>> >>>
>>
>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Manu Zhang <ow...@gmail.com>.
There's still https://github.com/apache/iceberg/pull/11216 under 1.8.0
milestone.
Do we want to include it?
On Sun, Feb 9, 2025 at 3:01 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Thanks Amogh
>
> I updated the PR with some cleanups.
>
> Regards
> JB
>
> On Sun, Feb 9, 2025 at 4:04 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
> >
> > Thanks JB I left a review, it'll be good to get another set of eyes on
> it! Thank you for surfacing and fixing these issues, it's very appreciated.
> >
> > Amogh Jahagirdar
> >
> > On Sat, Feb 8, 2025 at 12:50 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >>
> >> Here's the PR: https://github.com/apache/iceberg/pull/12195 for ref.
> >>
> >> Regards
> >> JB
> >>
> >> On Fri, Feb 7, 2025 at 5:48 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >> >
> >> > Hi Amogh,
> >> >
> >> > I found issues in the LICENSE/NOTICE from kafka-connect-runtime
> >> > distribution (what's in the distribution zip). AFAIR, we plan to
> >> > distribute this distribution, so it should be fixed.
> >> > I will open a PR about that today.
> >> >
> >> > Sorry about that.
> >> >
> >> > Regards
> >> > JB
> >> >
> >> > On Fri, Feb 7, 2025 at 11:35 AM Amogh Jahagirdar <2a...@gmail.com>
> wrote:
> >> > >
> >> > > Hey all,
> >> > >
> >> > > An update, the final License/Notice release blockers are merged
> (big thanks to JB, and Ryan/Fokko for helping review)! I'm in transit at
> the moment, but once I get to a place with stable wifi I will cut a release
> candidate.
> >> > >
> >> > > Thanks,
> >> > > Amogh Jahagirdar
> >> > >
> >> > > On Wed, Jan 29, 2025 at 2:23 AM Amogh Jahagirdar <2a...@gmail.com>
> wrote:
> >> > >>
> >> > >> Agreed, I wouldn't be opposed to looking into approaches to make
> release times more predictable. At the same time, I'd advocate that in the
> community, that anyone can propose a release at any point in time. Of
> course, we can discuss as a community and make sure there's a reasonable
> changeset, as well as focus review time on PRs which are close to being
> ready for that release.
> >> > >> To some degree this contradicts having a predictable release
> schedule, but I feel like we can really just have a hybrid "Periodic
> release + arbitrary off-cycle release" approach and things won't get too
> crazy. It's a way to get the best of both frequency of release and user
> expectations on release times.
> >> > >>
> >> > >> An update on 1.8 to the community, we're working on updating
> LICENSE/NOTICE files in the AWS/GCP/Azure bundles, thank you JB for driving
> that. It's something we need to get in for the release. Once that's in, I
> will cut the RC.
> >> > >>
> >> > >> Thanks,
> >> > >>
> >> > >> Amogh Jahagirdar
> >> > >>
> >> > >> On Sun, Jan 26, 2025 at 1:16 PM Jean-Baptiste Onofré <
> jb@nanthrax.net> wrote:
> >> > >>>
> >> > >>> Hi Amogh,
> >> > >>>
> >> > >>> Thanks !
> >> > >>>
> >> > >>> I agree we should have more frequent releases, but also more
> >> > >>> "predictable" release time and give visibility to the community
> >> > >>> (especially users).
> >> > >>> Some ASF projects are providing "tables" with release plans:
> >> > >>> - https://camel.apache.org/download/
> >> > >>> - https://karaf.apache.org/download.html
> >> > >>> - https://activemq.apache.org/components/classic/download/
> >> > >>> - ...
> >> > >>>
> >> > >>> Maybe we can provide something similar ?
> >> > >>>
> >> > >>> Thanks !
> >> > >>> Regards
> >> > >>> JB
> >> > >>>
> >> > >>> On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <
> 2amogh@gmail.com> wrote:
> >> > >>> >
> >> > >>> > Hey all,
> >> > >>> >
> >> > >>> > Just following up here with a bit of a status update, so in the
> past week or so, items in the 1.8 release milestone have been closing out.
> >> > >>> > I'm aiming to cut a release next Tuesday, Jan 28.
> >> > >>> >
> >> > >>> > I'd like to reiterate that for any changes that don't make the
> 1.8 release, we can do a fast follow 1.9 release, and from the last
> community sync that seems to be the direction.
> >> > >>> > In this particular case, the 1.8 release is a bit earlier than
> our typical release cadence and with the 1.9 being a fast follow on, I
> think we're well on track.
> >> > >>> > Please add the proposed changes to the 1.9 milestone so folks
> can review ahead of time!
> >> > >>> >
> >> > >>> > In general, I'd encourage more frequent releases, changes which
> are ready can just go out and with the smaller diff it reduces the risks
> that exist with larger updates.
> >> > >>> >
> >> > >>> > Thanks,
> >> > >>> > Amogh Jahagirdar
> >> > >>> >
> >> > >>> > On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <
> dweeks@apache.org> wrote:
> >> > >>> >>
> >> > >>> >> Robert,
> >> > >>> >>
> >> > >>> >> I hear your frustration with the progress on the Auth Manager
> work, but I believe everyone recognizes that this was a large refactor
> further complicated by the need to preserve backward compatibility and
> handling deprecations appropriately. This work has gone through many
> iterations as we explored how to make the changes cleanly. Eventually the
> scale of the change led to breaking up the PR for closer review, which I
> believe was the right decision because we identified multiple issues after
> taking that step. That may have slowed down progress, but a lot of hours
> have gone into discussing, reviewing, and validating the work in this area.
> >> > >>> >>
> >> > >>> >> As a project, we have leaned away from gating releases on
> specific features because it leads to slower release cycles and prevents
> other features that are ready from going out. We also don't want to rush
> features just to hit a release target, but rather release more frequently
> to make changes available as they are ready.
> >> > >>> >>
> >> > >>> >> At this point, I believe the plan is to follow up soon with a
> 1.9 release.
> >> > >>> >>
> >> > >>> >> -Dan
> >> > >>> >>
> >> > >>> >> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de>
> wrote:
> >> > >>> >>>
> >> > >>> >>> Hey,
> >> > >>> >>>
> >> > >>> >>> IMHO 1.8 should definitely include the Auth-Manager work,
> which tackles
> >> > >>> >>> actual bugs in the Iceberg code base wrt OAuth
> implementation. That work
> >> > >>> >>> was originally intended to go into 1.7 and now it shall be
> delayed again
> >> > >>> >>> for 1.9. The PR was originally opened in July 2024, about
> half a year
> >> > >>> >>> ago and is still getting reviewed. In the meantime there were
> more than
> >> > >>> >>> 600 other PRs that got reviewed and merged.
> >> > >>> >>>
> >> > >>> >>> The overall agreement around spring 2024, please correct me
> if I am
> >> > >>> >>> wrong, was the whole REST/OAuth area needs to be improved,
> and the oauth
> >> > >>> >>> endpoint removed entirely.
> >> > >>> >>>
> >> > >>> >>> Generally speaking, and I know I'm not alone, getting reviews
> from
> >> > >>> >>> Iceberg committers is extremely hard. A lot of issues and PRs
> just get
> >> > >>> >>> closed (by that stale bot) without having gotten _any_
> attention from an
> >> > >>> >>> Iceberg committer. This is not a new situation but going on
> for a long
> >> > >>> >>> time. I have been talking to two Iceberg PMC members in
> person many
> >> > >>> >>> months ago that this is a very disappointing situation that
> needs to be
> >> > >>> >>> fixed. The reply was always "we are already working on it" -
> but at
> >> > >>> >>> least from my personal POV the situation did not improve.
> >> > >>> >>>
> >> > >>> >>> Robert
> >> > >>> >>>
> >> > >>> >>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
> >> > >>> >>> > Hi folks,
> >> > >>> >>> >
> >> > >>> >>> > Following the Community Meeting yesterday, I would like to
> propose the
> >> > >>> >>> > following plan regarding releases:
> >> > >>> >>> >
> >> > >>> >>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0),
> as said by
> >> > >>> >>> > Ryan, we have to double check the NOTICE/LICENSE.
> Interestingly, I
> >> > >>> >>> > discussed this point with Fokko at the beginning of this
> week, because
> >> > >>> >>> > I have some doubts about LICENSE/NOTICE content in the
> "uber" jar
> >> > >>> >>> > artifacts where we shade dependencies. I'm doing a complete
> pass on
> >> > >>> >>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I
> should have a
> >> > >>> >>> > complete analysis by tomorrow. This is potentially a
> blocker for
> >> > >>> >>> > release votes.
> >> > >>> >>> >
> >> > >>> >>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote.
> I will work
> >> > >>> >>> > with Fokko on this one.
> >> > >>> >>> >
> >> > >>> >>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the
> release
> >> > >>> >>> > manager). Due to still some WIP, we "revisited" the 1.8.0
> release
> >> > >>> >>> > content: for instance, as best effort, we wanted to include
> REST Auth
> >> > >>> >>> > Manager improvement (OAuth2 Manager) but we preferred to
> postpone to
> >> > >>> >>> > 1.9.0. That's totally fine to me, however, I would propose
> to strongly
> >> > >>> >>> > focus on pending PRs for 1.9.0. Imho, we should "target"
> (again as
> >> > >>> >>> > clear best effort) on variant, partition stats and Auth
> Manager.
> >> > >>> >>> >
> >> > >>> >>> > 3. Assuming 1.8.0 will be released at the end of
> Jan/beginning of Feb,
> >> > >>> >>> > according to our "release cadence", what do you think about
> planning
> >> > >>> >>> > 1.9.0 in April ? Again with the main targets listed in (2).
> >> > >>> >>> >
> >> > >>> >>> > I tried to sum up what we discussed yesterday :)
> >> > >>> >>> > Thoughts ?
> >> > >>> >>> >
> >> > >>> >>> > Regards
> >> > >>> >>> > JB
> >> > >>> >>> >
> >> > >>> >>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <
> jb@nanthrax.net> wrote:
> >> > >>> >>> >> Hi folks,
> >> > >>> >>> >>
> >> > >>> >>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we
> want to keep
> >> > >>> >>> >> our release "pace", 1.8.0 should be released around mid
> February.
> >> > >>> >>> >>
> >> > >>> >>> >> I think we already have a good "train" of merged PRs (or
> should be
> >> > >>> >>> >> merged soon): default values, REST auth improvements,
> dependencies
> >> > >>> >>> >> updates, etc.
> >> > >>> >>> >>
> >> > >>> >>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update
> GitHub Issues
> >> > >>> >>> >> and PRs we would like to "target" to 1.8.0.
> >> > >>> >>> >>
> >> > >>> >>> >> Thoughts ?
> >> > >>> >>> >>
> >> > >>> >>> >> Regards
> >> > >>> >>> >> JB
> >> > >>> >>>
> >> > >>> >>> --
> >> > >>> >>> Robert Stupp
> >> > >>> >>> @snazy
> >> > >>> >>>
>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks Amogh
I updated the PR with some cleanups.
Regards
JB
On Sun, Feb 9, 2025 at 4:04 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
>
> Thanks JB I left a review, it'll be good to get another set of eyes on it! Thank you for surfacing and fixing these issues, it's very appreciated.
>
> Amogh Jahagirdar
>
> On Sat, Feb 8, 2025 at 12:50 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>>
>> Here's the PR: https://github.com/apache/iceberg/pull/12195 for ref.
>>
>> Regards
>> JB
>>
>> On Fri, Feb 7, 2025 at 5:48 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>> >
>> > Hi Amogh,
>> >
>> > I found issues in the LICENSE/NOTICE from kafka-connect-runtime
>> > distribution (what's in the distribution zip). AFAIR, we plan to
>> > distribute this distribution, so it should be fixed.
>> > I will open a PR about that today.
>> >
>> > Sorry about that.
>> >
>> > Regards
>> > JB
>> >
>> > On Fri, Feb 7, 2025 at 11:35 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
>> > >
>> > > Hey all,
>> > >
>> > > An update, the final License/Notice release blockers are merged (big thanks to JB, and Ryan/Fokko for helping review)! I'm in transit at the moment, but once I get to a place with stable wifi I will cut a release candidate.
>> > >
>> > > Thanks,
>> > > Amogh Jahagirdar
>> > >
>> > > On Wed, Jan 29, 2025 at 2:23 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
>> > >>
>> > >> Agreed, I wouldn't be opposed to looking into approaches to make release times more predictable. At the same time, I'd advocate that in the community, that anyone can propose a release at any point in time. Of course, we can discuss as a community and make sure there's a reasonable changeset, as well as focus review time on PRs which are close to being ready for that release.
>> > >> To some degree this contradicts having a predictable release schedule, but I feel like we can really just have a hybrid "Periodic release + arbitrary off-cycle release" approach and things won't get too crazy. It's a way to get the best of both frequency of release and user expectations on release times.
>> > >>
>> > >> An update on 1.8 to the community, we're working on updating LICENSE/NOTICE files in the AWS/GCP/Azure bundles, thank you JB for driving that. It's something we need to get in for the release. Once that's in, I will cut the RC.
>> > >>
>> > >> Thanks,
>> > >>
>> > >> Amogh Jahagirdar
>> > >>
>> > >> On Sun, Jan 26, 2025 at 1:16 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>> > >>>
>> > >>> Hi Amogh,
>> > >>>
>> > >>> Thanks !
>> > >>>
>> > >>> I agree we should have more frequent releases, but also more
>> > >>> "predictable" release time and give visibility to the community
>> > >>> (especially users).
>> > >>> Some ASF projects are providing "tables" with release plans:
>> > >>> - https://camel.apache.org/download/
>> > >>> - https://karaf.apache.org/download.html
>> > >>> - https://activemq.apache.org/components/classic/download/
>> > >>> - ...
>> > >>>
>> > >>> Maybe we can provide something similar ?
>> > >>>
>> > >>> Thanks !
>> > >>> Regards
>> > >>> JB
>> > >>>
>> > >>> On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
>> > >>> >
>> > >>> > Hey all,
>> > >>> >
>> > >>> > Just following up here with a bit of a status update, so in the past week or so, items in the 1.8 release milestone have been closing out.
>> > >>> > I'm aiming to cut a release next Tuesday, Jan 28.
>> > >>> >
>> > >>> > I'd like to reiterate that for any changes that don't make the 1.8 release, we can do a fast follow 1.9 release, and from the last community sync that seems to be the direction.
>> > >>> > In this particular case, the 1.8 release is a bit earlier than our typical release cadence and with the 1.9 being a fast follow on, I think we're well on track.
>> > >>> > Please add the proposed changes to the 1.9 milestone so folks can review ahead of time!
>> > >>> >
>> > >>> > In general, I'd encourage more frequent releases, changes which are ready can just go out and with the smaller diff it reduces the risks that exist with larger updates.
>> > >>> >
>> > >>> > Thanks,
>> > >>> > Amogh Jahagirdar
>> > >>> >
>> > >>> > On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <dw...@apache.org> wrote:
>> > >>> >>
>> > >>> >> Robert,
>> > >>> >>
>> > >>> >> I hear your frustration with the progress on the Auth Manager work, but I believe everyone recognizes that this was a large refactor further complicated by the need to preserve backward compatibility and handling deprecations appropriately. This work has gone through many iterations as we explored how to make the changes cleanly. Eventually the scale of the change led to breaking up the PR for closer review, which I believe was the right decision because we identified multiple issues after taking that step. That may have slowed down progress, but a lot of hours have gone into discussing, reviewing, and validating the work in this area.
>> > >>> >>
>> > >>> >> As a project, we have leaned away from gating releases on specific features because it leads to slower release cycles and prevents other features that are ready from going out. We also don't want to rush features just to hit a release target, but rather release more frequently to make changes available as they are ready.
>> > >>> >>
>> > >>> >> At this point, I believe the plan is to follow up soon with a 1.9 release.
>> > >>> >>
>> > >>> >> -Dan
>> > >>> >>
>> > >>> >> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de> wrote:
>> > >>> >>>
>> > >>> >>> Hey,
>> > >>> >>>
>> > >>> >>> IMHO 1.8 should definitely include the Auth-Manager work, which tackles
>> > >>> >>> actual bugs in the Iceberg code base wrt OAuth implementation. That work
>> > >>> >>> was originally intended to go into 1.7 and now it shall be delayed again
>> > >>> >>> for 1.9. The PR was originally opened in July 2024, about half a year
>> > >>> >>> ago and is still getting reviewed. In the meantime there were more than
>> > >>> >>> 600 other PRs that got reviewed and merged.
>> > >>> >>>
>> > >>> >>> The overall agreement around spring 2024, please correct me if I am
>> > >>> >>> wrong, was the whole REST/OAuth area needs to be improved, and the oauth
>> > >>> >>> endpoint removed entirely.
>> > >>> >>>
>> > >>> >>> Generally speaking, and I know I'm not alone, getting reviews from
>> > >>> >>> Iceberg committers is extremely hard. A lot of issues and PRs just get
>> > >>> >>> closed (by that stale bot) without having gotten _any_ attention from an
>> > >>> >>> Iceberg committer. This is not a new situation but going on for a long
>> > >>> >>> time. I have been talking to two Iceberg PMC members in person many
>> > >>> >>> months ago that this is a very disappointing situation that needs to be
>> > >>> >>> fixed. The reply was always "we are already working on it" - but at
>> > >>> >>> least from my personal POV the situation did not improve.
>> > >>> >>>
>> > >>> >>> Robert
>> > >>> >>>
>> > >>> >>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
>> > >>> >>> > Hi folks,
>> > >>> >>> >
>> > >>> >>> > Following the Community Meeting yesterday, I would like to propose the
>> > >>> >>> > following plan regarding releases:
>> > >>> >>> >
>> > >>> >>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
>> > >>> >>> > Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
>> > >>> >>> > discussed this point with Fokko at the beginning of this week, because
>> > >>> >>> > I have some doubts about LICENSE/NOTICE content in the "uber" jar
>> > >>> >>> > artifacts where we shade dependencies. I'm doing a complete pass on
>> > >>> >>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
>> > >>> >>> > complete analysis by tomorrow. This is potentially a blocker for
>> > >>> >>> > release votes.
>> > >>> >>> >
>> > >>> >>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will work
>> > >>> >>> > with Fokko on this one.
>> > >>> >>> >
>> > >>> >>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
>> > >>> >>> > manager). Due to still some WIP, we "revisited" the 1.8.0 release
>> > >>> >>> > content: for instance, as best effort, we wanted to include REST Auth
>> > >>> >>> > Manager improvement (OAuth2 Manager) but we preferred to postpone to
>> > >>> >>> > 1.9.0. That's totally fine to me, however, I would propose to strongly
>> > >>> >>> > focus on pending PRs for 1.9.0. Imho, we should "target" (again as
>> > >>> >>> > clear best effort) on variant, partition stats and Auth Manager.
>> > >>> >>> >
>> > >>> >>> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning of Feb,
>> > >>> >>> > according to our "release cadence", what do you think about planning
>> > >>> >>> > 1.9.0 in April ? Again with the main targets listed in (2).
>> > >>> >>> >
>> > >>> >>> > I tried to sum up what we discussed yesterday :)
>> > >>> >>> > Thoughts ?
>> > >>> >>> >
>> > >>> >>> > Regards
>> > >>> >>> > JB
>> > >>> >>> >
>> > >>> >>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>> > >>> >>> >> Hi folks,
>> > >>> >>> >>
>> > >>> >>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
>> > >>> >>> >> our release "pace", 1.8.0 should be released around mid February.
>> > >>> >>> >>
>> > >>> >>> >> I think we already have a good "train" of merged PRs (or should be
>> > >>> >>> >> merged soon): default values, REST auth improvements, dependencies
>> > >>> >>> >> updates, etc.
>> > >>> >>> >>
>> > >>> >>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>> > >>> >>> >> and PRs we would like to "target" to 1.8.0.
>> > >>> >>> >>
>> > >>> >>> >> Thoughts ?
>> > >>> >>> >>
>> > >>> >>> >> Regards
>> > >>> >>> >> JB
>> > >>> >>>
>> > >>> >>> --
>> > >>> >>> Robert Stupp
>> > >>> >>> @snazy
>> > >>> >>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Amogh Jahagirdar <2a...@gmail.com>.
Thanks JB I left a review, it'll be good to get another set of eyes on it!
Thank you for surfacing and fixing these issues, it's very appreciated.
Amogh Jahagirdar
On Sat, Feb 8, 2025 at 12:50 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Here's the PR: https://github.com/apache/iceberg/pull/12195 for ref.
>
> Regards
> JB
>
> On Fri, Feb 7, 2025 at 5:48 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >
> > Hi Amogh,
> >
> > I found issues in the LICENSE/NOTICE from kafka-connect-runtime
> > distribution (what's in the distribution zip). AFAIR, we plan to
> > distribute this distribution, so it should be fixed.
> > I will open a PR about that today.
> >
> > Sorry about that.
> >
> > Regards
> > JB
> >
> > On Fri, Feb 7, 2025 at 11:35 AM Amogh Jahagirdar <2a...@gmail.com>
> wrote:
> > >
> > > Hey all,
> > >
> > > An update, the final License/Notice release blockers are merged (big
> thanks to JB, and Ryan/Fokko for helping review)! I'm in transit at the
> moment, but once I get to a place with stable wifi I will cut a release
> candidate.
> > >
> > > Thanks,
> > > Amogh Jahagirdar
> > >
> > > On Wed, Jan 29, 2025 at 2:23 AM Amogh Jahagirdar <2a...@gmail.com>
> wrote:
> > >>
> > >> Agreed, I wouldn't be opposed to looking into approaches to make
> release times more predictable. At the same time, I'd advocate that in the
> community, that anyone can propose a release at any point in time. Of
> course, we can discuss as a community and make sure there's a reasonable
> changeset, as well as focus review time on PRs which are close to being
> ready for that release.
> > >> To some degree this contradicts having a predictable release
> schedule, but I feel like we can really just have a hybrid "Periodic
> release + arbitrary off-cycle release" approach and things won't get too
> crazy. It's a way to get the best of both frequency of release and user
> expectations on release times.
> > >>
> > >> An update on 1.8 to the community, we're working on updating
> LICENSE/NOTICE files in the AWS/GCP/Azure bundles, thank you JB for driving
> that. It's something we need to get in for the release. Once that's in, I
> will cut the RC.
> > >>
> > >> Thanks,
> > >>
> > >> Amogh Jahagirdar
> > >>
> > >> On Sun, Jan 26, 2025 at 1:16 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> > >>>
> > >>> Hi Amogh,
> > >>>
> > >>> Thanks !
> > >>>
> > >>> I agree we should have more frequent releases, but also more
> > >>> "predictable" release time and give visibility to the community
> > >>> (especially users).
> > >>> Some ASF projects are providing "tables" with release plans:
> > >>> - https://camel.apache.org/download/
> > >>> - https://karaf.apache.org/download.html
> > >>> - https://activemq.apache.org/components/classic/download/
> > >>> - ...
> > >>>
> > >>> Maybe we can provide something similar ?
> > >>>
> > >>> Thanks !
> > >>> Regards
> > >>> JB
> > >>>
> > >>> On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <2a...@gmail.com>
> wrote:
> > >>> >
> > >>> > Hey all,
> > >>> >
> > >>> > Just following up here with a bit of a status update, so in the
> past week or so, items in the 1.8 release milestone have been closing out.
> > >>> > I'm aiming to cut a release next Tuesday, Jan 28.
> > >>> >
> > >>> > I'd like to reiterate that for any changes that don't make the 1.8
> release, we can do a fast follow 1.9 release, and from the last community
> sync that seems to be the direction.
> > >>> > In this particular case, the 1.8 release is a bit earlier than our
> typical release cadence and with the 1.9 being a fast follow on, I think
> we're well on track.
> > >>> > Please add the proposed changes to the 1.9 milestone so folks can
> review ahead of time!
> > >>> >
> > >>> > In general, I'd encourage more frequent releases, changes which
> are ready can just go out and with the smaller diff it reduces the risks
> that exist with larger updates.
> > >>> >
> > >>> > Thanks,
> > >>> > Amogh Jahagirdar
> > >>> >
> > >>> > On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <dw...@apache.org>
> wrote:
> > >>> >>
> > >>> >> Robert,
> > >>> >>
> > >>> >> I hear your frustration with the progress on the Auth Manager
> work, but I believe everyone recognizes that this was a large refactor
> further complicated by the need to preserve backward compatibility and
> handling deprecations appropriately. This work has gone through many
> iterations as we explored how to make the changes cleanly. Eventually the
> scale of the change led to breaking up the PR for closer review, which I
> believe was the right decision because we identified multiple issues after
> taking that step. That may have slowed down progress, but a lot of hours
> have gone into discussing, reviewing, and validating the work in this area.
> > >>> >>
> > >>> >> As a project, we have leaned away from gating releases on
> specific features because it leads to slower release cycles and prevents
> other features that are ready from going out. We also don't want to rush
> features just to hit a release target, but rather release more frequently
> to make changes available as they are ready.
> > >>> >>
> > >>> >> At this point, I believe the plan is to follow up soon with a 1.9
> release.
> > >>> >>
> > >>> >> -Dan
> > >>> >>
> > >>> >> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de>
> wrote:
> > >>> >>>
> > >>> >>> Hey,
> > >>> >>>
> > >>> >>> IMHO 1.8 should definitely include the Auth-Manager work, which
> tackles
> > >>> >>> actual bugs in the Iceberg code base wrt OAuth implementation.
> That work
> > >>> >>> was originally intended to go into 1.7 and now it shall be
> delayed again
> > >>> >>> for 1.9. The PR was originally opened in July 2024, about half a
> year
> > >>> >>> ago and is still getting reviewed. In the meantime there were
> more than
> > >>> >>> 600 other PRs that got reviewed and merged.
> > >>> >>>
> > >>> >>> The overall agreement around spring 2024, please correct me if I
> am
> > >>> >>> wrong, was the whole REST/OAuth area needs to be improved, and
> the oauth
> > >>> >>> endpoint removed entirely.
> > >>> >>>
> > >>> >>> Generally speaking, and I know I'm not alone, getting reviews
> from
> > >>> >>> Iceberg committers is extremely hard. A lot of issues and PRs
> just get
> > >>> >>> closed (by that stale bot) without having gotten _any_ attention
> from an
> > >>> >>> Iceberg committer. This is not a new situation but going on for
> a long
> > >>> >>> time. I have been talking to two Iceberg PMC members in person
> many
> > >>> >>> months ago that this is a very disappointing situation that
> needs to be
> > >>> >>> fixed. The reply was always "we are already working on it" - but
> at
> > >>> >>> least from my personal POV the situation did not improve.
> > >>> >>>
> > >>> >>> Robert
> > >>> >>>
> > >>> >>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
> > >>> >>> > Hi folks,
> > >>> >>> >
> > >>> >>> > Following the Community Meeting yesterday, I would like to
> propose the
> > >>> >>> > following plan regarding releases:
> > >>> >>> >
> > >>> >>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as
> said by
> > >>> >>> > Ryan, we have to double check the NOTICE/LICENSE.
> Interestingly, I
> > >>> >>> > discussed this point with Fokko at the beginning of this week,
> because
> > >>> >>> > I have some doubts about LICENSE/NOTICE content in the "uber"
> jar
> > >>> >>> > artifacts where we shade dependencies. I'm doing a complete
> pass on
> > >>> >>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should
> have a
> > >>> >>> > complete analysis by tomorrow. This is potentially a blocker
> for
> > >>> >>> > release votes.
> > >>> >>> >
> > >>> >>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I
> will work
> > >>> >>> > with Fokko on this one.
> > >>> >>> >
> > >>> >>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the
> release
> > >>> >>> > manager). Due to still some WIP, we "revisited" the 1.8.0
> release
> > >>> >>> > content: for instance, as best effort, we wanted to include
> REST Auth
> > >>> >>> > Manager improvement (OAuth2 Manager) but we preferred to
> postpone to
> > >>> >>> > 1.9.0. That's totally fine to me, however, I would propose to
> strongly
> > >>> >>> > focus on pending PRs for 1.9.0. Imho, we should "target"
> (again as
> > >>> >>> > clear best effort) on variant, partition stats and Auth
> Manager.
> > >>> >>> >
> > >>> >>> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning
> of Feb,
> > >>> >>> > according to our "release cadence", what do you think about
> planning
> > >>> >>> > 1.9.0 in April ? Again with the main targets listed in (2).
> > >>> >>> >
> > >>> >>> > I tried to sum up what we discussed yesterday :)
> > >>> >>> > Thoughts ?
> > >>> >>> >
> > >>> >>> > Regards
> > >>> >>> > JB
> > >>> >>> >
> > >>> >>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <
> jb@nanthrax.net> wrote:
> > >>> >>> >> Hi folks,
> > >>> >>> >>
> > >>> >>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we
> want to keep
> > >>> >>> >> our release "pace", 1.8.0 should be released around mid
> February.
> > >>> >>> >>
> > >>> >>> >> I think we already have a good "train" of merged PRs (or
> should be
> > >>> >>> >> merged soon): default values, REST auth improvements,
> dependencies
> > >>> >>> >> updates, etc.
> > >>> >>> >>
> > >>> >>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub
> Issues
> > >>> >>> >> and PRs we would like to "target" to 1.8.0.
> > >>> >>> >>
> > >>> >>> >> Thoughts ?
> > >>> >>> >>
> > >>> >>> >> Regards
> > >>> >>> >> JB
> > >>> >>>
> > >>> >>> --
> > >>> >>> Robert Stupp
> > >>> >>> @snazy
> > >>> >>>
>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Here's the PR: https://github.com/apache/iceberg/pull/12195 for ref.
Regards
JB
On Fri, Feb 7, 2025 at 5:48 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> Hi Amogh,
>
> I found issues in the LICENSE/NOTICE from kafka-connect-runtime
> distribution (what's in the distribution zip). AFAIR, we plan to
> distribute this distribution, so it should be fixed.
> I will open a PR about that today.
>
> Sorry about that.
>
> Regards
> JB
>
> On Fri, Feb 7, 2025 at 11:35 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
> >
> > Hey all,
> >
> > An update, the final License/Notice release blockers are merged (big thanks to JB, and Ryan/Fokko for helping review)! I'm in transit at the moment, but once I get to a place with stable wifi I will cut a release candidate.
> >
> > Thanks,
> > Amogh Jahagirdar
> >
> > On Wed, Jan 29, 2025 at 2:23 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
> >>
> >> Agreed, I wouldn't be opposed to looking into approaches to make release times more predictable. At the same time, I'd advocate that in the community, that anyone can propose a release at any point in time. Of course, we can discuss as a community and make sure there's a reasonable changeset, as well as focus review time on PRs which are close to being ready for that release.
> >> To some degree this contradicts having a predictable release schedule, but I feel like we can really just have a hybrid "Periodic release + arbitrary off-cycle release" approach and things won't get too crazy. It's a way to get the best of both frequency of release and user expectations on release times.
> >>
> >> An update on 1.8 to the community, we're working on updating LICENSE/NOTICE files in the AWS/GCP/Azure bundles, thank you JB for driving that. It's something we need to get in for the release. Once that's in, I will cut the RC.
> >>
> >> Thanks,
> >>
> >> Amogh Jahagirdar
> >>
> >> On Sun, Jan 26, 2025 at 1:16 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> >>>
> >>> Hi Amogh,
> >>>
> >>> Thanks !
> >>>
> >>> I agree we should have more frequent releases, but also more
> >>> "predictable" release time and give visibility to the community
> >>> (especially users).
> >>> Some ASF projects are providing "tables" with release plans:
> >>> - https://camel.apache.org/download/
> >>> - https://karaf.apache.org/download.html
> >>> - https://activemq.apache.org/components/classic/download/
> >>> - ...
> >>>
> >>> Maybe we can provide something similar ?
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
> >>> >
> >>> > Hey all,
> >>> >
> >>> > Just following up here with a bit of a status update, so in the past week or so, items in the 1.8 release milestone have been closing out.
> >>> > I'm aiming to cut a release next Tuesday, Jan 28.
> >>> >
> >>> > I'd like to reiterate that for any changes that don't make the 1.8 release, we can do a fast follow 1.9 release, and from the last community sync that seems to be the direction.
> >>> > In this particular case, the 1.8 release is a bit earlier than our typical release cadence and with the 1.9 being a fast follow on, I think we're well on track.
> >>> > Please add the proposed changes to the 1.9 milestone so folks can review ahead of time!
> >>> >
> >>> > In general, I'd encourage more frequent releases, changes which are ready can just go out and with the smaller diff it reduces the risks that exist with larger updates.
> >>> >
> >>> > Thanks,
> >>> > Amogh Jahagirdar
> >>> >
> >>> > On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <dw...@apache.org> wrote:
> >>> >>
> >>> >> Robert,
> >>> >>
> >>> >> I hear your frustration with the progress on the Auth Manager work, but I believe everyone recognizes that this was a large refactor further complicated by the need to preserve backward compatibility and handling deprecations appropriately. This work has gone through many iterations as we explored how to make the changes cleanly. Eventually the scale of the change led to breaking up the PR for closer review, which I believe was the right decision because we identified multiple issues after taking that step. That may have slowed down progress, but a lot of hours have gone into discussing, reviewing, and validating the work in this area.
> >>> >>
> >>> >> As a project, we have leaned away from gating releases on specific features because it leads to slower release cycles and prevents other features that are ready from going out. We also don't want to rush features just to hit a release target, but rather release more frequently to make changes available as they are ready.
> >>> >>
> >>> >> At this point, I believe the plan is to follow up soon with a 1.9 release.
> >>> >>
> >>> >> -Dan
> >>> >>
> >>> >> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de> wrote:
> >>> >>>
> >>> >>> Hey,
> >>> >>>
> >>> >>> IMHO 1.8 should definitely include the Auth-Manager work, which tackles
> >>> >>> actual bugs in the Iceberg code base wrt OAuth implementation. That work
> >>> >>> was originally intended to go into 1.7 and now it shall be delayed again
> >>> >>> for 1.9. The PR was originally opened in July 2024, about half a year
> >>> >>> ago and is still getting reviewed. In the meantime there were more than
> >>> >>> 600 other PRs that got reviewed and merged.
> >>> >>>
> >>> >>> The overall agreement around spring 2024, please correct me if I am
> >>> >>> wrong, was the whole REST/OAuth area needs to be improved, and the oauth
> >>> >>> endpoint removed entirely.
> >>> >>>
> >>> >>> Generally speaking, and I know I'm not alone, getting reviews from
> >>> >>> Iceberg committers is extremely hard. A lot of issues and PRs just get
> >>> >>> closed (by that stale bot) without having gotten _any_ attention from an
> >>> >>> Iceberg committer. This is not a new situation but going on for a long
> >>> >>> time. I have been talking to two Iceberg PMC members in person many
> >>> >>> months ago that this is a very disappointing situation that needs to be
> >>> >>> fixed. The reply was always "we are already working on it" - but at
> >>> >>> least from my personal POV the situation did not improve.
> >>> >>>
> >>> >>> Robert
> >>> >>>
> >>> >>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
> >>> >>> > Hi folks,
> >>> >>> >
> >>> >>> > Following the Community Meeting yesterday, I would like to propose the
> >>> >>> > following plan regarding releases:
> >>> >>> >
> >>> >>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
> >>> >>> > Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
> >>> >>> > discussed this point with Fokko at the beginning of this week, because
> >>> >>> > I have some doubts about LICENSE/NOTICE content in the "uber" jar
> >>> >>> > artifacts where we shade dependencies. I'm doing a complete pass on
> >>> >>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
> >>> >>> > complete analysis by tomorrow. This is potentially a blocker for
> >>> >>> > release votes.
> >>> >>> >
> >>> >>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will work
> >>> >>> > with Fokko on this one.
> >>> >>> >
> >>> >>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
> >>> >>> > manager). Due to still some WIP, we "revisited" the 1.8.0 release
> >>> >>> > content: for instance, as best effort, we wanted to include REST Auth
> >>> >>> > Manager improvement (OAuth2 Manager) but we preferred to postpone to
> >>> >>> > 1.9.0. That's totally fine to me, however, I would propose to strongly
> >>> >>> > focus on pending PRs for 1.9.0. Imho, we should "target" (again as
> >>> >>> > clear best effort) on variant, partition stats and Auth Manager.
> >>> >>> >
> >>> >>> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning of Feb,
> >>> >>> > according to our "release cadence", what do you think about planning
> >>> >>> > 1.9.0 in April ? Again with the main targets listed in (2).
> >>> >>> >
> >>> >>> > I tried to sum up what we discussed yesterday :)
> >>> >>> > Thoughts ?
> >>> >>> >
> >>> >>> > Regards
> >>> >>> > JB
> >>> >>> >
> >>> >>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> >>> >>> >> Hi folks,
> >>> >>> >>
> >>> >>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
> >>> >>> >> our release "pace", 1.8.0 should be released around mid February.
> >>> >>> >>
> >>> >>> >> I think we already have a good "train" of merged PRs (or should be
> >>> >>> >> merged soon): default values, REST auth improvements, dependencies
> >>> >>> >> updates, etc.
> >>> >>> >>
> >>> >>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
> >>> >>> >> and PRs we would like to "target" to 1.8.0.
> >>> >>> >>
> >>> >>> >> Thoughts ?
> >>> >>> >>
> >>> >>> >> Regards
> >>> >>> >> JB
> >>> >>>
> >>> >>> --
> >>> >>> Robert Stupp
> >>> >>> @snazy
> >>> >>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Amogh,
I found issues in the LICENSE/NOTICE from kafka-connect-runtime
distribution (what's in the distribution zip). AFAIR, we plan to
distribute this distribution, so it should be fixed.
I will open a PR about that today.
Sorry about that.
Regards
JB
On Fri, Feb 7, 2025 at 11:35 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
>
> Hey all,
>
> An update, the final License/Notice release blockers are merged (big thanks to JB, and Ryan/Fokko for helping review)! I'm in transit at the moment, but once I get to a place with stable wifi I will cut a release candidate.
>
> Thanks,
> Amogh Jahagirdar
>
> On Wed, Jan 29, 2025 at 2:23 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
>>
>> Agreed, I wouldn't be opposed to looking into approaches to make release times more predictable. At the same time, I'd advocate that in the community, that anyone can propose a release at any point in time. Of course, we can discuss as a community and make sure there's a reasonable changeset, as well as focus review time on PRs which are close to being ready for that release.
>> To some degree this contradicts having a predictable release schedule, but I feel like we can really just have a hybrid "Periodic release + arbitrary off-cycle release" approach and things won't get too crazy. It's a way to get the best of both frequency of release and user expectations on release times.
>>
>> An update on 1.8 to the community, we're working on updating LICENSE/NOTICE files in the AWS/GCP/Azure bundles, thank you JB for driving that. It's something we need to get in for the release. Once that's in, I will cut the RC.
>>
>> Thanks,
>>
>> Amogh Jahagirdar
>>
>> On Sun, Jan 26, 2025 at 1:16 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>>>
>>> Hi Amogh,
>>>
>>> Thanks !
>>>
>>> I agree we should have more frequent releases, but also more
>>> "predictable" release time and give visibility to the community
>>> (especially users).
>>> Some ASF projects are providing "tables" with release plans:
>>> - https://camel.apache.org/download/
>>> - https://karaf.apache.org/download.html
>>> - https://activemq.apache.org/components/classic/download/
>>> - ...
>>>
>>> Maybe we can provide something similar ?
>>>
>>> Thanks !
>>> Regards
>>> JB
>>>
>>> On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
>>> >
>>> > Hey all,
>>> >
>>> > Just following up here with a bit of a status update, so in the past week or so, items in the 1.8 release milestone have been closing out.
>>> > I'm aiming to cut a release next Tuesday, Jan 28.
>>> >
>>> > I'd like to reiterate that for any changes that don't make the 1.8 release, we can do a fast follow 1.9 release, and from the last community sync that seems to be the direction.
>>> > In this particular case, the 1.8 release is a bit earlier than our typical release cadence and with the 1.9 being a fast follow on, I think we're well on track.
>>> > Please add the proposed changes to the 1.9 milestone so folks can review ahead of time!
>>> >
>>> > In general, I'd encourage more frequent releases, changes which are ready can just go out and with the smaller diff it reduces the risks that exist with larger updates.
>>> >
>>> > Thanks,
>>> > Amogh Jahagirdar
>>> >
>>> > On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <dw...@apache.org> wrote:
>>> >>
>>> >> Robert,
>>> >>
>>> >> I hear your frustration with the progress on the Auth Manager work, but I believe everyone recognizes that this was a large refactor further complicated by the need to preserve backward compatibility and handling deprecations appropriately. This work has gone through many iterations as we explored how to make the changes cleanly. Eventually the scale of the change led to breaking up the PR for closer review, which I believe was the right decision because we identified multiple issues after taking that step. That may have slowed down progress, but a lot of hours have gone into discussing, reviewing, and validating the work in this area.
>>> >>
>>> >> As a project, we have leaned away from gating releases on specific features because it leads to slower release cycles and prevents other features that are ready from going out. We also don't want to rush features just to hit a release target, but rather release more frequently to make changes available as they are ready.
>>> >>
>>> >> At this point, I believe the plan is to follow up soon with a 1.9 release.
>>> >>
>>> >> -Dan
>>> >>
>>> >> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de> wrote:
>>> >>>
>>> >>> Hey,
>>> >>>
>>> >>> IMHO 1.8 should definitely include the Auth-Manager work, which tackles
>>> >>> actual bugs in the Iceberg code base wrt OAuth implementation. That work
>>> >>> was originally intended to go into 1.7 and now it shall be delayed again
>>> >>> for 1.9. The PR was originally opened in July 2024, about half a year
>>> >>> ago and is still getting reviewed. In the meantime there were more than
>>> >>> 600 other PRs that got reviewed and merged.
>>> >>>
>>> >>> The overall agreement around spring 2024, please correct me if I am
>>> >>> wrong, was the whole REST/OAuth area needs to be improved, and the oauth
>>> >>> endpoint removed entirely.
>>> >>>
>>> >>> Generally speaking, and I know I'm not alone, getting reviews from
>>> >>> Iceberg committers is extremely hard. A lot of issues and PRs just get
>>> >>> closed (by that stale bot) without having gotten _any_ attention from an
>>> >>> Iceberg committer. This is not a new situation but going on for a long
>>> >>> time. I have been talking to two Iceberg PMC members in person many
>>> >>> months ago that this is a very disappointing situation that needs to be
>>> >>> fixed. The reply was always "we are already working on it" - but at
>>> >>> least from my personal POV the situation did not improve.
>>> >>>
>>> >>> Robert
>>> >>>
>>> >>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
>>> >>> > Hi folks,
>>> >>> >
>>> >>> > Following the Community Meeting yesterday, I would like to propose the
>>> >>> > following plan regarding releases:
>>> >>> >
>>> >>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
>>> >>> > Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
>>> >>> > discussed this point with Fokko at the beginning of this week, because
>>> >>> > I have some doubts about LICENSE/NOTICE content in the "uber" jar
>>> >>> > artifacts where we shade dependencies. I'm doing a complete pass on
>>> >>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
>>> >>> > complete analysis by tomorrow. This is potentially a blocker for
>>> >>> > release votes.
>>> >>> >
>>> >>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will work
>>> >>> > with Fokko on this one.
>>> >>> >
>>> >>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
>>> >>> > manager). Due to still some WIP, we "revisited" the 1.8.0 release
>>> >>> > content: for instance, as best effort, we wanted to include REST Auth
>>> >>> > Manager improvement (OAuth2 Manager) but we preferred to postpone to
>>> >>> > 1.9.0. That's totally fine to me, however, I would propose to strongly
>>> >>> > focus on pending PRs for 1.9.0. Imho, we should "target" (again as
>>> >>> > clear best effort) on variant, partition stats and Auth Manager.
>>> >>> >
>>> >>> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning of Feb,
>>> >>> > according to our "release cadence", what do you think about planning
>>> >>> > 1.9.0 in April ? Again with the main targets listed in (2).
>>> >>> >
>>> >>> > I tried to sum up what we discussed yesterday :)
>>> >>> > Thoughts ?
>>> >>> >
>>> >>> > Regards
>>> >>> > JB
>>> >>> >
>>> >>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>>> >>> >> Hi folks,
>>> >>> >>
>>> >>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
>>> >>> >> our release "pace", 1.8.0 should be released around mid February.
>>> >>> >>
>>> >>> >> I think we already have a good "train" of merged PRs (or should be
>>> >>> >> merged soon): default values, REST auth improvements, dependencies
>>> >>> >> updates, etc.
>>> >>> >>
>>> >>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>>> >>> >> and PRs we would like to "target" to 1.8.0.
>>> >>> >>
>>> >>> >> Thoughts ?
>>> >>> >>
>>> >>> >> Regards
>>> >>> >> JB
>>> >>>
>>> >>> --
>>> >>> Robert Stupp
>>> >>> @snazy
>>> >>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Amogh Jahagirdar <2a...@gmail.com>.
Hey all,
An update, the final License/Notice release blockers are merged (big thanks
to JB, and Ryan/Fokko for helping review)! I'm in transit at the moment,
but once I get to a place with stable wifi I will cut a release candidate.
Thanks,
Amogh Jahagirdar
On Wed, Jan 29, 2025 at 2:23 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
> Agreed, I wouldn't be opposed to looking into approaches to make release
> times more predictable. At the same time, I'd advocate that in the
> community, that anyone can propose a release at any point in time. Of
> course, we can discuss as a community and make sure there's a reasonable
> changeset, as well as focus review time on PRs which are close to being
> ready for that release.
> To some degree this contradicts having a predictable release schedule, but
> I feel like we can really just have a hybrid "Periodic release + arbitrary
> off-cycle release" approach and things won't get too crazy. It's a way to
> get the best of both frequency of release and user expectations on release
> times.
>
> An update on 1.8 to the community, we're working on updating
> LICENSE/NOTICE files in the AWS/GCP/Azure bundles
> <https://github.com/apache/iceberg/pull/12095/>, thank you JB for driving
> that. It's something we need to get in for the release. Once that's in, I
> will cut the RC.
>
> Thanks,
>
> Amogh Jahagirdar
>
> On Sun, Jan 26, 2025 at 1:16 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi Amogh,
>>
>> Thanks !
>>
>> I agree we should have more frequent releases, but also more
>> "predictable" release time and give visibility to the community
>> (especially users).
>> Some ASF projects are providing "tables" with release plans:
>> - https://camel.apache.org/download/
>> - https://karaf.apache.org/download.html
>> - https://activemq.apache.org/components/classic/download/
>> - ...
>>
>> Maybe we can provide something similar ?
>>
>> Thanks !
>> Regards
>> JB
>>
>> On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <2a...@gmail.com>
>> wrote:
>> >
>> > Hey all,
>> >
>> > Just following up here with a bit of a status update, so in the past
>> week or so, items in the 1.8 release milestone have been closing out.
>> > I'm aiming to cut a release next Tuesday, Jan 28.
>> >
>> > I'd like to reiterate that for any changes that don't make the 1.8
>> release, we can do a fast follow 1.9 release, and from the last community
>> sync that seems to be the direction.
>> > In this particular case, the 1.8 release is a bit earlier than our
>> typical release cadence and with the 1.9 being a fast follow on, I think
>> we're well on track.
>> > Please add the proposed changes to the 1.9 milestone so folks can
>> review ahead of time!
>> >
>> > In general, I'd encourage more frequent releases, changes which are
>> ready can just go out and with the smaller diff it reduces the risks that
>> exist with larger updates.
>> >
>> > Thanks,
>> > Amogh Jahagirdar
>> >
>> > On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <dw...@apache.org>
>> wrote:
>> >>
>> >> Robert,
>> >>
>> >> I hear your frustration with the progress on the Auth Manager work,
>> but I believe everyone recognizes that this was a large refactor further
>> complicated by the need to preserve backward compatibility and handling
>> deprecations appropriately. This work has gone through many iterations as
>> we explored how to make the changes cleanly. Eventually the scale of the
>> change led to breaking up the PR for closer review, which I believe was the
>> right decision because we identified multiple issues after taking that
>> step. That may have slowed down progress, but a lot of hours have gone
>> into discussing, reviewing, and validating the work in this area.
>> >>
>> >> As a project, we have leaned away from gating releases on specific
>> features because it leads to slower release cycles and prevents other
>> features that are ready from going out. We also don't want to rush
>> features just to hit a release target, but rather release more frequently
>> to make changes available as they are ready.
>> >>
>> >> At this point, I believe the plan is to follow up soon with a 1.9
>> release.
>> >>
>> >> -Dan
>> >>
>> >> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de> wrote:
>> >>>
>> >>> Hey,
>> >>>
>> >>> IMHO 1.8 should definitely include the Auth-Manager work, which
>> tackles
>> >>> actual bugs in the Iceberg code base wrt OAuth implementation. That
>> work
>> >>> was originally intended to go into 1.7 and now it shall be delayed
>> again
>> >>> for 1.9. The PR was originally opened in July 2024, about half a year
>> >>> ago and is still getting reviewed. In the meantime there were more
>> than
>> >>> 600 other PRs that got reviewed and merged.
>> >>>
>> >>> The overall agreement around spring 2024, please correct me if I am
>> >>> wrong, was the whole REST/OAuth area needs to be improved, and the
>> oauth
>> >>> endpoint removed entirely.
>> >>>
>> >>> Generally speaking, and I know I'm not alone, getting reviews from
>> >>> Iceberg committers is extremely hard. A lot of issues and PRs just get
>> >>> closed (by that stale bot) without having gotten _any_ attention from
>> an
>> >>> Iceberg committer. This is not a new situation but going on for a long
>> >>> time. I have been talking to two Iceberg PMC members in person many
>> >>> months ago that this is a very disappointing situation that needs to
>> be
>> >>> fixed. The reply was always "we are already working on it" - but at
>> >>> least from my personal POV the situation did not improve.
>> >>>
>> >>> Robert
>> >>>
>> >>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
>> >>> > Hi folks,
>> >>> >
>> >>> > Following the Community Meeting yesterday, I would like to propose
>> the
>> >>> > following plan regarding releases:
>> >>> >
>> >>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said
>> by
>> >>> > Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
>> >>> > discussed this point with Fokko at the beginning of this week,
>> because
>> >>> > I have some doubts about LICENSE/NOTICE content in the "uber" jar
>> >>> > artifacts where we shade dependencies. I'm doing a complete pass on
>> >>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
>> >>> > complete analysis by tomorrow. This is potentially a blocker for
>> >>> > release votes.
>> >>> >
>> >>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will
>> work
>> >>> > with Fokko on this one.
>> >>> >
>> >>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
>> >>> > manager). Due to still some WIP, we "revisited" the 1.8.0 release
>> >>> > content: for instance, as best effort, we wanted to include REST
>> Auth
>> >>> > Manager improvement (OAuth2 Manager) but we preferred to postpone to
>> >>> > 1.9.0. That's totally fine to me, however, I would propose to
>> strongly
>> >>> > focus on pending PRs for 1.9.0. Imho, we should "target" (again as
>> >>> > clear best effort) on variant, partition stats and Auth Manager.
>> >>> >
>> >>> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning of
>> Feb,
>> >>> > according to our "release cadence", what do you think about planning
>> >>> > 1.9.0 in April ? Again with the main targets listed in (2).
>> >>> >
>> >>> > I tried to sum up what we discussed yesterday :)
>> >>> > Thoughts ?
>> >>> >
>> >>> > Regards
>> >>> > JB
>> >>> >
>> >>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <
>> jb@nanthrax.net> wrote:
>> >>> >> Hi folks,
>> >>> >>
>> >>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to
>> keep
>> >>> >> our release "pace", 1.8.0 should be released around mid February.
>> >>> >>
>> >>> >> I think we already have a good "train" of merged PRs (or should be
>> >>> >> merged soon): default values, REST auth improvements, dependencies
>> >>> >> updates, etc.
>> >>> >>
>> >>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>> >>> >> and PRs we would like to "target" to 1.8.0.
>> >>> >>
>> >>> >> Thoughts ?
>> >>> >>
>> >>> >> Regards
>> >>> >> JB
>> >>>
>> >>> --
>> >>> Robert Stupp
>> >>> @snazy
>> >>>
>>
>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Amogh Jahagirdar <2a...@gmail.com>.
Agreed, I wouldn't be opposed to looking into approaches to make release
times more predictable. At the same time, I'd advocate that in the
community, that anyone can propose a release at any point in time. Of
course, we can discuss as a community and make sure there's a reasonable
changeset, as well as focus review time on PRs which are close to being
ready for that release.
To some degree this contradicts having a predictable release schedule, but
I feel like we can really just have a hybrid "Periodic release + arbitrary
off-cycle release" approach and things won't get too crazy. It's a way to
get the best of both frequency of release and user expectations on release
times.
An update on 1.8 to the community, we're working on updating LICENSE/NOTICE
files in the AWS/GCP/Azure bundles
<https://github.com/apache/iceberg/pull/12095/>, thank you JB for driving
that. It's something we need to get in for the release. Once that's in, I
will cut the RC.
Thanks,
Amogh Jahagirdar
On Sun, Jan 26, 2025 at 1:16 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi Amogh,
>
> Thanks !
>
> I agree we should have more frequent releases, but also more
> "predictable" release time and give visibility to the community
> (especially users).
> Some ASF projects are providing "tables" with release plans:
> - https://camel.apache.org/download/
> - https://karaf.apache.org/download.html
> - https://activemq.apache.org/components/classic/download/
> - ...
>
> Maybe we can provide something similar ?
>
> Thanks !
> Regards
> JB
>
> On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
> >
> > Hey all,
> >
> > Just following up here with a bit of a status update, so in the past
> week or so, items in the 1.8 release milestone have been closing out.
> > I'm aiming to cut a release next Tuesday, Jan 28.
> >
> > I'd like to reiterate that for any changes that don't make the 1.8
> release, we can do a fast follow 1.9 release, and from the last community
> sync that seems to be the direction.
> > In this particular case, the 1.8 release is a bit earlier than our
> typical release cadence and with the 1.9 being a fast follow on, I think
> we're well on track.
> > Please add the proposed changes to the 1.9 milestone so folks can review
> ahead of time!
> >
> > In general, I'd encourage more frequent releases, changes which are
> ready can just go out and with the smaller diff it reduces the risks that
> exist with larger updates.
> >
> > Thanks,
> > Amogh Jahagirdar
> >
> > On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <dw...@apache.org> wrote:
> >>
> >> Robert,
> >>
> >> I hear your frustration with the progress on the Auth Manager work, but
> I believe everyone recognizes that this was a large refactor further
> complicated by the need to preserve backward compatibility and handling
> deprecations appropriately. This work has gone through many iterations as
> we explored how to make the changes cleanly. Eventually the scale of the
> change led to breaking up the PR for closer review, which I believe was the
> right decision because we identified multiple issues after taking that
> step. That may have slowed down progress, but a lot of hours have gone
> into discussing, reviewing, and validating the work in this area.
> >>
> >> As a project, we have leaned away from gating releases on specific
> features because it leads to slower release cycles and prevents other
> features that are ready from going out. We also don't want to rush
> features just to hit a release target, but rather release more frequently
> to make changes available as they are ready.
> >>
> >> At this point, I believe the plan is to follow up soon with a 1.9
> release.
> >>
> >> -Dan
> >>
> >> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de> wrote:
> >>>
> >>> Hey,
> >>>
> >>> IMHO 1.8 should definitely include the Auth-Manager work, which tackles
> >>> actual bugs in the Iceberg code base wrt OAuth implementation. That
> work
> >>> was originally intended to go into 1.7 and now it shall be delayed
> again
> >>> for 1.9. The PR was originally opened in July 2024, about half a year
> >>> ago and is still getting reviewed. In the meantime there were more than
> >>> 600 other PRs that got reviewed and merged.
> >>>
> >>> The overall agreement around spring 2024, please correct me if I am
> >>> wrong, was the whole REST/OAuth area needs to be improved, and the
> oauth
> >>> endpoint removed entirely.
> >>>
> >>> Generally speaking, and I know I'm not alone, getting reviews from
> >>> Iceberg committers is extremely hard. A lot of issues and PRs just get
> >>> closed (by that stale bot) without having gotten _any_ attention from
> an
> >>> Iceberg committer. This is not a new situation but going on for a long
> >>> time. I have been talking to two Iceberg PMC members in person many
> >>> months ago that this is a very disappointing situation that needs to be
> >>> fixed. The reply was always "we are already working on it" - but at
> >>> least from my personal POV the situation did not improve.
> >>>
> >>> Robert
> >>>
> >>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
> >>> > Hi folks,
> >>> >
> >>> > Following the Community Meeting yesterday, I would like to propose
> the
> >>> > following plan regarding releases:
> >>> >
> >>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
> >>> > Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
> >>> > discussed this point with Fokko at the beginning of this week,
> because
> >>> > I have some doubts about LICENSE/NOTICE content in the "uber" jar
> >>> > artifacts where we shade dependencies. I'm doing a complete pass on
> >>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
> >>> > complete analysis by tomorrow. This is potentially a blocker for
> >>> > release votes.
> >>> >
> >>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will
> work
> >>> > with Fokko on this one.
> >>> >
> >>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
> >>> > manager). Due to still some WIP, we "revisited" the 1.8.0 release
> >>> > content: for instance, as best effort, we wanted to include REST Auth
> >>> > Manager improvement (OAuth2 Manager) but we preferred to postpone to
> >>> > 1.9.0. That's totally fine to me, however, I would propose to
> strongly
> >>> > focus on pending PRs for 1.9.0. Imho, we should "target" (again as
> >>> > clear best effort) on variant, partition stats and Auth Manager.
> >>> >
> >>> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning of
> Feb,
> >>> > according to our "release cadence", what do you think about planning
> >>> > 1.9.0 in April ? Again with the main targets listed in (2).
> >>> >
> >>> > I tried to sum up what we discussed yesterday :)
> >>> > Thoughts ?
> >>> >
> >>> > Regards
> >>> > JB
> >>> >
> >>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >>> >> Hi folks,
> >>> >>
> >>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to
> keep
> >>> >> our release "pace", 1.8.0 should be released around mid February.
> >>> >>
> >>> >> I think we already have a good "train" of merged PRs (or should be
> >>> >> merged soon): default values, REST auth improvements, dependencies
> >>> >> updates, etc.
> >>> >>
> >>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
> >>> >> and PRs we would like to "target" to 1.8.0.
> >>> >>
> >>> >> Thoughts ?
> >>> >>
> >>> >> Regards
> >>> >> JB
> >>>
> >>> --
> >>> Robert Stupp
> >>> @snazy
> >>>
>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Amogh,
Thanks !
I agree we should have more frequent releases, but also more
"predictable" release time and give visibility to the community
(especially users).
Some ASF projects are providing "tables" with release plans:
- https://camel.apache.org/download/
- https://karaf.apache.org/download.html
- https://activemq.apache.org/components/classic/download/
- ...
Maybe we can provide something similar ?
Thanks !
Regards
JB
On Sat, Jan 25, 2025 at 1:07 AM Amogh Jahagirdar <2a...@gmail.com> wrote:
>
> Hey all,
>
> Just following up here with a bit of a status update, so in the past week or so, items in the 1.8 release milestone have been closing out.
> I'm aiming to cut a release next Tuesday, Jan 28.
>
> I'd like to reiterate that for any changes that don't make the 1.8 release, we can do a fast follow 1.9 release, and from the last community sync that seems to be the direction.
> In this particular case, the 1.8 release is a bit earlier than our typical release cadence and with the 1.9 being a fast follow on, I think we're well on track.
> Please add the proposed changes to the 1.9 milestone so folks can review ahead of time!
>
> In general, I'd encourage more frequent releases, changes which are ready can just go out and with the smaller diff it reduces the risks that exist with larger updates.
>
> Thanks,
> Amogh Jahagirdar
>
> On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <dw...@apache.org> wrote:
>>
>> Robert,
>>
>> I hear your frustration with the progress on the Auth Manager work, but I believe everyone recognizes that this was a large refactor further complicated by the need to preserve backward compatibility and handling deprecations appropriately. This work has gone through many iterations as we explored how to make the changes cleanly. Eventually the scale of the change led to breaking up the PR for closer review, which I believe was the right decision because we identified multiple issues after taking that step. That may have slowed down progress, but a lot of hours have gone into discussing, reviewing, and validating the work in this area.
>>
>> As a project, we have leaned away from gating releases on specific features because it leads to slower release cycles and prevents other features that are ready from going out. We also don't want to rush features just to hit a release target, but rather release more frequently to make changes available as they are ready.
>>
>> At this point, I believe the plan is to follow up soon with a 1.9 release.
>>
>> -Dan
>>
>> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de> wrote:
>>>
>>> Hey,
>>>
>>> IMHO 1.8 should definitely include the Auth-Manager work, which tackles
>>> actual bugs in the Iceberg code base wrt OAuth implementation. That work
>>> was originally intended to go into 1.7 and now it shall be delayed again
>>> for 1.9. The PR was originally opened in July 2024, about half a year
>>> ago and is still getting reviewed. In the meantime there were more than
>>> 600 other PRs that got reviewed and merged.
>>>
>>> The overall agreement around spring 2024, please correct me if I am
>>> wrong, was the whole REST/OAuth area needs to be improved, and the oauth
>>> endpoint removed entirely.
>>>
>>> Generally speaking, and I know I'm not alone, getting reviews from
>>> Iceberg committers is extremely hard. A lot of issues and PRs just get
>>> closed (by that stale bot) without having gotten _any_ attention from an
>>> Iceberg committer. This is not a new situation but going on for a long
>>> time. I have been talking to two Iceberg PMC members in person many
>>> months ago that this is a very disappointing situation that needs to be
>>> fixed. The reply was always "we are already working on it" - but at
>>> least from my personal POV the situation did not improve.
>>>
>>> Robert
>>>
>>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
>>> > Hi folks,
>>> >
>>> > Following the Community Meeting yesterday, I would like to propose the
>>> > following plan regarding releases:
>>> >
>>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
>>> > Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
>>> > discussed this point with Fokko at the beginning of this week, because
>>> > I have some doubts about LICENSE/NOTICE content in the "uber" jar
>>> > artifacts where we shade dependencies. I'm doing a complete pass on
>>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
>>> > complete analysis by tomorrow. This is potentially a blocker for
>>> > release votes.
>>> >
>>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will work
>>> > with Fokko on this one.
>>> >
>>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
>>> > manager). Due to still some WIP, we "revisited" the 1.8.0 release
>>> > content: for instance, as best effort, we wanted to include REST Auth
>>> > Manager improvement (OAuth2 Manager) but we preferred to postpone to
>>> > 1.9.0. That's totally fine to me, however, I would propose to strongly
>>> > focus on pending PRs for 1.9.0. Imho, we should "target" (again as
>>> > clear best effort) on variant, partition stats and Auth Manager.
>>> >
>>> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning of Feb,
>>> > according to our "release cadence", what do you think about planning
>>> > 1.9.0 in April ? Again with the main targets listed in (2).
>>> >
>>> > I tried to sum up what we discussed yesterday :)
>>> > Thoughts ?
>>> >
>>> > Regards
>>> > JB
>>> >
>>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>>> >> Hi folks,
>>> >>
>>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
>>> >> our release "pace", 1.8.0 should be released around mid February.
>>> >>
>>> >> I think we already have a good "train" of merged PRs (or should be
>>> >> merged soon): default values, REST auth improvements, dependencies
>>> >> updates, etc.
>>> >>
>>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>>> >> and PRs we would like to "target" to 1.8.0.
>>> >>
>>> >> Thoughts ?
>>> >>
>>> >> Regards
>>> >> JB
>>>
>>> --
>>> Robert Stupp
>>> @snazy
>>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Fokko Driesprong <fo...@apache.org>.
Added!
+1 on the more frequent releases!
Kind regards,
Fokko
Op za 25 jan 2025 om 02:01 schreef Dmitri Bourlatchkov <di...@apache.org>:
> Thanks for the clarification, Amogh! Much appreciated!
>
> Would you mind adding these PRs to the 1.9 milestone, please?
>
> https://github.com/apache/iceberg/pull/11992
> https://github.com/apache/iceberg/pull/11995
>
> Thanks,
> Dmitri.
>
>
> On Fri, Jan 24, 2025 at 7:07 PM Amogh Jahagirdar <2a...@gmail.com> wrote:
>
>> Hey all,
>>
>> Just following up here with a bit of a status update, so in the past week
>> or so, items in the 1.8 release milestone
>> <https://github.com/apache/iceberg/milestone/50> have been closing out.
>> I'm aiming to cut a release next Tuesday, Jan 28.
>>
>> I'd like to reiterate that for any changes that don't make the 1.8
>> release, we can do a fast follow 1.9 release, and from the last community
>> sync that seems to be the direction.
>> In this particular case, the 1.8 release is a bit earlier than our
>> typical release cadence and with the 1.9 being a fast follow on, I think
>> we're well on track.
>> Please add the proposed changes to the 1.9 milestone
>> <https://github.com/apache/iceberg/milestone/53> so folks can review
>> ahead of time!
>>
>> In general, I'd encourage more frequent releases, changes which are ready
>> can just go out and with the smaller diff it reduces the risks that exist
>> with larger updates.
>>
>> Thanks,
>> Amogh Jahagirdar
>>
>> On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <dw...@apache.org> wrote:
>>
>>> Robert,
>>>
>>> I hear your frustration with the progress on the Auth Manager work, but
>>> I believe everyone recognizes that this was a large refactor further
>>> complicated by the need to preserve backward compatibility and handling
>>> deprecations appropriately. This work has gone through many iterations as
>>> we explored how to make the changes cleanly. Eventually the scale of the
>>> change led to breaking up the PR for closer review, which I believe was the
>>> right decision because we identified multiple issues after taking that
>>> step. That may have slowed down progress, but a lot of hours have gone
>>> into discussing, reviewing, and validating the work in this area.
>>>
>>> As a project, we have leaned away from gating releases on specific
>>> features because it leads to slower release cycles and prevents other
>>> features that are ready from going out. We also don't want to rush
>>> features just to hit a release target, but rather release more frequently
>>> to make changes available as they are ready.
>>>
>>> At this point, I believe the plan is to follow up soon with a 1.9
>>> release.
>>>
>>> -Dan
>>>
>>> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de> wrote:
>>>
>>>> Hey,
>>>>
>>>> IMHO 1.8 should definitely include the Auth-Manager work, which tackles
>>>> actual bugs in the Iceberg code base wrt OAuth implementation. That
>>>> work
>>>> was originally intended to go into 1.7 and now it shall be delayed
>>>> again
>>>> for 1.9. The PR was originally opened in July 2024, about half a year
>>>> ago and is still getting reviewed. In the meantime there were more than
>>>> 600 other PRs that got reviewed and merged.
>>>>
>>>> The overall agreement around spring 2024, please correct me if I am
>>>> wrong, was the whole REST/OAuth area needs to be improved, and the
>>>> oauth
>>>> endpoint removed entirely.
>>>>
>>>> Generally speaking, and I know I'm not alone, getting reviews from
>>>> Iceberg committers is extremely hard. A lot of issues and PRs just get
>>>> closed (by that stale bot) without having gotten _any_ attention from
>>>> an
>>>> Iceberg committer. This is not a new situation but going on for a long
>>>> time. I have been talking to two Iceberg PMC members in person many
>>>> months ago that this is a very disappointing situation that needs to be
>>>> fixed. The reply was always "we are already working on it" - but at
>>>> least from my personal POV the situation did not improve.
>>>>
>>>> Robert
>>>>
>>>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
>>>> > Hi folks,
>>>> >
>>>> > Following the Community Meeting yesterday, I would like to propose the
>>>> > following plan regarding releases:
>>>> >
>>>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
>>>> > Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
>>>> > discussed this point with Fokko at the beginning of this week, because
>>>> > I have some doubts about LICENSE/NOTICE content in the "uber" jar
>>>> > artifacts where we shade dependencies. I'm doing a complete pass on
>>>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
>>>> > complete analysis by tomorrow. This is potentially a blocker for
>>>> > release votes.
>>>> >
>>>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will work
>>>> > with Fokko on this one.
>>>> >
>>>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
>>>> > manager). Due to still some WIP, we "revisited" the 1.8.0 release
>>>> > content: for instance, as best effort, we wanted to include REST Auth
>>>> > Manager improvement (OAuth2 Manager) but we preferred to postpone to
>>>> > 1.9.0. That's totally fine to me, however, I would propose to strongly
>>>> > focus on pending PRs for 1.9.0. Imho, we should "target" (again as
>>>> > clear best effort) on variant, partition stats and Auth Manager.
>>>> >
>>>> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning of Feb,
>>>> > according to our "release cadence", what do you think about planning
>>>> > 1.9.0 in April ? Again with the main targets listed in (2).
>>>> >
>>>> > I tried to sum up what we discussed yesterday :)
>>>> > Thoughts ?
>>>> >
>>>> > Regards
>>>> > JB
>>>> >
>>>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>> wrote:
>>>> >> Hi folks,
>>>> >>
>>>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to
>>>> keep
>>>> >> our release "pace", 1.8.0 should be released around mid February.
>>>> >>
>>>> >> I think we already have a good "train" of merged PRs (or should be
>>>> >> merged soon): default values, REST auth improvements, dependencies
>>>> >> updates, etc.
>>>> >>
>>>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>>>> >> and PRs we would like to "target" to 1.8.0.
>>>> >>
>>>> >> Thoughts ?
>>>> >>
>>>> >> Regards
>>>> >> JB
>>>>
>>>> --
>>>> Robert Stupp
>>>> @snazy
>>>>
>>>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Dmitri Bourlatchkov <di...@apache.org>.
Thanks for the clarification, Amogh! Much appreciated!
Would you mind adding these PRs to the 1.9 milestone, please?
https://github.com/apache/iceberg/pull/11992
https://github.com/apache/iceberg/pull/11995
Thanks,
Dmitri.
On Fri, Jan 24, 2025 at 7:07 PM Amogh Jahagirdar <2a...@gmail.com> wrote:
> Hey all,
>
> Just following up here with a bit of a status update, so in the past week
> or so, items in the 1.8 release milestone
> <https://github.com/apache/iceberg/milestone/50> have been closing out.
> I'm aiming to cut a release next Tuesday, Jan 28.
>
> I'd like to reiterate that for any changes that don't make the 1.8
> release, we can do a fast follow 1.9 release, and from the last community
> sync that seems to be the direction.
> In this particular case, the 1.8 release is a bit earlier than our typical
> release cadence and with the 1.9 being a fast follow on, I think we're well
> on track.
> Please add the proposed changes to the 1.9 milestone
> <https://github.com/apache/iceberg/milestone/53> so folks can review
> ahead of time!
>
> In general, I'd encourage more frequent releases, changes which are ready
> can just go out and with the smaller diff it reduces the risks that exist
> with larger updates.
>
> Thanks,
> Amogh Jahagirdar
>
> On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <dw...@apache.org> wrote:
>
>> Robert,
>>
>> I hear your frustration with the progress on the Auth Manager work, but I
>> believe everyone recognizes that this was a large refactor further
>> complicated by the need to preserve backward compatibility and handling
>> deprecations appropriately. This work has gone through many iterations as
>> we explored how to make the changes cleanly. Eventually the scale of the
>> change led to breaking up the PR for closer review, which I believe was the
>> right decision because we identified multiple issues after taking that
>> step. That may have slowed down progress, but a lot of hours have gone
>> into discussing, reviewing, and validating the work in this area.
>>
>> As a project, we have leaned away from gating releases on specific
>> features because it leads to slower release cycles and prevents other
>> features that are ready from going out. We also don't want to rush
>> features just to hit a release target, but rather release more frequently
>> to make changes available as they are ready.
>>
>> At this point, I believe the plan is to follow up soon with a 1.9 release.
>>
>> -Dan
>>
>> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de> wrote:
>>
>>> Hey,
>>>
>>> IMHO 1.8 should definitely include the Auth-Manager work, which tackles
>>> actual bugs in the Iceberg code base wrt OAuth implementation. That work
>>> was originally intended to go into 1.7 and now it shall be delayed again
>>> for 1.9. The PR was originally opened in July 2024, about half a year
>>> ago and is still getting reviewed. In the meantime there were more than
>>> 600 other PRs that got reviewed and merged.
>>>
>>> The overall agreement around spring 2024, please correct me if I am
>>> wrong, was the whole REST/OAuth area needs to be improved, and the oauth
>>> endpoint removed entirely.
>>>
>>> Generally speaking, and I know I'm not alone, getting reviews from
>>> Iceberg committers is extremely hard. A lot of issues and PRs just get
>>> closed (by that stale bot) without having gotten _any_ attention from an
>>> Iceberg committer. This is not a new situation but going on for a long
>>> time. I have been talking to two Iceberg PMC members in person many
>>> months ago that this is a very disappointing situation that needs to be
>>> fixed. The reply was always "we are already working on it" - but at
>>> least from my personal POV the situation did not improve.
>>>
>>> Robert
>>>
>>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
>>> > Hi folks,
>>> >
>>> > Following the Community Meeting yesterday, I would like to propose the
>>> > following plan regarding releases:
>>> >
>>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
>>> > Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
>>> > discussed this point with Fokko at the beginning of this week, because
>>> > I have some doubts about LICENSE/NOTICE content in the "uber" jar
>>> > artifacts where we shade dependencies. I'm doing a complete pass on
>>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
>>> > complete analysis by tomorrow. This is potentially a blocker for
>>> > release votes.
>>> >
>>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will work
>>> > with Fokko on this one.
>>> >
>>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
>>> > manager). Due to still some WIP, we "revisited" the 1.8.0 release
>>> > content: for instance, as best effort, we wanted to include REST Auth
>>> > Manager improvement (OAuth2 Manager) but we preferred to postpone to
>>> > 1.9.0. That's totally fine to me, however, I would propose to strongly
>>> > focus on pending PRs for 1.9.0. Imho, we should "target" (again as
>>> > clear best effort) on variant, partition stats and Auth Manager.
>>> >
>>> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning of Feb,
>>> > according to our "release cadence", what do you think about planning
>>> > 1.9.0 in April ? Again with the main targets listed in (2).
>>> >
>>> > I tried to sum up what we discussed yesterday :)
>>> > Thoughts ?
>>> >
>>> > Regards
>>> > JB
>>> >
>>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>> >> Hi folks,
>>> >>
>>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
>>> >> our release "pace", 1.8.0 should be released around mid February.
>>> >>
>>> >> I think we already have a good "train" of merged PRs (or should be
>>> >> merged soon): default values, REST auth improvements, dependencies
>>> >> updates, etc.
>>> >>
>>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>>> >> and PRs we would like to "target" to 1.8.0.
>>> >>
>>> >> Thoughts ?
>>> >>
>>> >> Regards
>>> >> JB
>>>
>>> --
>>> Robert Stupp
>>> @snazy
>>>
>>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Amogh Jahagirdar <2a...@gmail.com>.
Hey all,
Just following up here with a bit of a status update, so in the past week
or so, items in the 1.8 release milestone
<https://github.com/apache/iceberg/milestone/50> have been closing out.
I'm aiming to cut a release next Tuesday, Jan 28.
I'd like to reiterate that for any changes that don't make the 1.8 release,
we can do a fast follow 1.9 release, and from the last community sync that
seems to be the direction.
In this particular case, the 1.8 release is a bit earlier than our typical
release cadence and with the 1.9 being a fast follow on, I think we're well
on track.
Please add the proposed changes to the 1.9 milestone
<https://github.com/apache/iceberg/milestone/53> so folks can review ahead
of time!
In general, I'd encourage more frequent releases, changes which are ready
can just go out and with the smaller diff it reduces the risks that exist
with larger updates.
Thanks,
Amogh Jahagirdar
On Thu, Jan 16, 2025 at 10:05 AM Daniel Weeks <dw...@apache.org> wrote:
> Robert,
>
> I hear your frustration with the progress on the Auth Manager work, but I
> believe everyone recognizes that this was a large refactor further
> complicated by the need to preserve backward compatibility and handling
> deprecations appropriately. This work has gone through many iterations as
> we explored how to make the changes cleanly. Eventually the scale of the
> change led to breaking up the PR for closer review, which I believe was the
> right decision because we identified multiple issues after taking that
> step. That may have slowed down progress, but a lot of hours have gone
> into discussing, reviewing, and validating the work in this area.
>
> As a project, we have leaned away from gating releases on specific
> features because it leads to slower release cycles and prevents other
> features that are ready from going out. We also don't want to rush
> features just to hit a release target, but rather release more frequently
> to make changes available as they are ready.
>
> At this point, I believe the plan is to follow up soon with a 1.9 release.
>
> -Dan
>
> On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de> wrote:
>
>> Hey,
>>
>> IMHO 1.8 should definitely include the Auth-Manager work, which tackles
>> actual bugs in the Iceberg code base wrt OAuth implementation. That work
>> was originally intended to go into 1.7 and now it shall be delayed again
>> for 1.9. The PR was originally opened in July 2024, about half a year
>> ago and is still getting reviewed. In the meantime there were more than
>> 600 other PRs that got reviewed and merged.
>>
>> The overall agreement around spring 2024, please correct me if I am
>> wrong, was the whole REST/OAuth area needs to be improved, and the oauth
>> endpoint removed entirely.
>>
>> Generally speaking, and I know I'm not alone, getting reviews from
>> Iceberg committers is extremely hard. A lot of issues and PRs just get
>> closed (by that stale bot) without having gotten _any_ attention from an
>> Iceberg committer. This is not a new situation but going on for a long
>> time. I have been talking to two Iceberg PMC members in person many
>> months ago that this is a very disappointing situation that needs to be
>> fixed. The reply was always "we are already working on it" - but at
>> least from my personal POV the situation did not improve.
>>
>> Robert
>>
>> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
>> > Hi folks,
>> >
>> > Following the Community Meeting yesterday, I would like to propose the
>> > following plan regarding releases:
>> >
>> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
>> > Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
>> > discussed this point with Fokko at the beginning of this week, because
>> > I have some doubts about LICENSE/NOTICE content in the "uber" jar
>> > artifacts where we shade dependencies. I'm doing a complete pass on
>> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
>> > complete analysis by tomorrow. This is potentially a blocker for
>> > release votes.
>> >
>> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will work
>> > with Fokko on this one.
>> >
>> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
>> > manager). Due to still some WIP, we "revisited" the 1.8.0 release
>> > content: for instance, as best effort, we wanted to include REST Auth
>> > Manager improvement (OAuth2 Manager) but we preferred to postpone to
>> > 1.9.0. That's totally fine to me, however, I would propose to strongly
>> > focus on pending PRs for 1.9.0. Imho, we should "target" (again as
>> > clear best effort) on variant, partition stats and Auth Manager.
>> >
>> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning of Feb,
>> > according to our "release cadence", what do you think about planning
>> > 1.9.0 in April ? Again with the main targets listed in (2).
>> >
>> > I tried to sum up what we discussed yesterday :)
>> > Thoughts ?
>> >
>> > Regards
>> > JB
>> >
>> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>> >> Hi folks,
>> >>
>> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
>> >> our release "pace", 1.8.0 should be released around mid February.
>> >>
>> >> I think we already have a good "train" of merged PRs (or should be
>> >> merged soon): default values, REST auth improvements, dependencies
>> >> updates, etc.
>> >>
>> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>> >> and PRs we would like to "target" to 1.8.0.
>> >>
>> >> Thoughts ?
>> >>
>> >> Regards
>> >> JB
>>
>> --
>> Robert Stupp
>> @snazy
>>
>>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Daniel Weeks <dw...@apache.org>.
Robert,
I hear your frustration with the progress on the Auth Manager work, but I
believe everyone recognizes that this was a large refactor further
complicated by the need to preserve backward compatibility and handling
deprecations appropriately. This work has gone through many iterations as
we explored how to make the changes cleanly. Eventually the scale of the
change led to breaking up the PR for closer review, which I believe was the
right decision because we identified multiple issues after taking that
step. That may have slowed down progress, but a lot of hours have gone
into discussing, reviewing, and validating the work in this area.
As a project, we have leaned away from gating releases on specific features
because it leads to slower release cycles and prevents other features that
are ready from going out. We also don't want to rush features just to hit
a release target, but rather release more frequently to make changes
available as they are ready.
At this point, I believe the plan is to follow up soon with a 1.9 release.
-Dan
On Thu, Jan 16, 2025 at 2:36 AM Robert Stupp <sn...@snazy.de> wrote:
> Hey,
>
> IMHO 1.8 should definitely include the Auth-Manager work, which tackles
> actual bugs in the Iceberg code base wrt OAuth implementation. That work
> was originally intended to go into 1.7 and now it shall be delayed again
> for 1.9. The PR was originally opened in July 2024, about half a year
> ago and is still getting reviewed. In the meantime there were more than
> 600 other PRs that got reviewed and merged.
>
> The overall agreement around spring 2024, please correct me if I am
> wrong, was the whole REST/OAuth area needs to be improved, and the oauth
> endpoint removed entirely.
>
> Generally speaking, and I know I'm not alone, getting reviews from
> Iceberg committers is extremely hard. A lot of issues and PRs just get
> closed (by that stale bot) without having gotten _any_ attention from an
> Iceberg committer. This is not a new situation but going on for a long
> time. I have been talking to two Iceberg PMC members in person many
> months ago that this is a very disappointing situation that needs to be
> fixed. The reply was always "we are already working on it" - but at
> least from my personal POV the situation did not improve.
>
> Robert
>
> On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
> > Hi folks,
> >
> > Following the Community Meeting yesterday, I would like to propose the
> > following plan regarding releases:
> >
> > 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
> > Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
> > discussed this point with Fokko at the beginning of this week, because
> > I have some doubts about LICENSE/NOTICE content in the "uber" jar
> > artifacts where we shade dependencies. I'm doing a complete pass on
> > all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
> > complete analysis by tomorrow. This is potentially a blocker for
> > release votes.
> >
> > 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will work
> > with Fokko on this one.
> >
> > 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
> > manager). Due to still some WIP, we "revisited" the 1.8.0 release
> > content: for instance, as best effort, we wanted to include REST Auth
> > Manager improvement (OAuth2 Manager) but we preferred to postpone to
> > 1.9.0. That's totally fine to me, however, I would propose to strongly
> > focus on pending PRs for 1.9.0. Imho, we should "target" (again as
> > clear best effort) on variant, partition stats and Auth Manager.
> >
> > 3. Assuming 1.8.0 will be released at the end of Jan/beginning of Feb,
> > according to our "release cadence", what do you think about planning
> > 1.9.0 in April ? Again with the main targets listed in (2).
> >
> > I tried to sum up what we discussed yesterday :)
> > Thoughts ?
> >
> > Regards
> > JB
> >
> > On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >> Hi folks,
> >>
> >> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
> >> our release "pace", 1.8.0 should be released around mid February.
> >>
> >> I think we already have a good "train" of merged PRs (or should be
> >> merged soon): default values, REST auth improvements, dependencies
> >> updates, etc.
> >>
> >> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
> >> and PRs we would like to "target" to 1.8.0.
> >>
> >> Thoughts ?
> >>
> >> Regards
> >> JB
>
> --
> Robert Stupp
> @snazy
>
>
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Robert Stupp <sn...@snazy.de>.
Hey,
IMHO 1.8 should definitely include the Auth-Manager work, which tackles
actual bugs in the Iceberg code base wrt OAuth implementation. That work
was originally intended to go into 1.7 and now it shall be delayed again
for 1.9. The PR was originally opened in July 2024, about half a year
ago and is still getting reviewed. In the meantime there were more than
600 other PRs that got reviewed and merged.
The overall agreement around spring 2024, please correct me if I am
wrong, was the whole REST/OAuth area needs to be improved, and the oauth
endpoint removed entirely.
Generally speaking, and I know I'm not alone, getting reviews from
Iceberg committers is extremely hard. A lot of issues and PRs just get
closed (by that stale bot) without having gotten _any_ attention from an
Iceberg committer. This is not a new situation but going on for a long
time. I have been talking to two Iceberg PMC members in person many
months ago that this is a very disappointing situation that needs to be
fixed. The reply was always "we are already working on it" - but at
least from my personal POV the situation did not improve.
Robert
On 16.01.25 10:56, Jean-Baptiste Onofré wrote:
> Hi folks,
>
> Following the Community Meeting yesterday, I would like to propose the
> following plan regarding releases:
>
> 0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
> Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
> discussed this point with Fokko at the beginning of this week, because
> I have some doubts about LICENSE/NOTICE content in the "uber" jar
> artifacts where we shade dependencies. I'm doing a complete pass on
> all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
> complete analysis by tomorrow. This is potentially a blocker for
> release votes.
>
> 1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will work
> with Fokko on this one.
>
> 2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
> manager). Due to still some WIP, we "revisited" the 1.8.0 release
> content: for instance, as best effort, we wanted to include REST Auth
> Manager improvement (OAuth2 Manager) but we preferred to postpone to
> 1.9.0. That's totally fine to me, however, I would propose to strongly
> focus on pending PRs for 1.9.0. Imho, we should "target" (again as
> clear best effort) on variant, partition stats and Auth Manager.
>
> 3. Assuming 1.8.0 will be released at the end of Jan/beginning of Feb,
> according to our "release cadence", what do you think about planning
> 1.9.0 in April ? Again with the main targets listed in (2).
>
> I tried to sum up what we discussed yesterday :)
> Thoughts ?
>
> Regards
> JB
>
> On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>> Hi folks,
>>
>> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
>> our release "pace", 1.8.0 should be released around mid February.
>>
>> I think we already have a good "train" of merged PRs (or should be
>> merged soon): default values, REST auth improvements, dependencies
>> updates, etc.
>>
>> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
>> and PRs we would like to "target" to 1.8.0.
>>
>> Thoughts ?
>>
>> Regards
>> JB
--
Robert Stupp
@snazy
Re: [DISCUSS] Apache Iceberg (java) 1.8.0 release
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi folks,
Following the Community Meeting yesterday, I would like to propose the
following plan regarding releases:
0. As a prerequisite to any release (1.7.2, 1.8.0, 1.9.0), as said by
Ryan, we have to double check the NOTICE/LICENSE. Interestingly, I
discussed this point with Fokko at the beginning of this week, because
I have some doubts about LICENSE/NOTICE content in the "uber" jar
artifacts where we shade dependencies. I'm doing a complete pass on
all artifacts in 1.7.2-SNAPSHOT and 1.8.0-SNAPSHOT. I should have a
complete analysis by tomorrow. This is potentially a blocker for
release votes.
1. As soon as (0) is done, 1.7.2 can be submitted to vote. I will work
with Fokko on this one.
2. We plan to do 1.8.0 in a couple of weeks (Amogh is the release
manager). Due to still some WIP, we "revisited" the 1.8.0 release
content: for instance, as best effort, we wanted to include REST Auth
Manager improvement (OAuth2 Manager) but we preferred to postpone to
1.9.0. That's totally fine to me, however, I would propose to strongly
focus on pending PRs for 1.9.0. Imho, we should "target" (again as
clear best effort) on variant, partition stats and Auth Manager.
3. Assuming 1.8.0 will be released at the end of Jan/beginning of Feb,
according to our "release cadence", what do you think about planning
1.9.0 in April ? Again with the main targets listed in (2).
I tried to sum up what we discussed yesterday :)
Thoughts ?
Regards
JB
On Thu, Jan 9, 2025 at 7:51 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> Hi folks,
>
> We did Apache Iceberg 1.7.0 release on Nov 8, 2024. If we want to keep
> our release "pace", 1.8.0 should be released around mid February.
>
> I think we already have a good "train" of merged PRs (or should be
> merged soon): default values, REST auth improvements, dependencies
> updates, etc.
>
> WDYT about 1.8.0 mid Feb ? If so, I propose we update GitHub Issues
> and PRs we would like to "target" to 1.8.0.
>
> Thoughts ?
>
> Regards
> JB