Re: Hadoop 3.2 Release Plan proposal

[Sunil G <su...@apache.org>]

Hi Folks,

Given all blockers are closed now, I am planning to do RC1 tomorrow.
branch-3.2.0 remains closed for commit. Please let me know if any issues
are missed.

Thanks,
Sunil G

On Tue, Nov 13, 2018 at 8:11 PM Sunil G <su...@apache.org> wrote:

> Hi Folks,
>
> All blockers are closed by last weekend and corrected jiras as well.
> Preparing RC0 now. Facing some issue with shaded jars file size. Hence
> respinning again.
> Planning to complete this by end of this week.
>
> Thanks
> Sunil
>
> On Fri, Oct 26, 2018 at 2:34 AM Konstantin Shvachko <sh...@gmail.com>
> wrote:
>
>> Another thing is that I see a bunch of jiras under HDFS-8707, which don't
>> have the Fix Version field listing 3.2, some have it just empty.
>> This means they will not be populated into release notes.
>>
>> Thanks,
>> --Konstantin
>>
>> On Thu, Oct 25, 2018 at 7:59 AM Sunil G <su...@apache.org> wrote:
>>
>> > Thanks Konstantin for pointing out.
>> > As 3.2 is pretty much on RC level, its better we try to find a good
>> > solution to this issue.
>> >
>> > I ll follow up on this in the jira.
>> >
>> > - Sunil
>> >
>> > On Thu, Oct 25, 2018 at 11:35 AM Konstantin Shvachko <
>> shv.hadoop@gmail.com>
>> > wrote:
>> >
>> >> I've tried to attract attention to an incompatibility issue through the
>> >> jira, but it didn't work. So pitching in in this thread.
>> >> https://issues.apache.org/jira/browse/HDFS-12026
>> >> It introduced binary incompatibility, which will prevent people from
>> >> upgrading from 3.1 to 3.2.
>> >> I think it can get messy if we release anything with this feature.
>> >>
>> >> Thanks,
>> >> --Konstantin
>> >>
>> >> On Mon, Oct 22, 2018 at 5:01 AM Steve Loughran <stevel@hortonworks.com
>> >
>> >> wrote:
>> >>
>> >>> its in.
>> >>>
>> >>> good catch!
>> >>>
>> >>> On 20 Oct 2018, at 01:35, Wei-Chiu Chuang <weichiu@cloudera.com
>> <mailto:
>> >>> weichiu@cloudera.com>> wrote:
>> >>>
>> >>> Thanks Sunil G for driving the release,
>> >>> I filed HADOOP-15866<
>> https://issues.apache.org/jira/browse/HADOOP-15866>
>> >>> for a compat fix. If any one has cycle please review it, as I think
>> it is
>> >>> needed for 3.2.0.
>> >>>
>> >>> On Thu, Oct 18, 2018 at 4:43 AM Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>> Hi Folks,
>> >>>
>> >>> As we previously communicated for 3.2.0 release, we have delayed due
>> to
>> >>> few
>> >>> blockers in our gate.
>> >>>
>> >>> I just cut branch-3.2.0 for release purpose. branch-3.2 will be open
>> for
>> >>> all bug fixes.
>> >>>
>> >>> - Sunil
>> >>>
>> >>>
>> >>> On Tue, Oct 16, 2018 at 8:59 AM Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>>
>> >>> > Hi Folks,
>> >>> >
>> >>> > We are now close to RC as other blocker issues are now merged to
>> trunk
>> >>> and
>> >>> > branch-3.2. Last 2 critical issues are closer to merge and will be
>> >>> > committed in few hours.
>> >>> > With this, I will be creating 3.2.0 branch today and will go ahead
>> >>> with RC
>> >>> > related process.
>> >>> >
>> >>> > - Sunil
>> >>> >
>> >>> > On Mon, Oct 15, 2018 at 11:43 PM Jonathan Bender <
>> jonbender@stripe.com
>> >>> <ma...@stripe.com>>
>> >>> > wrote:
>> >>> >
>> >>> >> Hello, were there any updates around the 3.2.0 RC timing? All I
>> see in
>> >>> >> the current blockers are related to the new Submarine subproject,
>> >>> wasn't
>> >>> >> sure if that is what is holding things up.
>> >>> >>
>> >>> >> Cheers,
>> >>> >> Jon
>> >>> >>
>> >>> >> On Tue, Oct 2, 2018 at 7:13 PM, Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>> >>
>> >>> >>> Thanks Robert and Haibo for quickly correcting same.
>> >>> >>> Sigh, I somehow missed one file while committing the change. Sorry
>> >>> for
>> >>> >>> the
>> >>> >>> trouble.
>> >>> >>>
>> >>> >>> - Sunil
>> >>> >>>
>> >>> >>> On Wed, Oct 3, 2018 at 5:22 AM Robert Kanter <
>> rkanter@cloudera.com
>> >>> <ma...@cloudera.com>>
>> >>> >>> wrote:
>> >>> >>>
>> >>> >>> > Looks like there's two that weren't updated:
>> >>> >>> > >> [115] 16:32 : hadoop-common (trunk) :: grep "3.2.0-SNAPSHOT"
>> .
>> >>> -r
>> >>> >>> > --include=pom.xml
>> >>> >>> > ./hadoop-project/pom.xml:
>> >>> >>> >
>> >>> <hadoop.assemblies.version>3.2.0-SNAPSHOT</hadoop.assemblies.version>
>> >>> >>> > ./pom.xml:    <hadoop.version>3.2.0-SNAPSHOT</hadoop.version>
>> >>> >>> >
>> >>> >>> > I've just pushed in an addendum commit to fix those.
>> >>> >>> > In the future, please make sure to do a sanity compile when
>> >>> updating
>> >>> >>> poms.
>> >>> >>> >
>> >>> >>> > thanks
>> >>> >>> > - Robert
>> >>> >>> >
>> >>> >>> > On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri
>> >>> >>> <fa...@cloudera.com.invalid>>
>> >>> >>> > wrote:
>> >>> >>> >
>> >>> >>> >> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in
>> >>> the
>> >>> >>> >> top-level pom.xml?
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> On Tue, Oct 2, 2018 at 10:16 AM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>> wrote:
>> >>> >>> >>
>> >>> >>> >> > Hi All
>> >>> >>> >> >
>> >>> >>> >> > As mentioned in earlier mail, I have cut branch-3.2 and reset
>> >>> trunk
>> >>> >>> to
>> >>> >>> >> > 3.3.0-SNAPSHOT. I will share the RC details sooner once all
>> >>> >>> necessary
>> >>> >>> >> > patches are pulled into branch-3.2.
>> >>> >>> >> >
>> >>> >>> >> > Thank You
>> >>> >>> >> > - Sunil
>> >>> >>> >> >
>> >>> >>> >> >
>> >>> >>> >> > On Mon, Sep 24, 2018 at 2:00 PM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>> wrote:
>> >>> >>> >> >
>> >>> >>> >> > > Hi All
>> >>> >>> >> > >
>> >>> >>> >> > > We are now down to the last Blocker and HADOOP-15407 is
>> >>> merged to
>> >>> >>> >> trunk.
>> >>> >>> >> > > Thanks for the support.
>> >>> >>> >> > >
>> >>> >>> >> > > *Plan for RC*
>> >>> >>> >> > > 3.2 branch cut and reset trunk : *25th Tuesday*
>> >>> >>> >> > > RC0 for 3.2: *28th Friday*
>> >>> >>> >> > >
>> >>> >>> >> > > Thank You
>> >>> >>> >> > > Sunil
>> >>> >>> >> > >
>> >>> >>> >> > >
>> >>> >>> >> > > On Mon, Sep 17, 2018 at 3:21 PM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >
>> >>> >>> >> > >> Hi All
>> >>> >>> >> > >>
>> >>> >>> >> > >> We are down to 3 Blockers and 4 Critical now. Thanks all
>> of
>> >>> you
>> >>> >>> for
>> >>> >>> >> > >> helping in this. I am following up on these tickets, once
>> its
>> >>> >>> closed
>> >>> >>> >> we
>> >>> >>> >> > >> will cut the 3.2 branch.
>> >>> >>> >> > >>
>> >>> >>> >> > >> Thanks
>> >>> >>> >> > >> Sunil Govindan
>> >>> >>> >> > >>
>> >>> >>> >> > >>
>> >>> >>> >> > >> On Wed, Sep 12, 2018 at 5:10 PM Sunil G <
>> sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >>
>> >>> >>> >> > >>> Hi All,
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Inline with the original 3.2 communication proposal dated
>> >>> 17th
>> >>> >>> July
>> >>> >>> >> > >>> 2018, I would like to provide more updates.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> We are approaching previously proposed code freeze date
>> >>> >>> (September
>> >>> >>> >> 14,
>> >>> >>> >> > >>> 2018). So I would like to cut 3.2 branch on 17th Sept and
>> >>> point
>> >>> >>> >> > existing
>> >>> >>> >> > >>> trunk to 3.3 if there are no issues.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> *Current Release Plan:*
>> >>> >>> >> > >>> Feature freeze date : all features to merge by September
>> 7,
>> >>> >>> 2018.
>> >>> >>> >> > >>> Code freeze date : blockers/critical only, no
>> improvements
>> >>> and
>> >>> >>> >> > >>> blocker/critical bug-fixes September 14, 2018.
>> >>> >>> >> > >>> Release date: September 28, 2018
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> If any critical/blocker tickets which are targeted to
>> >>> 3.2.0, we
>> >>> >>> >> need to
>> >>> >>> >> > >>> backport to 3.2 post branch cut.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Here's an updated 3.2.0 feature status:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 1. Merged & Completed features:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> DeepLearning
>> >>> >>> >> > >>> workloads Initial cut.
>> >>> >>> >> > >>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>> Scheduler.
>> >>> >>> >> > >>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> YARN
>> >>> >>> Service
>> >>> >>> >> > API
>> >>> >>> >> > >>> and CLI.
>> >>> >>> >> > >>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> >> > >>> - (Inigo) HDFS-12615: Router-based HDFS federation.
>> >>> Improvement
>> >>> >>> >> works.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 2. Features close to finish:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> - (Steve) S3Guard Phase III. Close to commit.
>> >>> >>> >> > >>> - (Steve) S3a phase V. Close to commit.
>> >>> >>> >> > >>> - (Steve) Support Windows Azure Storage. Close to commit.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 3. Tentative/Cancelled features for 3.2:
>> >>> >>> >> > >>> - (Rohith) YARN-5742: Serve aggregated logs of historical
>> >>> apps
>> >>> >>> from
>> >>> >>> >> > >>> ATSv2. Patch in progress.
>> >>> >>> >> > >>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
>> >>> >>> >> challenging to
>> >>> >>> >> > >>> be done before Aug 2018.
>> >>> >>> >> > >>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>> 19 Blocker and Critical issues [1] are open, I am
>> following
>> >>> up
>> >>> >>> with
>> >>> >>> >> > >>> owners to get status on each of them to get in by Code
>> >>> Freeze
>> >>> >>> date.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> priority
>> >>> in
>> >>> >>> >> > (Blocker,
>> >>> >>> >> > >>> Critical) AND resolution = Unresolved AND "Target
>> >>> Version/s" =
>> >>> >>> 3.2.0
>> >>> >>> >> > ORDER
>> >>> >>> >> > >>> BY priority DESC
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Thanks,
>> >>> >>> >> > >>> Sunil
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> On Thu, Aug 30, 2018 at 9:59 PM Sunil G <
>> sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>> Hi All,
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Inline with earlier communication dated 17th July 2018,
>> I
>> >>> would
>> >>> >>> >> like
>> >>> >>> >> > to
>> >>> >>> >> > >>>> provide some updates.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> We are approaching previously proposed code freeze date
>> >>> (Aug
>> >>> >>> 31).
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> One of the critical feature Node Attributes feature
>> merge
>> >>> >>> >> > >>>> discussion/vote is ongoing. Also few other Blocker bugs
>> >>> need a
>> >>> >>> bit
>> >>> >>> >> > more
>> >>> >>> >> > >>>> time. With regard to this, suggesting to push the
>> >>> feature/code
>> >>> >>> >> freeze
>> >>> >>> >> > for 2
>> >>> >>> >> > >>>> more weeks to accommodate these jiras too.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Proposing Updated changes in plan inline with this:
>> >>> >>> >> > >>>> Feature freeze date : all features to merge by
>> September 7,
>> >>> >>> 2018.
>> >>> >>> >> > >>>> Code freeze date : blockers/critical only, no
>> improvements
>> >>> and
>> >>> >>> >> > >>>>  blocker/critical bug-fixes September 14, 2018.
>> >>> >>> >> > >>>> Release date: September 28, 2018
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> If any features in branch which are targeted to 3.2.0,
>> >>> please
>> >>> >>> >> reply to
>> >>> >>> >> > >>>> this email thread.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> *Here's an updated 3.2.0 feature status:*
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 1. Merged & Completed features:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> DeepLearning
>> >>> >>> >> > >>>> workloads Initial cut.
>> >>> >>> >> > >>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>>> Scheduler.
>> >>> >>> >> > >>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> >>> YARN
>> >>> >>> >> Service
>> >>> >>> >> > >>>> API and CLI.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 2. Features close to finish:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> >> Merge/Vote
>> >>> >>> >> > >>>> Ongoing.
>> >>> >>> >> > >>>> - (Rohith) YARN-5742: Serve aggregated logs of
>> historical
>> >>> apps
>> >>> >>> from
>> >>> >>> >> > >>>> ATSv2. Patch in progress.
>> >>> >>> >> > >>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
>> >>> >>> Improvement
>> >>> >>> >> > works.
>> >>> >>> >> > >>>> - (Steve) S3Guard Phase III, S3a phase V, Support
>> Windows
>> >>> Azure
>> >>> >>> >> > >>>> Storage. In progress.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 3. Tentative features:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
>> >>> >>> >> challenging
>> >>> >>> >> > to
>> >>> >>> >> > >>>> be done before Aug 2018.
>> >>> >>> >> > >>>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 26 Blocker and Critical issues [1] are open, I am
>> >>> following up
>> >>> >>> with
>> >>> >>> >> > >>>> owners to get status on each of them to get in by Code
>> >>> Freeze
>> >>> >>> date.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> >>> priority in
>> >>> >>> >> > >>>> (Blocker, Critical) AND resolution = Unresolved AND
>> "Target
>> >>> >>> >> > Version/s" =
>> >>> >>> >> > >>>> 3.2.0 ORDER BY priority DESC
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Thanks,
>> >>> >>> >> > >>>> Sunil
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> On Tue, Aug 14, 2018 at 10:30 PM Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> wrote:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>>> Hi All,
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> Thanks for the feedbacks. Inline with earlier
>> >>> communication
>> >>> >>> dated
>> >>> >>> >> > 17th
>> >>> >>> >> > >>>>> July 2018, I would like to provide some updates.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> We are approaching previously proposed feature freeze
>> date
>> >>> >>> (Aug
>> >>> >>> >> 21,
>> >>> >>> >> > >>>>> about 7 days from today).
>> >>> >>> >> > >>>>> If any features in branch which are targeted to 3.2.0,
>> >>> please
>> >>> >>> >> reply
>> >>> >>> >> > to
>> >>> >>> >> > >>>>> this email thread.
>> >>> >>> >> > >>>>> Steve has mentioned about the s3 features which will
>> come
>> >>> >>> close to
>> >>> >>> >> > >>>>> Code Freeze Date (Aug 31st).
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> *Here's an updated 3.2.0 feature status:*
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 1. Merged & Completed features:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> >>> DeepLearning
>> >>> >>> >> > >>>>> workloads Initial cut.
>> >>> >>> >> > >>>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 2. Features close to finish:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> Major
>> >>> >>> >> > >>>>> patches are all in, only one last
>> >>> >>> >> > >>>>> patch is in review state.
>> >>> >>> >> > >>>>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>>>> Scheduler. Close to commit.
>> >>> >>> >> > >>>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> >>> YARN
>> >>> >>> >> Service
>> >>> >>> >> > >>>>> API and CLI. 2 patches are pending
>> >>> >>> >> > >>>>> which will be closed by Feature freeze date.
>> >>> >>> >> > >>>>> - (Rohith) YARN-5742: Serve aggregated logs of
>> historical
>> >>> apps
>> >>> >>> >> from
>> >>> >>> >> > >>>>> ATSv2. Patch in progress.
>> >>> >>> >> > >>>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
>> >>> >>> Improvement
>> >>> >>> >> > >>>>> works.
>> >>> >>> >> > >>>>> - (Steve) S3Guard Phase III, S3a phase V, Support
>> Windows
>> >>> >>> Azure
>> >>> >>> >> > >>>>> Storage. In progress.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 3. Tentative features:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Haibo Chen) YARN-1011: Resource overcommitment.
>> Looks
>> >>> >>> >> challenging
>> >>> >>> >> > >>>>> to be done before Aug 2018.
>> >>> >>> >> > >>>>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>>>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 39 Blocker and Critical issues [1] are open, I am
>> checking
>> >>> >>> with
>> >>> >>> >> > owners
>> >>> >>> >> > >>>>> to get status on each of them to get in by Code Freeze
>> >>> date.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> >>> priority in
>> >>> >>> >> > >>>>> (Blocker, Critical) AND resolution = Unresolved AND
>> >>> "Target
>> >>> >>> >> > Version/s" =
>> >>> >>> >> > >>>>> 3.2.0 ORDER BY priority DESC
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> Thanks,
>> >>> >>> >> > >>>>> Sunil
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> On Fri, Jul 20, 2018 at 8:03 AM Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> wrote:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>>> Thanks Subru for the thoughts.
>> >>> >>> >> > >>>>>> One of the main reason for a major release is to push
>> out
>> >>> >>> >> critical
>> >>> >>> >> > >>>>>> features with a faster cadence to the users. If we are
>> >>> >>> pulling
>> >>> >>> >> more
>> >>> >>> >> > and
>> >>> >>> >> > >>>>>> more different types of features to a minor release,
>> that
>> >>> >>> branch
>> >>> >>> >> > will
>> >>> >>> >> > >>>>>> become more destabilized and it may be tough to say
>> that
>> >>> >>> 3.1.2 is
>> >>> >>> >> > stable
>> >>> >>> >> > >>>>>> that 3.1.1 for eg. We always tend to improve and
>> >>> stabilize
>> >>> >>> >> features
>> >>> >>> >> > in
>> >>> >>> >> > >>>>>> subsequent minor release.
>> >>> >>> >> > >>>>>> For few companies, it makes sense to push out these
>> new
>> >>> >>> features
>> >>> >>> >> > >>>>>> faster to make a reach to the users. Adding to the
>> point
>> >>> to
>> >>> >>> the
>> >>> >>> >> > backporting
>> >>> >>> >> > >>>>>> issues, I agree that its a pain and we can workaround
>> >>> that
>> >>> >>> with
>> >>> >>> >> > some git
>> >>> >>> >> > >>>>>> scripts. If we can make such scripts available to
>> >>> committers,
>> >>> >>> >> > backport will
>> >>> >>> >> > >>>>>> be seem-less across branches and we can achieve the
>> >>> faster
>> >>> >>> >> release
>> >>> >>> >> > cadence
>> >>> >>> >> > >>>>>> also.
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> Thoughts?
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> - Sunil
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> On Fri, Jul 20, 2018 at 3:37 AM Subru Krishnan <
>> >>> >>> subru@apache.org<ma...@apache.org>
>> >>> >>> >> >
>> >>> >>> >> > >>>>>> wrote:
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>>> Thanks Sunil for volunteering to lead the release
>> >>> effort. I
>> >>> >>> am
>> >>> >>> >> > >>>>>>> generally
>> >>> >>> >> > >>>>>>> supportive of a release but -1 on a 3.2 (prefer a
>> >>> 3.1.x) as
>> >>> >>> >> feel we
>> >>> >>> >> > >>>>>>> already
>> >>> >>> >> > >>>>>>> have too many branches to be maintained. I already
>> see
>> >>> many
>> >>> >>> >> commits
>> >>> >>> >> > >>>>>>> are in
>> >>> >>> >> > >>>>>>> different branches with no apparent rationale, for
>> e.g:
>> >>> 3.1
>> >>> >>> has
>> >>> >>> >> > >>>>>>> commits
>> >>> >>> >> > >>>>>>> which are absent in 3.0 etc.
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> Additionally AFAIK 3.x has not been deployed in any
>> >>> major
>> >>> >>> >> > production
>> >>> >>> >> > >>>>>>> setting so the cost of adding features should be
>> >>> minimal.
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> Thoughts?
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> -Subru
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> On Thu, Jul 19, 2018 at 12:31 AM, Sunil G <
>> >>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> > wrote:
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> > Thanks Steve, Aaron, Wangda for sharing thoughts.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Yes, important changes and features are much
>> needed,
>> >>> >>> hence we
>> >>> >>> >> > will
>> >>> >>> >> > >>>>>>> be
>> >>> >>> >> > >>>>>>> > keeping the door open for them as possible. Also
>> >>> >>> considering
>> >>> >>> >> few
>> >>> >>> >> > >>>>>>> more
>> >>> >>> >> > >>>>>>> > offline requests from other folks, I think
>> extending
>> >>> the
>> >>> >>> >> > timeframe
>> >>> >>> >> > >>>>>>> by
>> >>> >>> >> > >>>>>>> > couple of weeks makes sense (including a second RC
>> >>> >>> buffer) and
>> >>> >>> >> > >>>>>>> this should
>> >>> >>> >> > >>>>>>> > ideally help us to ship this by September itself.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Revised dates (I will be updating same in Roadmap
>> >>> wiki as
>> >>> >>> >> well)
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Feature freeze date : all features to merge by
>> >>> August
>> >>> >>> 21,
>> >>> >>> >> 2018.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Code freeze date : blockers/critical only, no
>> >>> >>> improvements
>> >>> >>> >> and
>> >>> >>> >> > >>>>>>> non
>> >>> >>> >> > >>>>>>> > blocker/critical
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > bug-fixes  August 31, 2018.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Release date: September 15, 2018
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Thank Eric and Zian, I think Wangda has already
>> >>> answered
>> >>> >>> your
>> >>> >>> >> > >>>>>>> questions.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Thanks
>> >>> >>> >> > >>>>>>> > Sunil
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > On Thu, Jul 19, 2018 at 12:13 PM Wangda Tan <
>> >>> >>> >> wheeleast@gmail.com<ma...@gmail.com>
>> >>> >>> >> > >
>> >>> >>> >> > >>>>>>> wrote:
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > > Thanks Sunil for volunteering to be RM of 3.2
>> >>> release,
>> >>> >>> +1
>> >>> >>> >> for
>> >>> >>> >> > >>>>>>> that.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > To concerns from Steve,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > It is a good idea to keep the door open to get
>> >>> important
>> >>> >>> >> > changes
>> >>> >>> >> > >>>>>>> /
>> >>> >>> >> > >>>>>>> > > features in before cutoff. I would prefer to keep
>> >>> the
>> >>> >>> >> proposed
>> >>> >>> >> > >>>>>>> release
>> >>> >>> >> > >>>>>>> > date
>> >>> >>> >> > >>>>>>> > > to make sure things can happen earlier instead of
>> >>> last
>> >>> >>> >> minute
>> >>> >>> >> > >>>>>>> and we all
>> >>> >>> >> > >>>>>>> > > know that releases are always get delayed :). I'm
>> >>> also
>> >>> >>> fine
>> >>> >>> >> if
>> >>> >>> >> > >>>>>>> we want
>> >>> >>> >> > >>>>>>> > get
>> >>> >>> >> > >>>>>>> > > another several weeks time.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Regarding of 3.3 release, I would suggest doing
>> that
>> >>> >>> before
>> >>> >>> >> > >>>>>>> thanksgiving.
>> >>> >>> >> > >>>>>>> > > Do you think is it good or too early / late?
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Eric,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > The YARN-8220 will be replaced by YARN-8135, if
>> >>> >>> YARN-8135
>> >>> >>> >> can
>> >>> >>> >> > >>>>>>> get merged
>> >>> >>> >> > >>>>>>> > > in time, we probably not need the YARN-8220.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Sunil,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Could u update
>> >>> >>> >> > >>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> >>> >>> >> > >>>>>>> > Roadmap
>> >>> >>> >> > >>>>>>> > > with proposed plan as well? We can fill feature
>> list
>> >>> >>> first
>> >>> >>> >> > >>>>>>> before getting
>> >>> >>> >> > >>>>>>> > > consensus of time.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Thanks,
>> >>> >>> >> > >>>>>>> > > Wangda
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > On Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri
>> >>> >>> >> > >>>>>>> <fabbri@cloudera.com.invalid<mailto:
>> >>> fabbri@cloudera.com.invalid>
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > wrote:
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > >> On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran <
>> >>> >>> >> > >>>>>>> stevel@hortonworks.com<mailto:stevel@hortonworks.com
>> >>
>> >>> >>> >> > >>>>>>> > >> wrote:
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > On 16 Jul 2018, at 23:45, Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>
>> >>> >>> >> > <mailto:
>> >>> >>> >> > >>>>>>> > >> > sunilg@apache.org<ma...@apache.org>>>
>> >>> wrote:
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I would also would like to take this
>> opportunity
>> >>> to
>> >>> >>> come
>> >>> >>> >> up
>> >>> >>> >> > >>>>>>> with a
>> >>> >>> >> > >>>>>>> > >> detailed
>> >>> >>> >> > >>>>>>> > >> > plan.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > - Feature freeze date : all features should be
>> >>> >>> merged by
>> >>> >>> >> > >>>>>>> August 10,
>> >>> >>> >> > >>>>>>> > >> 2018.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > <snip>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > Please let me know if I missed any features
>> >>> targeted
>> >>> >>> to
>> >>> >>> >> 3.2
>> >>> >>> >> > >>>>>>> per this
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > Well there these big todo lists for S3 &
>> S3Guard.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15226
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15220
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > There's a bigger bit of work coming on for
>> Azure
>> >>> >>> Datalake
>> >>> >>> >> > Gen
>> >>> >>> >> > >>>>>>> 2
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15407
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I don't think this is quite ready yet, I've
>> been
>> >>> >>> doing
>> >>> >>> >> work
>> >>> >>> >> > >>>>>>> on it, but
>> >>> >>> >> > >>>>>>> > >> if
>> >>> >>> >> > >>>>>>> > >> > we have a 3 week deadline, I'm going to expect
>> >>> some
>> >>> >>> >> timely
>> >>> >>> >> > >>>>>>> reviews on
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15546
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I've uprated that to a blocker feature; will
>> >>> review
>> >>> >>> the
>> >>> >>> >> S3 &
>> >>> >>> >> > >>>>>>> S3Guard
>> >>> >>> >> > >>>>>>> > >> JIRAs
>> >>> >>> >> > >>>>>>> > >> > to see which of those are blocking. Then there
>> >>> are
>> >>> >>> some
>> >>> >>> >> > >>>>>>> pressing
>> >>> >>> >> > >>>>>>> > "guave,
>> >>> >>> >> > >>>>>>> > >> > java 9 prep"
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >>  I can help with this part if you like.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > timeline. I would like to volunteer myself as
>> >>> release
>> >>> >>> >> > manager
>> >>> >>> >> > >>>>>>> of 3.2.0
>> >>> >>> >> > >>>>>>> > >> > release.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > well volunteered!
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> Yes, thank you for stepping up.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I think this raises a good q: what timetable
>> >>> should
>> >>> >>> we
>> >>> >>> >> have
>> >>> >>> >> > >>>>>>> for the
>> >>> >>> >> > >>>>>>> > >> 3.2. &
>> >>> >>> >> > >>>>>>> > >> > 3.3 releases; if we do want a faster cadence,
>> >>> then
>> >>> >>> having
>> >>> >>> >> > the
>> >>> >>> >> > >>>>>>> outline
>> >>> >>> >> > >>>>>>> > >> time
>> >>> >>> >> > >>>>>>> > >> > from the 3.2 to the 3.3 release means that
>> >>> there's
>> >>> >>> less
>> >>> >>> >> > >>>>>>> concern about
>> >>> >>> >> > >>>>>>> > >> > things not making the 3.2 dealine
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > -Steve
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> Good idea to mitigate the short deadline.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> -AF
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>
>> >>> >>> >> >
>> >>> >>> >>
>> >>> >>> >
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>>
>> >>>
>> >>> --
>> >>> A very happy Clouderan
>> >>>
>> >>>
>>
>