You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Sunil G <su...@apache.org> on 2018/11/13 14:41:06 UTC

Re: Hadoop 3.2 Release Plan proposal

Hi Folks,

All blockers are closed by last weekend and corrected jiras as well.
Preparing RC0 now. Facing some issue with shaded jars file size. Hence
respinning again.
Planning to complete this by end of this week.

Thanks
Sunil

On Fri, Oct 26, 2018 at 2:34 AM Konstantin Shvachko <sh...@gmail.com>
wrote:

> Another thing is that I see a bunch of jiras under HDFS-8707, which don't
> have the Fix Version field listing 3.2, some have it just empty.
> This means they will not be populated into release notes.
>
> Thanks,
> --Konstantin
>
> On Thu, Oct 25, 2018 at 7:59 AM Sunil G <su...@apache.org> wrote:
>
> > Thanks Konstantin for pointing out.
> > As 3.2 is pretty much on RC level, its better we try to find a good
> > solution to this issue.
> >
> > I ll follow up on this in the jira.
> >
> > - Sunil
> >
> > On Thu, Oct 25, 2018 at 11:35 AM Konstantin Shvachko <
> shv.hadoop@gmail.com>
> > wrote:
> >
> >> I've tried to attract attention to an incompatibility issue through the
> >> jira, but it didn't work. So pitching in in this thread.
> >> https://issues.apache.org/jira/browse/HDFS-12026
> >> It introduced binary incompatibility, which will prevent people from
> >> upgrading from 3.1 to 3.2.
> >> I think it can get messy if we release anything with this feature.
> >>
> >> Thanks,
> >> --Konstantin
> >>
> >> On Mon, Oct 22, 2018 at 5:01 AM Steve Loughran <st...@hortonworks.com>
> >> wrote:
> >>
> >>> its in.
> >>>
> >>> good catch!
> >>>
> >>> On 20 Oct 2018, at 01:35, Wei-Chiu Chuang <weichiu@cloudera.com
> <mailto:
> >>> weichiu@cloudera.com>> wrote:
> >>>
> >>> Thanks Sunil G for driving the release,
> >>> I filed HADOOP-15866<
> https://issues.apache.org/jira/browse/HADOOP-15866>
> >>> for a compat fix. If any one has cycle please review it, as I think it
> is
> >>> needed for 3.2.0.
> >>>
> >>> On Thu, Oct 18, 2018 at 4:43 AM Sunil G <sunilg@apache.org<mailto:
> >>> sunilg@apache.org>> wrote:
> >>> Hi Folks,
> >>>
> >>> As we previously communicated for 3.2.0 release, we have delayed due to
> >>> few
> >>> blockers in our gate.
> >>>
> >>> I just cut branch-3.2.0 for release purpose. branch-3.2 will be open
> for
> >>> all bug fixes.
> >>>
> >>> - Sunil
> >>>
> >>>
> >>> On Tue, Oct 16, 2018 at 8:59 AM Sunil G <sunilg@apache.org<mailto:
> >>> sunilg@apache.org>> wrote:
> >>>
> >>> > Hi Folks,
> >>> >
> >>> > We are now close to RC as other blocker issues are now merged to
> trunk
> >>> and
> >>> > branch-3.2. Last 2 critical issues are closer to merge and will be
> >>> > committed in few hours.
> >>> > With this, I will be creating 3.2.0 branch today and will go ahead
> >>> with RC
> >>> > related process.
> >>> >
> >>> > - Sunil
> >>> >
> >>> > On Mon, Oct 15, 2018 at 11:43 PM Jonathan Bender <
> jonbender@stripe.com
> >>> <ma...@stripe.com>>
> >>> > wrote:
> >>> >
> >>> >> Hello, were there any updates around the 3.2.0 RC timing? All I see
> in
> >>> >> the current blockers are related to the new Submarine subproject,
> >>> wasn't
> >>> >> sure if that is what is holding things up.
> >>> >>
> >>> >> Cheers,
> >>> >> Jon
> >>> >>
> >>> >> On Tue, Oct 2, 2018 at 7:13 PM, Sunil G <sunilg@apache.org<mailto:
> >>> sunilg@apache.org>> wrote:
> >>> >>
> >>> >>> Thanks Robert and Haibo for quickly correcting same.
> >>> >>> Sigh, I somehow missed one file while committing the change. Sorry
> >>> for
> >>> >>> the
> >>> >>> trouble.
> >>> >>>
> >>> >>> - Sunil
> >>> >>>
> >>> >>> On Wed, Oct 3, 2018 at 5:22 AM Robert Kanter <rkanter@cloudera.com
> >>> <ma...@cloudera.com>>
> >>> >>> wrote:
> >>> >>>
> >>> >>> > Looks like there's two that weren't updated:
> >>> >>> > >> [115] 16:32 : hadoop-common (trunk) :: grep "3.2.0-SNAPSHOT" .
> >>> -r
> >>> >>> > --include=pom.xml
> >>> >>> > ./hadoop-project/pom.xml:
> >>> >>> >
> >>> <hadoop.assemblies.version>3.2.0-SNAPSHOT</hadoop.assemblies.version>
> >>> >>> > ./pom.xml:    <hadoop.version>3.2.0-SNAPSHOT</hadoop.version>
> >>> >>> >
> >>> >>> > I've just pushed in an addendum commit to fix those.
> >>> >>> > In the future, please make sure to do a sanity compile when
> >>> updating
> >>> >>> poms.
> >>> >>> >
> >>> >>> > thanks
> >>> >>> > - Robert
> >>> >>> >
> >>> >>> > On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri
> >>> >>> <fa...@cloudera.com.invalid>>
> >>> >>> > wrote:
> >>> >>> >
> >>> >>> >> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in
> >>> the
> >>> >>> >> top-level pom.xml?
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> On Tue, Oct 2, 2018 at 10:16 AM Sunil G <sunilg@apache.org
> >>> <ma...@apache.org>> wrote:
> >>> >>> >>
> >>> >>> >> > Hi All
> >>> >>> >> >
> >>> >>> >> > As mentioned in earlier mail, I have cut branch-3.2 and reset
> >>> trunk
> >>> >>> to
> >>> >>> >> > 3.3.0-SNAPSHOT. I will share the RC details sooner once all
> >>> >>> necessary
> >>> >>> >> > patches are pulled into branch-3.2.
> >>> >>> >> >
> >>> >>> >> > Thank You
> >>> >>> >> > - Sunil
> >>> >>> >> >
> >>> >>> >> >
> >>> >>> >> > On Mon, Sep 24, 2018 at 2:00 PM Sunil G <sunilg@apache.org
> >>> <ma...@apache.org>> wrote:
> >>> >>> >> >
> >>> >>> >> > > Hi All
> >>> >>> >> > >
> >>> >>> >> > > We are now down to the last Blocker and HADOOP-15407 is
> >>> merged to
> >>> >>> >> trunk.
> >>> >>> >> > > Thanks for the support.
> >>> >>> >> > >
> >>> >>> >> > > *Plan for RC*
> >>> >>> >> > > 3.2 branch cut and reset trunk : *25th Tuesday*
> >>> >>> >> > > RC0 for 3.2: *28th Friday*
> >>> >>> >> > >
> >>> >>> >> > > Thank You
> >>> >>> >> > > Sunil
> >>> >>> >> > >
> >>> >>> >> > >
> >>> >>> >> > > On Mon, Sep 17, 2018 at 3:21 PM Sunil G <sunilg@apache.org
> >>> <ma...@apache.org>>
> >>> >>> wrote:
> >>> >>> >> > >
> >>> >>> >> > >> Hi All
> >>> >>> >> > >>
> >>> >>> >> > >> We are down to 3 Blockers and 4 Critical now. Thanks all of
> >>> you
> >>> >>> for
> >>> >>> >> > >> helping in this. I am following up on these tickets, once
> its
> >>> >>> closed
> >>> >>> >> we
> >>> >>> >> > >> will cut the 3.2 branch.
> >>> >>> >> > >>
> >>> >>> >> > >> Thanks
> >>> >>> >> > >> Sunil Govindan
> >>> >>> >> > >>
> >>> >>> >> > >>
> >>> >>> >> > >> On Wed, Sep 12, 2018 at 5:10 PM Sunil G <sunilg@apache.org
> >>> <ma...@apache.org>>
> >>> >>> wrote:
> >>> >>> >> > >>
> >>> >>> >> > >>> Hi All,
> >>> >>> >> > >>>
> >>> >>> >> > >>> Inline with the original 3.2 communication proposal dated
> >>> 17th
> >>> >>> July
> >>> >>> >> > >>> 2018, I would like to provide more updates.
> >>> >>> >> > >>>
> >>> >>> >> > >>> We are approaching previously proposed code freeze date
> >>> >>> (September
> >>> >>> >> 14,
> >>> >>> >> > >>> 2018). So I would like to cut 3.2 branch on 17th Sept and
> >>> point
> >>> >>> >> > existing
> >>> >>> >> > >>> trunk to 3.3 if there are no issues.
> >>> >>> >> > >>>
> >>> >>> >> > >>> *Current Release Plan:*
> >>> >>> >> > >>> Feature freeze date : all features to merge by September
> 7,
> >>> >>> 2018.
> >>> >>> >> > >>> Code freeze date : blockers/critical only, no improvements
> >>> and
> >>> >>> >> > >>> blocker/critical bug-fixes September 14, 2018.
> >>> >>> >> > >>> Release date: September 28, 2018
> >>> >>> >> > >>>
> >>> >>> >> > >>> If any critical/blocker tickets which are targeted to
> >>> 3.2.0, we
> >>> >>> >> need to
> >>> >>> >> > >>> backport to 3.2 post branch cut.
> >>> >>> >> > >>>
> >>> >>> >> > >>> Here's an updated 3.2.0 feature status:
> >>> >>> >> > >>>
> >>> >>> >> > >>> 1. Merged & Completed features:
> >>> >>> >> > >>>
> >>> >>> >> > >>> - (Wangda) YARN-8561: Hadoop Submarine project for
> >>> DeepLearning
> >>> >>> >> > >>> workloads Initial cut.
> >>> >>> >> > >>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
> >>> >>> >> > >>> - (Sunil) YARN-7494: Multi Node scheduling support in
> >>> Capacity
> >>> >>> >> > >>> Scheduler.
> >>> >>> >> > >>> - (Chandni/Eric) YARN-7512: Support service upgrade via
> YARN
> >>> >>> Service
> >>> >>> >> > API
> >>> >>> >> > >>> and CLI.
> >>> >>> >> > >>> - (Naga/Sunil) YARN-3409: Node Attributes support in YARN.
> >>> >>> >> > >>> - (Inigo) HDFS-12615: Router-based HDFS federation.
> >>> Improvement
> >>> >>> >> works.
> >>> >>> >> > >>>
> >>> >>> >> > >>> 2. Features close to finish:
> >>> >>> >> > >>>
> >>> >>> >> > >>> - (Steve) S3Guard Phase III. Close to commit.
> >>> >>> >> > >>> - (Steve) S3a phase V. Close to commit.
> >>> >>> >> > >>> - (Steve) Support Windows Azure Storage. Close to commit.
> >>> >>> >> > >>>
> >>> >>> >> > >>> 3. Tentative/Cancelled features for 3.2:
> >>> >>> >> > >>> - (Rohith) YARN-5742: Serve aggregated logs of historical
> >>> apps
> >>> >>> from
> >>> >>> >> > >>> ATSv2. Patch in progress.
> >>> >>> >> > >>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
> >>> >>> >> challenging to
> >>> >>> >> > >>> be done before Aug 2018.
> >>> >>> >> > >>> - (Eric) YARN-7129: Application Catalog for YARN
> >>> applications.
> >>> >>> >> > >>> Challenging as more discussions are on-going.
> >>> >>> >> > >>>
> >>> >>> >> > >>> *Summary of 3.2.0 issues status:*
> >>> >>> >> > >>> 19 Blocker and Critical issues [1] are open, I am
> following
> >>> up
> >>> >>> with
> >>> >>> >> > >>> owners to get status on each of them to get in by Code
> >>> Freeze
> >>> >>> date.
> >>> >>> >> > >>>
> >>> >>> >> > >>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
> priority
> >>> in
> >>> >>> >> > (Blocker,
> >>> >>> >> > >>> Critical) AND resolution = Unresolved AND "Target
> >>> Version/s" =
> >>> >>> 3.2.0
> >>> >>> >> > ORDER
> >>> >>> >> > >>> BY priority DESC
> >>> >>> >> > >>>
> >>> >>> >> > >>> Thanks,
> >>> >>> >> > >>> Sunil
> >>> >>> >> > >>>
> >>> >>> >> > >>>
> >>> >>> >> > >>>
> >>> >>> >> > >>> On Thu, Aug 30, 2018 at 9:59 PM Sunil G <
> sunilg@apache.org
> >>> <ma...@apache.org>>
> >>> >>> wrote:
> >>> >>> >> > >>>
> >>> >>> >> > >>>> Hi All,
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> Inline with earlier communication dated 17th July 2018, I
> >>> would
> >>> >>> >> like
> >>> >>> >> > to
> >>> >>> >> > >>>> provide some updates.
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> We are approaching previously proposed code freeze date
> >>> (Aug
> >>> >>> 31).
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> One of the critical feature Node Attributes feature merge
> >>> >>> >> > >>>> discussion/vote is ongoing. Also few other Blocker bugs
> >>> need a
> >>> >>> bit
> >>> >>> >> > more
> >>> >>> >> > >>>> time. With regard to this, suggesting to push the
> >>> feature/code
> >>> >>> >> freeze
> >>> >>> >> > for 2
> >>> >>> >> > >>>> more weeks to accommodate these jiras too.
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> Proposing Updated changes in plan inline with this:
> >>> >>> >> > >>>> Feature freeze date : all features to merge by September
> 7,
> >>> >>> 2018.
> >>> >>> >> > >>>> Code freeze date : blockers/critical only, no
> improvements
> >>> and
> >>> >>> >> > >>>>  blocker/critical bug-fixes September 14, 2018.
> >>> >>> >> > >>>> Release date: September 28, 2018
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> If any features in branch which are targeted to 3.2.0,
> >>> please
> >>> >>> >> reply to
> >>> >>> >> > >>>> this email thread.
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> *Here's an updated 3.2.0 feature status:*
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> 1. Merged & Completed features:
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> - (Wangda) YARN-8561: Hadoop Submarine project for
> >>> DeepLearning
> >>> >>> >> > >>>> workloads Initial cut.
> >>> >>> >> > >>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
> >>> >>> >> > >>>> - (Sunil) YARN-7494: Multi Node scheduling support in
> >>> Capacity
> >>> >>> >> > >>>> Scheduler.
> >>> >>> >> > >>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
> >>> YARN
> >>> >>> >> Service
> >>> >>> >> > >>>> API and CLI.
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> 2. Features close to finish:
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
> YARN.
> >>> >>> >> Merge/Vote
> >>> >>> >> > >>>> Ongoing.
> >>> >>> >> > >>>> - (Rohith) YARN-5742: Serve aggregated logs of historical
> >>> apps
> >>> >>> from
> >>> >>> >> > >>>> ATSv2. Patch in progress.
> >>> >>> >> > >>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
> >>> >>> Improvement
> >>> >>> >> > works.
> >>> >>> >> > >>>> - (Steve) S3Guard Phase III, S3a phase V, Support Windows
> >>> Azure
> >>> >>> >> > >>>> Storage. In progress.
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> 3. Tentative features:
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
> >>> >>> >> challenging
> >>> >>> >> > to
> >>> >>> >> > >>>> be done before Aug 2018.
> >>> >>> >> > >>>> - (Eric) YARN-7129: Application Catalog for YARN
> >>> applications.
> >>> >>> >> > >>>> Challenging as more discussions are on-going.
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> *Summary of 3.2.0 issues status:*
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> 26 Blocker and Critical issues [1] are open, I am
> >>> following up
> >>> >>> with
> >>> >>> >> > >>>> owners to get status on each of them to get in by Code
> >>> Freeze
> >>> >>> date.
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
> >>> priority in
> >>> >>> >> > >>>> (Blocker, Critical) AND resolution = Unresolved AND
> "Target
> >>> >>> >> > Version/s" =
> >>> >>> >> > >>>> 3.2.0 ORDER BY priority DESC
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> Thanks,
> >>> >>> >> > >>>> Sunil
> >>> >>> >> > >>>>
> >>> >>> >> > >>>> On Tue, Aug 14, 2018 at 10:30 PM Sunil G <
> >>> sunilg@apache.org<ma...@apache.org>>
> >>> >>> >> wrote:
> >>> >>> >> > >>>>
> >>> >>> >> > >>>>> Hi All,
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> Thanks for the feedbacks. Inline with earlier
> >>> communication
> >>> >>> dated
> >>> >>> >> > 17th
> >>> >>> >> > >>>>> July 2018, I would like to provide some updates.
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> We are approaching previously proposed feature freeze
> date
> >>> >>> (Aug
> >>> >>> >> 21,
> >>> >>> >> > >>>>> about 7 days from today).
> >>> >>> >> > >>>>> If any features in branch which are targeted to 3.2.0,
> >>> please
> >>> >>> >> reply
> >>> >>> >> > to
> >>> >>> >> > >>>>> this email thread.
> >>> >>> >> > >>>>> Steve has mentioned about the s3 features which will
> come
> >>> >>> close to
> >>> >>> >> > >>>>> Code Freeze Date (Aug 31st).
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> *Here's an updated 3.2.0 feature status:*
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> 1. Merged & Completed features:
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> - (Wangda) YARN-8561: Hadoop Submarine project for
> >>> >>> DeepLearning
> >>> >>> >> > >>>>> workloads Initial cut.
> >>> >>> >> > >>>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> 2. Features close to finish:
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
> YARN.
> >>> >>> Major
> >>> >>> >> > >>>>> patches are all in, only one last
> >>> >>> >> > >>>>> patch is in review state.
> >>> >>> >> > >>>>> - (Sunil) YARN-7494: Multi Node scheduling support in
> >>> Capacity
> >>> >>> >> > >>>>> Scheduler. Close to commit.
> >>> >>> >> > >>>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
> >>> YARN
> >>> >>> >> Service
> >>> >>> >> > >>>>> API and CLI. 2 patches are pending
> >>> >>> >> > >>>>> which will be closed by Feature freeze date.
> >>> >>> >> > >>>>> - (Rohith) YARN-5742: Serve aggregated logs of
> historical
> >>> apps
> >>> >>> >> from
> >>> >>> >> > >>>>> ATSv2. Patch in progress.
> >>> >>> >> > >>>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
> >>> >>> Improvement
> >>> >>> >> > >>>>> works.
> >>> >>> >> > >>>>> - (Steve) S3Guard Phase III, S3a phase V, Support
> Windows
> >>> >>> Azure
> >>> >>> >> > >>>>> Storage. In progress.
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> 3. Tentative features:
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
> >>> >>> >> challenging
> >>> >>> >> > >>>>> to be done before Aug 2018.
> >>> >>> >> > >>>>> - (Eric) YARN-7129: Application Catalog for YARN
> >>> applications.
> >>> >>> >> > >>>>> Challenging as more discussions are on-going.
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> *Summary of 3.2.0 issues status:*
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> 39 Blocker and Critical issues [1] are open, I am
> checking
> >>> >>> with
> >>> >>> >> > owners
> >>> >>> >> > >>>>> to get status on each of them to get in by Code Freeze
> >>> date.
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
> >>> priority in
> >>> >>> >> > >>>>> (Blocker, Critical) AND resolution = Unresolved AND
> >>> "Target
> >>> >>> >> > Version/s" =
> >>> >>> >> > >>>>> 3.2.0 ORDER BY priority DESC
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> Thanks,
> >>> >>> >> > >>>>> Sunil
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>> On Fri, Jul 20, 2018 at 8:03 AM Sunil G <
> >>> sunilg@apache.org<ma...@apache.org>>
> >>> >>> >> wrote:
> >>> >>> >> > >>>>>
> >>> >>> >> > >>>>>> Thanks Subru for the thoughts.
> >>> >>> >> > >>>>>> One of the main reason for a major release is to push
> out
> >>> >>> >> critical
> >>> >>> >> > >>>>>> features with a faster cadence to the users. If we are
> >>> >>> pulling
> >>> >>> >> more
> >>> >>> >> > and
> >>> >>> >> > >>>>>> more different types of features to a minor release,
> that
> >>> >>> branch
> >>> >>> >> > will
> >>> >>> >> > >>>>>> become more destabilized and it may be tough to say
> that
> >>> >>> 3.1.2 is
> >>> >>> >> > stable
> >>> >>> >> > >>>>>> that 3.1.1 for eg. We always tend to improve and
> >>> stabilize
> >>> >>> >> features
> >>> >>> >> > in
> >>> >>> >> > >>>>>> subsequent minor release.
> >>> >>> >> > >>>>>> For few companies, it makes sense to push out these new
> >>> >>> features
> >>> >>> >> > >>>>>> faster to make a reach to the users. Adding to the
> point
> >>> to
> >>> >>> the
> >>> >>> >> > backporting
> >>> >>> >> > >>>>>> issues, I agree that its a pain and we can workaround
> >>> that
> >>> >>> with
> >>> >>> >> > some git
> >>> >>> >> > >>>>>> scripts. If we can make such scripts available to
> >>> committers,
> >>> >>> >> > backport will
> >>> >>> >> > >>>>>> be seem-less across branches and we can achieve the
> >>> faster
> >>> >>> >> release
> >>> >>> >> > cadence
> >>> >>> >> > >>>>>> also.
> >>> >>> >> > >>>>>>
> >>> >>> >> > >>>>>> Thoughts?
> >>> >>> >> > >>>>>>
> >>> >>> >> > >>>>>> - Sunil
> >>> >>> >> > >>>>>>
> >>> >>> >> > >>>>>>
> >>> >>> >> > >>>>>> On Fri, Jul 20, 2018 at 3:37 AM Subru Krishnan <
> >>> >>> subru@apache.org<ma...@apache.org>
> >>> >>> >> >
> >>> >>> >> > >>>>>> wrote:
> >>> >>> >> > >>>>>>
> >>> >>> >> > >>>>>>> Thanks Sunil for volunteering to lead the release
> >>> effort. I
> >>> >>> am
> >>> >>> >> > >>>>>>> generally
> >>> >>> >> > >>>>>>> supportive of a release but -1 on a 3.2 (prefer a
> >>> 3.1.x) as
> >>> >>> >> feel we
> >>> >>> >> > >>>>>>> already
> >>> >>> >> > >>>>>>> have too many branches to be maintained. I already see
> >>> many
> >>> >>> >> commits
> >>> >>> >> > >>>>>>> are in
> >>> >>> >> > >>>>>>> different branches with no apparent rationale, for
> e.g:
> >>> 3.1
> >>> >>> has
> >>> >>> >> > >>>>>>> commits
> >>> >>> >> > >>>>>>> which are absent in 3.0 etc.
> >>> >>> >> > >>>>>>>
> >>> >>> >> > >>>>>>> Additionally AFAIK 3.x has not been deployed in any
> >>> major
> >>> >>> >> > production
> >>> >>> >> > >>>>>>> setting so the cost of adding features should be
> >>> minimal.
> >>> >>> >> > >>>>>>>
> >>> >>> >> > >>>>>>> Thoughts?
> >>> >>> >> > >>>>>>>
> >>> >>> >> > >>>>>>> -Subru
> >>> >>> >> > >>>>>>>
> >>> >>> >> > >>>>>>> On Thu, Jul 19, 2018 at 12:31 AM, Sunil G <
> >>> >>> sunilg@apache.org<ma...@apache.org>>
> >>> >>> >> > wrote:
> >>> >>> >> > >>>>>>>
> >>> >>> >> > >>>>>>> > Thanks Steve, Aaron, Wangda for sharing thoughts.
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> > Yes, important changes and features are much needed,
> >>> >>> hence we
> >>> >>> >> > will
> >>> >>> >> > >>>>>>> be
> >>> >>> >> > >>>>>>> > keeping the door open for them as possible. Also
> >>> >>> considering
> >>> >>> >> few
> >>> >>> >> > >>>>>>> more
> >>> >>> >> > >>>>>>> > offline requests from other folks, I think extending
> >>> the
> >>> >>> >> > timeframe
> >>> >>> >> > >>>>>>> by
> >>> >>> >> > >>>>>>> > couple of weeks makes sense (including a second RC
> >>> >>> buffer) and
> >>> >>> >> > >>>>>>> this should
> >>> >>> >> > >>>>>>> > ideally help us to ship this by September itself.
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> > Revised dates (I will be updating same in Roadmap
> >>> wiki as
> >>> >>> >> well)
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> > - Feature freeze date : all features to merge by
> >>> August
> >>> >>> 21,
> >>> >>> >> 2018.
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> > - Code freeze date : blockers/critical only, no
> >>> >>> improvements
> >>> >>> >> and
> >>> >>> >> > >>>>>>> non
> >>> >>> >> > >>>>>>> > blocker/critical
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> > bug-fixes  August 31, 2018.
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> > - Release date: September 15, 2018
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> > Thank Eric and Zian, I think Wangda has already
> >>> answered
> >>> >>> your
> >>> >>> >> > >>>>>>> questions.
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> > Thanks
> >>> >>> >> > >>>>>>> > Sunil
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> > On Thu, Jul 19, 2018 at 12:13 PM Wangda Tan <
> >>> >>> >> wheeleast@gmail.com<ma...@gmail.com>
> >>> >>> >> > >
> >>> >>> >> > >>>>>>> wrote:
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>> > > Thanks Sunil for volunteering to be RM of 3.2
> >>> release,
> >>> >>> +1
> >>> >>> >> for
> >>> >>> >> > >>>>>>> that.
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > > To concerns from Steve,
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > > It is a good idea to keep the door open to get
> >>> important
> >>> >>> >> > changes
> >>> >>> >> > >>>>>>> /
> >>> >>> >> > >>>>>>> > > features in before cutoff. I would prefer to keep
> >>> the
> >>> >>> >> proposed
> >>> >>> >> > >>>>>>> release
> >>> >>> >> > >>>>>>> > date
> >>> >>> >> > >>>>>>> > > to make sure things can happen earlier instead of
> >>> last
> >>> >>> >> minute
> >>> >>> >> > >>>>>>> and we all
> >>> >>> >> > >>>>>>> > > know that releases are always get delayed :). I'm
> >>> also
> >>> >>> fine
> >>> >>> >> if
> >>> >>> >> > >>>>>>> we want
> >>> >>> >> > >>>>>>> > get
> >>> >>> >> > >>>>>>> > > another several weeks time.
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > > Regarding of 3.3 release, I would suggest doing
> that
> >>> >>> before
> >>> >>> >> > >>>>>>> thanksgiving.
> >>> >>> >> > >>>>>>> > > Do you think is it good or too early / late?
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > > Eric,
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > > The YARN-8220 will be replaced by YARN-8135, if
> >>> >>> YARN-8135
> >>> >>> >> can
> >>> >>> >> > >>>>>>> get merged
> >>> >>> >> > >>>>>>> > > in time, we probably not need the YARN-8220.
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > > Sunil,
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > > Could u update
> >>> >>> >> > >>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> >>> >>> >> > >>>>>>> > Roadmap
> >>> >>> >> > >>>>>>> > > with proposed plan as well? We can fill feature
> list
> >>> >>> first
> >>> >>> >> > >>>>>>> before getting
> >>> >>> >> > >>>>>>> > > consensus of time.
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > > Thanks,
> >>> >>> >> > >>>>>>> > > Wangda
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > > On Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri
> >>> >>> >> > >>>>>>> <fabbri@cloudera.com.invalid<mailto:
> >>> fabbri@cloudera.com.invalid>
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > > wrote:
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> > >> On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran <
> >>> >>> >> > >>>>>>> stevel@hortonworks.com<mailto:stevel@hortonworks.com
> >>
> >>> >>> >> > >>>>>>> > >> wrote:
> >>> >>> >> > >>>>>>> > >>
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > On 16 Jul 2018, at 23:45, Sunil G <
> >>> sunilg@apache.org<ma...@apache.org>
> >>> >>> >> > <mailto:
> >>> >>> >> > >>>>>>> > >> > sunilg@apache.org<ma...@apache.org>>>
> >>> wrote:
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > I would also would like to take this
> opportunity
> >>> to
> >>> >>> come
> >>> >>> >> up
> >>> >>> >> > >>>>>>> with a
> >>> >>> >> > >>>>>>> > >> detailed
> >>> >>> >> > >>>>>>> > >> > plan.
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > - Feature freeze date : all features should be
> >>> >>> merged by
> >>> >>> >> > >>>>>>> August 10,
> >>> >>> >> > >>>>>>> > >> 2018.
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > <snip>
> >>> >>> >> > >>>>>>> > >>
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > Please let me know if I missed any features
> >>> targeted
> >>> >>> to
> >>> >>> >> 3.2
> >>> >>> >> > >>>>>>> per this
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > Well there these big todo lists for S3 &
> S3Guard.
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> https://issues.apache.org/jira/browse/HADOOP-15226
> >>> >>> >> > >>>>>>> > >> >
> >>> https://issues.apache.org/jira/browse/HADOOP-15220
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > There's a bigger bit of work coming on for
> Azure
> >>> >>> Datalake
> >>> >>> >> > Gen
> >>> >>> >> > >>>>>>> 2
> >>> >>> >> > >>>>>>> > >> >
> >>> https://issues.apache.org/jira/browse/HADOOP-15407
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > I don't think this is quite ready yet, I've
> been
> >>> >>> doing
> >>> >>> >> work
> >>> >>> >> > >>>>>>> on it, but
> >>> >>> >> > >>>>>>> > >> if
> >>> >>> >> > >>>>>>> > >> > we have a 3 week deadline, I'm going to expect
> >>> some
> >>> >>> >> timely
> >>> >>> >> > >>>>>>> reviews on
> >>> >>> >> > >>>>>>> > >> >
> >>> https://issues.apache.org/jira/browse/HADOOP-15546
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > I've uprated that to a blocker feature; will
> >>> review
> >>> >>> the
> >>> >>> >> S3 &
> >>> >>> >> > >>>>>>> S3Guard
> >>> >>> >> > >>>>>>> > >> JIRAs
> >>> >>> >> > >>>>>>> > >> > to see which of those are blocking. Then there
> >>> are
> >>> >>> some
> >>> >>> >> > >>>>>>> pressing
> >>> >>> >> > >>>>>>> > "guave,
> >>> >>> >> > >>>>>>> > >> > java 9 prep"
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >>  I can help with this part if you like.
> >>> >>> >> > >>>>>>> > >>
> >>> >>> >> > >>>>>>> > >>
> >>> >>> >> > >>>>>>> > >>
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > timeline. I would like to volunteer myself as
> >>> release
> >>> >>> >> > manager
> >>> >>> >> > >>>>>>> of 3.2.0
> >>> >>> >> > >>>>>>> > >> > release.
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > well volunteered!
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> Yes, thank you for stepping up.
> >>> >>> >> > >>>>>>> > >>
> >>> >>> >> > >>>>>>> > >>
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > I think this raises a good q: what timetable
> >>> should
> >>> >>> we
> >>> >>> >> have
> >>> >>> >> > >>>>>>> for the
> >>> >>> >> > >>>>>>> > >> 3.2. &
> >>> >>> >> > >>>>>>> > >> > 3.3 releases; if we do want a faster cadence,
> >>> then
> >>> >>> having
> >>> >>> >> > the
> >>> >>> >> > >>>>>>> outline
> >>> >>> >> > >>>>>>> > >> time
> >>> >>> >> > >>>>>>> > >> > from the 3.2 to the 3.3 release means that
> >>> there's
> >>> >>> less
> >>> >>> >> > >>>>>>> concern about
> >>> >>> >> > >>>>>>> > >> > things not making the 3.2 dealine
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> > -Steve
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> >
> >>> >>> >> > >>>>>>> > >> Good idea to mitigate the short deadline.
> >>> >>> >> > >>>>>>> > >>
> >>> >>> >> > >>>>>>> > >> -AF
> >>> >>> >> > >>>>>>> > >>
> >>> >>> >> > >>>>>>> > >
> >>> >>> >> > >>>>>>> >
> >>> >>> >> > >>>>>>>
> >>> >>> >> > >>>>>>
> >>> >>> >> >
> >>> >>> >>
> >>> >>> >
> >>> >>>
> >>> >>
> >>> >>
> >>>
> >>>
> >>> --
> >>> A very happy Clouderan
> >>>
> >>>
>

Re: Hadoop 3.2 Release Plan proposal

Posted by Sunil G <su...@apache.org>.
Hi Folks,

Given all blockers are closed now, I am planning to do RC1 tomorrow.
branch-3.2.0 remains closed for commit. Please let me know if any issues
are missed.

Thanks,
Sunil G

On Tue, Nov 13, 2018 at 8:11 PM Sunil G <su...@apache.org> wrote:

> Hi Folks,
>
> All blockers are closed by last weekend and corrected jiras as well.
> Preparing RC0 now. Facing some issue with shaded jars file size. Hence
> respinning again.
> Planning to complete this by end of this week.
>
> Thanks
> Sunil
>
> On Fri, Oct 26, 2018 at 2:34 AM Konstantin Shvachko <sh...@gmail.com>
> wrote:
>
>> Another thing is that I see a bunch of jiras under HDFS-8707, which don't
>> have the Fix Version field listing 3.2, some have it just empty.
>> This means they will not be populated into release notes.
>>
>> Thanks,
>> --Konstantin
>>
>> On Thu, Oct 25, 2018 at 7:59 AM Sunil G <su...@apache.org> wrote:
>>
>> > Thanks Konstantin for pointing out.
>> > As 3.2 is pretty much on RC level, its better we try to find a good
>> > solution to this issue.
>> >
>> > I ll follow up on this in the jira.
>> >
>> > - Sunil
>> >
>> > On Thu, Oct 25, 2018 at 11:35 AM Konstantin Shvachko <
>> shv.hadoop@gmail.com>
>> > wrote:
>> >
>> >> I've tried to attract attention to an incompatibility issue through the
>> >> jira, but it didn't work. So pitching in in this thread.
>> >> https://issues.apache.org/jira/browse/HDFS-12026
>> >> It introduced binary incompatibility, which will prevent people from
>> >> upgrading from 3.1 to 3.2.
>> >> I think it can get messy if we release anything with this feature.
>> >>
>> >> Thanks,
>> >> --Konstantin
>> >>
>> >> On Mon, Oct 22, 2018 at 5:01 AM Steve Loughran <stevel@hortonworks.com
>> >
>> >> wrote:
>> >>
>> >>> its in.
>> >>>
>> >>> good catch!
>> >>>
>> >>> On 20 Oct 2018, at 01:35, Wei-Chiu Chuang <weichiu@cloudera.com
>> <mailto:
>> >>> weichiu@cloudera.com>> wrote:
>> >>>
>> >>> Thanks Sunil G for driving the release,
>> >>> I filed HADOOP-15866<
>> https://issues.apache.org/jira/browse/HADOOP-15866>
>> >>> for a compat fix. If any one has cycle please review it, as I think
>> it is
>> >>> needed for 3.2.0.
>> >>>
>> >>> On Thu, Oct 18, 2018 at 4:43 AM Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>> Hi Folks,
>> >>>
>> >>> As we previously communicated for 3.2.0 release, we have delayed due
>> to
>> >>> few
>> >>> blockers in our gate.
>> >>>
>> >>> I just cut branch-3.2.0 for release purpose. branch-3.2 will be open
>> for
>> >>> all bug fixes.
>> >>>
>> >>> - Sunil
>> >>>
>> >>>
>> >>> On Tue, Oct 16, 2018 at 8:59 AM Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>>
>> >>> > Hi Folks,
>> >>> >
>> >>> > We are now close to RC as other blocker issues are now merged to
>> trunk
>> >>> and
>> >>> > branch-3.2. Last 2 critical issues are closer to merge and will be
>> >>> > committed in few hours.
>> >>> > With this, I will be creating 3.2.0 branch today and will go ahead
>> >>> with RC
>> >>> > related process.
>> >>> >
>> >>> > - Sunil
>> >>> >
>> >>> > On Mon, Oct 15, 2018 at 11:43 PM Jonathan Bender <
>> jonbender@stripe.com
>> >>> <ma...@stripe.com>>
>> >>> > wrote:
>> >>> >
>> >>> >> Hello, were there any updates around the 3.2.0 RC timing? All I
>> see in
>> >>> >> the current blockers are related to the new Submarine subproject,
>> >>> wasn't
>> >>> >> sure if that is what is holding things up.
>> >>> >>
>> >>> >> Cheers,
>> >>> >> Jon
>> >>> >>
>> >>> >> On Tue, Oct 2, 2018 at 7:13 PM, Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>> >>
>> >>> >>> Thanks Robert and Haibo for quickly correcting same.
>> >>> >>> Sigh, I somehow missed one file while committing the change. Sorry
>> >>> for
>> >>> >>> the
>> >>> >>> trouble.
>> >>> >>>
>> >>> >>> - Sunil
>> >>> >>>
>> >>> >>> On Wed, Oct 3, 2018 at 5:22 AM Robert Kanter <
>> rkanter@cloudera.com
>> >>> <ma...@cloudera.com>>
>> >>> >>> wrote:
>> >>> >>>
>> >>> >>> > Looks like there's two that weren't updated:
>> >>> >>> > >> [115] 16:32 : hadoop-common (trunk) :: grep "3.2.0-SNAPSHOT"
>> .
>> >>> -r
>> >>> >>> > --include=pom.xml
>> >>> >>> > ./hadoop-project/pom.xml:
>> >>> >>> >
>> >>> <hadoop.assemblies.version>3.2.0-SNAPSHOT</hadoop.assemblies.version>
>> >>> >>> > ./pom.xml:    <hadoop.version>3.2.0-SNAPSHOT</hadoop.version>
>> >>> >>> >
>> >>> >>> > I've just pushed in an addendum commit to fix those.
>> >>> >>> > In the future, please make sure to do a sanity compile when
>> >>> updating
>> >>> >>> poms.
>> >>> >>> >
>> >>> >>> > thanks
>> >>> >>> > - Robert
>> >>> >>> >
>> >>> >>> > On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri
>> >>> >>> <fa...@cloudera.com.invalid>>
>> >>> >>> > wrote:
>> >>> >>> >
>> >>> >>> >> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in
>> >>> the
>> >>> >>> >> top-level pom.xml?
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> On Tue, Oct 2, 2018 at 10:16 AM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>> wrote:
>> >>> >>> >>
>> >>> >>> >> > Hi All
>> >>> >>> >> >
>> >>> >>> >> > As mentioned in earlier mail, I have cut branch-3.2 and reset
>> >>> trunk
>> >>> >>> to
>> >>> >>> >> > 3.3.0-SNAPSHOT. I will share the RC details sooner once all
>> >>> >>> necessary
>> >>> >>> >> > patches are pulled into branch-3.2.
>> >>> >>> >> >
>> >>> >>> >> > Thank You
>> >>> >>> >> > - Sunil
>> >>> >>> >> >
>> >>> >>> >> >
>> >>> >>> >> > On Mon, Sep 24, 2018 at 2:00 PM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>> wrote:
>> >>> >>> >> >
>> >>> >>> >> > > Hi All
>> >>> >>> >> > >
>> >>> >>> >> > > We are now down to the last Blocker and HADOOP-15407 is
>> >>> merged to
>> >>> >>> >> trunk.
>> >>> >>> >> > > Thanks for the support.
>> >>> >>> >> > >
>> >>> >>> >> > > *Plan for RC*
>> >>> >>> >> > > 3.2 branch cut and reset trunk : *25th Tuesday*
>> >>> >>> >> > > RC0 for 3.2: *28th Friday*
>> >>> >>> >> > >
>> >>> >>> >> > > Thank You
>> >>> >>> >> > > Sunil
>> >>> >>> >> > >
>> >>> >>> >> > >
>> >>> >>> >> > > On Mon, Sep 17, 2018 at 3:21 PM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >
>> >>> >>> >> > >> Hi All
>> >>> >>> >> > >>
>> >>> >>> >> > >> We are down to 3 Blockers and 4 Critical now. Thanks all
>> of
>> >>> you
>> >>> >>> for
>> >>> >>> >> > >> helping in this. I am following up on these tickets, once
>> its
>> >>> >>> closed
>> >>> >>> >> we
>> >>> >>> >> > >> will cut the 3.2 branch.
>> >>> >>> >> > >>
>> >>> >>> >> > >> Thanks
>> >>> >>> >> > >> Sunil Govindan
>> >>> >>> >> > >>
>> >>> >>> >> > >>
>> >>> >>> >> > >> On Wed, Sep 12, 2018 at 5:10 PM Sunil G <
>> sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >>
>> >>> >>> >> > >>> Hi All,
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Inline with the original 3.2 communication proposal dated
>> >>> 17th
>> >>> >>> July
>> >>> >>> >> > >>> 2018, I would like to provide more updates.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> We are approaching previously proposed code freeze date
>> >>> >>> (September
>> >>> >>> >> 14,
>> >>> >>> >> > >>> 2018). So I would like to cut 3.2 branch on 17th Sept and
>> >>> point
>> >>> >>> >> > existing
>> >>> >>> >> > >>> trunk to 3.3 if there are no issues.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> *Current Release Plan:*
>> >>> >>> >> > >>> Feature freeze date : all features to merge by September
>> 7,
>> >>> >>> 2018.
>> >>> >>> >> > >>> Code freeze date : blockers/critical only, no
>> improvements
>> >>> and
>> >>> >>> >> > >>> blocker/critical bug-fixes September 14, 2018.
>> >>> >>> >> > >>> Release date: September 28, 2018
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> If any critical/blocker tickets which are targeted to
>> >>> 3.2.0, we
>> >>> >>> >> need to
>> >>> >>> >> > >>> backport to 3.2 post branch cut.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Here's an updated 3.2.0 feature status:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 1. Merged & Completed features:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> DeepLearning
>> >>> >>> >> > >>> workloads Initial cut.
>> >>> >>> >> > >>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>> Scheduler.
>> >>> >>> >> > >>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> YARN
>> >>> >>> Service
>> >>> >>> >> > API
>> >>> >>> >> > >>> and CLI.
>> >>> >>> >> > >>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> >> > >>> - (Inigo) HDFS-12615: Router-based HDFS federation.
>> >>> Improvement
>> >>> >>> >> works.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 2. Features close to finish:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> - (Steve) S3Guard Phase III. Close to commit.
>> >>> >>> >> > >>> - (Steve) S3a phase V. Close to commit.
>> >>> >>> >> > >>> - (Steve) Support Windows Azure Storage. Close to commit.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 3. Tentative/Cancelled features for 3.2:
>> >>> >>> >> > >>> - (Rohith) YARN-5742: Serve aggregated logs of historical
>> >>> apps
>> >>> >>> from
>> >>> >>> >> > >>> ATSv2. Patch in progress.
>> >>> >>> >> > >>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
>> >>> >>> >> challenging to
>> >>> >>> >> > >>> be done before Aug 2018.
>> >>> >>> >> > >>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>> 19 Blocker and Critical issues [1] are open, I am
>> following
>> >>> up
>> >>> >>> with
>> >>> >>> >> > >>> owners to get status on each of them to get in by Code
>> >>> Freeze
>> >>> >>> date.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> priority
>> >>> in
>> >>> >>> >> > (Blocker,
>> >>> >>> >> > >>> Critical) AND resolution = Unresolved AND "Target
>> >>> Version/s" =
>> >>> >>> 3.2.0
>> >>> >>> >> > ORDER
>> >>> >>> >> > >>> BY priority DESC
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Thanks,
>> >>> >>> >> > >>> Sunil
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> On Thu, Aug 30, 2018 at 9:59 PM Sunil G <
>> sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>> Hi All,
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Inline with earlier communication dated 17th July 2018,
>> I
>> >>> would
>> >>> >>> >> like
>> >>> >>> >> > to
>> >>> >>> >> > >>>> provide some updates.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> We are approaching previously proposed code freeze date
>> >>> (Aug
>> >>> >>> 31).
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> One of the critical feature Node Attributes feature
>> merge
>> >>> >>> >> > >>>> discussion/vote is ongoing. Also few other Blocker bugs
>> >>> need a
>> >>> >>> bit
>> >>> >>> >> > more
>> >>> >>> >> > >>>> time. With regard to this, suggesting to push the
>> >>> feature/code
>> >>> >>> >> freeze
>> >>> >>> >> > for 2
>> >>> >>> >> > >>>> more weeks to accommodate these jiras too.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Proposing Updated changes in plan inline with this:
>> >>> >>> >> > >>>> Feature freeze date : all features to merge by
>> September 7,
>> >>> >>> 2018.
>> >>> >>> >> > >>>> Code freeze date : blockers/critical only, no
>> improvements
>> >>> and
>> >>> >>> >> > >>>>  blocker/critical bug-fixes September 14, 2018.
>> >>> >>> >> > >>>> Release date: September 28, 2018
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> If any features in branch which are targeted to 3.2.0,
>> >>> please
>> >>> >>> >> reply to
>> >>> >>> >> > >>>> this email thread.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> *Here's an updated 3.2.0 feature status:*
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 1. Merged & Completed features:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> DeepLearning
>> >>> >>> >> > >>>> workloads Initial cut.
>> >>> >>> >> > >>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>>> Scheduler.
>> >>> >>> >> > >>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> >>> YARN
>> >>> >>> >> Service
>> >>> >>> >> > >>>> API and CLI.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 2. Features close to finish:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> >> Merge/Vote
>> >>> >>> >> > >>>> Ongoing.
>> >>> >>> >> > >>>> - (Rohith) YARN-5742: Serve aggregated logs of
>> historical
>> >>> apps
>> >>> >>> from
>> >>> >>> >> > >>>> ATSv2. Patch in progress.
>> >>> >>> >> > >>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
>> >>> >>> Improvement
>> >>> >>> >> > works.
>> >>> >>> >> > >>>> - (Steve) S3Guard Phase III, S3a phase V, Support
>> Windows
>> >>> Azure
>> >>> >>> >> > >>>> Storage. In progress.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 3. Tentative features:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
>> >>> >>> >> challenging
>> >>> >>> >> > to
>> >>> >>> >> > >>>> be done before Aug 2018.
>> >>> >>> >> > >>>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 26 Blocker and Critical issues [1] are open, I am
>> >>> following up
>> >>> >>> with
>> >>> >>> >> > >>>> owners to get status on each of them to get in by Code
>> >>> Freeze
>> >>> >>> date.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> >>> priority in
>> >>> >>> >> > >>>> (Blocker, Critical) AND resolution = Unresolved AND
>> "Target
>> >>> >>> >> > Version/s" =
>> >>> >>> >> > >>>> 3.2.0 ORDER BY priority DESC
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Thanks,
>> >>> >>> >> > >>>> Sunil
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> On Tue, Aug 14, 2018 at 10:30 PM Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> wrote:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>>> Hi All,
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> Thanks for the feedbacks. Inline with earlier
>> >>> communication
>> >>> >>> dated
>> >>> >>> >> > 17th
>> >>> >>> >> > >>>>> July 2018, I would like to provide some updates.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> We are approaching previously proposed feature freeze
>> date
>> >>> >>> (Aug
>> >>> >>> >> 21,
>> >>> >>> >> > >>>>> about 7 days from today).
>> >>> >>> >> > >>>>> If any features in branch which are targeted to 3.2.0,
>> >>> please
>> >>> >>> >> reply
>> >>> >>> >> > to
>> >>> >>> >> > >>>>> this email thread.
>> >>> >>> >> > >>>>> Steve has mentioned about the s3 features which will
>> come
>> >>> >>> close to
>> >>> >>> >> > >>>>> Code Freeze Date (Aug 31st).
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> *Here's an updated 3.2.0 feature status:*
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 1. Merged & Completed features:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> >>> DeepLearning
>> >>> >>> >> > >>>>> workloads Initial cut.
>> >>> >>> >> > >>>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 2. Features close to finish:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> Major
>> >>> >>> >> > >>>>> patches are all in, only one last
>> >>> >>> >> > >>>>> patch is in review state.
>> >>> >>> >> > >>>>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>>>> Scheduler. Close to commit.
>> >>> >>> >> > >>>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> >>> YARN
>> >>> >>> >> Service
>> >>> >>> >> > >>>>> API and CLI. 2 patches are pending
>> >>> >>> >> > >>>>> which will be closed by Feature freeze date.
>> >>> >>> >> > >>>>> - (Rohith) YARN-5742: Serve aggregated logs of
>> historical
>> >>> apps
>> >>> >>> >> from
>> >>> >>> >> > >>>>> ATSv2. Patch in progress.
>> >>> >>> >> > >>>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
>> >>> >>> Improvement
>> >>> >>> >> > >>>>> works.
>> >>> >>> >> > >>>>> - (Steve) S3Guard Phase III, S3a phase V, Support
>> Windows
>> >>> >>> Azure
>> >>> >>> >> > >>>>> Storage. In progress.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 3. Tentative features:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Haibo Chen) YARN-1011: Resource overcommitment.
>> Looks
>> >>> >>> >> challenging
>> >>> >>> >> > >>>>> to be done before Aug 2018.
>> >>> >>> >> > >>>>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>>>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 39 Blocker and Critical issues [1] are open, I am
>> checking
>> >>> >>> with
>> >>> >>> >> > owners
>> >>> >>> >> > >>>>> to get status on each of them to get in by Code Freeze
>> >>> date.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> >>> priority in
>> >>> >>> >> > >>>>> (Blocker, Critical) AND resolution = Unresolved AND
>> >>> "Target
>> >>> >>> >> > Version/s" =
>> >>> >>> >> > >>>>> 3.2.0 ORDER BY priority DESC
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> Thanks,
>> >>> >>> >> > >>>>> Sunil
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> On Fri, Jul 20, 2018 at 8:03 AM Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> wrote:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>>> Thanks Subru for the thoughts.
>> >>> >>> >> > >>>>>> One of the main reason for a major release is to push
>> out
>> >>> >>> >> critical
>> >>> >>> >> > >>>>>> features with a faster cadence to the users. If we are
>> >>> >>> pulling
>> >>> >>> >> more
>> >>> >>> >> > and
>> >>> >>> >> > >>>>>> more different types of features to a minor release,
>> that
>> >>> >>> branch
>> >>> >>> >> > will
>> >>> >>> >> > >>>>>> become more destabilized and it may be tough to say
>> that
>> >>> >>> 3.1.2 is
>> >>> >>> >> > stable
>> >>> >>> >> > >>>>>> that 3.1.1 for eg. We always tend to improve and
>> >>> stabilize
>> >>> >>> >> features
>> >>> >>> >> > in
>> >>> >>> >> > >>>>>> subsequent minor release.
>> >>> >>> >> > >>>>>> For few companies, it makes sense to push out these
>> new
>> >>> >>> features
>> >>> >>> >> > >>>>>> faster to make a reach to the users. Adding to the
>> point
>> >>> to
>> >>> >>> the
>> >>> >>> >> > backporting
>> >>> >>> >> > >>>>>> issues, I agree that its a pain and we can workaround
>> >>> that
>> >>> >>> with
>> >>> >>> >> > some git
>> >>> >>> >> > >>>>>> scripts. If we can make such scripts available to
>> >>> committers,
>> >>> >>> >> > backport will
>> >>> >>> >> > >>>>>> be seem-less across branches and we can achieve the
>> >>> faster
>> >>> >>> >> release
>> >>> >>> >> > cadence
>> >>> >>> >> > >>>>>> also.
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> Thoughts?
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> - Sunil
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> On Fri, Jul 20, 2018 at 3:37 AM Subru Krishnan <
>> >>> >>> subru@apache.org<ma...@apache.org>
>> >>> >>> >> >
>> >>> >>> >> > >>>>>> wrote:
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>>> Thanks Sunil for volunteering to lead the release
>> >>> effort. I
>> >>> >>> am
>> >>> >>> >> > >>>>>>> generally
>> >>> >>> >> > >>>>>>> supportive of a release but -1 on a 3.2 (prefer a
>> >>> 3.1.x) as
>> >>> >>> >> feel we
>> >>> >>> >> > >>>>>>> already
>> >>> >>> >> > >>>>>>> have too many branches to be maintained. I already
>> see
>> >>> many
>> >>> >>> >> commits
>> >>> >>> >> > >>>>>>> are in
>> >>> >>> >> > >>>>>>> different branches with no apparent rationale, for
>> e.g:
>> >>> 3.1
>> >>> >>> has
>> >>> >>> >> > >>>>>>> commits
>> >>> >>> >> > >>>>>>> which are absent in 3.0 etc.
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> Additionally AFAIK 3.x has not been deployed in any
>> >>> major
>> >>> >>> >> > production
>> >>> >>> >> > >>>>>>> setting so the cost of adding features should be
>> >>> minimal.
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> Thoughts?
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> -Subru
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> On Thu, Jul 19, 2018 at 12:31 AM, Sunil G <
>> >>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> > wrote:
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> > Thanks Steve, Aaron, Wangda for sharing thoughts.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Yes, important changes and features are much
>> needed,
>> >>> >>> hence we
>> >>> >>> >> > will
>> >>> >>> >> > >>>>>>> be
>> >>> >>> >> > >>>>>>> > keeping the door open for them as possible. Also
>> >>> >>> considering
>> >>> >>> >> few
>> >>> >>> >> > >>>>>>> more
>> >>> >>> >> > >>>>>>> > offline requests from other folks, I think
>> extending
>> >>> the
>> >>> >>> >> > timeframe
>> >>> >>> >> > >>>>>>> by
>> >>> >>> >> > >>>>>>> > couple of weeks makes sense (including a second RC
>> >>> >>> buffer) and
>> >>> >>> >> > >>>>>>> this should
>> >>> >>> >> > >>>>>>> > ideally help us to ship this by September itself.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Revised dates (I will be updating same in Roadmap
>> >>> wiki as
>> >>> >>> >> well)
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Feature freeze date : all features to merge by
>> >>> August
>> >>> >>> 21,
>> >>> >>> >> 2018.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Code freeze date : blockers/critical only, no
>> >>> >>> improvements
>> >>> >>> >> and
>> >>> >>> >> > >>>>>>> non
>> >>> >>> >> > >>>>>>> > blocker/critical
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > bug-fixes  August 31, 2018.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Release date: September 15, 2018
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Thank Eric and Zian, I think Wangda has already
>> >>> answered
>> >>> >>> your
>> >>> >>> >> > >>>>>>> questions.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Thanks
>> >>> >>> >> > >>>>>>> > Sunil
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > On Thu, Jul 19, 2018 at 12:13 PM Wangda Tan <
>> >>> >>> >> wheeleast@gmail.com<ma...@gmail.com>
>> >>> >>> >> > >
>> >>> >>> >> > >>>>>>> wrote:
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > > Thanks Sunil for volunteering to be RM of 3.2
>> >>> release,
>> >>> >>> +1
>> >>> >>> >> for
>> >>> >>> >> > >>>>>>> that.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > To concerns from Steve,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > It is a good idea to keep the door open to get
>> >>> important
>> >>> >>> >> > changes
>> >>> >>> >> > >>>>>>> /
>> >>> >>> >> > >>>>>>> > > features in before cutoff. I would prefer to keep
>> >>> the
>> >>> >>> >> proposed
>> >>> >>> >> > >>>>>>> release
>> >>> >>> >> > >>>>>>> > date
>> >>> >>> >> > >>>>>>> > > to make sure things can happen earlier instead of
>> >>> last
>> >>> >>> >> minute
>> >>> >>> >> > >>>>>>> and we all
>> >>> >>> >> > >>>>>>> > > know that releases are always get delayed :). I'm
>> >>> also
>> >>> >>> fine
>> >>> >>> >> if
>> >>> >>> >> > >>>>>>> we want
>> >>> >>> >> > >>>>>>> > get
>> >>> >>> >> > >>>>>>> > > another several weeks time.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Regarding of 3.3 release, I would suggest doing
>> that
>> >>> >>> before
>> >>> >>> >> > >>>>>>> thanksgiving.
>> >>> >>> >> > >>>>>>> > > Do you think is it good or too early / late?
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Eric,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > The YARN-8220 will be replaced by YARN-8135, if
>> >>> >>> YARN-8135
>> >>> >>> >> can
>> >>> >>> >> > >>>>>>> get merged
>> >>> >>> >> > >>>>>>> > > in time, we probably not need the YARN-8220.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Sunil,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Could u update
>> >>> >>> >> > >>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> >>> >>> >> > >>>>>>> > Roadmap
>> >>> >>> >> > >>>>>>> > > with proposed plan as well? We can fill feature
>> list
>> >>> >>> first
>> >>> >>> >> > >>>>>>> before getting
>> >>> >>> >> > >>>>>>> > > consensus of time.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Thanks,
>> >>> >>> >> > >>>>>>> > > Wangda
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > On Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri
>> >>> >>> >> > >>>>>>> <fabbri@cloudera.com.invalid<mailto:
>> >>> fabbri@cloudera.com.invalid>
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > wrote:
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > >> On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran <
>> >>> >>> >> > >>>>>>> stevel@hortonworks.com<mailto:stevel@hortonworks.com
>> >>
>> >>> >>> >> > >>>>>>> > >> wrote:
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > On 16 Jul 2018, at 23:45, Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>
>> >>> >>> >> > <mailto:
>> >>> >>> >> > >>>>>>> > >> > sunilg@apache.org<ma...@apache.org>>>
>> >>> wrote:
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I would also would like to take this
>> opportunity
>> >>> to
>> >>> >>> come
>> >>> >>> >> up
>> >>> >>> >> > >>>>>>> with a
>> >>> >>> >> > >>>>>>> > >> detailed
>> >>> >>> >> > >>>>>>> > >> > plan.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > - Feature freeze date : all features should be
>> >>> >>> merged by
>> >>> >>> >> > >>>>>>> August 10,
>> >>> >>> >> > >>>>>>> > >> 2018.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > <snip>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > Please let me know if I missed any features
>> >>> targeted
>> >>> >>> to
>> >>> >>> >> 3.2
>> >>> >>> >> > >>>>>>> per this
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > Well there these big todo lists for S3 &
>> S3Guard.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15226
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15220
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > There's a bigger bit of work coming on for
>> Azure
>> >>> >>> Datalake
>> >>> >>> >> > Gen
>> >>> >>> >> > >>>>>>> 2
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15407
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I don't think this is quite ready yet, I've
>> been
>> >>> >>> doing
>> >>> >>> >> work
>> >>> >>> >> > >>>>>>> on it, but
>> >>> >>> >> > >>>>>>> > >> if
>> >>> >>> >> > >>>>>>> > >> > we have a 3 week deadline, I'm going to expect
>> >>> some
>> >>> >>> >> timely
>> >>> >>> >> > >>>>>>> reviews on
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15546
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I've uprated that to a blocker feature; will
>> >>> review
>> >>> >>> the
>> >>> >>> >> S3 &
>> >>> >>> >> > >>>>>>> S3Guard
>> >>> >>> >> > >>>>>>> > >> JIRAs
>> >>> >>> >> > >>>>>>> > >> > to see which of those are blocking. Then there
>> >>> are
>> >>> >>> some
>> >>> >>> >> > >>>>>>> pressing
>> >>> >>> >> > >>>>>>> > "guave,
>> >>> >>> >> > >>>>>>> > >> > java 9 prep"
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >>  I can help with this part if you like.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > timeline. I would like to volunteer myself as
>> >>> release
>> >>> >>> >> > manager
>> >>> >>> >> > >>>>>>> of 3.2.0
>> >>> >>> >> > >>>>>>> > >> > release.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > well volunteered!
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> Yes, thank you for stepping up.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I think this raises a good q: what timetable
>> >>> should
>> >>> >>> we
>> >>> >>> >> have
>> >>> >>> >> > >>>>>>> for the
>> >>> >>> >> > >>>>>>> > >> 3.2. &
>> >>> >>> >> > >>>>>>> > >> > 3.3 releases; if we do want a faster cadence,
>> >>> then
>> >>> >>> having
>> >>> >>> >> > the
>> >>> >>> >> > >>>>>>> outline
>> >>> >>> >> > >>>>>>> > >> time
>> >>> >>> >> > >>>>>>> > >> > from the 3.2 to the 3.3 release means that
>> >>> there's
>> >>> >>> less
>> >>> >>> >> > >>>>>>> concern about
>> >>> >>> >> > >>>>>>> > >> > things not making the 3.2 dealine
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > -Steve
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> Good idea to mitigate the short deadline.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> -AF
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>
>> >>> >>> >> >
>> >>> >>> >>
>> >>> >>> >
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>>
>> >>>
>> >>> --
>> >>> A very happy Clouderan
>> >>>
>> >>>
>>
>

Re: Hadoop 3.2 Release Plan proposal

Posted by Sunil G <su...@apache.org>.
Hi Folks,

Given all blockers are closed now, I am planning to do RC1 tomorrow.
branch-3.2.0 remains closed for commit. Please let me know if any issues
are missed.

Thanks,
Sunil G

On Tue, Nov 13, 2018 at 8:11 PM Sunil G <su...@apache.org> wrote:

> Hi Folks,
>
> All blockers are closed by last weekend and corrected jiras as well.
> Preparing RC0 now. Facing some issue with shaded jars file size. Hence
> respinning again.
> Planning to complete this by end of this week.
>
> Thanks
> Sunil
>
> On Fri, Oct 26, 2018 at 2:34 AM Konstantin Shvachko <sh...@gmail.com>
> wrote:
>
>> Another thing is that I see a bunch of jiras under HDFS-8707, which don't
>> have the Fix Version field listing 3.2, some have it just empty.
>> This means they will not be populated into release notes.
>>
>> Thanks,
>> --Konstantin
>>
>> On Thu, Oct 25, 2018 at 7:59 AM Sunil G <su...@apache.org> wrote:
>>
>> > Thanks Konstantin for pointing out.
>> > As 3.2 is pretty much on RC level, its better we try to find a good
>> > solution to this issue.
>> >
>> > I ll follow up on this in the jira.
>> >
>> > - Sunil
>> >
>> > On Thu, Oct 25, 2018 at 11:35 AM Konstantin Shvachko <
>> shv.hadoop@gmail.com>
>> > wrote:
>> >
>> >> I've tried to attract attention to an incompatibility issue through the
>> >> jira, but it didn't work. So pitching in in this thread.
>> >> https://issues.apache.org/jira/browse/HDFS-12026
>> >> It introduced binary incompatibility, which will prevent people from
>> >> upgrading from 3.1 to 3.2.
>> >> I think it can get messy if we release anything with this feature.
>> >>
>> >> Thanks,
>> >> --Konstantin
>> >>
>> >> On Mon, Oct 22, 2018 at 5:01 AM Steve Loughran <stevel@hortonworks.com
>> >
>> >> wrote:
>> >>
>> >>> its in.
>> >>>
>> >>> good catch!
>> >>>
>> >>> On 20 Oct 2018, at 01:35, Wei-Chiu Chuang <weichiu@cloudera.com
>> <mailto:
>> >>> weichiu@cloudera.com>> wrote:
>> >>>
>> >>> Thanks Sunil G for driving the release,
>> >>> I filed HADOOP-15866<
>> https://issues.apache.org/jira/browse/HADOOP-15866>
>> >>> for a compat fix. If any one has cycle please review it, as I think
>> it is
>> >>> needed for 3.2.0.
>> >>>
>> >>> On Thu, Oct 18, 2018 at 4:43 AM Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>> Hi Folks,
>> >>>
>> >>> As we previously communicated for 3.2.0 release, we have delayed due
>> to
>> >>> few
>> >>> blockers in our gate.
>> >>>
>> >>> I just cut branch-3.2.0 for release purpose. branch-3.2 will be open
>> for
>> >>> all bug fixes.
>> >>>
>> >>> - Sunil
>> >>>
>> >>>
>> >>> On Tue, Oct 16, 2018 at 8:59 AM Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>>
>> >>> > Hi Folks,
>> >>> >
>> >>> > We are now close to RC as other blocker issues are now merged to
>> trunk
>> >>> and
>> >>> > branch-3.2. Last 2 critical issues are closer to merge and will be
>> >>> > committed in few hours.
>> >>> > With this, I will be creating 3.2.0 branch today and will go ahead
>> >>> with RC
>> >>> > related process.
>> >>> >
>> >>> > - Sunil
>> >>> >
>> >>> > On Mon, Oct 15, 2018 at 11:43 PM Jonathan Bender <
>> jonbender@stripe.com
>> >>> <ma...@stripe.com>>
>> >>> > wrote:
>> >>> >
>> >>> >> Hello, were there any updates around the 3.2.0 RC timing? All I
>> see in
>> >>> >> the current blockers are related to the new Submarine subproject,
>> >>> wasn't
>> >>> >> sure if that is what is holding things up.
>> >>> >>
>> >>> >> Cheers,
>> >>> >> Jon
>> >>> >>
>> >>> >> On Tue, Oct 2, 2018 at 7:13 PM, Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>> >>
>> >>> >>> Thanks Robert and Haibo for quickly correcting same.
>> >>> >>> Sigh, I somehow missed one file while committing the change. Sorry
>> >>> for
>> >>> >>> the
>> >>> >>> trouble.
>> >>> >>>
>> >>> >>> - Sunil
>> >>> >>>
>> >>> >>> On Wed, Oct 3, 2018 at 5:22 AM Robert Kanter <
>> rkanter@cloudera.com
>> >>> <ma...@cloudera.com>>
>> >>> >>> wrote:
>> >>> >>>
>> >>> >>> > Looks like there's two that weren't updated:
>> >>> >>> > >> [115] 16:32 : hadoop-common (trunk) :: grep "3.2.0-SNAPSHOT"
>> .
>> >>> -r
>> >>> >>> > --include=pom.xml
>> >>> >>> > ./hadoop-project/pom.xml:
>> >>> >>> >
>> >>> <hadoop.assemblies.version>3.2.0-SNAPSHOT</hadoop.assemblies.version>
>> >>> >>> > ./pom.xml:    <hadoop.version>3.2.0-SNAPSHOT</hadoop.version>
>> >>> >>> >
>> >>> >>> > I've just pushed in an addendum commit to fix those.
>> >>> >>> > In the future, please make sure to do a sanity compile when
>> >>> updating
>> >>> >>> poms.
>> >>> >>> >
>> >>> >>> > thanks
>> >>> >>> > - Robert
>> >>> >>> >
>> >>> >>> > On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri
>> >>> >>> <fa...@cloudera.com.invalid>>
>> >>> >>> > wrote:
>> >>> >>> >
>> >>> >>> >> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in
>> >>> the
>> >>> >>> >> top-level pom.xml?
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> On Tue, Oct 2, 2018 at 10:16 AM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>> wrote:
>> >>> >>> >>
>> >>> >>> >> > Hi All
>> >>> >>> >> >
>> >>> >>> >> > As mentioned in earlier mail, I have cut branch-3.2 and reset
>> >>> trunk
>> >>> >>> to
>> >>> >>> >> > 3.3.0-SNAPSHOT. I will share the RC details sooner once all
>> >>> >>> necessary
>> >>> >>> >> > patches are pulled into branch-3.2.
>> >>> >>> >> >
>> >>> >>> >> > Thank You
>> >>> >>> >> > - Sunil
>> >>> >>> >> >
>> >>> >>> >> >
>> >>> >>> >> > On Mon, Sep 24, 2018 at 2:00 PM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>> wrote:
>> >>> >>> >> >
>> >>> >>> >> > > Hi All
>> >>> >>> >> > >
>> >>> >>> >> > > We are now down to the last Blocker and HADOOP-15407 is
>> >>> merged to
>> >>> >>> >> trunk.
>> >>> >>> >> > > Thanks for the support.
>> >>> >>> >> > >
>> >>> >>> >> > > *Plan for RC*
>> >>> >>> >> > > 3.2 branch cut and reset trunk : *25th Tuesday*
>> >>> >>> >> > > RC0 for 3.2: *28th Friday*
>> >>> >>> >> > >
>> >>> >>> >> > > Thank You
>> >>> >>> >> > > Sunil
>> >>> >>> >> > >
>> >>> >>> >> > >
>> >>> >>> >> > > On Mon, Sep 17, 2018 at 3:21 PM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >
>> >>> >>> >> > >> Hi All
>> >>> >>> >> > >>
>> >>> >>> >> > >> We are down to 3 Blockers and 4 Critical now. Thanks all
>> of
>> >>> you
>> >>> >>> for
>> >>> >>> >> > >> helping in this. I am following up on these tickets, once
>> its
>> >>> >>> closed
>> >>> >>> >> we
>> >>> >>> >> > >> will cut the 3.2 branch.
>> >>> >>> >> > >>
>> >>> >>> >> > >> Thanks
>> >>> >>> >> > >> Sunil Govindan
>> >>> >>> >> > >>
>> >>> >>> >> > >>
>> >>> >>> >> > >> On Wed, Sep 12, 2018 at 5:10 PM Sunil G <
>> sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >>
>> >>> >>> >> > >>> Hi All,
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Inline with the original 3.2 communication proposal dated
>> >>> 17th
>> >>> >>> July
>> >>> >>> >> > >>> 2018, I would like to provide more updates.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> We are approaching previously proposed code freeze date
>> >>> >>> (September
>> >>> >>> >> 14,
>> >>> >>> >> > >>> 2018). So I would like to cut 3.2 branch on 17th Sept and
>> >>> point
>> >>> >>> >> > existing
>> >>> >>> >> > >>> trunk to 3.3 if there are no issues.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> *Current Release Plan:*
>> >>> >>> >> > >>> Feature freeze date : all features to merge by September
>> 7,
>> >>> >>> 2018.
>> >>> >>> >> > >>> Code freeze date : blockers/critical only, no
>> improvements
>> >>> and
>> >>> >>> >> > >>> blocker/critical bug-fixes September 14, 2018.
>> >>> >>> >> > >>> Release date: September 28, 2018
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> If any critical/blocker tickets which are targeted to
>> >>> 3.2.0, we
>> >>> >>> >> need to
>> >>> >>> >> > >>> backport to 3.2 post branch cut.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Here's an updated 3.2.0 feature status:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 1. Merged & Completed features:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> DeepLearning
>> >>> >>> >> > >>> workloads Initial cut.
>> >>> >>> >> > >>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>> Scheduler.
>> >>> >>> >> > >>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> YARN
>> >>> >>> Service
>> >>> >>> >> > API
>> >>> >>> >> > >>> and CLI.
>> >>> >>> >> > >>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> >> > >>> - (Inigo) HDFS-12615: Router-based HDFS federation.
>> >>> Improvement
>> >>> >>> >> works.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 2. Features close to finish:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> - (Steve) S3Guard Phase III. Close to commit.
>> >>> >>> >> > >>> - (Steve) S3a phase V. Close to commit.
>> >>> >>> >> > >>> - (Steve) Support Windows Azure Storage. Close to commit.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 3. Tentative/Cancelled features for 3.2:
>> >>> >>> >> > >>> - (Rohith) YARN-5742: Serve aggregated logs of historical
>> >>> apps
>> >>> >>> from
>> >>> >>> >> > >>> ATSv2. Patch in progress.
>> >>> >>> >> > >>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
>> >>> >>> >> challenging to
>> >>> >>> >> > >>> be done before Aug 2018.
>> >>> >>> >> > >>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>> 19 Blocker and Critical issues [1] are open, I am
>> following
>> >>> up
>> >>> >>> with
>> >>> >>> >> > >>> owners to get status on each of them to get in by Code
>> >>> Freeze
>> >>> >>> date.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> priority
>> >>> in
>> >>> >>> >> > (Blocker,
>> >>> >>> >> > >>> Critical) AND resolution = Unresolved AND "Target
>> >>> Version/s" =
>> >>> >>> 3.2.0
>> >>> >>> >> > ORDER
>> >>> >>> >> > >>> BY priority DESC
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Thanks,
>> >>> >>> >> > >>> Sunil
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> On Thu, Aug 30, 2018 at 9:59 PM Sunil G <
>> sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>> Hi All,
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Inline with earlier communication dated 17th July 2018,
>> I
>> >>> would
>> >>> >>> >> like
>> >>> >>> >> > to
>> >>> >>> >> > >>>> provide some updates.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> We are approaching previously proposed code freeze date
>> >>> (Aug
>> >>> >>> 31).
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> One of the critical feature Node Attributes feature
>> merge
>> >>> >>> >> > >>>> discussion/vote is ongoing. Also few other Blocker bugs
>> >>> need a
>> >>> >>> bit
>> >>> >>> >> > more
>> >>> >>> >> > >>>> time. With regard to this, suggesting to push the
>> >>> feature/code
>> >>> >>> >> freeze
>> >>> >>> >> > for 2
>> >>> >>> >> > >>>> more weeks to accommodate these jiras too.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Proposing Updated changes in plan inline with this:
>> >>> >>> >> > >>>> Feature freeze date : all features to merge by
>> September 7,
>> >>> >>> 2018.
>> >>> >>> >> > >>>> Code freeze date : blockers/critical only, no
>> improvements
>> >>> and
>> >>> >>> >> > >>>>  blocker/critical bug-fixes September 14, 2018.
>> >>> >>> >> > >>>> Release date: September 28, 2018
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> If any features in branch which are targeted to 3.2.0,
>> >>> please
>> >>> >>> >> reply to
>> >>> >>> >> > >>>> this email thread.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> *Here's an updated 3.2.0 feature status:*
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 1. Merged & Completed features:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> DeepLearning
>> >>> >>> >> > >>>> workloads Initial cut.
>> >>> >>> >> > >>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>>> Scheduler.
>> >>> >>> >> > >>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> >>> YARN
>> >>> >>> >> Service
>> >>> >>> >> > >>>> API and CLI.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 2. Features close to finish:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> >> Merge/Vote
>> >>> >>> >> > >>>> Ongoing.
>> >>> >>> >> > >>>> - (Rohith) YARN-5742: Serve aggregated logs of
>> historical
>> >>> apps
>> >>> >>> from
>> >>> >>> >> > >>>> ATSv2. Patch in progress.
>> >>> >>> >> > >>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
>> >>> >>> Improvement
>> >>> >>> >> > works.
>> >>> >>> >> > >>>> - (Steve) S3Guard Phase III, S3a phase V, Support
>> Windows
>> >>> Azure
>> >>> >>> >> > >>>> Storage. In progress.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 3. Tentative features:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
>> >>> >>> >> challenging
>> >>> >>> >> > to
>> >>> >>> >> > >>>> be done before Aug 2018.
>> >>> >>> >> > >>>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 26 Blocker and Critical issues [1] are open, I am
>> >>> following up
>> >>> >>> with
>> >>> >>> >> > >>>> owners to get status on each of them to get in by Code
>> >>> Freeze
>> >>> >>> date.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> >>> priority in
>> >>> >>> >> > >>>> (Blocker, Critical) AND resolution = Unresolved AND
>> "Target
>> >>> >>> >> > Version/s" =
>> >>> >>> >> > >>>> 3.2.0 ORDER BY priority DESC
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Thanks,
>> >>> >>> >> > >>>> Sunil
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> On Tue, Aug 14, 2018 at 10:30 PM Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> wrote:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>>> Hi All,
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> Thanks for the feedbacks. Inline with earlier
>> >>> communication
>> >>> >>> dated
>> >>> >>> >> > 17th
>> >>> >>> >> > >>>>> July 2018, I would like to provide some updates.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> We are approaching previously proposed feature freeze
>> date
>> >>> >>> (Aug
>> >>> >>> >> 21,
>> >>> >>> >> > >>>>> about 7 days from today).
>> >>> >>> >> > >>>>> If any features in branch which are targeted to 3.2.0,
>> >>> please
>> >>> >>> >> reply
>> >>> >>> >> > to
>> >>> >>> >> > >>>>> this email thread.
>> >>> >>> >> > >>>>> Steve has mentioned about the s3 features which will
>> come
>> >>> >>> close to
>> >>> >>> >> > >>>>> Code Freeze Date (Aug 31st).
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> *Here's an updated 3.2.0 feature status:*
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 1. Merged & Completed features:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> >>> DeepLearning
>> >>> >>> >> > >>>>> workloads Initial cut.
>> >>> >>> >> > >>>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 2. Features close to finish:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> Major
>> >>> >>> >> > >>>>> patches are all in, only one last
>> >>> >>> >> > >>>>> patch is in review state.
>> >>> >>> >> > >>>>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>>>> Scheduler. Close to commit.
>> >>> >>> >> > >>>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> >>> YARN
>> >>> >>> >> Service
>> >>> >>> >> > >>>>> API and CLI. 2 patches are pending
>> >>> >>> >> > >>>>> which will be closed by Feature freeze date.
>> >>> >>> >> > >>>>> - (Rohith) YARN-5742: Serve aggregated logs of
>> historical
>> >>> apps
>> >>> >>> >> from
>> >>> >>> >> > >>>>> ATSv2. Patch in progress.
>> >>> >>> >> > >>>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
>> >>> >>> Improvement
>> >>> >>> >> > >>>>> works.
>> >>> >>> >> > >>>>> - (Steve) S3Guard Phase III, S3a phase V, Support
>> Windows
>> >>> >>> Azure
>> >>> >>> >> > >>>>> Storage. In progress.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 3. Tentative features:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Haibo Chen) YARN-1011: Resource overcommitment.
>> Looks
>> >>> >>> >> challenging
>> >>> >>> >> > >>>>> to be done before Aug 2018.
>> >>> >>> >> > >>>>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>>>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 39 Blocker and Critical issues [1] are open, I am
>> checking
>> >>> >>> with
>> >>> >>> >> > owners
>> >>> >>> >> > >>>>> to get status on each of them to get in by Code Freeze
>> >>> date.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> >>> priority in
>> >>> >>> >> > >>>>> (Blocker, Critical) AND resolution = Unresolved AND
>> >>> "Target
>> >>> >>> >> > Version/s" =
>> >>> >>> >> > >>>>> 3.2.0 ORDER BY priority DESC
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> Thanks,
>> >>> >>> >> > >>>>> Sunil
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> On Fri, Jul 20, 2018 at 8:03 AM Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> wrote:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>>> Thanks Subru for the thoughts.
>> >>> >>> >> > >>>>>> One of the main reason for a major release is to push
>> out
>> >>> >>> >> critical
>> >>> >>> >> > >>>>>> features with a faster cadence to the users. If we are
>> >>> >>> pulling
>> >>> >>> >> more
>> >>> >>> >> > and
>> >>> >>> >> > >>>>>> more different types of features to a minor release,
>> that
>> >>> >>> branch
>> >>> >>> >> > will
>> >>> >>> >> > >>>>>> become more destabilized and it may be tough to say
>> that
>> >>> >>> 3.1.2 is
>> >>> >>> >> > stable
>> >>> >>> >> > >>>>>> that 3.1.1 for eg. We always tend to improve and
>> >>> stabilize
>> >>> >>> >> features
>> >>> >>> >> > in
>> >>> >>> >> > >>>>>> subsequent minor release.
>> >>> >>> >> > >>>>>> For few companies, it makes sense to push out these
>> new
>> >>> >>> features
>> >>> >>> >> > >>>>>> faster to make a reach to the users. Adding to the
>> point
>> >>> to
>> >>> >>> the
>> >>> >>> >> > backporting
>> >>> >>> >> > >>>>>> issues, I agree that its a pain and we can workaround
>> >>> that
>> >>> >>> with
>> >>> >>> >> > some git
>> >>> >>> >> > >>>>>> scripts. If we can make such scripts available to
>> >>> committers,
>> >>> >>> >> > backport will
>> >>> >>> >> > >>>>>> be seem-less across branches and we can achieve the
>> >>> faster
>> >>> >>> >> release
>> >>> >>> >> > cadence
>> >>> >>> >> > >>>>>> also.
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> Thoughts?
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> - Sunil
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> On Fri, Jul 20, 2018 at 3:37 AM Subru Krishnan <
>> >>> >>> subru@apache.org<ma...@apache.org>
>> >>> >>> >> >
>> >>> >>> >> > >>>>>> wrote:
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>>> Thanks Sunil for volunteering to lead the release
>> >>> effort. I
>> >>> >>> am
>> >>> >>> >> > >>>>>>> generally
>> >>> >>> >> > >>>>>>> supportive of a release but -1 on a 3.2 (prefer a
>> >>> 3.1.x) as
>> >>> >>> >> feel we
>> >>> >>> >> > >>>>>>> already
>> >>> >>> >> > >>>>>>> have too many branches to be maintained. I already
>> see
>> >>> many
>> >>> >>> >> commits
>> >>> >>> >> > >>>>>>> are in
>> >>> >>> >> > >>>>>>> different branches with no apparent rationale, for
>> e.g:
>> >>> 3.1
>> >>> >>> has
>> >>> >>> >> > >>>>>>> commits
>> >>> >>> >> > >>>>>>> which are absent in 3.0 etc.
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> Additionally AFAIK 3.x has not been deployed in any
>> >>> major
>> >>> >>> >> > production
>> >>> >>> >> > >>>>>>> setting so the cost of adding features should be
>> >>> minimal.
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> Thoughts?
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> -Subru
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> On Thu, Jul 19, 2018 at 12:31 AM, Sunil G <
>> >>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> > wrote:
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> > Thanks Steve, Aaron, Wangda for sharing thoughts.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Yes, important changes and features are much
>> needed,
>> >>> >>> hence we
>> >>> >>> >> > will
>> >>> >>> >> > >>>>>>> be
>> >>> >>> >> > >>>>>>> > keeping the door open for them as possible. Also
>> >>> >>> considering
>> >>> >>> >> few
>> >>> >>> >> > >>>>>>> more
>> >>> >>> >> > >>>>>>> > offline requests from other folks, I think
>> extending
>> >>> the
>> >>> >>> >> > timeframe
>> >>> >>> >> > >>>>>>> by
>> >>> >>> >> > >>>>>>> > couple of weeks makes sense (including a second RC
>> >>> >>> buffer) and
>> >>> >>> >> > >>>>>>> this should
>> >>> >>> >> > >>>>>>> > ideally help us to ship this by September itself.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Revised dates (I will be updating same in Roadmap
>> >>> wiki as
>> >>> >>> >> well)
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Feature freeze date : all features to merge by
>> >>> August
>> >>> >>> 21,
>> >>> >>> >> 2018.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Code freeze date : blockers/critical only, no
>> >>> >>> improvements
>> >>> >>> >> and
>> >>> >>> >> > >>>>>>> non
>> >>> >>> >> > >>>>>>> > blocker/critical
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > bug-fixes  August 31, 2018.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Release date: September 15, 2018
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Thank Eric and Zian, I think Wangda has already
>> >>> answered
>> >>> >>> your
>> >>> >>> >> > >>>>>>> questions.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Thanks
>> >>> >>> >> > >>>>>>> > Sunil
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > On Thu, Jul 19, 2018 at 12:13 PM Wangda Tan <
>> >>> >>> >> wheeleast@gmail.com<ma...@gmail.com>
>> >>> >>> >> > >
>> >>> >>> >> > >>>>>>> wrote:
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > > Thanks Sunil for volunteering to be RM of 3.2
>> >>> release,
>> >>> >>> +1
>> >>> >>> >> for
>> >>> >>> >> > >>>>>>> that.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > To concerns from Steve,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > It is a good idea to keep the door open to get
>> >>> important
>> >>> >>> >> > changes
>> >>> >>> >> > >>>>>>> /
>> >>> >>> >> > >>>>>>> > > features in before cutoff. I would prefer to keep
>> >>> the
>> >>> >>> >> proposed
>> >>> >>> >> > >>>>>>> release
>> >>> >>> >> > >>>>>>> > date
>> >>> >>> >> > >>>>>>> > > to make sure things can happen earlier instead of
>> >>> last
>> >>> >>> >> minute
>> >>> >>> >> > >>>>>>> and we all
>> >>> >>> >> > >>>>>>> > > know that releases are always get delayed :). I'm
>> >>> also
>> >>> >>> fine
>> >>> >>> >> if
>> >>> >>> >> > >>>>>>> we want
>> >>> >>> >> > >>>>>>> > get
>> >>> >>> >> > >>>>>>> > > another several weeks time.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Regarding of 3.3 release, I would suggest doing
>> that
>> >>> >>> before
>> >>> >>> >> > >>>>>>> thanksgiving.
>> >>> >>> >> > >>>>>>> > > Do you think is it good or too early / late?
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Eric,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > The YARN-8220 will be replaced by YARN-8135, if
>> >>> >>> YARN-8135
>> >>> >>> >> can
>> >>> >>> >> > >>>>>>> get merged
>> >>> >>> >> > >>>>>>> > > in time, we probably not need the YARN-8220.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Sunil,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Could u update
>> >>> >>> >> > >>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> >>> >>> >> > >>>>>>> > Roadmap
>> >>> >>> >> > >>>>>>> > > with proposed plan as well? We can fill feature
>> list
>> >>> >>> first
>> >>> >>> >> > >>>>>>> before getting
>> >>> >>> >> > >>>>>>> > > consensus of time.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Thanks,
>> >>> >>> >> > >>>>>>> > > Wangda
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > On Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri
>> >>> >>> >> > >>>>>>> <fabbri@cloudera.com.invalid<mailto:
>> >>> fabbri@cloudera.com.invalid>
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > wrote:
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > >> On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran <
>> >>> >>> >> > >>>>>>> stevel@hortonworks.com<mailto:stevel@hortonworks.com
>> >>
>> >>> >>> >> > >>>>>>> > >> wrote:
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > On 16 Jul 2018, at 23:45, Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>
>> >>> >>> >> > <mailto:
>> >>> >>> >> > >>>>>>> > >> > sunilg@apache.org<ma...@apache.org>>>
>> >>> wrote:
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I would also would like to take this
>> opportunity
>> >>> to
>> >>> >>> come
>> >>> >>> >> up
>> >>> >>> >> > >>>>>>> with a
>> >>> >>> >> > >>>>>>> > >> detailed
>> >>> >>> >> > >>>>>>> > >> > plan.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > - Feature freeze date : all features should be
>> >>> >>> merged by
>> >>> >>> >> > >>>>>>> August 10,
>> >>> >>> >> > >>>>>>> > >> 2018.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > <snip>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > Please let me know if I missed any features
>> >>> targeted
>> >>> >>> to
>> >>> >>> >> 3.2
>> >>> >>> >> > >>>>>>> per this
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > Well there these big todo lists for S3 &
>> S3Guard.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15226
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15220
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > There's a bigger bit of work coming on for
>> Azure
>> >>> >>> Datalake
>> >>> >>> >> > Gen
>> >>> >>> >> > >>>>>>> 2
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15407
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I don't think this is quite ready yet, I've
>> been
>> >>> >>> doing
>> >>> >>> >> work
>> >>> >>> >> > >>>>>>> on it, but
>> >>> >>> >> > >>>>>>> > >> if
>> >>> >>> >> > >>>>>>> > >> > we have a 3 week deadline, I'm going to expect
>> >>> some
>> >>> >>> >> timely
>> >>> >>> >> > >>>>>>> reviews on
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15546
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I've uprated that to a blocker feature; will
>> >>> review
>> >>> >>> the
>> >>> >>> >> S3 &
>> >>> >>> >> > >>>>>>> S3Guard
>> >>> >>> >> > >>>>>>> > >> JIRAs
>> >>> >>> >> > >>>>>>> > >> > to see which of those are blocking. Then there
>> >>> are
>> >>> >>> some
>> >>> >>> >> > >>>>>>> pressing
>> >>> >>> >> > >>>>>>> > "guave,
>> >>> >>> >> > >>>>>>> > >> > java 9 prep"
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >>  I can help with this part if you like.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > timeline. I would like to volunteer myself as
>> >>> release
>> >>> >>> >> > manager
>> >>> >>> >> > >>>>>>> of 3.2.0
>> >>> >>> >> > >>>>>>> > >> > release.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > well volunteered!
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> Yes, thank you for stepping up.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I think this raises a good q: what timetable
>> >>> should
>> >>> >>> we
>> >>> >>> >> have
>> >>> >>> >> > >>>>>>> for the
>> >>> >>> >> > >>>>>>> > >> 3.2. &
>> >>> >>> >> > >>>>>>> > >> > 3.3 releases; if we do want a faster cadence,
>> >>> then
>> >>> >>> having
>> >>> >>> >> > the
>> >>> >>> >> > >>>>>>> outline
>> >>> >>> >> > >>>>>>> > >> time
>> >>> >>> >> > >>>>>>> > >> > from the 3.2 to the 3.3 release means that
>> >>> there's
>> >>> >>> less
>> >>> >>> >> > >>>>>>> concern about
>> >>> >>> >> > >>>>>>> > >> > things not making the 3.2 dealine
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > -Steve
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> Good idea to mitigate the short deadline.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> -AF
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>
>> >>> >>> >> >
>> >>> >>> >>
>> >>> >>> >
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>>
>> >>>
>> >>> --
>> >>> A very happy Clouderan
>> >>>
>> >>>
>>
>

Re: Hadoop 3.2 Release Plan proposal

Posted by Sunil G <su...@apache.org>.
Hi Folks,

Given all blockers are closed now, I am planning to do RC1 tomorrow.
branch-3.2.0 remains closed for commit. Please let me know if any issues
are missed.

Thanks,
Sunil G

On Tue, Nov 13, 2018 at 8:11 PM Sunil G <su...@apache.org> wrote:

> Hi Folks,
>
> All blockers are closed by last weekend and corrected jiras as well.
> Preparing RC0 now. Facing some issue with shaded jars file size. Hence
> respinning again.
> Planning to complete this by end of this week.
>
> Thanks
> Sunil
>
> On Fri, Oct 26, 2018 at 2:34 AM Konstantin Shvachko <sh...@gmail.com>
> wrote:
>
>> Another thing is that I see a bunch of jiras under HDFS-8707, which don't
>> have the Fix Version field listing 3.2, some have it just empty.
>> This means they will not be populated into release notes.
>>
>> Thanks,
>> --Konstantin
>>
>> On Thu, Oct 25, 2018 at 7:59 AM Sunil G <su...@apache.org> wrote:
>>
>> > Thanks Konstantin for pointing out.
>> > As 3.2 is pretty much on RC level, its better we try to find a good
>> > solution to this issue.
>> >
>> > I ll follow up on this in the jira.
>> >
>> > - Sunil
>> >
>> > On Thu, Oct 25, 2018 at 11:35 AM Konstantin Shvachko <
>> shv.hadoop@gmail.com>
>> > wrote:
>> >
>> >> I've tried to attract attention to an incompatibility issue through the
>> >> jira, but it didn't work. So pitching in in this thread.
>> >> https://issues.apache.org/jira/browse/HDFS-12026
>> >> It introduced binary incompatibility, which will prevent people from
>> >> upgrading from 3.1 to 3.2.
>> >> I think it can get messy if we release anything with this feature.
>> >>
>> >> Thanks,
>> >> --Konstantin
>> >>
>> >> On Mon, Oct 22, 2018 at 5:01 AM Steve Loughran <stevel@hortonworks.com
>> >
>> >> wrote:
>> >>
>> >>> its in.
>> >>>
>> >>> good catch!
>> >>>
>> >>> On 20 Oct 2018, at 01:35, Wei-Chiu Chuang <weichiu@cloudera.com
>> <mailto:
>> >>> weichiu@cloudera.com>> wrote:
>> >>>
>> >>> Thanks Sunil G for driving the release,
>> >>> I filed HADOOP-15866<
>> https://issues.apache.org/jira/browse/HADOOP-15866>
>> >>> for a compat fix. If any one has cycle please review it, as I think
>> it is
>> >>> needed for 3.2.0.
>> >>>
>> >>> On Thu, Oct 18, 2018 at 4:43 AM Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>> Hi Folks,
>> >>>
>> >>> As we previously communicated for 3.2.0 release, we have delayed due
>> to
>> >>> few
>> >>> blockers in our gate.
>> >>>
>> >>> I just cut branch-3.2.0 for release purpose. branch-3.2 will be open
>> for
>> >>> all bug fixes.
>> >>>
>> >>> - Sunil
>> >>>
>> >>>
>> >>> On Tue, Oct 16, 2018 at 8:59 AM Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>>
>> >>> > Hi Folks,
>> >>> >
>> >>> > We are now close to RC as other blocker issues are now merged to
>> trunk
>> >>> and
>> >>> > branch-3.2. Last 2 critical issues are closer to merge and will be
>> >>> > committed in few hours.
>> >>> > With this, I will be creating 3.2.0 branch today and will go ahead
>> >>> with RC
>> >>> > related process.
>> >>> >
>> >>> > - Sunil
>> >>> >
>> >>> > On Mon, Oct 15, 2018 at 11:43 PM Jonathan Bender <
>> jonbender@stripe.com
>> >>> <ma...@stripe.com>>
>> >>> > wrote:
>> >>> >
>> >>> >> Hello, were there any updates around the 3.2.0 RC timing? All I
>> see in
>> >>> >> the current blockers are related to the new Submarine subproject,
>> >>> wasn't
>> >>> >> sure if that is what is holding things up.
>> >>> >>
>> >>> >> Cheers,
>> >>> >> Jon
>> >>> >>
>> >>> >> On Tue, Oct 2, 2018 at 7:13 PM, Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>> >>
>> >>> >>> Thanks Robert and Haibo for quickly correcting same.
>> >>> >>> Sigh, I somehow missed one file while committing the change. Sorry
>> >>> for
>> >>> >>> the
>> >>> >>> trouble.
>> >>> >>>
>> >>> >>> - Sunil
>> >>> >>>
>> >>> >>> On Wed, Oct 3, 2018 at 5:22 AM Robert Kanter <
>> rkanter@cloudera.com
>> >>> <ma...@cloudera.com>>
>> >>> >>> wrote:
>> >>> >>>
>> >>> >>> > Looks like there's two that weren't updated:
>> >>> >>> > >> [115] 16:32 : hadoop-common (trunk) :: grep "3.2.0-SNAPSHOT"
>> .
>> >>> -r
>> >>> >>> > --include=pom.xml
>> >>> >>> > ./hadoop-project/pom.xml:
>> >>> >>> >
>> >>> <hadoop.assemblies.version>3.2.0-SNAPSHOT</hadoop.assemblies.version>
>> >>> >>> > ./pom.xml:    <hadoop.version>3.2.0-SNAPSHOT</hadoop.version>
>> >>> >>> >
>> >>> >>> > I've just pushed in an addendum commit to fix those.
>> >>> >>> > In the future, please make sure to do a sanity compile when
>> >>> updating
>> >>> >>> poms.
>> >>> >>> >
>> >>> >>> > thanks
>> >>> >>> > - Robert
>> >>> >>> >
>> >>> >>> > On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri
>> >>> >>> <fa...@cloudera.com.invalid>>
>> >>> >>> > wrote:
>> >>> >>> >
>> >>> >>> >> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in
>> >>> the
>> >>> >>> >> top-level pom.xml?
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> On Tue, Oct 2, 2018 at 10:16 AM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>> wrote:
>> >>> >>> >>
>> >>> >>> >> > Hi All
>> >>> >>> >> >
>> >>> >>> >> > As mentioned in earlier mail, I have cut branch-3.2 and reset
>> >>> trunk
>> >>> >>> to
>> >>> >>> >> > 3.3.0-SNAPSHOT. I will share the RC details sooner once all
>> >>> >>> necessary
>> >>> >>> >> > patches are pulled into branch-3.2.
>> >>> >>> >> >
>> >>> >>> >> > Thank You
>> >>> >>> >> > - Sunil
>> >>> >>> >> >
>> >>> >>> >> >
>> >>> >>> >> > On Mon, Sep 24, 2018 at 2:00 PM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>> wrote:
>> >>> >>> >> >
>> >>> >>> >> > > Hi All
>> >>> >>> >> > >
>> >>> >>> >> > > We are now down to the last Blocker and HADOOP-15407 is
>> >>> merged to
>> >>> >>> >> trunk.
>> >>> >>> >> > > Thanks for the support.
>> >>> >>> >> > >
>> >>> >>> >> > > *Plan for RC*
>> >>> >>> >> > > 3.2 branch cut and reset trunk : *25th Tuesday*
>> >>> >>> >> > > RC0 for 3.2: *28th Friday*
>> >>> >>> >> > >
>> >>> >>> >> > > Thank You
>> >>> >>> >> > > Sunil
>> >>> >>> >> > >
>> >>> >>> >> > >
>> >>> >>> >> > > On Mon, Sep 17, 2018 at 3:21 PM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >
>> >>> >>> >> > >> Hi All
>> >>> >>> >> > >>
>> >>> >>> >> > >> We are down to 3 Blockers and 4 Critical now. Thanks all
>> of
>> >>> you
>> >>> >>> for
>> >>> >>> >> > >> helping in this. I am following up on these tickets, once
>> its
>> >>> >>> closed
>> >>> >>> >> we
>> >>> >>> >> > >> will cut the 3.2 branch.
>> >>> >>> >> > >>
>> >>> >>> >> > >> Thanks
>> >>> >>> >> > >> Sunil Govindan
>> >>> >>> >> > >>
>> >>> >>> >> > >>
>> >>> >>> >> > >> On Wed, Sep 12, 2018 at 5:10 PM Sunil G <
>> sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >>
>> >>> >>> >> > >>> Hi All,
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Inline with the original 3.2 communication proposal dated
>> >>> 17th
>> >>> >>> July
>> >>> >>> >> > >>> 2018, I would like to provide more updates.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> We are approaching previously proposed code freeze date
>> >>> >>> (September
>> >>> >>> >> 14,
>> >>> >>> >> > >>> 2018). So I would like to cut 3.2 branch on 17th Sept and
>> >>> point
>> >>> >>> >> > existing
>> >>> >>> >> > >>> trunk to 3.3 if there are no issues.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> *Current Release Plan:*
>> >>> >>> >> > >>> Feature freeze date : all features to merge by September
>> 7,
>> >>> >>> 2018.
>> >>> >>> >> > >>> Code freeze date : blockers/critical only, no
>> improvements
>> >>> and
>> >>> >>> >> > >>> blocker/critical bug-fixes September 14, 2018.
>> >>> >>> >> > >>> Release date: September 28, 2018
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> If any critical/blocker tickets which are targeted to
>> >>> 3.2.0, we
>> >>> >>> >> need to
>> >>> >>> >> > >>> backport to 3.2 post branch cut.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Here's an updated 3.2.0 feature status:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 1. Merged & Completed features:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> DeepLearning
>> >>> >>> >> > >>> workloads Initial cut.
>> >>> >>> >> > >>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>> Scheduler.
>> >>> >>> >> > >>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> YARN
>> >>> >>> Service
>> >>> >>> >> > API
>> >>> >>> >> > >>> and CLI.
>> >>> >>> >> > >>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> >> > >>> - (Inigo) HDFS-12615: Router-based HDFS federation.
>> >>> Improvement
>> >>> >>> >> works.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 2. Features close to finish:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> - (Steve) S3Guard Phase III. Close to commit.
>> >>> >>> >> > >>> - (Steve) S3a phase V. Close to commit.
>> >>> >>> >> > >>> - (Steve) Support Windows Azure Storage. Close to commit.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 3. Tentative/Cancelled features for 3.2:
>> >>> >>> >> > >>> - (Rohith) YARN-5742: Serve aggregated logs of historical
>> >>> apps
>> >>> >>> from
>> >>> >>> >> > >>> ATSv2. Patch in progress.
>> >>> >>> >> > >>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
>> >>> >>> >> challenging to
>> >>> >>> >> > >>> be done before Aug 2018.
>> >>> >>> >> > >>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>> 19 Blocker and Critical issues [1] are open, I am
>> following
>> >>> up
>> >>> >>> with
>> >>> >>> >> > >>> owners to get status on each of them to get in by Code
>> >>> Freeze
>> >>> >>> date.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> priority
>> >>> in
>> >>> >>> >> > (Blocker,
>> >>> >>> >> > >>> Critical) AND resolution = Unresolved AND "Target
>> >>> Version/s" =
>> >>> >>> 3.2.0
>> >>> >>> >> > ORDER
>> >>> >>> >> > >>> BY priority DESC
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Thanks,
>> >>> >>> >> > >>> Sunil
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> On Thu, Aug 30, 2018 at 9:59 PM Sunil G <
>> sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>> Hi All,
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Inline with earlier communication dated 17th July 2018,
>> I
>> >>> would
>> >>> >>> >> like
>> >>> >>> >> > to
>> >>> >>> >> > >>>> provide some updates.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> We are approaching previously proposed code freeze date
>> >>> (Aug
>> >>> >>> 31).
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> One of the critical feature Node Attributes feature
>> merge
>> >>> >>> >> > >>>> discussion/vote is ongoing. Also few other Blocker bugs
>> >>> need a
>> >>> >>> bit
>> >>> >>> >> > more
>> >>> >>> >> > >>>> time. With regard to this, suggesting to push the
>> >>> feature/code
>> >>> >>> >> freeze
>> >>> >>> >> > for 2
>> >>> >>> >> > >>>> more weeks to accommodate these jiras too.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Proposing Updated changes in plan inline with this:
>> >>> >>> >> > >>>> Feature freeze date : all features to merge by
>> September 7,
>> >>> >>> 2018.
>> >>> >>> >> > >>>> Code freeze date : blockers/critical only, no
>> improvements
>> >>> and
>> >>> >>> >> > >>>>  blocker/critical bug-fixes September 14, 2018.
>> >>> >>> >> > >>>> Release date: September 28, 2018
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> If any features in branch which are targeted to 3.2.0,
>> >>> please
>> >>> >>> >> reply to
>> >>> >>> >> > >>>> this email thread.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> *Here's an updated 3.2.0 feature status:*
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 1. Merged & Completed features:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> DeepLearning
>> >>> >>> >> > >>>> workloads Initial cut.
>> >>> >>> >> > >>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>>> Scheduler.
>> >>> >>> >> > >>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> >>> YARN
>> >>> >>> >> Service
>> >>> >>> >> > >>>> API and CLI.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 2. Features close to finish:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> >> Merge/Vote
>> >>> >>> >> > >>>> Ongoing.
>> >>> >>> >> > >>>> - (Rohith) YARN-5742: Serve aggregated logs of
>> historical
>> >>> apps
>> >>> >>> from
>> >>> >>> >> > >>>> ATSv2. Patch in progress.
>> >>> >>> >> > >>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
>> >>> >>> Improvement
>> >>> >>> >> > works.
>> >>> >>> >> > >>>> - (Steve) S3Guard Phase III, S3a phase V, Support
>> Windows
>> >>> Azure
>> >>> >>> >> > >>>> Storage. In progress.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 3. Tentative features:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
>> >>> >>> >> challenging
>> >>> >>> >> > to
>> >>> >>> >> > >>>> be done before Aug 2018.
>> >>> >>> >> > >>>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 26 Blocker and Critical issues [1] are open, I am
>> >>> following up
>> >>> >>> with
>> >>> >>> >> > >>>> owners to get status on each of them to get in by Code
>> >>> Freeze
>> >>> >>> date.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> >>> priority in
>> >>> >>> >> > >>>> (Blocker, Critical) AND resolution = Unresolved AND
>> "Target
>> >>> >>> >> > Version/s" =
>> >>> >>> >> > >>>> 3.2.0 ORDER BY priority DESC
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Thanks,
>> >>> >>> >> > >>>> Sunil
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> On Tue, Aug 14, 2018 at 10:30 PM Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> wrote:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>>> Hi All,
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> Thanks for the feedbacks. Inline with earlier
>> >>> communication
>> >>> >>> dated
>> >>> >>> >> > 17th
>> >>> >>> >> > >>>>> July 2018, I would like to provide some updates.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> We are approaching previously proposed feature freeze
>> date
>> >>> >>> (Aug
>> >>> >>> >> 21,
>> >>> >>> >> > >>>>> about 7 days from today).
>> >>> >>> >> > >>>>> If any features in branch which are targeted to 3.2.0,
>> >>> please
>> >>> >>> >> reply
>> >>> >>> >> > to
>> >>> >>> >> > >>>>> this email thread.
>> >>> >>> >> > >>>>> Steve has mentioned about the s3 features which will
>> come
>> >>> >>> close to
>> >>> >>> >> > >>>>> Code Freeze Date (Aug 31st).
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> *Here's an updated 3.2.0 feature status:*
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 1. Merged & Completed features:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> >>> DeepLearning
>> >>> >>> >> > >>>>> workloads Initial cut.
>> >>> >>> >> > >>>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 2. Features close to finish:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> Major
>> >>> >>> >> > >>>>> patches are all in, only one last
>> >>> >>> >> > >>>>> patch is in review state.
>> >>> >>> >> > >>>>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>>>> Scheduler. Close to commit.
>> >>> >>> >> > >>>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> >>> YARN
>> >>> >>> >> Service
>> >>> >>> >> > >>>>> API and CLI. 2 patches are pending
>> >>> >>> >> > >>>>> which will be closed by Feature freeze date.
>> >>> >>> >> > >>>>> - (Rohith) YARN-5742: Serve aggregated logs of
>> historical
>> >>> apps
>> >>> >>> >> from
>> >>> >>> >> > >>>>> ATSv2. Patch in progress.
>> >>> >>> >> > >>>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
>> >>> >>> Improvement
>> >>> >>> >> > >>>>> works.
>> >>> >>> >> > >>>>> - (Steve) S3Guard Phase III, S3a phase V, Support
>> Windows
>> >>> >>> Azure
>> >>> >>> >> > >>>>> Storage. In progress.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 3. Tentative features:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Haibo Chen) YARN-1011: Resource overcommitment.
>> Looks
>> >>> >>> >> challenging
>> >>> >>> >> > >>>>> to be done before Aug 2018.
>> >>> >>> >> > >>>>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>>>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 39 Blocker and Critical issues [1] are open, I am
>> checking
>> >>> >>> with
>> >>> >>> >> > owners
>> >>> >>> >> > >>>>> to get status on each of them to get in by Code Freeze
>> >>> date.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> >>> priority in
>> >>> >>> >> > >>>>> (Blocker, Critical) AND resolution = Unresolved AND
>> >>> "Target
>> >>> >>> >> > Version/s" =
>> >>> >>> >> > >>>>> 3.2.0 ORDER BY priority DESC
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> Thanks,
>> >>> >>> >> > >>>>> Sunil
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> On Fri, Jul 20, 2018 at 8:03 AM Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> wrote:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>>> Thanks Subru for the thoughts.
>> >>> >>> >> > >>>>>> One of the main reason for a major release is to push
>> out
>> >>> >>> >> critical
>> >>> >>> >> > >>>>>> features with a faster cadence to the users. If we are
>> >>> >>> pulling
>> >>> >>> >> more
>> >>> >>> >> > and
>> >>> >>> >> > >>>>>> more different types of features to a minor release,
>> that
>> >>> >>> branch
>> >>> >>> >> > will
>> >>> >>> >> > >>>>>> become more destabilized and it may be tough to say
>> that
>> >>> >>> 3.1.2 is
>> >>> >>> >> > stable
>> >>> >>> >> > >>>>>> that 3.1.1 for eg. We always tend to improve and
>> >>> stabilize
>> >>> >>> >> features
>> >>> >>> >> > in
>> >>> >>> >> > >>>>>> subsequent minor release.
>> >>> >>> >> > >>>>>> For few companies, it makes sense to push out these
>> new
>> >>> >>> features
>> >>> >>> >> > >>>>>> faster to make a reach to the users. Adding to the
>> point
>> >>> to
>> >>> >>> the
>> >>> >>> >> > backporting
>> >>> >>> >> > >>>>>> issues, I agree that its a pain and we can workaround
>> >>> that
>> >>> >>> with
>> >>> >>> >> > some git
>> >>> >>> >> > >>>>>> scripts. If we can make such scripts available to
>> >>> committers,
>> >>> >>> >> > backport will
>> >>> >>> >> > >>>>>> be seem-less across branches and we can achieve the
>> >>> faster
>> >>> >>> >> release
>> >>> >>> >> > cadence
>> >>> >>> >> > >>>>>> also.
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> Thoughts?
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> - Sunil
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> On Fri, Jul 20, 2018 at 3:37 AM Subru Krishnan <
>> >>> >>> subru@apache.org<ma...@apache.org>
>> >>> >>> >> >
>> >>> >>> >> > >>>>>> wrote:
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>>> Thanks Sunil for volunteering to lead the release
>> >>> effort. I
>> >>> >>> am
>> >>> >>> >> > >>>>>>> generally
>> >>> >>> >> > >>>>>>> supportive of a release but -1 on a 3.2 (prefer a
>> >>> 3.1.x) as
>> >>> >>> >> feel we
>> >>> >>> >> > >>>>>>> already
>> >>> >>> >> > >>>>>>> have too many branches to be maintained. I already
>> see
>> >>> many
>> >>> >>> >> commits
>> >>> >>> >> > >>>>>>> are in
>> >>> >>> >> > >>>>>>> different branches with no apparent rationale, for
>> e.g:
>> >>> 3.1
>> >>> >>> has
>> >>> >>> >> > >>>>>>> commits
>> >>> >>> >> > >>>>>>> which are absent in 3.0 etc.
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> Additionally AFAIK 3.x has not been deployed in any
>> >>> major
>> >>> >>> >> > production
>> >>> >>> >> > >>>>>>> setting so the cost of adding features should be
>> >>> minimal.
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> Thoughts?
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> -Subru
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> On Thu, Jul 19, 2018 at 12:31 AM, Sunil G <
>> >>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> > wrote:
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> > Thanks Steve, Aaron, Wangda for sharing thoughts.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Yes, important changes and features are much
>> needed,
>> >>> >>> hence we
>> >>> >>> >> > will
>> >>> >>> >> > >>>>>>> be
>> >>> >>> >> > >>>>>>> > keeping the door open for them as possible. Also
>> >>> >>> considering
>> >>> >>> >> few
>> >>> >>> >> > >>>>>>> more
>> >>> >>> >> > >>>>>>> > offline requests from other folks, I think
>> extending
>> >>> the
>> >>> >>> >> > timeframe
>> >>> >>> >> > >>>>>>> by
>> >>> >>> >> > >>>>>>> > couple of weeks makes sense (including a second RC
>> >>> >>> buffer) and
>> >>> >>> >> > >>>>>>> this should
>> >>> >>> >> > >>>>>>> > ideally help us to ship this by September itself.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Revised dates (I will be updating same in Roadmap
>> >>> wiki as
>> >>> >>> >> well)
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Feature freeze date : all features to merge by
>> >>> August
>> >>> >>> 21,
>> >>> >>> >> 2018.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Code freeze date : blockers/critical only, no
>> >>> >>> improvements
>> >>> >>> >> and
>> >>> >>> >> > >>>>>>> non
>> >>> >>> >> > >>>>>>> > blocker/critical
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > bug-fixes  August 31, 2018.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Release date: September 15, 2018
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Thank Eric and Zian, I think Wangda has already
>> >>> answered
>> >>> >>> your
>> >>> >>> >> > >>>>>>> questions.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Thanks
>> >>> >>> >> > >>>>>>> > Sunil
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > On Thu, Jul 19, 2018 at 12:13 PM Wangda Tan <
>> >>> >>> >> wheeleast@gmail.com<ma...@gmail.com>
>> >>> >>> >> > >
>> >>> >>> >> > >>>>>>> wrote:
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > > Thanks Sunil for volunteering to be RM of 3.2
>> >>> release,
>> >>> >>> +1
>> >>> >>> >> for
>> >>> >>> >> > >>>>>>> that.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > To concerns from Steve,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > It is a good idea to keep the door open to get
>> >>> important
>> >>> >>> >> > changes
>> >>> >>> >> > >>>>>>> /
>> >>> >>> >> > >>>>>>> > > features in before cutoff. I would prefer to keep
>> >>> the
>> >>> >>> >> proposed
>> >>> >>> >> > >>>>>>> release
>> >>> >>> >> > >>>>>>> > date
>> >>> >>> >> > >>>>>>> > > to make sure things can happen earlier instead of
>> >>> last
>> >>> >>> >> minute
>> >>> >>> >> > >>>>>>> and we all
>> >>> >>> >> > >>>>>>> > > know that releases are always get delayed :). I'm
>> >>> also
>> >>> >>> fine
>> >>> >>> >> if
>> >>> >>> >> > >>>>>>> we want
>> >>> >>> >> > >>>>>>> > get
>> >>> >>> >> > >>>>>>> > > another several weeks time.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Regarding of 3.3 release, I would suggest doing
>> that
>> >>> >>> before
>> >>> >>> >> > >>>>>>> thanksgiving.
>> >>> >>> >> > >>>>>>> > > Do you think is it good or too early / late?
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Eric,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > The YARN-8220 will be replaced by YARN-8135, if
>> >>> >>> YARN-8135
>> >>> >>> >> can
>> >>> >>> >> > >>>>>>> get merged
>> >>> >>> >> > >>>>>>> > > in time, we probably not need the YARN-8220.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Sunil,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Could u update
>> >>> >>> >> > >>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> >>> >>> >> > >>>>>>> > Roadmap
>> >>> >>> >> > >>>>>>> > > with proposed plan as well? We can fill feature
>> list
>> >>> >>> first
>> >>> >>> >> > >>>>>>> before getting
>> >>> >>> >> > >>>>>>> > > consensus of time.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Thanks,
>> >>> >>> >> > >>>>>>> > > Wangda
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > On Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri
>> >>> >>> >> > >>>>>>> <fabbri@cloudera.com.invalid<mailto:
>> >>> fabbri@cloudera.com.invalid>
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > wrote:
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > >> On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran <
>> >>> >>> >> > >>>>>>> stevel@hortonworks.com<mailto:stevel@hortonworks.com
>> >>
>> >>> >>> >> > >>>>>>> > >> wrote:
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > On 16 Jul 2018, at 23:45, Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>
>> >>> >>> >> > <mailto:
>> >>> >>> >> > >>>>>>> > >> > sunilg@apache.org<ma...@apache.org>>>
>> >>> wrote:
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I would also would like to take this
>> opportunity
>> >>> to
>> >>> >>> come
>> >>> >>> >> up
>> >>> >>> >> > >>>>>>> with a
>> >>> >>> >> > >>>>>>> > >> detailed
>> >>> >>> >> > >>>>>>> > >> > plan.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > - Feature freeze date : all features should be
>> >>> >>> merged by
>> >>> >>> >> > >>>>>>> August 10,
>> >>> >>> >> > >>>>>>> > >> 2018.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > <snip>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > Please let me know if I missed any features
>> >>> targeted
>> >>> >>> to
>> >>> >>> >> 3.2
>> >>> >>> >> > >>>>>>> per this
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > Well there these big todo lists for S3 &
>> S3Guard.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15226
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15220
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > There's a bigger bit of work coming on for
>> Azure
>> >>> >>> Datalake
>> >>> >>> >> > Gen
>> >>> >>> >> > >>>>>>> 2
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15407
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I don't think this is quite ready yet, I've
>> been
>> >>> >>> doing
>> >>> >>> >> work
>> >>> >>> >> > >>>>>>> on it, but
>> >>> >>> >> > >>>>>>> > >> if
>> >>> >>> >> > >>>>>>> > >> > we have a 3 week deadline, I'm going to expect
>> >>> some
>> >>> >>> >> timely
>> >>> >>> >> > >>>>>>> reviews on
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15546
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I've uprated that to a blocker feature; will
>> >>> review
>> >>> >>> the
>> >>> >>> >> S3 &
>> >>> >>> >> > >>>>>>> S3Guard
>> >>> >>> >> > >>>>>>> > >> JIRAs
>> >>> >>> >> > >>>>>>> > >> > to see which of those are blocking. Then there
>> >>> are
>> >>> >>> some
>> >>> >>> >> > >>>>>>> pressing
>> >>> >>> >> > >>>>>>> > "guave,
>> >>> >>> >> > >>>>>>> > >> > java 9 prep"
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >>  I can help with this part if you like.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > timeline. I would like to volunteer myself as
>> >>> release
>> >>> >>> >> > manager
>> >>> >>> >> > >>>>>>> of 3.2.0
>> >>> >>> >> > >>>>>>> > >> > release.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > well volunteered!
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> Yes, thank you for stepping up.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I think this raises a good q: what timetable
>> >>> should
>> >>> >>> we
>> >>> >>> >> have
>> >>> >>> >> > >>>>>>> for the
>> >>> >>> >> > >>>>>>> > >> 3.2. &
>> >>> >>> >> > >>>>>>> > >> > 3.3 releases; if we do want a faster cadence,
>> >>> then
>> >>> >>> having
>> >>> >>> >> > the
>> >>> >>> >> > >>>>>>> outline
>> >>> >>> >> > >>>>>>> > >> time
>> >>> >>> >> > >>>>>>> > >> > from the 3.2 to the 3.3 release means that
>> >>> there's
>> >>> >>> less
>> >>> >>> >> > >>>>>>> concern about
>> >>> >>> >> > >>>>>>> > >> > things not making the 3.2 dealine
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > -Steve
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> Good idea to mitigate the short deadline.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> -AF
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>
>> >>> >>> >> >
>> >>> >>> >>
>> >>> >>> >
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>>
>> >>>
>> >>> --
>> >>> A very happy Clouderan
>> >>>
>> >>>
>>
>

Re: Hadoop 3.2 Release Plan proposal

Posted by Sunil G <su...@apache.org>.
Hi Folks,

Given all blockers are closed now, I am planning to do RC1 tomorrow.
branch-3.2.0 remains closed for commit. Please let me know if any issues
are missed.

Thanks,
Sunil G

On Tue, Nov 13, 2018 at 8:11 PM Sunil G <su...@apache.org> wrote:

> Hi Folks,
>
> All blockers are closed by last weekend and corrected jiras as well.
> Preparing RC0 now. Facing some issue with shaded jars file size. Hence
> respinning again.
> Planning to complete this by end of this week.
>
> Thanks
> Sunil
>
> On Fri, Oct 26, 2018 at 2:34 AM Konstantin Shvachko <sh...@gmail.com>
> wrote:
>
>> Another thing is that I see a bunch of jiras under HDFS-8707, which don't
>> have the Fix Version field listing 3.2, some have it just empty.
>> This means they will not be populated into release notes.
>>
>> Thanks,
>> --Konstantin
>>
>> On Thu, Oct 25, 2018 at 7:59 AM Sunil G <su...@apache.org> wrote:
>>
>> > Thanks Konstantin for pointing out.
>> > As 3.2 is pretty much on RC level, its better we try to find a good
>> > solution to this issue.
>> >
>> > I ll follow up on this in the jira.
>> >
>> > - Sunil
>> >
>> > On Thu, Oct 25, 2018 at 11:35 AM Konstantin Shvachko <
>> shv.hadoop@gmail.com>
>> > wrote:
>> >
>> >> I've tried to attract attention to an incompatibility issue through the
>> >> jira, but it didn't work. So pitching in in this thread.
>> >> https://issues.apache.org/jira/browse/HDFS-12026
>> >> It introduced binary incompatibility, which will prevent people from
>> >> upgrading from 3.1 to 3.2.
>> >> I think it can get messy if we release anything with this feature.
>> >>
>> >> Thanks,
>> >> --Konstantin
>> >>
>> >> On Mon, Oct 22, 2018 at 5:01 AM Steve Loughran <stevel@hortonworks.com
>> >
>> >> wrote:
>> >>
>> >>> its in.
>> >>>
>> >>> good catch!
>> >>>
>> >>> On 20 Oct 2018, at 01:35, Wei-Chiu Chuang <weichiu@cloudera.com
>> <mailto:
>> >>> weichiu@cloudera.com>> wrote:
>> >>>
>> >>> Thanks Sunil G for driving the release,
>> >>> I filed HADOOP-15866<
>> https://issues.apache.org/jira/browse/HADOOP-15866>
>> >>> for a compat fix. If any one has cycle please review it, as I think
>> it is
>> >>> needed for 3.2.0.
>> >>>
>> >>> On Thu, Oct 18, 2018 at 4:43 AM Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>> Hi Folks,
>> >>>
>> >>> As we previously communicated for 3.2.0 release, we have delayed due
>> to
>> >>> few
>> >>> blockers in our gate.
>> >>>
>> >>> I just cut branch-3.2.0 for release purpose. branch-3.2 will be open
>> for
>> >>> all bug fixes.
>> >>>
>> >>> - Sunil
>> >>>
>> >>>
>> >>> On Tue, Oct 16, 2018 at 8:59 AM Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>>
>> >>> > Hi Folks,
>> >>> >
>> >>> > We are now close to RC as other blocker issues are now merged to
>> trunk
>> >>> and
>> >>> > branch-3.2. Last 2 critical issues are closer to merge and will be
>> >>> > committed in few hours.
>> >>> > With this, I will be creating 3.2.0 branch today and will go ahead
>> >>> with RC
>> >>> > related process.
>> >>> >
>> >>> > - Sunil
>> >>> >
>> >>> > On Mon, Oct 15, 2018 at 11:43 PM Jonathan Bender <
>> jonbender@stripe.com
>> >>> <ma...@stripe.com>>
>> >>> > wrote:
>> >>> >
>> >>> >> Hello, were there any updates around the 3.2.0 RC timing? All I
>> see in
>> >>> >> the current blockers are related to the new Submarine subproject,
>> >>> wasn't
>> >>> >> sure if that is what is holding things up.
>> >>> >>
>> >>> >> Cheers,
>> >>> >> Jon
>> >>> >>
>> >>> >> On Tue, Oct 2, 2018 at 7:13 PM, Sunil G <sunilg@apache.org<mailto:
>> >>> sunilg@apache.org>> wrote:
>> >>> >>
>> >>> >>> Thanks Robert and Haibo for quickly correcting same.
>> >>> >>> Sigh, I somehow missed one file while committing the change. Sorry
>> >>> for
>> >>> >>> the
>> >>> >>> trouble.
>> >>> >>>
>> >>> >>> - Sunil
>> >>> >>>
>> >>> >>> On Wed, Oct 3, 2018 at 5:22 AM Robert Kanter <
>> rkanter@cloudera.com
>> >>> <ma...@cloudera.com>>
>> >>> >>> wrote:
>> >>> >>>
>> >>> >>> > Looks like there's two that weren't updated:
>> >>> >>> > >> [115] 16:32 : hadoop-common (trunk) :: grep "3.2.0-SNAPSHOT"
>> .
>> >>> -r
>> >>> >>> > --include=pom.xml
>> >>> >>> > ./hadoop-project/pom.xml:
>> >>> >>> >
>> >>> <hadoop.assemblies.version>3.2.0-SNAPSHOT</hadoop.assemblies.version>
>> >>> >>> > ./pom.xml:    <hadoop.version>3.2.0-SNAPSHOT</hadoop.version>
>> >>> >>> >
>> >>> >>> > I've just pushed in an addendum commit to fix those.
>> >>> >>> > In the future, please make sure to do a sanity compile when
>> >>> updating
>> >>> >>> poms.
>> >>> >>> >
>> >>> >>> > thanks
>> >>> >>> > - Robert
>> >>> >>> >
>> >>> >>> > On Tue, Oct 2, 2018 at 11:44 AM Aaron Fabbri
>> >>> >>> <fa...@cloudera.com.invalid>>
>> >>> >>> > wrote:
>> >>> >>> >
>> >>> >>> >> Trunk is not building for me.. Did you miss a 3.2.0-SNAPSHOT in
>> >>> the
>> >>> >>> >> top-level pom.xml?
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> On Tue, Oct 2, 2018 at 10:16 AM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>> wrote:
>> >>> >>> >>
>> >>> >>> >> > Hi All
>> >>> >>> >> >
>> >>> >>> >> > As mentioned in earlier mail, I have cut branch-3.2 and reset
>> >>> trunk
>> >>> >>> to
>> >>> >>> >> > 3.3.0-SNAPSHOT. I will share the RC details sooner once all
>> >>> >>> necessary
>> >>> >>> >> > patches are pulled into branch-3.2.
>> >>> >>> >> >
>> >>> >>> >> > Thank You
>> >>> >>> >> > - Sunil
>> >>> >>> >> >
>> >>> >>> >> >
>> >>> >>> >> > On Mon, Sep 24, 2018 at 2:00 PM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>> wrote:
>> >>> >>> >> >
>> >>> >>> >> > > Hi All
>> >>> >>> >> > >
>> >>> >>> >> > > We are now down to the last Blocker and HADOOP-15407 is
>> >>> merged to
>> >>> >>> >> trunk.
>> >>> >>> >> > > Thanks for the support.
>> >>> >>> >> > >
>> >>> >>> >> > > *Plan for RC*
>> >>> >>> >> > > 3.2 branch cut and reset trunk : *25th Tuesday*
>> >>> >>> >> > > RC0 for 3.2: *28th Friday*
>> >>> >>> >> > >
>> >>> >>> >> > > Thank You
>> >>> >>> >> > > Sunil
>> >>> >>> >> > >
>> >>> >>> >> > >
>> >>> >>> >> > > On Mon, Sep 17, 2018 at 3:21 PM Sunil G <sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >
>> >>> >>> >> > >> Hi All
>> >>> >>> >> > >>
>> >>> >>> >> > >> We are down to 3 Blockers and 4 Critical now. Thanks all
>> of
>> >>> you
>> >>> >>> for
>> >>> >>> >> > >> helping in this. I am following up on these tickets, once
>> its
>> >>> >>> closed
>> >>> >>> >> we
>> >>> >>> >> > >> will cut the 3.2 branch.
>> >>> >>> >> > >>
>> >>> >>> >> > >> Thanks
>> >>> >>> >> > >> Sunil Govindan
>> >>> >>> >> > >>
>> >>> >>> >> > >>
>> >>> >>> >> > >> On Wed, Sep 12, 2018 at 5:10 PM Sunil G <
>> sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >>
>> >>> >>> >> > >>> Hi All,
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Inline with the original 3.2 communication proposal dated
>> >>> 17th
>> >>> >>> July
>> >>> >>> >> > >>> 2018, I would like to provide more updates.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> We are approaching previously proposed code freeze date
>> >>> >>> (September
>> >>> >>> >> 14,
>> >>> >>> >> > >>> 2018). So I would like to cut 3.2 branch on 17th Sept and
>> >>> point
>> >>> >>> >> > existing
>> >>> >>> >> > >>> trunk to 3.3 if there are no issues.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> *Current Release Plan:*
>> >>> >>> >> > >>> Feature freeze date : all features to merge by September
>> 7,
>> >>> >>> 2018.
>> >>> >>> >> > >>> Code freeze date : blockers/critical only, no
>> improvements
>> >>> and
>> >>> >>> >> > >>> blocker/critical bug-fixes September 14, 2018.
>> >>> >>> >> > >>> Release date: September 28, 2018
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> If any critical/blocker tickets which are targeted to
>> >>> 3.2.0, we
>> >>> >>> >> need to
>> >>> >>> >> > >>> backport to 3.2 post branch cut.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Here's an updated 3.2.0 feature status:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 1. Merged & Completed features:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> DeepLearning
>> >>> >>> >> > >>> workloads Initial cut.
>> >>> >>> >> > >>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>> Scheduler.
>> >>> >>> >> > >>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> YARN
>> >>> >>> Service
>> >>> >>> >> > API
>> >>> >>> >> > >>> and CLI.
>> >>> >>> >> > >>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> >> > >>> - (Inigo) HDFS-12615: Router-based HDFS federation.
>> >>> Improvement
>> >>> >>> >> works.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 2. Features close to finish:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> - (Steve) S3Guard Phase III. Close to commit.
>> >>> >>> >> > >>> - (Steve) S3a phase V. Close to commit.
>> >>> >>> >> > >>> - (Steve) Support Windows Azure Storage. Close to commit.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> 3. Tentative/Cancelled features for 3.2:
>> >>> >>> >> > >>> - (Rohith) YARN-5742: Serve aggregated logs of historical
>> >>> apps
>> >>> >>> from
>> >>> >>> >> > >>> ATSv2. Patch in progress.
>> >>> >>> >> > >>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
>> >>> >>> >> challenging to
>> >>> >>> >> > >>> be done before Aug 2018.
>> >>> >>> >> > >>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>> 19 Blocker and Critical issues [1] are open, I am
>> following
>> >>> up
>> >>> >>> with
>> >>> >>> >> > >>> owners to get status on each of them to get in by Code
>> >>> Freeze
>> >>> >>> date.
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> priority
>> >>> in
>> >>> >>> >> > (Blocker,
>> >>> >>> >> > >>> Critical) AND resolution = Unresolved AND "Target
>> >>> Version/s" =
>> >>> >>> 3.2.0
>> >>> >>> >> > ORDER
>> >>> >>> >> > >>> BY priority DESC
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> Thanks,
>> >>> >>> >> > >>> Sunil
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>
>> >>> >>> >> > >>> On Thu, Aug 30, 2018 at 9:59 PM Sunil G <
>> sunilg@apache.org
>> >>> <ma...@apache.org>>
>> >>> >>> wrote:
>> >>> >>> >> > >>>
>> >>> >>> >> > >>>> Hi All,
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Inline with earlier communication dated 17th July 2018,
>> I
>> >>> would
>> >>> >>> >> like
>> >>> >>> >> > to
>> >>> >>> >> > >>>> provide some updates.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> We are approaching previously proposed code freeze date
>> >>> (Aug
>> >>> >>> 31).
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> One of the critical feature Node Attributes feature
>> merge
>> >>> >>> >> > >>>> discussion/vote is ongoing. Also few other Blocker bugs
>> >>> need a
>> >>> >>> bit
>> >>> >>> >> > more
>> >>> >>> >> > >>>> time. With regard to this, suggesting to push the
>> >>> feature/code
>> >>> >>> >> freeze
>> >>> >>> >> > for 2
>> >>> >>> >> > >>>> more weeks to accommodate these jiras too.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Proposing Updated changes in plan inline with this:
>> >>> >>> >> > >>>> Feature freeze date : all features to merge by
>> September 7,
>> >>> >>> 2018.
>> >>> >>> >> > >>>> Code freeze date : blockers/critical only, no
>> improvements
>> >>> and
>> >>> >>> >> > >>>>  blocker/critical bug-fixes September 14, 2018.
>> >>> >>> >> > >>>> Release date: September 28, 2018
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> If any features in branch which are targeted to 3.2.0,
>> >>> please
>> >>> >>> >> reply to
>> >>> >>> >> > >>>> this email thread.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> *Here's an updated 3.2.0 feature status:*
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 1. Merged & Completed features:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> DeepLearning
>> >>> >>> >> > >>>> workloads Initial cut.
>> >>> >>> >> > >>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>>> Scheduler.
>> >>> >>> >> > >>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> >>> YARN
>> >>> >>> >> Service
>> >>> >>> >> > >>>> API and CLI.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 2. Features close to finish:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> >> Merge/Vote
>> >>> >>> >> > >>>> Ongoing.
>> >>> >>> >> > >>>> - (Rohith) YARN-5742: Serve aggregated logs of
>> historical
>> >>> apps
>> >>> >>> from
>> >>> >>> >> > >>>> ATSv2. Patch in progress.
>> >>> >>> >> > >>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
>> >>> >>> Improvement
>> >>> >>> >> > works.
>> >>> >>> >> > >>>> - (Steve) S3Guard Phase III, S3a phase V, Support
>> Windows
>> >>> Azure
>> >>> >>> >> > >>>> Storage. In progress.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 3. Tentative features:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> - (Haibo Chen) YARN-1011: Resource overcommitment. Looks
>> >>> >>> >> challenging
>> >>> >>> >> > to
>> >>> >>> >> > >>>> be done before Aug 2018.
>> >>> >>> >> > >>>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> 26 Blocker and Critical issues [1] are open, I am
>> >>> following up
>> >>> >>> with
>> >>> >>> >> > >>>> owners to get status on each of them to get in by Code
>> >>> Freeze
>> >>> >>> date.
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> >>> priority in
>> >>> >>> >> > >>>> (Blocker, Critical) AND resolution = Unresolved AND
>> "Target
>> >>> >>> >> > Version/s" =
>> >>> >>> >> > >>>> 3.2.0 ORDER BY priority DESC
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> Thanks,
>> >>> >>> >> > >>>> Sunil
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>> On Tue, Aug 14, 2018 at 10:30 PM Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> wrote:
>> >>> >>> >> > >>>>
>> >>> >>> >> > >>>>> Hi All,
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> Thanks for the feedbacks. Inline with earlier
>> >>> communication
>> >>> >>> dated
>> >>> >>> >> > 17th
>> >>> >>> >> > >>>>> July 2018, I would like to provide some updates.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> We are approaching previously proposed feature freeze
>> date
>> >>> >>> (Aug
>> >>> >>> >> 21,
>> >>> >>> >> > >>>>> about 7 days from today).
>> >>> >>> >> > >>>>> If any features in branch which are targeted to 3.2.0,
>> >>> please
>> >>> >>> >> reply
>> >>> >>> >> > to
>> >>> >>> >> > >>>>> this email thread.
>> >>> >>> >> > >>>>> Steve has mentioned about the s3 features which will
>> come
>> >>> >>> close to
>> >>> >>> >> > >>>>> Code Freeze Date (Aug 31st).
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> *Here's an updated 3.2.0 feature status:*
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 1. Merged & Completed features:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Wangda) YARN-8561: Hadoop Submarine project for
>> >>> >>> DeepLearning
>> >>> >>> >> > >>>>> workloads Initial cut.
>> >>> >>> >> > >>>>> - (Uma) HDFS-10285: HDFS Storage Policy Satisfier
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 2. Features close to finish:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Naga/Sunil) YARN-3409: Node Attributes support in
>> YARN.
>> >>> >>> Major
>> >>> >>> >> > >>>>> patches are all in, only one last
>> >>> >>> >> > >>>>> patch is in review state.
>> >>> >>> >> > >>>>> - (Sunil) YARN-7494: Multi Node scheduling support in
>> >>> Capacity
>> >>> >>> >> > >>>>> Scheduler. Close to commit.
>> >>> >>> >> > >>>>> - (Chandni/Eric) YARN-7512: Support service upgrade via
>> >>> YARN
>> >>> >>> >> Service
>> >>> >>> >> > >>>>> API and CLI. 2 patches are pending
>> >>> >>> >> > >>>>> which will be closed by Feature freeze date.
>> >>> >>> >> > >>>>> - (Rohith) YARN-5742: Serve aggregated logs of
>> historical
>> >>> apps
>> >>> >>> >> from
>> >>> >>> >> > >>>>> ATSv2. Patch in progress.
>> >>> >>> >> > >>>>> - (Virajit) HDFS-12615: Router-based HDFS federation.
>> >>> >>> Improvement
>> >>> >>> >> > >>>>> works.
>> >>> >>> >> > >>>>> - (Steve) S3Guard Phase III, S3a phase V, Support
>> Windows
>> >>> >>> Azure
>> >>> >>> >> > >>>>> Storage. In progress.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 3. Tentative features:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> - (Haibo Chen) YARN-1011: Resource overcommitment.
>> Looks
>> >>> >>> >> challenging
>> >>> >>> >> > >>>>> to be done before Aug 2018.
>> >>> >>> >> > >>>>> - (Eric) YARN-7129: Application Catalog for YARN
>> >>> applications.
>> >>> >>> >> > >>>>> Challenging as more discussions are on-going.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> *Summary of 3.2.0 issues status:*
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> 39 Blocker and Critical issues [1] are open, I am
>> checking
>> >>> >>> with
>> >>> >>> >> > owners
>> >>> >>> >> > >>>>> to get status on each of them to get in by Code Freeze
>> >>> date.
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
>> >>> priority in
>> >>> >>> >> > >>>>> (Blocker, Critical) AND resolution = Unresolved AND
>> >>> "Target
>> >>> >>> >> > Version/s" =
>> >>> >>> >> > >>>>> 3.2.0 ORDER BY priority DESC
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> Thanks,
>> >>> >>> >> > >>>>> Sunil
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>> On Fri, Jul 20, 2018 at 8:03 AM Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> wrote:
>> >>> >>> >> > >>>>>
>> >>> >>> >> > >>>>>> Thanks Subru for the thoughts.
>> >>> >>> >> > >>>>>> One of the main reason for a major release is to push
>> out
>> >>> >>> >> critical
>> >>> >>> >> > >>>>>> features with a faster cadence to the users. If we are
>> >>> >>> pulling
>> >>> >>> >> more
>> >>> >>> >> > and
>> >>> >>> >> > >>>>>> more different types of features to a minor release,
>> that
>> >>> >>> branch
>> >>> >>> >> > will
>> >>> >>> >> > >>>>>> become more destabilized and it may be tough to say
>> that
>> >>> >>> 3.1.2 is
>> >>> >>> >> > stable
>> >>> >>> >> > >>>>>> that 3.1.1 for eg. We always tend to improve and
>> >>> stabilize
>> >>> >>> >> features
>> >>> >>> >> > in
>> >>> >>> >> > >>>>>> subsequent minor release.
>> >>> >>> >> > >>>>>> For few companies, it makes sense to push out these
>> new
>> >>> >>> features
>> >>> >>> >> > >>>>>> faster to make a reach to the users. Adding to the
>> point
>> >>> to
>> >>> >>> the
>> >>> >>> >> > backporting
>> >>> >>> >> > >>>>>> issues, I agree that its a pain and we can workaround
>> >>> that
>> >>> >>> with
>> >>> >>> >> > some git
>> >>> >>> >> > >>>>>> scripts. If we can make such scripts available to
>> >>> committers,
>> >>> >>> >> > backport will
>> >>> >>> >> > >>>>>> be seem-less across branches and we can achieve the
>> >>> faster
>> >>> >>> >> release
>> >>> >>> >> > cadence
>> >>> >>> >> > >>>>>> also.
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> Thoughts?
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> - Sunil
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>> On Fri, Jul 20, 2018 at 3:37 AM Subru Krishnan <
>> >>> >>> subru@apache.org<ma...@apache.org>
>> >>> >>> >> >
>> >>> >>> >> > >>>>>> wrote:
>> >>> >>> >> > >>>>>>
>> >>> >>> >> > >>>>>>> Thanks Sunil for volunteering to lead the release
>> >>> effort. I
>> >>> >>> am
>> >>> >>> >> > >>>>>>> generally
>> >>> >>> >> > >>>>>>> supportive of a release but -1 on a 3.2 (prefer a
>> >>> 3.1.x) as
>> >>> >>> >> feel we
>> >>> >>> >> > >>>>>>> already
>> >>> >>> >> > >>>>>>> have too many branches to be maintained. I already
>> see
>> >>> many
>> >>> >>> >> commits
>> >>> >>> >> > >>>>>>> are in
>> >>> >>> >> > >>>>>>> different branches with no apparent rationale, for
>> e.g:
>> >>> 3.1
>> >>> >>> has
>> >>> >>> >> > >>>>>>> commits
>> >>> >>> >> > >>>>>>> which are absent in 3.0 etc.
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> Additionally AFAIK 3.x has not been deployed in any
>> >>> major
>> >>> >>> >> > production
>> >>> >>> >> > >>>>>>> setting so the cost of adding features should be
>> >>> minimal.
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> Thoughts?
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> -Subru
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> On Thu, Jul 19, 2018 at 12:31 AM, Sunil G <
>> >>> >>> sunilg@apache.org<ma...@apache.org>>
>> >>> >>> >> > wrote:
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>> > Thanks Steve, Aaron, Wangda for sharing thoughts.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Yes, important changes and features are much
>> needed,
>> >>> >>> hence we
>> >>> >>> >> > will
>> >>> >>> >> > >>>>>>> be
>> >>> >>> >> > >>>>>>> > keeping the door open for them as possible. Also
>> >>> >>> considering
>> >>> >>> >> few
>> >>> >>> >> > >>>>>>> more
>> >>> >>> >> > >>>>>>> > offline requests from other folks, I think
>> extending
>> >>> the
>> >>> >>> >> > timeframe
>> >>> >>> >> > >>>>>>> by
>> >>> >>> >> > >>>>>>> > couple of weeks makes sense (including a second RC
>> >>> >>> buffer) and
>> >>> >>> >> > >>>>>>> this should
>> >>> >>> >> > >>>>>>> > ideally help us to ship this by September itself.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Revised dates (I will be updating same in Roadmap
>> >>> wiki as
>> >>> >>> >> well)
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Feature freeze date : all features to merge by
>> >>> August
>> >>> >>> 21,
>> >>> >>> >> 2018.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Code freeze date : blockers/critical only, no
>> >>> >>> improvements
>> >>> >>> >> and
>> >>> >>> >> > >>>>>>> non
>> >>> >>> >> > >>>>>>> > blocker/critical
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > bug-fixes  August 31, 2018.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > - Release date: September 15, 2018
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Thank Eric and Zian, I think Wangda has already
>> >>> answered
>> >>> >>> your
>> >>> >>> >> > >>>>>>> questions.
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > Thanks
>> >>> >>> >> > >>>>>>> > Sunil
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > On Thu, Jul 19, 2018 at 12:13 PM Wangda Tan <
>> >>> >>> >> wheeleast@gmail.com<ma...@gmail.com>
>> >>> >>> >> > >
>> >>> >>> >> > >>>>>>> wrote:
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>> > > Thanks Sunil for volunteering to be RM of 3.2
>> >>> release,
>> >>> >>> +1
>> >>> >>> >> for
>> >>> >>> >> > >>>>>>> that.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > To concerns from Steve,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > It is a good idea to keep the door open to get
>> >>> important
>> >>> >>> >> > changes
>> >>> >>> >> > >>>>>>> /
>> >>> >>> >> > >>>>>>> > > features in before cutoff. I would prefer to keep
>> >>> the
>> >>> >>> >> proposed
>> >>> >>> >> > >>>>>>> release
>> >>> >>> >> > >>>>>>> > date
>> >>> >>> >> > >>>>>>> > > to make sure things can happen earlier instead of
>> >>> last
>> >>> >>> >> minute
>> >>> >>> >> > >>>>>>> and we all
>> >>> >>> >> > >>>>>>> > > know that releases are always get delayed :). I'm
>> >>> also
>> >>> >>> fine
>> >>> >>> >> if
>> >>> >>> >> > >>>>>>> we want
>> >>> >>> >> > >>>>>>> > get
>> >>> >>> >> > >>>>>>> > > another several weeks time.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Regarding of 3.3 release, I would suggest doing
>> that
>> >>> >>> before
>> >>> >>> >> > >>>>>>> thanksgiving.
>> >>> >>> >> > >>>>>>> > > Do you think is it good or too early / late?
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Eric,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > The YARN-8220 will be replaced by YARN-8135, if
>> >>> >>> YARN-8135
>> >>> >>> >> can
>> >>> >>> >> > >>>>>>> get merged
>> >>> >>> >> > >>>>>>> > > in time, we probably not need the YARN-8220.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Sunil,
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Could u update
>> >>> >>> >> > >>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> >>> >>> >> > >>>>>>> > Roadmap
>> >>> >>> >> > >>>>>>> > > with proposed plan as well? We can fill feature
>> list
>> >>> >>> first
>> >>> >>> >> > >>>>>>> before getting
>> >>> >>> >> > >>>>>>> > > consensus of time.
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > Thanks,
>> >>> >>> >> > >>>>>>> > > Wangda
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > On Wed, Jul 18, 2018 at 6:20 PM Aaron Fabbri
>> >>> >>> >> > >>>>>>> <fabbri@cloudera.com.invalid<mailto:
>> >>> fabbri@cloudera.com.invalid>
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > > wrote:
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> > >> On Tue, Jul 17, 2018 at 7:21 PM Steve Loughran <
>> >>> >>> >> > >>>>>>> stevel@hortonworks.com<mailto:stevel@hortonworks.com
>> >>
>> >>> >>> >> > >>>>>>> > >> wrote:
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > On 16 Jul 2018, at 23:45, Sunil G <
>> >>> sunilg@apache.org<ma...@apache.org>
>> >>> >>> >> > <mailto:
>> >>> >>> >> > >>>>>>> > >> > sunilg@apache.org<ma...@apache.org>>>
>> >>> wrote:
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I would also would like to take this
>> opportunity
>> >>> to
>> >>> >>> come
>> >>> >>> >> up
>> >>> >>> >> > >>>>>>> with a
>> >>> >>> >> > >>>>>>> > >> detailed
>> >>> >>> >> > >>>>>>> > >> > plan.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > - Feature freeze date : all features should be
>> >>> >>> merged by
>> >>> >>> >> > >>>>>>> August 10,
>> >>> >>> >> > >>>>>>> > >> 2018.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > <snip>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > Please let me know if I missed any features
>> >>> targeted
>> >>> >>> to
>> >>> >>> >> 3.2
>> >>> >>> >> > >>>>>>> per this
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > Well there these big todo lists for S3 &
>> S3Guard.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15226
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15220
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > There's a bigger bit of work coming on for
>> Azure
>> >>> >>> Datalake
>> >>> >>> >> > Gen
>> >>> >>> >> > >>>>>>> 2
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15407
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I don't think this is quite ready yet, I've
>> been
>> >>> >>> doing
>> >>> >>> >> work
>> >>> >>> >> > >>>>>>> on it, but
>> >>> >>> >> > >>>>>>> > >> if
>> >>> >>> >> > >>>>>>> > >> > we have a 3 week deadline, I'm going to expect
>> >>> some
>> >>> >>> >> timely
>> >>> >>> >> > >>>>>>> reviews on
>> >>> >>> >> > >>>>>>> > >> >
>> >>> https://issues.apache.org/jira/browse/HADOOP-15546
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I've uprated that to a blocker feature; will
>> >>> review
>> >>> >>> the
>> >>> >>> >> S3 &
>> >>> >>> >> > >>>>>>> S3Guard
>> >>> >>> >> > >>>>>>> > >> JIRAs
>> >>> >>> >> > >>>>>>> > >> > to see which of those are blocking. Then there
>> >>> are
>> >>> >>> some
>> >>> >>> >> > >>>>>>> pressing
>> >>> >>> >> > >>>>>>> > "guave,
>> >>> >>> >> > >>>>>>> > >> > java 9 prep"
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >>  I can help with this part if you like.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > timeline. I would like to volunteer myself as
>> >>> release
>> >>> >>> >> > manager
>> >>> >>> >> > >>>>>>> of 3.2.0
>> >>> >>> >> > >>>>>>> > >> > release.
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > well volunteered!
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> Yes, thank you for stepping up.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > I think this raises a good q: what timetable
>> >>> should
>> >>> >>> we
>> >>> >>> >> have
>> >>> >>> >> > >>>>>>> for the
>> >>> >>> >> > >>>>>>> > >> 3.2. &
>> >>> >>> >> > >>>>>>> > >> > 3.3 releases; if we do want a faster cadence,
>> >>> then
>> >>> >>> having
>> >>> >>> >> > the
>> >>> >>> >> > >>>>>>> outline
>> >>> >>> >> > >>>>>>> > >> time
>> >>> >>> >> > >>>>>>> > >> > from the 3.2 to the 3.3 release means that
>> >>> there's
>> >>> >>> less
>> >>> >>> >> > >>>>>>> concern about
>> >>> >>> >> > >>>>>>> > >> > things not making the 3.2 dealine
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> > -Steve
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> >
>> >>> >>> >> > >>>>>>> > >> Good idea to mitigate the short deadline.
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >> -AF
>> >>> >>> >> > >>>>>>> > >>
>> >>> >>> >> > >>>>>>> > >
>> >>> >>> >> > >>>>>>> >
>> >>> >>> >> > >>>>>>>
>> >>> >>> >> > >>>>>>
>> >>> >>> >> >
>> >>> >>> >>
>> >>> >>> >
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>>
>> >>>
>> >>> --
>> >>> A very happy Clouderan
>> >>>
>> >>>
>>
>