Release plan for Apache Hadoop 2.6.3

[Junping Du <jd...@hortonworks.com>]

Hi all,
   Based on the feedback from previous email threads and JIRAs, we are in good time for releasing 2.6.3!

Things Done/TODO:

  + Branch
    -- Based on branch-2.6, which has been open to 2.6.3 commits for a while, I already branch out branch-2.6.3 today.
    -- New commits coming as fix to 2.6.3 need to explicitly cherry-pick to branch-2.6.3 as well as branch-2.6.

  + Patches
    -- 2.6.3 already has 31 fixes as list of [1] shows.
    -- The list of open tickets [2] on 2.6.3 is relative small (3 for now) as we already move non-blocker / non-critical ones out to 2.6.4. I'll keep triaging and see what we can get in within a week or so.

  + Release
    -- Given the open ticket list is relative short and all of them are patch available, I plan to start RC process for voting early next week with leaving aside some buffer time.

Thoughts?

Appreciate any help in moving forward open tickets [2].​

Thanks,

Junping

[1] 2.6.3 Fixed Tickets:
http://s.apache.org/Ylq

[2] 2.6.3 Tickets:
http://s.apache.org/9A7



________________________________
From: Junping Du
Sent: Tuesday, December 01, 2015 12:26 PM
To: Vinod Kumar Vavilapalli
Cc: Sangjin Lee; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Hadoop Common; hdfs-dev@hadoop.apache.org; Haohui Mai; Chris Trezzo; junping_du@apache.org
Subject: Re: continuing releases on Apache Hadoop 2.6.x


Thanks Vinod!

I went through all 31 fixes (Common: 8, HDFS: 6, MAPREDUCE: 8, YARN: 9) landed on branch-2.6​ now, and all outstanding patches (>=5KB) are listed below and they are all bug fix.

Haven't found any high risky patches so far but I noticed we have 3 significant fixes (MAPREDUCE-5948, MAPREDUCE-6481 and MAPREDUCE-6549) for LineRecordReader.  Among them, MAPREDUCE-5948 and MAPREDUCE-6481 are completed fixes and we will ship them in 2.7.2, so I think there is no risk with them. MAPREDUCE-6549, together with an open JIRA: MAPREDUCE-6558, sounds like fix the same issue for uncompressed/compressed input file. I already ping the JIRA owners to get suggestions on if we could release MAPREDUCE-6549 independently or together with MAPREDUCE-6558 in next release.

I already pinged all owners for open JIRAs that targeted for 2.6.3 to move it to 2.6.4. Based on the feedback from JIRAs (and voices here too), I will create branch-2.6.3 in next 1 day or 2. Thanks!​


List of Outstanding Patches:


    HADOOP-10134 [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. (apurtell via stevel) 38K

    HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in commitBlock. (Chang Li via zhz) 12K
    HDFS-9470. Encryption zone on root not loaded from fsimage after NN restart. (Xiao Chen via wang) 5K

    MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state. (zhihai xu via devaraj) 5K
    MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well (Vinayakumar B, Rushabh Shah, and Akira AJISAKA via jlowe) 20K
    MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. (Zhihai Xu via jlowe) 19K
    MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter) 31K

    YARN-2859. ApplicationHistoryServer binds to default port 8188 in MiniYARNCluster.(Vinod Kumar Vavilapalli via xgong) 6K (with addendum)
    YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations (Varun Vasudev via jlowe) 12K
    YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. (zhihai xu via jlowe) 7K


Thanks,


Junping


________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>
Sent: Monday, November 30, 2015 9:23 PM
To: Junping Du
Cc: Sangjin Lee; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Hadoop Common; hdfs-dev@hadoop.apache.org; Haohui Mai; Chris Trezzo
Subject: Re: continuing releases on Apache Hadoop 2.6.x

+1, this sounds good to me.

Thanks
+Vinod


On Nov 25, 2015, at 11:44 AM, Junping Du <jd...@hortonworks.com>> wrote:

May be we should quickly go through landed patches in 2.6.3 list and postpone any risky patches to 2.6.4?