You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by Zhijie Shen <zs...@hortonworks.com> on 2014/01/17 21:53:48 UTC

[VOTE] Merge YARN-321 Generic Application History Service to trunk

Hi folks,

As previously discussed here (http://markmail.org/message/iscvp7cedrtvmd6p),
I would like to call a vote to merge the YARN-321 branch for Generic
Application History Server into trunk.

*Scope of the changes*

The changes enable ResourceManager to record the historic information of
the application, the application attempt and the container in terms of
events via a history writer. In addition, the changes setup up an
application history server, which allows users to access the recorded
information via RPC interface, web UI and REST APIs.

*Details of development*

- Development of the feature is tracked in the jira -
https://issues.apache.org/jira/browse/YARN-321.
- Development has been done in a separate branch -
https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
- The feature development involved about 35 subtasks.
- The up-to-date design is posted at -
https://issues.apache.org/jira/secure/attachment/12619638/Generic
Application
History - Design-20131219.pdf<https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf>

- The uber merge patch Jira -
https://issues.apache.org/jira/browse/YARN-1587

*Testing*

A number of unit tests have been added as a part of the feature. In
addition, we’ve also done end-to-end functional tests, and performance
tests for HDFS-based history storage and history events processing. Last
but not least, we have updated branch YARN-321 against the latest trunk,
edited merge conflicts, fixed test failures caused by merge, and corrected
a bunch of bad source code issues. The uber merge patch that contains all
the diff between branch YARN-321 and trunk has been run through Jenkins.

*Pending work*

- Make it work in secure mode
- Pending bug fixes

We wish to merge the branch now instead of waiting for later. The main
reason for this is that as the branch grew in size, the cost of its
maintenance became huge. Once the feature is merged into trunk, we will
continue to work on pending work like security stuff, to test and fix any
bugs that may be found on the trunk, and to refactor the code about to
share some pieces in PRC and web interfaces.

*Release status*

If the security stuff and the pending fixes arrive by the time everything
else planned for Release 2.4 is done, we can include it as well. This is
what we are striving for. Otherwise, we will call AHS not-feature-complete
and not stable.

The bulk of the design and implementation was done by Mayank Bansal and me
with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
others. Also, thanks to Robert Joseph Evans and Sandy Ryza for providing
feedback on the design discussions.

This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.

Thanks,
Zhijie

--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Chen He <ai...@gmail.com>.

As a cluster resource management platform, generic history service is
necessary for job-tracing, troubleshooting, fault tolerance, and
performance tuning.
Like StarterLog in Condor, accounting logs in Torque, and so on. However,
generic history service should avoid getting involved into too many details
since some applications may have modules to trace their own histories.


+1  non-binding


On Fri, Jan 17, 2014 at 2:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:

> Hi folks,
>
> As previously discussed here (http://markmail.org/message/iscvp7cedrtvmd6p
> ),
> I would like to call a vote to merge the YARN-321 branch for Generic
> Application History Server into trunk.
>
> *Scope of the changes*
>
>
> The changes enable ResourceManager to record the historic information of
> the application, the application attempt and the container in terms of
> events via a history writer. In addition, the changes setup up an
> application history server, which allows users to access the recorded
> information via RPC interface, web UI and REST APIs.
>
> *Details of development*
>
>
>  - Development of the feature is tracked in the jira -
> https://issues.apache.org/jira/browse/YARN-321.
>  - Development has been done in a separate branch -
> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>  - The feature development involved about 35 subtasks.
>  - The up-to-date design is posted at -
> https://issues.apache.org/jira/secure/attachment/12619638/Generic
> Application
> History - Design-20131219.pdf<
> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
> >
>
>  - The uber merge patch Jira -
> https://issues.apache.org/jira/browse/YARN-1587
>
> *Testing*
>
>
> A number of unit tests have been added as a part of the feature. In
> addition, we’ve also done end-to-end functional tests, and performance
> tests for HDFS-based history storage and history events processing. Last
> but not least, we have updated branch YARN-321 against the latest trunk,
> edited merge conflicts, fixed test failures caused by merge, and corrected
> a bunch of bad source code issues. The uber merge patch that contains all
> the diff between branch YARN-321 and trunk has been run through Jenkins.
>
> *Pending work*
>
>
>  - Make it work in secure mode
>  - Pending bug fixes
>
> We wish to merge the branch now instead of waiting for later. The main
> reason for this is that as the branch grew in size, the cost of its
> maintenance became huge. Once the feature is merged into trunk, we will
> continue to work on pending work like security stuff, to test and fix any
> bugs that may be found on the trunk, and to refactor the code about to
> share some pieces in PRC and web interfaces.
>
> *Release status*
>
>
> If the security stuff and the pending fixes arrive by the time everything
> else planned for Release 2.4 is done, we can include it as well. This is
> what we are striving for. Otherwise, we will call AHS not-feature-complete
> and not stable.
>
> The bulk of the design and implementation was done by Mayank Bansal and me
> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for providing
> feedback on the design discussions.
>
>
>
> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>
> Thanks,
> Zhijie
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.

Thanks Zhijie!

I've been involved in its design. And reviewed almost every patch. Contributed a little w.r.t tests etc.

The code is in a decent shape bar some bugs and security. The merge should let us avoid the maintenance burden.

Tested the latest code on my single node setup. It mostly looks good.

+1 (binding).

Thanks,
+Vinod


On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:

> Hi folks,
> 
> As previously discussed here (http://markmail.org/message/iscvp7cedrtvmd6p),
> I would like to call a vote to merge the YARN-321 branch for Generic
> Application History Server into trunk.
> 
> *Scope of the changes*
> 
> 
> The changes enable ResourceManager to record the historic information of
> the application, the application attempt and the container in terms of
> events via a history writer. In addition, the changes setup up an
> application history server, which allows users to access the recorded
> information via RPC interface, web UI and REST APIs.
> 
> *Details of development*
> 
> 
> - Development of the feature is tracked in the jira -
> https://issues.apache.org/jira/browse/YARN-321.
> - Development has been done in a separate branch -
> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
> - The feature development involved about 35 subtasks.
> - The up-to-date design is posted at -
> https://issues.apache.org/jira/secure/attachment/12619638/Generic
> Application
> History - Design-20131219.pdf<https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf>
> 
> - The uber merge patch Jira -
> https://issues.apache.org/jira/browse/YARN-1587
> 
> *Testing*
> 
> 
> A number of unit tests have been added as a part of the feature. In
> addition, we’ve also done end-to-end functional tests, and performance
> tests for HDFS-based history storage and history events processing. Last
> but not least, we have updated branch YARN-321 against the latest trunk,
> edited merge conflicts, fixed test failures caused by merge, and corrected
> a bunch of bad source code issues. The uber merge patch that contains all
> the diff between branch YARN-321 and trunk has been run through Jenkins.
> 
> *Pending work*
> 
> 
> - Make it work in secure mode
> - Pending bug fixes
> 
> We wish to merge the branch now instead of waiting for later. The main
> reason for this is that as the branch grew in size, the cost of its
> maintenance became huge. Once the feature is merged into trunk, we will
> continue to work on pending work like security stuff, to test and fix any
> bugs that may be found on the trunk, and to refactor the code about to
> share some pieces in PRC and web interfaces.
> 
> *Release status*
> 
> 
> If the security stuff and the pending fixes arrive by the time everything
> else planned for Release 2.4 is done, we can include it as well. This is
> what we are striving for. Otherwise, we will call AHS not-feature-complete
> and not stable.
> 
> The bulk of the design and implementation was done by Mayank Bansal and me
> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for providing
> feedback on the design discussions.
> 
> 
> 
> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
> 
> Thanks,
> Zhijie
> 
> -- 
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.

Yup, I was building and running tests in the branch.

ALL CLEAR now. YARN-321 merged. I could build branch-2. Ran all the YARN tests successfully.

Let me know if any more issues blow up and/or file JIRA tickets.

+Vinod

On Jan 28, 2014, at 1:08 PM, Sandy Ryza <sa...@cloudera.com> wrote:

> It looks like Vinod just fixed this.  Thanks!
> 
> 
> On Tue, Jan 28, 2014 at 1:00 PM, Sandy Ryza <sa...@cloudera.com> wrote:
> 
>> I'm getting build errors on branch-2 because the pom for the new module
>> still refers to 3.0.0-SNAPSHOT:
>> 
>>  <parent>
>>    <artifactId>hadoop-yarn-server</artifactId>
>>    <groupId>org.apache.hadoop</groupId>
>>    <version>3.0.0-SNAPSHOT</version>
>>  </parent>
>>  <modelVersion>4.0.0</modelVersion>
>>  <groupId>org.apache.hadoop</groupId>
>>  <artifactId>hadoop-yarn-server-applicationhistoryservice</artifactId>
>>  <version>3.0.0-SNAPSHOT</version>
>>  <name>hadoop-yarn-server-applicationhistoryservice</name>
>> 
>> -Sandy
>> 
>> 
>> On Tue, Jan 28, 2014 at 10:37 AM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>> 
>>> 
>>> Trunk build seems stable now after the merge. A couple of yarn trunk
>>> builds also ran fine (
>>> https://builds.apache.org/job/Hadoop-Yarn-trunk/462/ and
>>> https://builds.apache.org/job/Hadoop-Yarn-trunk/463/). Branch 2.3 is
>>> also cut and is out of the way.
>>> 
>>> I'm now merging this into branch-2 so as to minimize YARN dev effort
>>> across branches as per the discussion thread on YARN-321 merge.
>>> 
>>> Thanks,
>>> +Vinod Kumar Vavilapalli
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> On Jan 25, 2014, at 8:55 PM, Vinod Kumar Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>> 
>>> Done. Ran trunk compilation and the new module's (small number of) YARN
>>> tests. Seems fine. Please let Zhijie/Mayank or me know if something major
>>> is broken. And/or file tickets.
>>> 
>>> Thanks,
>>> +Vinod
>>> 
>>> On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>> 
>>> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm
>>> merging YARN-321 into trunk now.
>>> 
>>> Thanks everyone for voting!
>>> 
>>> +Vinod
>>> 
>>> On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:
>>> 
>>> +1
>>> 
>>> 
>>> 
>>> Thanks
>>> Devaraj K
>>> 
>>> 
>>> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
>>> 
>>> +1
>>> 
>>> Thanks,
>>> Mayank
>>> 
>>> 
>>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
>>> wrote:
>>> 
>>> +1 (binding)
>>> 
>>> Arun
>>> 
>>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>>> 
>>> Hi folks,
>>> 
>>> As previously discussed here (
>>> 
>>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>> 
>>> I would like to call a vote to merge the YARN-321 branch for Generic
>>> Application History Server into trunk.
>>> 
>>> *Scope of the changes*
>>> 
>>> 
>>> The changes enable ResourceManager to record the historic information
>>> 
>>> of
>>> 
>>> the application, the application attempt and the container in terms of
>>> events via a history writer. In addition, the changes setup up an
>>> application history server, which allows users to access the recorded
>>> information via RPC interface, web UI and REST APIs.
>>> 
>>> *Details of development*
>>> 
>>> 
>>> - Development of the feature is tracked in the jira -
>>> https://issues.apache.org/jira/browse/YARN-321.
>>> - Development has been done in a separate branch -
>>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>>> - The feature development involved about 35 subtasks.
>>> - The up-to-date design is posted at -
>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>>> Application
>>> History - Design-20131219.pdf<
>>> 
>>> 
>>> 
>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>> 
>>> 
>>> 
>>> - The uber merge patch Jira -
>>> https://issues.apache.org/jira/browse/YARN-1587
>>> 
>>> *Testing*
>>> 
>>> 
>>> A number of unit tests have been added as a part of the feature. In
>>> addition, we've also done end-to-end functional tests, and performance
>>> tests for HDFS-based history storage and history events processing.
>>> 
>>> Last
>>> 
>>> but not least, we have updated branch YARN-321 against the latest
>>> 
>>> trunk,
>>> 
>>> edited merge conflicts, fixed test failures caused by merge, and
>>> 
>>> corrected
>>> 
>>> a bunch of bad source code issues. The uber merge patch that contains
>>> 
>>> all
>>> 
>>> the diff between branch YARN-321 and trunk has been run through
>>> 
>>> Jenkins.
>>> 
>>> 
>>> *Pending work*
>>> 
>>> 
>>> - Make it work in secure mode
>>> - Pending bug fixes
>>> 
>>> We wish to merge the branch now instead of waiting for later. The main
>>> reason for this is that as the branch grew in size, the cost of its
>>> maintenance became huge. Once the feature is merged into trunk, we will
>>> continue to work on pending work like security stuff, to test and fix
>>> 
>>> any
>>> 
>>> bugs that may be found on the trunk, and to refactor the code about to
>>> share some pieces in PRC and web interfaces.
>>> 
>>> *Release status*
>>> 
>>> 
>>> If the security stuff and the pending fixes arrive by the time
>>> 
>>> everything
>>> 
>>> else planned for Release 2.4 is done, we can include it as well. This
>>> 
>>> is
>>> 
>>> what we are striving for. Otherwise, we will call AHS
>>> 
>>> not-feature-complete
>>> 
>>> and not stable.
>>> 
>>> The bulk of the design and implementation was done by Mayank Bansal and
>>> 
>>> me
>>> 
>>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>>> 
>>> providing
>>> 
>>> feedback on the design discussions.
>>> 
>>> 
>>> 
>>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>> 
>>> Thanks,
>>> Zhijie
>>> 
>>> --
>>> Zhijie Shen
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or
>>> 
>>> entity
>>> 
>>> to
>>> 
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the
>>> 
>>> reader
>>> 
>>> of this message is not the intended recipient, you are hereby notified
>>> 
>>> that
>>> 
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>>> 
>>> immediately
>>> 
>>> and delete it from your system. Thank You.
>>> 
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> 
>>> to
>>> 
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>>> 
>>> that
>>> 
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>>> 
>>> immediately
>>> 
>>> and delete it from your system. Thank You.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>> 
>> 
>> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.

Yup, I was building and running tests in the branch.

ALL CLEAR now. YARN-321 merged. I could build branch-2. Ran all the YARN tests successfully.

Let me know if any more issues blow up and/or file JIRA tickets.

+Vinod

On Jan 28, 2014, at 1:08 PM, Sandy Ryza <sa...@cloudera.com> wrote:

> It looks like Vinod just fixed this.  Thanks!
> 
> 
> On Tue, Jan 28, 2014 at 1:00 PM, Sandy Ryza <sa...@cloudera.com> wrote:
> 
>> I'm getting build errors on branch-2 because the pom for the new module
>> still refers to 3.0.0-SNAPSHOT:
>> 
>>  <parent>
>>    <artifactId>hadoop-yarn-server</artifactId>
>>    <groupId>org.apache.hadoop</groupId>
>>    <version>3.0.0-SNAPSHOT</version>
>>  </parent>
>>  <modelVersion>4.0.0</modelVersion>
>>  <groupId>org.apache.hadoop</groupId>
>>  <artifactId>hadoop-yarn-server-applicationhistoryservice</artifactId>
>>  <version>3.0.0-SNAPSHOT</version>
>>  <name>hadoop-yarn-server-applicationhistoryservice</name>
>> 
>> -Sandy
>> 
>> 
>> On Tue, Jan 28, 2014 at 10:37 AM, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org> wrote:
>> 
>>> 
>>> Trunk build seems stable now after the merge. A couple of yarn trunk
>>> builds also ran fine (
>>> https://builds.apache.org/job/Hadoop-Yarn-trunk/462/ and
>>> https://builds.apache.org/job/Hadoop-Yarn-trunk/463/). Branch 2.3 is
>>> also cut and is out of the way.
>>> 
>>> I'm now merging this into branch-2 so as to minimize YARN dev effort
>>> across branches as per the discussion thread on YARN-321 merge.
>>> 
>>> Thanks,
>>> +Vinod Kumar Vavilapalli
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> On Jan 25, 2014, at 8:55 PM, Vinod Kumar Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>> 
>>> Done. Ran trunk compilation and the new module's (small number of) YARN
>>> tests. Seems fine. Please let Zhijie/Mayank or me know if something major
>>> is broken. And/or file tickets.
>>> 
>>> Thanks,
>>> +Vinod
>>> 
>>> On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>> 
>>> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm
>>> merging YARN-321 into trunk now.
>>> 
>>> Thanks everyone for voting!
>>> 
>>> +Vinod
>>> 
>>> On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:
>>> 
>>> +1
>>> 
>>> 
>>> 
>>> Thanks
>>> Devaraj K
>>> 
>>> 
>>> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
>>> 
>>> +1
>>> 
>>> Thanks,
>>> Mayank
>>> 
>>> 
>>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
>>> wrote:
>>> 
>>> +1 (binding)
>>> 
>>> Arun
>>> 
>>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>>> 
>>> Hi folks,
>>> 
>>> As previously discussed here (
>>> 
>>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>> 
>>> I would like to call a vote to merge the YARN-321 branch for Generic
>>> Application History Server into trunk.
>>> 
>>> *Scope of the changes*
>>> 
>>> 
>>> The changes enable ResourceManager to record the historic information
>>> 
>>> of
>>> 
>>> the application, the application attempt and the container in terms of
>>> events via a history writer. In addition, the changes setup up an
>>> application history server, which allows users to access the recorded
>>> information via RPC interface, web UI and REST APIs.
>>> 
>>> *Details of development*
>>> 
>>> 
>>> - Development of the feature is tracked in the jira -
>>> https://issues.apache.org/jira/browse/YARN-321.
>>> - Development has been done in a separate branch -
>>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>>> - The feature development involved about 35 subtasks.
>>> - The up-to-date design is posted at -
>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>>> Application
>>> History - Design-20131219.pdf<
>>> 
>>> 
>>> 
>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>> 
>>> 
>>> 
>>> - The uber merge patch Jira -
>>> https://issues.apache.org/jira/browse/YARN-1587
>>> 
>>> *Testing*
>>> 
>>> 
>>> A number of unit tests have been added as a part of the feature. In
>>> addition, we've also done end-to-end functional tests, and performance
>>> tests for HDFS-based history storage and history events processing.
>>> 
>>> Last
>>> 
>>> but not least, we have updated branch YARN-321 against the latest
>>> 
>>> trunk,
>>> 
>>> edited merge conflicts, fixed test failures caused by merge, and
>>> 
>>> corrected
>>> 
>>> a bunch of bad source code issues. The uber merge patch that contains
>>> 
>>> all
>>> 
>>> the diff between branch YARN-321 and trunk has been run through
>>> 
>>> Jenkins.
>>> 
>>> 
>>> *Pending work*
>>> 
>>> 
>>> - Make it work in secure mode
>>> - Pending bug fixes
>>> 
>>> We wish to merge the branch now instead of waiting for later. The main
>>> reason for this is that as the branch grew in size, the cost of its
>>> maintenance became huge. Once the feature is merged into trunk, we will
>>> continue to work on pending work like security stuff, to test and fix
>>> 
>>> any
>>> 
>>> bugs that may be found on the trunk, and to refactor the code about to
>>> share some pieces in PRC and web interfaces.
>>> 
>>> *Release status*
>>> 
>>> 
>>> If the security stuff and the pending fixes arrive by the time
>>> 
>>> everything
>>> 
>>> else planned for Release 2.4 is done, we can include it as well. This
>>> 
>>> is
>>> 
>>> what we are striving for. Otherwise, we will call AHS
>>> 
>>> not-feature-complete
>>> 
>>> and not stable.
>>> 
>>> The bulk of the design and implementation was done by Mayank Bansal and
>>> 
>>> me
>>> 
>>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>>> 
>>> providing
>>> 
>>> feedback on the design discussions.
>>> 
>>> 
>>> 
>>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>> 
>>> Thanks,
>>> Zhijie
>>> 
>>> --
>>> Zhijie Shen
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or
>>> 
>>> entity
>>> 
>>> to
>>> 
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the
>>> 
>>> reader
>>> 
>>> of this message is not the intended recipient, you are hereby notified
>>> 
>>> that
>>> 
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>>> 
>>> immediately
>>> 
>>> and delete it from your system. Thank You.
>>> 
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> 
>>> to
>>> 
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>>> 
>>> that
>>> 
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>>> 
>>> immediately
>>> 
>>> and delete it from your system. Thank You.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>> 
>> 
>> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Sandy Ryza <sa...@cloudera.com>.

It looks like Vinod just fixed this.  Thanks!


On Tue, Jan 28, 2014 at 1:00 PM, Sandy Ryza <sa...@cloudera.com> wrote:

> I'm getting build errors on branch-2 because the pom for the new module
> still refers to 3.0.0-SNAPSHOT:
>
>   <parent>
>     <artifactId>hadoop-yarn-server</artifactId>
>     <groupId>org.apache.hadoop</groupId>
>     <version>3.0.0-SNAPSHOT</version>
>   </parent>
>   <modelVersion>4.0.0</modelVersion>
>   <groupId>org.apache.hadoop</groupId>
>   <artifactId>hadoop-yarn-server-applicationhistoryservice</artifactId>
>   <version>3.0.0-SNAPSHOT</version>
>   <name>hadoop-yarn-server-applicationhistoryservice</name>
>
> -Sandy
>
>
> On Tue, Jan 28, 2014 at 10:37 AM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>>
>> Trunk build seems stable now after the merge. A couple of yarn trunk
>> builds also ran fine (
>> https://builds.apache.org/job/Hadoop-Yarn-trunk/462/ and
>> https://builds.apache.org/job/Hadoop-Yarn-trunk/463/). Branch 2.3 is
>> also cut and is out of the way.
>>
>> I'm now merging this into branch-2 so as to minimize YARN dev effort
>> across branches as per the discussion thread on YARN-321 merge.
>>
>>  Thanks,
>> +Vinod Kumar Vavilapalli
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>> On Jan 25, 2014, at 8:55 PM, Vinod Kumar Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>> Done. Ran trunk compilation and the new module's (small number of) YARN
>> tests. Seems fine. Please let Zhijie/Mayank or me know if something major
>> is broken. And/or file tickets.
>>
>> Thanks,
>> +Vinod
>>
>> On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm
>> merging YARN-321 into trunk now.
>>
>> Thanks everyone for voting!
>>
>> +Vinod
>>
>> On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:
>>
>> +1
>>
>>
>>
>> Thanks
>> Devaraj K
>>
>>
>> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
>>
>> +1
>>
>> Thanks,
>> Mayank
>>
>>
>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>>
>> +1 (binding)
>>
>> Arun
>>
>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>>
>> Hi folks,
>>
>> As previously discussed here (
>>
>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>
>> I would like to call a vote to merge the YARN-321 branch for Generic
>> Application History Server into trunk.
>>
>> *Scope of the changes*
>>
>>
>> The changes enable ResourceManager to record the historic information
>>
>> of
>>
>> the application, the application attempt and the container in terms of
>> events via a history writer. In addition, the changes setup up an
>> application history server, which allows users to access the recorded
>> information via RPC interface, web UI and REST APIs.
>>
>> *Details of development*
>>
>>
>> - Development of the feature is tracked in the jira -
>> https://issues.apache.org/jira/browse/YARN-321.
>> - Development has been done in a separate branch -
>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>> - The feature development involved about 35 subtasks.
>> - The up-to-date design is posted at -
>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>> Application
>> History - Design-20131219.pdf<
>>
>>
>>
>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>
>>
>>
>> - The uber merge patch Jira -
>> https://issues.apache.org/jira/browse/YARN-1587
>>
>> *Testing*
>>
>>
>> A number of unit tests have been added as a part of the feature. In
>> addition, we've also done end-to-end functional tests, and performance
>> tests for HDFS-based history storage and history events processing.
>>
>> Last
>>
>> but not least, we have updated branch YARN-321 against the latest
>>
>> trunk,
>>
>> edited merge conflicts, fixed test failures caused by merge, and
>>
>> corrected
>>
>> a bunch of bad source code issues. The uber merge patch that contains
>>
>> all
>>
>> the diff between branch YARN-321 and trunk has been run through
>>
>> Jenkins.
>>
>>
>> *Pending work*
>>
>>
>> - Make it work in secure mode
>> - Pending bug fixes
>>
>> We wish to merge the branch now instead of waiting for later. The main
>> reason for this is that as the branch grew in size, the cost of its
>> maintenance became huge. Once the feature is merged into trunk, we will
>> continue to work on pending work like security stuff, to test and fix
>>
>> any
>>
>> bugs that may be found on the trunk, and to refactor the code about to
>> share some pieces in PRC and web interfaces.
>>
>> *Release status*
>>
>>
>> If the security stuff and the pending fixes arrive by the time
>>
>> everything
>>
>> else planned for Release 2.4 is done, we can include it as well. This
>>
>> is
>>
>> what we are striving for. Otherwise, we will call AHS
>>
>> not-feature-complete
>>
>> and not stable.
>>
>> The bulk of the design and implementation was done by Mayank Bansal and
>>
>> me
>>
>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>>
>> providing
>>
>> feedback on the design discussions.
>>
>>
>>
>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>
>> Thanks,
>> Zhijie
>>
>> --
>> Zhijie Shen
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or
>>
>> entity
>>
>> to
>>
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the
>>
>> reader
>>
>> of this message is not the intended recipient, you are hereby notified
>>
>> that
>>
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>>
>> immediately
>>
>> and delete it from your system. Thank You.
>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>>
>> to
>>
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>>
>> that
>>
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>>
>> immediately
>>
>> and delete it from your system. Thank You.
>>
>>
>>
>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
>

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Sandy Ryza <sa...@cloudera.com>.

It looks like Vinod just fixed this.  Thanks!


On Tue, Jan 28, 2014 at 1:00 PM, Sandy Ryza <sa...@cloudera.com> wrote:

> I'm getting build errors on branch-2 because the pom for the new module
> still refers to 3.0.0-SNAPSHOT:
>
>   <parent>
>     <artifactId>hadoop-yarn-server</artifactId>
>     <groupId>org.apache.hadoop</groupId>
>     <version>3.0.0-SNAPSHOT</version>
>   </parent>
>   <modelVersion>4.0.0</modelVersion>
>   <groupId>org.apache.hadoop</groupId>
>   <artifactId>hadoop-yarn-server-applicationhistoryservice</artifactId>
>   <version>3.0.0-SNAPSHOT</version>
>   <name>hadoop-yarn-server-applicationhistoryservice</name>
>
> -Sandy
>
>
> On Tue, Jan 28, 2014 at 10:37 AM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>>
>> Trunk build seems stable now after the merge. A couple of yarn trunk
>> builds also ran fine (
>> https://builds.apache.org/job/Hadoop-Yarn-trunk/462/ and
>> https://builds.apache.org/job/Hadoop-Yarn-trunk/463/). Branch 2.3 is
>> also cut and is out of the way.
>>
>> I'm now merging this into branch-2 so as to minimize YARN dev effort
>> across branches as per the discussion thread on YARN-321 merge.
>>
>>  Thanks,
>> +Vinod Kumar Vavilapalli
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>> On Jan 25, 2014, at 8:55 PM, Vinod Kumar Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>> Done. Ran trunk compilation and the new module's (small number of) YARN
>> tests. Seems fine. Please let Zhijie/Mayank or me know if something major
>> is broken. And/or file tickets.
>>
>> Thanks,
>> +Vinod
>>
>> On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm
>> merging YARN-321 into trunk now.
>>
>> Thanks everyone for voting!
>>
>> +Vinod
>>
>> On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:
>>
>> +1
>>
>>
>>
>> Thanks
>> Devaraj K
>>
>>
>> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
>>
>> +1
>>
>> Thanks,
>> Mayank
>>
>>
>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>>
>> +1 (binding)
>>
>> Arun
>>
>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>>
>> Hi folks,
>>
>> As previously discussed here (
>>
>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>
>> I would like to call a vote to merge the YARN-321 branch for Generic
>> Application History Server into trunk.
>>
>> *Scope of the changes*
>>
>>
>> The changes enable ResourceManager to record the historic information
>>
>> of
>>
>> the application, the application attempt and the container in terms of
>> events via a history writer. In addition, the changes setup up an
>> application history server, which allows users to access the recorded
>> information via RPC interface, web UI and REST APIs.
>>
>> *Details of development*
>>
>>
>> - Development of the feature is tracked in the jira -
>> https://issues.apache.org/jira/browse/YARN-321.
>> - Development has been done in a separate branch -
>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>> - The feature development involved about 35 subtasks.
>> - The up-to-date design is posted at -
>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>> Application
>> History - Design-20131219.pdf<
>>
>>
>>
>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>
>>
>>
>> - The uber merge patch Jira -
>> https://issues.apache.org/jira/browse/YARN-1587
>>
>> *Testing*
>>
>>
>> A number of unit tests have been added as a part of the feature. In
>> addition, we've also done end-to-end functional tests, and performance
>> tests for HDFS-based history storage and history events processing.
>>
>> Last
>>
>> but not least, we have updated branch YARN-321 against the latest
>>
>> trunk,
>>
>> edited merge conflicts, fixed test failures caused by merge, and
>>
>> corrected
>>
>> a bunch of bad source code issues. The uber merge patch that contains
>>
>> all
>>
>> the diff between branch YARN-321 and trunk has been run through
>>
>> Jenkins.
>>
>>
>> *Pending work*
>>
>>
>> - Make it work in secure mode
>> - Pending bug fixes
>>
>> We wish to merge the branch now instead of waiting for later. The main
>> reason for this is that as the branch grew in size, the cost of its
>> maintenance became huge. Once the feature is merged into trunk, we will
>> continue to work on pending work like security stuff, to test and fix
>>
>> any
>>
>> bugs that may be found on the trunk, and to refactor the code about to
>> share some pieces in PRC and web interfaces.
>>
>> *Release status*
>>
>>
>> If the security stuff and the pending fixes arrive by the time
>>
>> everything
>>
>> else planned for Release 2.4 is done, we can include it as well. This
>>
>> is
>>
>> what we are striving for. Otherwise, we will call AHS
>>
>> not-feature-complete
>>
>> and not stable.
>>
>> The bulk of the design and implementation was done by Mayank Bansal and
>>
>> me
>>
>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>>
>> providing
>>
>> feedback on the design discussions.
>>
>>
>>
>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>
>> Thanks,
>> Zhijie
>>
>> --
>> Zhijie Shen
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or
>>
>> entity
>>
>> to
>>
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the
>>
>> reader
>>
>> of this message is not the intended recipient, you are hereby notified
>>
>> that
>>
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>>
>> immediately
>>
>> and delete it from your system. Thank You.
>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>>
>> to
>>
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>>
>> that
>>
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>>
>> immediately
>>
>> and delete it from your system. Thank You.
>>
>>
>>
>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
>

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Sandy Ryza <sa...@cloudera.com>.

I'm getting build errors on branch-2 because the pom for the new module
still refers to 3.0.0-SNAPSHOT:

  <parent>
    <artifactId>hadoop-yarn-server</artifactId>
    <groupId>org.apache.hadoop</groupId>
    <version>3.0.0-SNAPSHOT</version>
  </parent>
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.apache.hadoop</groupId>
  <artifactId>hadoop-yarn-server-applicationhistoryservice</artifactId>
  <version>3.0.0-SNAPSHOT</version>
  <name>hadoop-yarn-server-applicationhistoryservice</name>

-Sandy


On Tue, Jan 28, 2014 at 10:37 AM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:

>
> Trunk build seems stable now after the merge. A couple of yarn trunk
> builds also ran fine (https://builds.apache.org/job/Hadoop-Yarn-trunk/462/and
> https://builds.apache.org/job/Hadoop-Yarn-trunk/463/). Branch 2.3 is also
> cut and is out of the way.
>
> I'm now merging this into branch-2 so as to minimize YARN dev effort
> across branches as per the discussion thread on YARN-321 merge.
>
> Thanks,
> +Vinod Kumar Vavilapalli
> Hortonworks Inc.
> http://hortonworks.com/
>
>
> On Jan 25, 2014, at 8:55 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
> Done. Ran trunk compilation and the new module's (small number of) YARN
> tests. Seems fine. Please let Zhijie/Mayank or me know if something major
> is broken. And/or file tickets.
>
> Thanks,
> +Vinod
>
> On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm
> merging YARN-321 into trunk now.
>
> Thanks everyone for voting!
>
> +Vinod
>
> On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:
>
> +1
>
>
>
> Thanks
> Devaraj K
>
>
> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
>
> +1
>
> Thanks,
> Mayank
>
>
> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
> wrote:
>
> +1 (binding)
>
> Arun
>
> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>
> Hi folks,
>
> As previously discussed here (
>
> http://markmail.org/message/iscvp7cedrtvmd6p),
>
> I would like to call a vote to merge the YARN-321 branch for Generic
> Application History Server into trunk.
>
> *Scope of the changes*
>
>
> The changes enable ResourceManager to record the historic information
>
> of
>
> the application, the application attempt and the container in terms of
> events via a history writer. In addition, the changes setup up an
> application history server, which allows users to access the recorded
> information via RPC interface, web UI and REST APIs.
>
> *Details of development*
>
>
> - Development of the feature is tracked in the jira -
> https://issues.apache.org/jira/browse/YARN-321.
> - Development has been done in a separate branch -
> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
> - The feature development involved about 35 subtasks.
> - The up-to-date design is posted at -
> https://issues.apache.org/jira/secure/attachment/12619638/Generic
> Application
> History - Design-20131219.pdf<
>
>
>
> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>
>
>
> - The uber merge patch Jira -
> https://issues.apache.org/jira/browse/YARN-1587
>
> *Testing*
>
>
> A number of unit tests have been added as a part of the feature. In
> addition, we've also done end-to-end functional tests, and performance
> tests for HDFS-based history storage and history events processing.
>
> Last
>
> but not least, we have updated branch YARN-321 against the latest
>
> trunk,
>
> edited merge conflicts, fixed test failures caused by merge, and
>
> corrected
>
> a bunch of bad source code issues. The uber merge patch that contains
>
> all
>
> the diff between branch YARN-321 and trunk has been run through
>
> Jenkins.
>
>
> *Pending work*
>
>
> - Make it work in secure mode
> - Pending bug fixes
>
> We wish to merge the branch now instead of waiting for later. The main
> reason for this is that as the branch grew in size, the cost of its
> maintenance became huge. Once the feature is merged into trunk, we will
> continue to work on pending work like security stuff, to test and fix
>
> any
>
> bugs that may be found on the trunk, and to refactor the code about to
> share some pieces in PRC and web interfaces.
>
> *Release status*
>
>
> If the security stuff and the pending fixes arrive by the time
>
> everything
>
> else planned for Release 2.4 is done, we can include it as well. This
>
> is
>
> what we are striving for. Otherwise, we will call AHS
>
> not-feature-complete
>
> and not stable.
>
> The bulk of the design and implementation was done by Mayank Bansal and
>
> me
>
> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>
> providing
>
> feedback on the design discussions.
>
>
>
> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>
> Thanks,
> Zhijie
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or
>
> entity
>
> to
>
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the
>
> reader
>
> of this message is not the intended recipient, you are hereby notified
>
> that
>
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender
>
> immediately
>
> and delete it from your system. Thank You.
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
>
> to
>
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified
>
> that
>
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender
>
> immediately
>
> and delete it from your system. Thank You.
>
>
>
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Sandy Ryza <sa...@cloudera.com>.

I'm getting build errors on branch-2 because the pom for the new module
still refers to 3.0.0-SNAPSHOT:

  <parent>
    <artifactId>hadoop-yarn-server</artifactId>
    <groupId>org.apache.hadoop</groupId>
    <version>3.0.0-SNAPSHOT</version>
  </parent>
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.apache.hadoop</groupId>
  <artifactId>hadoop-yarn-server-applicationhistoryservice</artifactId>
  <version>3.0.0-SNAPSHOT</version>
  <name>hadoop-yarn-server-applicationhistoryservice</name>

-Sandy


On Tue, Jan 28, 2014 at 10:37 AM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:

>
> Trunk build seems stable now after the merge. A couple of yarn trunk
> builds also ran fine (https://builds.apache.org/job/Hadoop-Yarn-trunk/462/and
> https://builds.apache.org/job/Hadoop-Yarn-trunk/463/). Branch 2.3 is also
> cut and is out of the way.
>
> I'm now merging this into branch-2 so as to minimize YARN dev effort
> across branches as per the discussion thread on YARN-321 merge.
>
> Thanks,
> +Vinod Kumar Vavilapalli
> Hortonworks Inc.
> http://hortonworks.com/
>
>
> On Jan 25, 2014, at 8:55 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
> Done. Ran trunk compilation and the new module's (small number of) YARN
> tests. Seems fine. Please let Zhijie/Mayank or me know if something major
> is broken. And/or file tickets.
>
> Thanks,
> +Vinod
>
> On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm
> merging YARN-321 into trunk now.
>
> Thanks everyone for voting!
>
> +Vinod
>
> On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:
>
> +1
>
>
>
> Thanks
> Devaraj K
>
>
> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
>
> +1
>
> Thanks,
> Mayank
>
>
> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
> wrote:
>
> +1 (binding)
>
> Arun
>
> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>
> Hi folks,
>
> As previously discussed here (
>
> http://markmail.org/message/iscvp7cedrtvmd6p),
>
> I would like to call a vote to merge the YARN-321 branch for Generic
> Application History Server into trunk.
>
> *Scope of the changes*
>
>
> The changes enable ResourceManager to record the historic information
>
> of
>
> the application, the application attempt and the container in terms of
> events via a history writer. In addition, the changes setup up an
> application history server, which allows users to access the recorded
> information via RPC interface, web UI and REST APIs.
>
> *Details of development*
>
>
> - Development of the feature is tracked in the jira -
> https://issues.apache.org/jira/browse/YARN-321.
> - Development has been done in a separate branch -
> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
> - The feature development involved about 35 subtasks.
> - The up-to-date design is posted at -
> https://issues.apache.org/jira/secure/attachment/12619638/Generic
> Application
> History - Design-20131219.pdf<
>
>
>
> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>
>
>
> - The uber merge patch Jira -
> https://issues.apache.org/jira/browse/YARN-1587
>
> *Testing*
>
>
> A number of unit tests have been added as a part of the feature. In
> addition, we've also done end-to-end functional tests, and performance
> tests for HDFS-based history storage and history events processing.
>
> Last
>
> but not least, we have updated branch YARN-321 against the latest
>
> trunk,
>
> edited merge conflicts, fixed test failures caused by merge, and
>
> corrected
>
> a bunch of bad source code issues. The uber merge patch that contains
>
> all
>
> the diff between branch YARN-321 and trunk has been run through
>
> Jenkins.
>
>
> *Pending work*
>
>
> - Make it work in secure mode
> - Pending bug fixes
>
> We wish to merge the branch now instead of waiting for later. The main
> reason for this is that as the branch grew in size, the cost of its
> maintenance became huge. Once the feature is merged into trunk, we will
> continue to work on pending work like security stuff, to test and fix
>
> any
>
> bugs that may be found on the trunk, and to refactor the code about to
> share some pieces in PRC and web interfaces.
>
> *Release status*
>
>
> If the security stuff and the pending fixes arrive by the time
>
> everything
>
> else planned for Release 2.4 is done, we can include it as well. This
>
> is
>
> what we are striving for. Otherwise, we will call AHS
>
> not-feature-complete
>
> and not stable.
>
> The bulk of the design and implementation was done by Mayank Bansal and
>
> me
>
> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>
> providing
>
> feedback on the design discussions.
>
>
>
> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>
> Thanks,
> Zhijie
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or
>
> entity
>
> to
>
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the
>
> reader
>
> of this message is not the intended recipient, you are hereby notified
>
> that
>
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender
>
> immediately
>
> and delete it from your system. Thank You.
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
>
> to
>
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified
>
> that
>
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender
>
> immediately
>
> and delete it from your system. Thank You.
>
>
>
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Trunk build seems stable now after the merge. A couple of yarn trunk builds also ran fine (https://builds.apache.org/job/Hadoop-Yarn-trunk/462/ and https://builds.apache.org/job/Hadoop-Yarn-trunk/463/). Branch 2.3 is also cut and is out of the way.

I'm now merging this into branch-2 so as to minimize YARN dev effort across branches as per the discussion thread on YARN-321 merge.

Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/


On Jan 25, 2014, at 8:55 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:

> Done. Ran trunk compilation and the new module's (small number of) YARN tests. Seems fine. Please let Zhijie/Mayank or me know if something major is broken. And/or file tickets.
> 
> Thanks,
> +Vinod
> 
> On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
> 
>> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm merging YARN-321 into trunk now.
>> 
>> Thanks everyone for voting!
>> 
>> +Vinod
>> 
>> On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:
>> 
>>> +1
>>> 
>>> 
>>> 
>>> Thanks
>>> Devaraj K
>>> 
>>> 
>>> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
>>> 
>>>> +1
>>>> 
>>>> Thanks,
>>>> Mayank
>>>> 
>>>> 
>>>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
>>>> wrote:
>>>> 
>>>>> +1 (binding)
>>>>> 
>>>>> Arun
>>>>> 
>>>>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>>>>> 
>>>>>> Hi folks,
>>>>>> 
>>>>>> As previously discussed here (
>>>>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>>>>> I would like to call a vote to merge the YARN-321 branch for Generic
>>>>>> Application History Server into trunk.
>>>>>> 
>>>>>> *Scope of the changes*
>>>>>> 
>>>>>> 
>>>>>> The changes enable ResourceManager to record the historic information
>>>> of
>>>>>> the application, the application attempt and the container in terms of
>>>>>> events via a history writer. In addition, the changes setup up an
>>>>>> application history server, which allows users to access the recorded
>>>>>> information via RPC interface, web UI and REST APIs.
>>>>>> 
>>>>>> *Details of development*
>>>>>> 
>>>>>> 
>>>>>> - Development of the feature is tracked in the jira -
>>>>>> https://issues.apache.org/jira/browse/YARN-321.
>>>>>> - Development has been done in a separate branch -
>>>>>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>>>>>> - The feature development involved about 35 subtasks.
>>>>>> - The up-to-date design is posted at -
>>>>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>>>>>> Application
>>>>>> History - Design-20131219.pdf<
>>>>> 
>>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>>>>> 
>>>>>> 
>>>>>> - The uber merge patch Jira -
>>>>>> https://issues.apache.org/jira/browse/YARN-1587
>>>>>> 
>>>>>> *Testing*
>>>>>> 
>>>>>> 
>>>>>> A number of unit tests have been added as a part of the feature. In
>>>>>> addition, we’ve also done end-to-end functional tests, and performance
>>>>>> tests for HDFS-based history storage and history events processing.
>>>> Last
>>>>>> but not least, we have updated branch YARN-321 against the latest
>>>> trunk,
>>>>>> edited merge conflicts, fixed test failures caused by merge, and
>>>>> corrected
>>>>>> a bunch of bad source code issues. The uber merge patch that contains
>>>> all
>>>>>> the diff between branch YARN-321 and trunk has been run through
>>>> Jenkins.
>>>>>> 
>>>>>> *Pending work*
>>>>>> 
>>>>>> 
>>>>>> - Make it work in secure mode
>>>>>> - Pending bug fixes
>>>>>> 
>>>>>> We wish to merge the branch now instead of waiting for later. The main
>>>>>> reason for this is that as the branch grew in size, the cost of its
>>>>>> maintenance became huge. Once the feature is merged into trunk, we will
>>>>>> continue to work on pending work like security stuff, to test and fix
>>>> any
>>>>>> bugs that may be found on the trunk, and to refactor the code about to
>>>>>> share some pieces in PRC and web interfaces.
>>>>>> 
>>>>>> *Release status*
>>>>>> 
>>>>>> 
>>>>>> If the security stuff and the pending fixes arrive by the time
>>>> everything
>>>>>> else planned for Release 2.4 is done, we can include it as well. This
>>>> is
>>>>>> what we are striving for. Otherwise, we will call AHS
>>>>> not-feature-complete
>>>>>> and not stable.
>>>>>> 
>>>>>> The bulk of the design and implementation was done by Mayank Bansal and
>>>>> me
>>>>>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>>>>>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>>>> providing
>>>>>> feedback on the design discussions.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>>>>> 
>>>>>> Thanks,
>>>>>> Zhijie
>>>>>> 
>>>>>> --
>>>>>> Zhijie Shen
>>>>>> Hortonworks Inc.
>>>>>> http://hortonworks.com/
>>>>>> 
>>>>>> --
>>>>>> CONFIDENTIALITY NOTICE
>>>>>> NOTICE: This message is intended for the use of the individual or
>>>> entity
>>>>> to
>>>>>> which it is addressed and may contain information that is confidential,
>>>>>> privileged and exempt from disclosure under applicable law. If the
>>>> reader
>>>>>> of this message is not the intended recipient, you are hereby notified
>>>>> that
>>>>>> any printing, copying, dissemination, distribution, disclosure or
>>>>>> forwarding of this communication is strictly prohibited. If you have
>>>>>> received this communication in error, please contact the sender
>>>>> immediately
>>>>>> and delete it from your system. Thank You.
>>>>> 
>>>>> --
>>>>> Arun C. Murthy
>>>>> Hortonworks Inc.
>>>>> http://hortonworks.com/
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> CONFIDENTIALITY NOTICE
>>>>> NOTICE: This message is intended for the use of the individual or entity
>>>> to
>>>>> which it is addressed and may contain information that is confidential,
>>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>>> of this message is not the intended recipient, you are hereby notified
>>>> that
>>>>> any printing, copying, dissemination, distribution, disclosure or
>>>>> forwarding of this communication is strictly prohibited. If you have
>>>>> received this communication in error, please contact the sender
>>>> immediately
>>>>> and delete it from your system. Thank You.
>>>>> 
>>>> 
>> 
> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.

Trunk build seems stable now after the merge. A couple of yarn trunk builds also ran fine (https://builds.apache.org/job/Hadoop-Yarn-trunk/462/ and https://builds.apache.org/job/Hadoop-Yarn-trunk/463/). Branch 2.3 is also cut and is out of the way.

I'm now merging this into branch-2 so as to minimize YARN dev effort across branches as per the discussion thread on YARN-321 merge.

Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/


On Jan 25, 2014, at 8:55 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:

> Done. Ran trunk compilation and the new module's (small number of) YARN tests. Seems fine. Please let Zhijie/Mayank or me know if something major is broken. And/or file tickets.
> 
> Thanks,
> +Vinod
> 
> On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
> 
>> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm merging YARN-321 into trunk now.
>> 
>> Thanks everyone for voting!
>> 
>> +Vinod
>> 
>> On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:
>> 
>>> +1
>>> 
>>> 
>>> 
>>> Thanks
>>> Devaraj K
>>> 
>>> 
>>> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
>>> 
>>>> +1
>>>> 
>>>> Thanks,
>>>> Mayank
>>>> 
>>>> 
>>>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
>>>> wrote:
>>>> 
>>>>> +1 (binding)
>>>>> 
>>>>> Arun
>>>>> 
>>>>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>>>>> 
>>>>>> Hi folks,
>>>>>> 
>>>>>> As previously discussed here (
>>>>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>>>>> I would like to call a vote to merge the YARN-321 branch for Generic
>>>>>> Application History Server into trunk.
>>>>>> 
>>>>>> *Scope of the changes*
>>>>>> 
>>>>>> 
>>>>>> The changes enable ResourceManager to record the historic information
>>>> of
>>>>>> the application, the application attempt and the container in terms of
>>>>>> events via a history writer. In addition, the changes setup up an
>>>>>> application history server, which allows users to access the recorded
>>>>>> information via RPC interface, web UI and REST APIs.
>>>>>> 
>>>>>> *Details of development*
>>>>>> 
>>>>>> 
>>>>>> - Development of the feature is tracked in the jira -
>>>>>> https://issues.apache.org/jira/browse/YARN-321.
>>>>>> - Development has been done in a separate branch -
>>>>>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>>>>>> - The feature development involved about 35 subtasks.
>>>>>> - The up-to-date design is posted at -
>>>>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>>>>>> Application
>>>>>> History - Design-20131219.pdf<
>>>>> 
>>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>>>>> 
>>>>>> 
>>>>>> - The uber merge patch Jira -
>>>>>> https://issues.apache.org/jira/browse/YARN-1587
>>>>>> 
>>>>>> *Testing*
>>>>>> 
>>>>>> 
>>>>>> A number of unit tests have been added as a part of the feature. In
>>>>>> addition, we’ve also done end-to-end functional tests, and performance
>>>>>> tests for HDFS-based history storage and history events processing.
>>>> Last
>>>>>> but not least, we have updated branch YARN-321 against the latest
>>>> trunk,
>>>>>> edited merge conflicts, fixed test failures caused by merge, and
>>>>> corrected
>>>>>> a bunch of bad source code issues. The uber merge patch that contains
>>>> all
>>>>>> the diff between branch YARN-321 and trunk has been run through
>>>> Jenkins.
>>>>>> 
>>>>>> *Pending work*
>>>>>> 
>>>>>> 
>>>>>> - Make it work in secure mode
>>>>>> - Pending bug fixes
>>>>>> 
>>>>>> We wish to merge the branch now instead of waiting for later. The main
>>>>>> reason for this is that as the branch grew in size, the cost of its
>>>>>> maintenance became huge. Once the feature is merged into trunk, we will
>>>>>> continue to work on pending work like security stuff, to test and fix
>>>> any
>>>>>> bugs that may be found on the trunk, and to refactor the code about to
>>>>>> share some pieces in PRC and web interfaces.
>>>>>> 
>>>>>> *Release status*
>>>>>> 
>>>>>> 
>>>>>> If the security stuff and the pending fixes arrive by the time
>>>> everything
>>>>>> else planned for Release 2.4 is done, we can include it as well. This
>>>> is
>>>>>> what we are striving for. Otherwise, we will call AHS
>>>>> not-feature-complete
>>>>>> and not stable.
>>>>>> 
>>>>>> The bulk of the design and implementation was done by Mayank Bansal and
>>>>> me
>>>>>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>>>>>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>>>> providing
>>>>>> feedback on the design discussions.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>>>>> 
>>>>>> Thanks,
>>>>>> Zhijie
>>>>>> 
>>>>>> --
>>>>>> Zhijie Shen
>>>>>> Hortonworks Inc.
>>>>>> http://hortonworks.com/
>>>>>> 
>>>>>> --
>>>>>> CONFIDENTIALITY NOTICE
>>>>>> NOTICE: This message is intended for the use of the individual or
>>>> entity
>>>>> to
>>>>>> which it is addressed and may contain information that is confidential,
>>>>>> privileged and exempt from disclosure under applicable law. If the
>>>> reader
>>>>>> of this message is not the intended recipient, you are hereby notified
>>>>> that
>>>>>> any printing, copying, dissemination, distribution, disclosure or
>>>>>> forwarding of this communication is strictly prohibited. If you have
>>>>>> received this communication in error, please contact the sender
>>>>> immediately
>>>>>> and delete it from your system. Thank You.
>>>>> 
>>>>> --
>>>>> Arun C. Murthy
>>>>> Hortonworks Inc.
>>>>> http://hortonworks.com/
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> CONFIDENTIALITY NOTICE
>>>>> NOTICE: This message is intended for the use of the individual or entity
>>>> to
>>>>> which it is addressed and may contain information that is confidential,
>>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>>> of this message is not the intended recipient, you are hereby notified
>>>> that
>>>>> any printing, copying, dissemination, distribution, disclosure or
>>>>> forwarding of this communication is strictly prohibited. If you have
>>>>> received this communication in error, please contact the sender
>>>> immediately
>>>>> and delete it from your system. Thank You.
>>>>> 
>>>> 
>> 
> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.

Done. Ran trunk compilation and the new module's (small number of) YARN tests. Seems fine. Please let Zhijie/Mayank or me know if something major is broken. And/or file tickets.

Thanks,
+Vinod

On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:

> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm merging YARN-321 into trunk now.
> 
> Thanks everyone for voting!
> 
> +Vinod
> 
> On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:
> 
>> +1
>> 
>> 
>> 
>> Thanks
>> Devaraj K
>> 
>> 
>> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
>> 
>>> +1
>>> 
>>> Thanks,
>>> Mayank
>>> 
>>> 
>>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
>>> wrote:
>>> 
>>>> +1 (binding)
>>>> 
>>>> Arun
>>>> 
>>>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>>>> 
>>>>> Hi folks,
>>>>> 
>>>>> As previously discussed here (
>>>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>>>> I would like to call a vote to merge the YARN-321 branch for Generic
>>>>> Application History Server into trunk.
>>>>> 
>>>>> *Scope of the changes*
>>>>> 
>>>>> 
>>>>> The changes enable ResourceManager to record the historic information
>>> of
>>>>> the application, the application attempt and the container in terms of
>>>>> events via a history writer. In addition, the changes setup up an
>>>>> application history server, which allows users to access the recorded
>>>>> information via RPC interface, web UI and REST APIs.
>>>>> 
>>>>> *Details of development*
>>>>> 
>>>>> 
>>>>> - Development of the feature is tracked in the jira -
>>>>> https://issues.apache.org/jira/browse/YARN-321.
>>>>> - Development has been done in a separate branch -
>>>>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>>>>> - The feature development involved about 35 subtasks.
>>>>> - The up-to-date design is posted at -
>>>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>>>>> Application
>>>>> History - Design-20131219.pdf<
>>>> 
>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>>>> 
>>>>> 
>>>>> - The uber merge patch Jira -
>>>>> https://issues.apache.org/jira/browse/YARN-1587
>>>>> 
>>>>> *Testing*
>>>>> 
>>>>> 
>>>>> A number of unit tests have been added as a part of the feature. In
>>>>> addition, we’ve also done end-to-end functional tests, and performance
>>>>> tests for HDFS-based history storage and history events processing.
>>> Last
>>>>> but not least, we have updated branch YARN-321 against the latest
>>> trunk,
>>>>> edited merge conflicts, fixed test failures caused by merge, and
>>>> corrected
>>>>> a bunch of bad source code issues. The uber merge patch that contains
>>> all
>>>>> the diff between branch YARN-321 and trunk has been run through
>>> Jenkins.
>>>>> 
>>>>> *Pending work*
>>>>> 
>>>>> 
>>>>> - Make it work in secure mode
>>>>> - Pending bug fixes
>>>>> 
>>>>> We wish to merge the branch now instead of waiting for later. The main
>>>>> reason for this is that as the branch grew in size, the cost of its
>>>>> maintenance became huge. Once the feature is merged into trunk, we will
>>>>> continue to work on pending work like security stuff, to test and fix
>>> any
>>>>> bugs that may be found on the trunk, and to refactor the code about to
>>>>> share some pieces in PRC and web interfaces.
>>>>> 
>>>>> *Release status*
>>>>> 
>>>>> 
>>>>> If the security stuff and the pending fixes arrive by the time
>>> everything
>>>>> else planned for Release 2.4 is done, we can include it as well. This
>>> is
>>>>> what we are striving for. Otherwise, we will call AHS
>>>> not-feature-complete
>>>>> and not stable.
>>>>> 
>>>>> The bulk of the design and implementation was done by Mayank Bansal and
>>>> me
>>>>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>>>>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>>> providing
>>>>> feedback on the design discussions.
>>>>> 
>>>>> 
>>>>> 
>>>>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>>>> 
>>>>> Thanks,
>>>>> Zhijie
>>>>> 
>>>>> --
>>>>> Zhijie Shen
>>>>> Hortonworks Inc.
>>>>> http://hortonworks.com/
>>>>> 
>>>>> --
>>>>> CONFIDENTIALITY NOTICE
>>>>> NOTICE: This message is intended for the use of the individual or
>>> entity
>>>> to
>>>>> which it is addressed and may contain information that is confidential,
>>>>> privileged and exempt from disclosure under applicable law. If the
>>> reader
>>>>> of this message is not the intended recipient, you are hereby notified
>>>> that
>>>>> any printing, copying, dissemination, distribution, disclosure or
>>>>> forwarding of this communication is strictly prohibited. If you have
>>>>> received this communication in error, please contact the sender
>>>> immediately
>>>>> and delete it from your system. Thank You.
>>>> 
>>>> --
>>>> Arun C. Murthy
>>>> Hortonworks Inc.
>>>> http://hortonworks.com/
>>>> 
>>>> 
>>>> 
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or entity
>>> to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender
>>> immediately
>>>> and delete it from your system. Thank You.
>>>> 
>>> 
> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.

Done. Ran trunk compilation and the new module's (small number of) YARN tests. Seems fine. Please let Zhijie/Mayank or me know if something major is broken. And/or file tickets.

Thanks,
+Vinod

On Jan 25, 2014, at 8:21 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:

> With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm merging YARN-321 into trunk now.
> 
> Thanks everyone for voting!
> 
> +Vinod
> 
> On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:
> 
>> +1
>> 
>> 
>> 
>> Thanks
>> Devaraj K
>> 
>> 
>> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
>> 
>>> +1
>>> 
>>> Thanks,
>>> Mayank
>>> 
>>> 
>>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
>>> wrote:
>>> 
>>>> +1 (binding)
>>>> 
>>>> Arun
>>>> 
>>>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>>>> 
>>>>> Hi folks,
>>>>> 
>>>>> As previously discussed here (
>>>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>>>> I would like to call a vote to merge the YARN-321 branch for Generic
>>>>> Application History Server into trunk.
>>>>> 
>>>>> *Scope of the changes*
>>>>> 
>>>>> 
>>>>> The changes enable ResourceManager to record the historic information
>>> of
>>>>> the application, the application attempt and the container in terms of
>>>>> events via a history writer. In addition, the changes setup up an
>>>>> application history server, which allows users to access the recorded
>>>>> information via RPC interface, web UI and REST APIs.
>>>>> 
>>>>> *Details of development*
>>>>> 
>>>>> 
>>>>> - Development of the feature is tracked in the jira -
>>>>> https://issues.apache.org/jira/browse/YARN-321.
>>>>> - Development has been done in a separate branch -
>>>>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>>>>> - The feature development involved about 35 subtasks.
>>>>> - The up-to-date design is posted at -
>>>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>>>>> Application
>>>>> History - Design-20131219.pdf<
>>>> 
>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>>>> 
>>>>> 
>>>>> - The uber merge patch Jira -
>>>>> https://issues.apache.org/jira/browse/YARN-1587
>>>>> 
>>>>> *Testing*
>>>>> 
>>>>> 
>>>>> A number of unit tests have been added as a part of the feature. In
>>>>> addition, we’ve also done end-to-end functional tests, and performance
>>>>> tests for HDFS-based history storage and history events processing.
>>> Last
>>>>> but not least, we have updated branch YARN-321 against the latest
>>> trunk,
>>>>> edited merge conflicts, fixed test failures caused by merge, and
>>>> corrected
>>>>> a bunch of bad source code issues. The uber merge patch that contains
>>> all
>>>>> the diff between branch YARN-321 and trunk has been run through
>>> Jenkins.
>>>>> 
>>>>> *Pending work*
>>>>> 
>>>>> 
>>>>> - Make it work in secure mode
>>>>> - Pending bug fixes
>>>>> 
>>>>> We wish to merge the branch now instead of waiting for later. The main
>>>>> reason for this is that as the branch grew in size, the cost of its
>>>>> maintenance became huge. Once the feature is merged into trunk, we will
>>>>> continue to work on pending work like security stuff, to test and fix
>>> any
>>>>> bugs that may be found on the trunk, and to refactor the code about to
>>>>> share some pieces in PRC and web interfaces.
>>>>> 
>>>>> *Release status*
>>>>> 
>>>>> 
>>>>> If the security stuff and the pending fixes arrive by the time
>>> everything
>>>>> else planned for Release 2.4 is done, we can include it as well. This
>>> is
>>>>> what we are striving for. Otherwise, we will call AHS
>>>> not-feature-complete
>>>>> and not stable.
>>>>> 
>>>>> The bulk of the design and implementation was done by Mayank Bansal and
>>>> me
>>>>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>>>>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>>> providing
>>>>> feedback on the design discussions.
>>>>> 
>>>>> 
>>>>> 
>>>>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>>>> 
>>>>> Thanks,
>>>>> Zhijie
>>>>> 
>>>>> --
>>>>> Zhijie Shen
>>>>> Hortonworks Inc.
>>>>> http://hortonworks.com/
>>>>> 
>>>>> --
>>>>> CONFIDENTIALITY NOTICE
>>>>> NOTICE: This message is intended for the use of the individual or
>>> entity
>>>> to
>>>>> which it is addressed and may contain information that is confidential,
>>>>> privileged and exempt from disclosure under applicable law. If the
>>> reader
>>>>> of this message is not the intended recipient, you are hereby notified
>>>> that
>>>>> any printing, copying, dissemination, distribution, disclosure or
>>>>> forwarding of this communication is strictly prohibited. If you have
>>>>> received this communication in error, please contact the sender
>>>> immediately
>>>>> and delete it from your system. Thank You.
>>>> 
>>>> --
>>>> Arun C. Murthy
>>>> Hortonworks Inc.
>>>> http://hortonworks.com/
>>>> 
>>>> 
>>>> 
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or entity
>>> to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender
>>> immediately
>>>> and delete it from your system. Thank You.
>>>> 
>>> 
> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.

With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm merging YARN-321 into trunk now.

Thanks everyone for voting!

+Vinod

On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:

> +1
> 
> 
> 
> Thanks
> Devaraj K
> 
> 
> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
> 
>> +1
>> 
>> Thanks,
>> Mayank
>> 
>> 
>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>> 
>>> +1 (binding)
>>> 
>>> Arun
>>> 
>>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>>> 
>>>> Hi folks,
>>>> 
>>>> As previously discussed here (
>>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>>> I would like to call a vote to merge the YARN-321 branch for Generic
>>>> Application History Server into trunk.
>>>> 
>>>> *Scope of the changes*
>>>> 
>>>> 
>>>> The changes enable ResourceManager to record the historic information
>> of
>>>> the application, the application attempt and the container in terms of
>>>> events via a history writer. In addition, the changes setup up an
>>>> application history server, which allows users to access the recorded
>>>> information via RPC interface, web UI and REST APIs.
>>>> 
>>>> *Details of development*
>>>> 
>>>> 
>>>> - Development of the feature is tracked in the jira -
>>>> https://issues.apache.org/jira/browse/YARN-321.
>>>> - Development has been done in a separate branch -
>>>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>>>> - The feature development involved about 35 subtasks.
>>>> - The up-to-date design is posted at -
>>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>>>> Application
>>>> History - Design-20131219.pdf<
>>> 
>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>>> 
>>>> 
>>>> - The uber merge patch Jira -
>>>> https://issues.apache.org/jira/browse/YARN-1587
>>>> 
>>>> *Testing*
>>>> 
>>>> 
>>>> A number of unit tests have been added as a part of the feature. In
>>>> addition, we’ve also done end-to-end functional tests, and performance
>>>> tests for HDFS-based history storage and history events processing.
>> Last
>>>> but not least, we have updated branch YARN-321 against the latest
>> trunk,
>>>> edited merge conflicts, fixed test failures caused by merge, and
>>> corrected
>>>> a bunch of bad source code issues. The uber merge patch that contains
>> all
>>>> the diff between branch YARN-321 and trunk has been run through
>> Jenkins.
>>>> 
>>>> *Pending work*
>>>> 
>>>> 
>>>> - Make it work in secure mode
>>>> - Pending bug fixes
>>>> 
>>>> We wish to merge the branch now instead of waiting for later. The main
>>>> reason for this is that as the branch grew in size, the cost of its
>>>> maintenance became huge. Once the feature is merged into trunk, we will
>>>> continue to work on pending work like security stuff, to test and fix
>> any
>>>> bugs that may be found on the trunk, and to refactor the code about to
>>>> share some pieces in PRC and web interfaces.
>>>> 
>>>> *Release status*
>>>> 
>>>> 
>>>> If the security stuff and the pending fixes arrive by the time
>> everything
>>>> else planned for Release 2.4 is done, we can include it as well. This
>> is
>>>> what we are striving for. Otherwise, we will call AHS
>>> not-feature-complete
>>>> and not stable.
>>>> 
>>>> The bulk of the design and implementation was done by Mayank Bansal and
>>> me
>>>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>>>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>> providing
>>>> feedback on the design discussions.
>>>> 
>>>> 
>>>> 
>>>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>>> 
>>>> Thanks,
>>>> Zhijie
>>>> 
>>>> --
>>>> Zhijie Shen
>>>> Hortonworks Inc.
>>>> http://hortonworks.com/
>>>> 
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or
>> entity
>>> to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the
>> reader
>>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender
>>> immediately
>>>> and delete it from your system. Thank You.
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>> to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>> that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>> immediately
>>> and delete it from your system. Thank You.
>>> 
>> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.

With 5 binding +1s, one non-binding +1 and no -1s, this vote passes. I'm merging YARN-321 into trunk now.

Thanks everyone for voting!

+Vinod

On Jan 22, 2014, at 3:29 AM, Devaraj K <de...@apache.org> wrote:

> +1
> 
> 
> 
> Thanks
> Devaraj K
> 
> 
> On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:
> 
>> +1
>> 
>> Thanks,
>> Mayank
>> 
>> 
>> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>> 
>>> +1 (binding)
>>> 
>>> Arun
>>> 
>>> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>>> 
>>>> Hi folks,
>>>> 
>>>> As previously discussed here (
>>> http://markmail.org/message/iscvp7cedrtvmd6p),
>>>> I would like to call a vote to merge the YARN-321 branch for Generic
>>>> Application History Server into trunk.
>>>> 
>>>> *Scope of the changes*
>>>> 
>>>> 
>>>> The changes enable ResourceManager to record the historic information
>> of
>>>> the application, the application attempt and the container in terms of
>>>> events via a history writer. In addition, the changes setup up an
>>>> application history server, which allows users to access the recorded
>>>> information via RPC interface, web UI and REST APIs.
>>>> 
>>>> *Details of development*
>>>> 
>>>> 
>>>> - Development of the feature is tracked in the jira -
>>>> https://issues.apache.org/jira/browse/YARN-321.
>>>> - Development has been done in a separate branch -
>>>> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
>>>> - The feature development involved about 35 subtasks.
>>>> - The up-to-date design is posted at -
>>>> https://issues.apache.org/jira/secure/attachment/12619638/Generic
>>>> Application
>>>> History - Design-20131219.pdf<
>>> 
>> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
>>>> 
>>>> 
>>>> - The uber merge patch Jira -
>>>> https://issues.apache.org/jira/browse/YARN-1587
>>>> 
>>>> *Testing*
>>>> 
>>>> 
>>>> A number of unit tests have been added as a part of the feature. In
>>>> addition, we’ve also done end-to-end functional tests, and performance
>>>> tests for HDFS-based history storage and history events processing.
>> Last
>>>> but not least, we have updated branch YARN-321 against the latest
>> trunk,
>>>> edited merge conflicts, fixed test failures caused by merge, and
>>> corrected
>>>> a bunch of bad source code issues. The uber merge patch that contains
>> all
>>>> the diff between branch YARN-321 and trunk has been run through
>> Jenkins.
>>>> 
>>>> *Pending work*
>>>> 
>>>> 
>>>> - Make it work in secure mode
>>>> - Pending bug fixes
>>>> 
>>>> We wish to merge the branch now instead of waiting for later. The main
>>>> reason for this is that as the branch grew in size, the cost of its
>>>> maintenance became huge. Once the feature is merged into trunk, we will
>>>> continue to work on pending work like security stuff, to test and fix
>> any
>>>> bugs that may be found on the trunk, and to refactor the code about to
>>>> share some pieces in PRC and web interfaces.
>>>> 
>>>> *Release status*
>>>> 
>>>> 
>>>> If the security stuff and the pending fixes arrive by the time
>> everything
>>>> else planned for Release 2.4 is done, we can include it as well. This
>> is
>>>> what we are striving for. Otherwise, we will call AHS
>>> not-feature-complete
>>>> and not stable.
>>>> 
>>>> The bulk of the design and implementation was done by Mayank Bansal and
>>> me
>>>> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
>>>> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
>> providing
>>>> feedback on the design discussions.
>>>> 
>>>> 
>>>> 
>>>> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
>>>> 
>>>> Thanks,
>>>> Zhijie
>>>> 
>>>> --
>>>> Zhijie Shen
>>>> Hortonworks Inc.
>>>> http://hortonworks.com/
>>>> 
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or
>> entity
>>> to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the
>> reader
>>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender
>>> immediately
>>>> and delete it from your system. Thank You.
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>> to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>> that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>> immediately
>>> and delete it from your system. Thank You.
>>> 
>> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Devaraj K <de...@apache.org>.

+1



Thanks
Devaraj K


On Wed, Jan 22, 2014 at 1:05 AM, Mayank Bansal <ma...@apache.org> wrote:

> +1
>
> Thanks,
> Mayank
>
>
> On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com>
> wrote:
>
> > +1 (binding)
> >
> > Arun
> >
> > On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
> >
> > > Hi folks,
> > >
> > > As previously discussed here (
> > http://markmail.org/message/iscvp7cedrtvmd6p),
> > > I would like to call a vote to merge the YARN-321 branch for Generic
> > > Application History Server into trunk.
> > >
> > > *Scope of the changes*
> > >
> > >
> > > The changes enable ResourceManager to record the historic information
> of
> > > the application, the application attempt and the container in terms of
> > > events via a history writer. In addition, the changes setup up an
> > > application history server, which allows users to access the recorded
> > > information via RPC interface, web UI and REST APIs.
> > >
> > > *Details of development*
> > >
> > >
> > > - Development of the feature is tracked in the jira -
> > > https://issues.apache.org/jira/browse/YARN-321.
> > > - Development has been done in a separate branch -
> > > https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
> > > - The feature development involved about 35 subtasks.
> > > - The up-to-date design is posted at -
> > > https://issues.apache.org/jira/secure/attachment/12619638/Generic
> > > Application
> > > History - Design-20131219.pdf<
> >
> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
> > >
> > >
> > > - The uber merge patch Jira -
> > > https://issues.apache.org/jira/browse/YARN-1587
> > >
> > > *Testing*
> > >
> > >
> > > A number of unit tests have been added as a part of the feature. In
> > > addition, we’ve also done end-to-end functional tests, and performance
> > > tests for HDFS-based history storage and history events processing.
> Last
> > > but not least, we have updated branch YARN-321 against the latest
> trunk,
> > > edited merge conflicts, fixed test failures caused by merge, and
> > corrected
> > > a bunch of bad source code issues. The uber merge patch that contains
> all
> > > the diff between branch YARN-321 and trunk has been run through
> Jenkins.
> > >
> > > *Pending work*
> > >
> > >
> > > - Make it work in secure mode
> > > - Pending bug fixes
> > >
> > > We wish to merge the branch now instead of waiting for later. The main
> > > reason for this is that as the branch grew in size, the cost of its
> > > maintenance became huge. Once the feature is merged into trunk, we will
> > > continue to work on pending work like security stuff, to test and fix
> any
> > > bugs that may be found on the trunk, and to refactor the code about to
> > > share some pieces in PRC and web interfaces.
> > >
> > > *Release status*
> > >
> > >
> > > If the security stuff and the pending fixes arrive by the time
> everything
> > > else planned for Release 2.4 is done, we can include it as well. This
> is
> > > what we are striving for. Otherwise, we will call AHS
> > not-feature-complete
> > > and not stable.
> > >
> > > The bulk of the design and implementation was done by Mayank Bansal and
> > me
> > > with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> > > others. Also, thanks to Robert Joseph Evans and Sandy Ryza for
> providing
> > > feedback on the design discussions.
> > >
> > >
> > >
> > > This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
> > >
> > > Thanks,
> > > Zhijie
> > >
> > > --
> > > Zhijie Shen
> > > Hortonworks Inc.
> > > http://hortonworks.com/
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Mayank Bansal <ma...@apache.org>.

+1

Thanks,
Mayank


On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com> wrote:

> +1 (binding)
>
> Arun
>
> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>
> > Hi folks,
> >
> > As previously discussed here (
> http://markmail.org/message/iscvp7cedrtvmd6p),
> > I would like to call a vote to merge the YARN-321 branch for Generic
> > Application History Server into trunk.
> >
> > *Scope of the changes*
> >
> >
> > The changes enable ResourceManager to record the historic information of
> > the application, the application attempt and the container in terms of
> > events via a history writer. In addition, the changes setup up an
> > application history server, which allows users to access the recorded
> > information via RPC interface, web UI and REST APIs.
> >
> > *Details of development*
> >
> >
> > - Development of the feature is tracked in the jira -
> > https://issues.apache.org/jira/browse/YARN-321.
> > - Development has been done in a separate branch -
> > https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
> > - The feature development involved about 35 subtasks.
> > - The up-to-date design is posted at -
> > https://issues.apache.org/jira/secure/attachment/12619638/Generic
> > Application
> > History - Design-20131219.pdf<
> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
> >
> >
> > - The uber merge patch Jira -
> > https://issues.apache.org/jira/browse/YARN-1587
> >
> > *Testing*
> >
> >
> > A number of unit tests have been added as a part of the feature. In
> > addition, we’ve also done end-to-end functional tests, and performance
> > tests for HDFS-based history storage and history events processing. Last
> > but not least, we have updated branch YARN-321 against the latest trunk,
> > edited merge conflicts, fixed test failures caused by merge, and
> corrected
> > a bunch of bad source code issues. The uber merge patch that contains all
> > the diff between branch YARN-321 and trunk has been run through Jenkins.
> >
> > *Pending work*
> >
> >
> > - Make it work in secure mode
> > - Pending bug fixes
> >
> > We wish to merge the branch now instead of waiting for later. The main
> > reason for this is that as the branch grew in size, the cost of its
> > maintenance became huge. Once the feature is merged into trunk, we will
> > continue to work on pending work like security stuff, to test and fix any
> > bugs that may be found on the trunk, and to refactor the code about to
> > share some pieces in PRC and web interfaces.
> >
> > *Release status*
> >
> >
> > If the security stuff and the pending fixes arrive by the time everything
> > else planned for Release 2.4 is done, we can include it as well. This is
> > what we are striving for. Otherwise, we will call AHS
> not-feature-complete
> > and not stable.
> >
> > The bulk of the design and implementation was done by Mayank Bansal and
> me
> > with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> > others. Also, thanks to Robert Joseph Evans and Sandy Ryza for providing
> > feedback on the design discussions.
> >
> >
> >
> > This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
> >
> > Thanks,
> > Zhijie
> >
> > --
> > Zhijie Shen
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Mayank Bansal <ma...@apache.org>.

+1

Thanks,
Mayank


On Tue, Jan 21, 2014 at 11:11 AM, Arun C Murthy <ac...@hortonworks.com> wrote:

> +1 (binding)
>
> Arun
>
> On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
>
> > Hi folks,
> >
> > As previously discussed here (
> http://markmail.org/message/iscvp7cedrtvmd6p),
> > I would like to call a vote to merge the YARN-321 branch for Generic
> > Application History Server into trunk.
> >
> > *Scope of the changes*
> >
> >
> > The changes enable ResourceManager to record the historic information of
> > the application, the application attempt and the container in terms of
> > events via a history writer. In addition, the changes setup up an
> > application history server, which allows users to access the recorded
> > information via RPC interface, web UI and REST APIs.
> >
> > *Details of development*
> >
> >
> > - Development of the feature is tracked in the jira -
> > https://issues.apache.org/jira/browse/YARN-321.
> > - Development has been done in a separate branch -
> > https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
> > - The feature development involved about 35 subtasks.
> > - The up-to-date design is posted at -
> > https://issues.apache.org/jira/secure/attachment/12619638/Generic
> > Application
> > History - Design-20131219.pdf<
> https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf
> >
> >
> > - The uber merge patch Jira -
> > https://issues.apache.org/jira/browse/YARN-1587
> >
> > *Testing*
> >
> >
> > A number of unit tests have been added as a part of the feature. In
> > addition, we’ve also done end-to-end functional tests, and performance
> > tests for HDFS-based history storage and history events processing. Last
> > but not least, we have updated branch YARN-321 against the latest trunk,
> > edited merge conflicts, fixed test failures caused by merge, and
> corrected
> > a bunch of bad source code issues. The uber merge patch that contains all
> > the diff between branch YARN-321 and trunk has been run through Jenkins.
> >
> > *Pending work*
> >
> >
> > - Make it work in secure mode
> > - Pending bug fixes
> >
> > We wish to merge the branch now instead of waiting for later. The main
> > reason for this is that as the branch grew in size, the cost of its
> > maintenance became huge. Once the feature is merged into trunk, we will
> > continue to work on pending work like security stuff, to test and fix any
> > bugs that may be found on the trunk, and to refactor the code about to
> > share some pieces in PRC and web interfaces.
> >
> > *Release status*
> >
> >
> > If the security stuff and the pending fixes arrive by the time everything
> > else planned for Release 2.4 is done, we can include it as well. This is
> > what we are striving for. Otherwise, we will call AHS
> not-feature-complete
> > and not stable.
> >
> > The bulk of the design and implementation was done by Mayank Bansal and
> me
> > with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> > others. Also, thanks to Robert Joseph Evans and Sandy Ryza for providing
> > feedback on the design discussions.
> >
> >
> >
> > This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
> >
> > Thanks,
> > Zhijie
> >
> > --
> > Zhijie Shen
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Arun C Murthy <ac...@hortonworks.com>.

+1 (binding)

Arun

On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:

> Hi folks,
> 
> As previously discussed here (http://markmail.org/message/iscvp7cedrtvmd6p),
> I would like to call a vote to merge the YARN-321 branch for Generic
> Application History Server into trunk.
> 
> *Scope of the changes*
> 
> 
> The changes enable ResourceManager to record the historic information of
> the application, the application attempt and the container in terms of
> events via a history writer. In addition, the changes setup up an
> application history server, which allows users to access the recorded
> information via RPC interface, web UI and REST APIs.
> 
> *Details of development*
> 
> 
> - Development of the feature is tracked in the jira -
> https://issues.apache.org/jira/browse/YARN-321.
> - Development has been done in a separate branch -
> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
> - The feature development involved about 35 subtasks.
> - The up-to-date design is posted at -
> https://issues.apache.org/jira/secure/attachment/12619638/Generic
> Application
> History - Design-20131219.pdf<https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf>
> 
> - The uber merge patch Jira -
> https://issues.apache.org/jira/browse/YARN-1587
> 
> *Testing*
> 
> 
> A number of unit tests have been added as a part of the feature. In
> addition, we’ve also done end-to-end functional tests, and performance
> tests for HDFS-based history storage and history events processing. Last
> but not least, we have updated branch YARN-321 against the latest trunk,
> edited merge conflicts, fixed test failures caused by merge, and corrected
> a bunch of bad source code issues. The uber merge patch that contains all
> the diff between branch YARN-321 and trunk has been run through Jenkins.
> 
> *Pending work*
> 
> 
> - Make it work in secure mode
> - Pending bug fixes
> 
> We wish to merge the branch now instead of waiting for later. The main
> reason for this is that as the branch grew in size, the cost of its
> maintenance became huge. Once the feature is merged into trunk, we will
> continue to work on pending work like security stuff, to test and fix any
> bugs that may be found on the trunk, and to refactor the code about to
> share some pieces in PRC and web interfaces.
> 
> *Release status*
> 
> 
> If the security stuff and the pending fixes arrive by the time everything
> else planned for Release 2.4 is done, we can include it as well. This is
> what we are striving for. Otherwise, we will call AHS not-feature-complete
> and not stable.
> 
> The bulk of the design and implementation was done by Mayank Bansal and me
> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for providing
> feedback on the design discussions.
> 
> 
> 
> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
> 
> Thanks,
> Zhijie
> 
> -- 
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Arun C Murthy <ac...@hortonworks.com>.

+1 (binding)

Arun

On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:

> Hi folks,
> 
> As previously discussed here (http://markmail.org/message/iscvp7cedrtvmd6p),
> I would like to call a vote to merge the YARN-321 branch for Generic
> Application History Server into trunk.
> 
> *Scope of the changes*
> 
> 
> The changes enable ResourceManager to record the historic information of
> the application, the application attempt and the container in terms of
> events via a history writer. In addition, the changes setup up an
> application history server, which allows users to access the recorded
> information via RPC interface, web UI and REST APIs.
> 
> *Details of development*
> 
> 
> - Development of the feature is tracked in the jira -
> https://issues.apache.org/jira/browse/YARN-321.
> - Development has been done in a separate branch -
> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
> - The feature development involved about 35 subtasks.
> - The up-to-date design is posted at -
> https://issues.apache.org/jira/secure/attachment/12619638/Generic
> Application
> History - Design-20131219.pdf<https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf>
> 
> - The uber merge patch Jira -
> https://issues.apache.org/jira/browse/YARN-1587
> 
> *Testing*
> 
> 
> A number of unit tests have been added as a part of the feature. In
> addition, we’ve also done end-to-end functional tests, and performance
> tests for HDFS-based history storage and history events processing. Last
> but not least, we have updated branch YARN-321 against the latest trunk,
> edited merge conflicts, fixed test failures caused by merge, and corrected
> a bunch of bad source code issues. The uber merge patch that contains all
> the diff between branch YARN-321 and trunk has been run through Jenkins.
> 
> *Pending work*
> 
> 
> - Make it work in secure mode
> - Pending bug fixes
> 
> We wish to merge the branch now instead of waiting for later. The main
> reason for this is that as the branch grew in size, the cost of its
> maintenance became huge. Once the feature is merged into trunk, we will
> continue to work on pending work like security stuff, to test and fix any
> bugs that may be found on the trunk, and to refactor the code about to
> share some pieces in PRC and web interfaces.
> 
> *Release status*
> 
> 
> If the security stuff and the pending fixes arrive by the time everything
> else planned for Release 2.4 is done, we can include it as well. This is
> what we are striving for. Otherwise, we will call AHS not-feature-complete
> and not stable.
> 
> The bulk of the design and implementation was done by Mayank Bansal and me
> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for providing
> feedback on the design discussions.
> 
> 
> 
> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
> 
> Thanks,
> Zhijie
> 
> -- 
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [VOTE] Merge YARN-321 Generic Application History Service to trunk

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.

Thanks Zhijie!

I've been involved in its design. And reviewed almost every patch. Contributed a little w.r.t tests etc.

The code is in a decent shape bar some bugs and security. The merge should let us avoid the maintenance burden.

Tested the latest code on my single node setup. It mostly looks good.

+1 (binding).

Thanks,
+Vinod


On Jan 17, 2014, at 12:53 PM, Zhijie Shen <zs...@hortonworks.com> wrote:

> Hi folks,
> 
> As previously discussed here (http://markmail.org/message/iscvp7cedrtvmd6p),
> I would like to call a vote to merge the YARN-321 branch for Generic
> Application History Server into trunk.
> 
> *Scope of the changes*
> 
> 
> The changes enable ResourceManager to record the historic information of
> the application, the application attempt and the container in terms of
> events via a history writer. In addition, the changes setup up an
> application history server, which allows users to access the recorded
> information via RPC interface, web UI and REST APIs.
> 
> *Details of development*
> 
> 
> - Development of the feature is tracked in the jira -
> https://issues.apache.org/jira/browse/YARN-321.
> - Development has been done in a separate branch -
> https://svn.apache.org/repos/asf/hadoop/common/branches/YARN-321.
> - The feature development involved about 35 subtasks.
> - The up-to-date design is posted at -
> https://issues.apache.org/jira/secure/attachment/12619638/Generic
> Application
> History - Design-20131219.pdf<https://issues.apache.org/jira/secure/attachment/12619638/Generic%C2%A0Application%20History%20-%20Design-20131219.pdf>
> 
> - The uber merge patch Jira -
> https://issues.apache.org/jira/browse/YARN-1587
> 
> *Testing*
> 
> 
> A number of unit tests have been added as a part of the feature. In
> addition, we’ve also done end-to-end functional tests, and performance
> tests for HDFS-based history storage and history events processing. Last
> but not least, we have updated branch YARN-321 against the latest trunk,
> edited merge conflicts, fixed test failures caused by merge, and corrected
> a bunch of bad source code issues. The uber merge patch that contains all
> the diff between branch YARN-321 and trunk has been run through Jenkins.
> 
> *Pending work*
> 
> 
> - Make it work in secure mode
> - Pending bug fixes
> 
> We wish to merge the branch now instead of waiting for later. The main
> reason for this is that as the branch grew in size, the cost of its
> maintenance became huge. Once the feature is merged into trunk, we will
> continue to work on pending work like security stuff, to test and fix any
> bugs that may be found on the trunk, and to refactor the code about to
> share some pieces in PRC and web interfaces.
> 
> *Release status*
> 
> 
> If the security stuff and the pending fixes arrive by the time everything
> else planned for Release 2.4 is done, we can include it as well. This is
> what we are striving for. Otherwise, we will call AHS not-feature-complete
> and not stable.
> 
> The bulk of the design and implementation was done by Mayank Bansal and me
> with contributions from Devaraj K and Vinod Kumar Vavilapalli amongst
> others. Also, thanks to Robert Joseph Evans and Sandy Ryza for providing
> feedback on the design discussions.
> 
> 
> 
> This vote runs for a week and closes on 1/24/2014 at 11:59 pm PT.
> 
> Thanks,
> Zhijie
> 
> -- 
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.