You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2015/05/08 00:23:21 UTC
Review Request 33957: YARN Upgrade Pack For HDP-2.2 To HDP-2.3
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33957/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez and Nate Cole.
Bugs: AMBARI-11006
https://issues.apache.org/jira/browse/AMBARI-11006
Repository: ambari
Description
-------
HDP 2.2 to 2.3 upgrade pack changes. Additionally, changes the configure task to be able to take defaults when transferring properties which was needed by YARN and MapR.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java a812169
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java 1f921c9
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 225cc63
ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/GLUSTERFS/configuration/hadoop-env.xml bce6b53
ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/HIVE/configuration/webhcat-site.xml a7caae4
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml cb641d6
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/ranger-yarn-plugin-properties.xml 93035d1
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java ecd55db
Diff: https://reviews.apache.org/r/33957/diff/
Testing
-------
mvn clean test
Thanks,
Jonathan Hurley
Re: Review Request 33957: YARN Upgrade Pack For HDP-2.2 To HDP-2.3
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33957/#review82954
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On May 7, 2015, 6:23 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33957/
> -----------------------------------------------------------
>
> (Updated May 7, 2015, 6:23 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-11006
> https://issues.apache.org/jira/browse/AMBARI-11006
>
>
> Repository: ambari
>
>
> Description
> -------
>
> HDP 2.2 to 2.3 upgrade pack changes. Additionally, changes the configure task to be able to take defaults when transferring properties which was needed by YARN and MapR.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java a812169
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java 1f921c9
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 225cc63
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/GLUSTERFS/configuration/hadoop-env.xml bce6b53
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/HIVE/configuration/webhcat-site.xml a7caae4
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml cb641d6
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/ranger-yarn-plugin-properties.xml 93035d1
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java ecd55db
>
> Diff: https://reviews.apache.org/r/33957/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 33957: YARN Upgrade Pack For HDP-2.2 To HDP-2.3
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33957/#review82964
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On May 7, 2015, 10:23 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33957/
> -----------------------------------------------------------
>
> (Updated May 7, 2015, 10:23 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-11006
> https://issues.apache.org/jira/browse/AMBARI-11006
>
>
> Repository: ambari
>
>
> Description
> -------
>
> HDP 2.2 to 2.3 upgrade pack changes. Additionally, changes the configure task to be able to take defaults when transferring properties which was needed by YARN and MapR.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java a812169
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java 1f921c9
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 225cc63
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/GLUSTERFS/configuration/hadoop-env.xml bce6b53
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/HIVE/configuration/webhcat-site.xml a7caae4
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml cb641d6
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/ranger-yarn-plugin-properties.xml 93035d1
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java ecd55db
>
> Diff: https://reviews.apache.org/r/33957/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 33957: YARN Upgrade Pack For HDP-2.2 To HDP-2.3
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On May 7, 2015, 6:33 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml, lines 383-387
> > <https://reviews.apache.org/r/33957/diff/1/?file=952611#file952611line383>
> >
> > Hmm, by the time this gets executed, it will be too late. I'm forced to add this because it's also a prerequisite check. Could likely remove this step from the process.
>
> Jonathan Hurley wrote:
> Are you sure about this? This property didn't exist in HDP 2.2. Why would it need to be a pre-req?
I went back over the instructions for upgrade and I think I see where the confusion is. HDP 2.2.0.0 does not have `yarn.timeline-service.recovery.enabled` and therefore setting it before upgrade has no effect. It however is listed as a required YARN property for YARN running on HDP 2.3. So, for upgrades from 2.2.[1-3].x to 2.3, the only step here is the configuration task.
HDP 2.2.4+ however does support this property and it should be set before upgrade. That's where `YarnTimelineServerStatePreservingCheck` comes in. It only triggers a FAIL if HDP stack is >= 2.2.4. If upgrading from HDP 2.2.4, then this value must be set to true. The configure task in this case would NOOP because the value already exists.
Let me know if you still think I missed something.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33957/#review82932
-----------------------------------------------------------
On May 7, 2015, 6:23 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33957/
> -----------------------------------------------------------
>
> (Updated May 7, 2015, 6:23 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-11006
> https://issues.apache.org/jira/browse/AMBARI-11006
>
>
> Repository: ambari
>
>
> Description
> -------
>
> HDP 2.2 to 2.3 upgrade pack changes. Additionally, changes the configure task to be able to take defaults when transferring properties which was needed by YARN and MapR.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java a812169
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java 1f921c9
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 225cc63
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/GLUSTERFS/configuration/hadoop-env.xml bce6b53
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/HIVE/configuration/webhcat-site.xml a7caae4
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml cb641d6
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/ranger-yarn-plugin-properties.xml 93035d1
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java ecd55db
>
> Diff: https://reviews.apache.org/r/33957/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 33957: YARN Upgrade Pack For HDP-2.2 To HDP-2.3
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On May 7, 2015, 6:33 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml, lines 383-387
> > <https://reviews.apache.org/r/33957/diff/1/?file=952611#file952611line383>
> >
> > Hmm, by the time this gets executed, it will be too late. I'm forced to add this because it's also a prerequisite check. Could likely remove this step from the process.
Are you sure about this? This property didn't exist in HDP 2.2. Why would it need to be a pre-req?
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33957/#review82932
-----------------------------------------------------------
On May 7, 2015, 6:23 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33957/
> -----------------------------------------------------------
>
> (Updated May 7, 2015, 6:23 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-11006
> https://issues.apache.org/jira/browse/AMBARI-11006
>
>
> Repository: ambari
>
>
> Description
> -------
>
> HDP 2.2 to 2.3 upgrade pack changes. Additionally, changes the configure task to be able to take defaults when transferring properties which was needed by YARN and MapR.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java a812169
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java 1f921c9
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 225cc63
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/GLUSTERFS/configuration/hadoop-env.xml bce6b53
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/HIVE/configuration/webhcat-site.xml a7caae4
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml cb641d6
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/ranger-yarn-plugin-properties.xml 93035d1
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java ecd55db
>
> Diff: https://reviews.apache.org/r/33957/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 33957: YARN Upgrade Pack For HDP-2.2 To HDP-2.3
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33957/#review82932
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml
<https://reviews.apache.org/r/33957/#comment133785>
Hmm, by the time this gets executed, it will be too late. I'm forced to add this because it's also a prerequisite check. Could likely remove this step from the process.
- Nate Cole
On May 7, 2015, 6:23 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33957/
> -----------------------------------------------------------
>
> (Updated May 7, 2015, 6:23 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
>
>
> Bugs: AMBARI-11006
> https://issues.apache.org/jira/browse/AMBARI-11006
>
>
> Repository: ambari
>
>
> Description
> -------
>
> HDP 2.2 to 2.3 upgrade pack changes. Additionally, changes the configure task to be able to take defaults when transferring properties which was needed by YARN and MapR.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java a812169
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ConfigureTask.java 1f921c9
> ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 225cc63
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/GLUSTERFS/configuration/hadoop-env.xml bce6b53
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/HIVE/configuration/webhcat-site.xml a7caae4
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml cb641d6
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/ranger-yarn-plugin-properties.xml 93035d1
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/ConfigureActionTest.java ecd55db
>
> Diff: https://reviews.apache.org/r/33957/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Jonathan Hurley
>
>