You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2015/05/28 04:58:20 UTC
Review Request 34750: RU from Dal to Dal+ has failures in First Oozie
Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
Bugs: AMBARI-11476
https://issues.apache.org/jira/browse/AMBARI-11476
Repository: ambari
Description
-------
This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
Diff: https://reviews.apache.org/r/34750/diff/
Testing
-------
Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
----------------------------------------------------------------------
Total run:743
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85554
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On May 27, 2015, 10:58 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 27, 2015, 10:58 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85522
-----------------------------------------------------------
Ship it!
Ship It!
- Dmitro Lisnichenko
On May 28, 2015, 2:58 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 28, 2015, 2:58 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On May 28, 2015, 3:06 a.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py, line 51
> > <https://reviews.apache.org/r/34750/diff/1/?file=973307#file973307line51>
> >
> > This is needed because yarn-site.xml is used to retrieve yarn.resourcemanager.address in order to populate the JOBTRACKER value in the /usr/hdp/current/oozie-client/doc/examples/apps/map-reduce/job.properties file
>
> Andrew Onischuk wrote:
> Hey Alejandro. I thought that we populated the value in code, and the file yarn-site.xml didn't need to be to the machine. Meaning something like this was enough:
>
> var/lib/ambari-agent/ambari-sudo.sh sed -i "s|jobTracker=localhost:8021|jobTracker=$JOBTRACKER|g" examples/apps/map-reduce/job.properties
> /var/lib/ambari-agent/ambari-sudo.sh sed -i "s|jobTracker=localhost:9001|jobTracker=$JOBTRACKER|g" examples/apps/map-reduce/job.properties
> /var/lib/ambari-agent/ambari-sudo.sh sed -i "s|jobTracker=localhost:8032|jobTracker=$JOBTRACKER|g" examples/apps/map-reduce/job.properties
>
> before.
>
> Did something changed upon that?
In prepareOozieHdfsDirectories.sh, it has, export JOBTRACKER=`getValueFromField ${hadoop_conf_dir}/yarn-site.xml yarn.resourcemanager.address`
so it expects yarn-site.xml to be present.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85500
-----------------------------------------------------------
On May 28, 2015, 2:58 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 28, 2015, 2:58 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Andrew Onischuk <ao...@hortonworks.com>.
> On May 28, 2015, 3:06 a.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py, line 51
> > <https://reviews.apache.org/r/34750/diff/1/?file=973307#file973307line51>
> >
> > This is needed because yarn-site.xml is used to retrieve yarn.resourcemanager.address in order to populate the JOBTRACKER value in the /usr/hdp/current/oozie-client/doc/examples/apps/map-reduce/job.properties file
Hey Alejandro. I thought that we populated the value in code, and the file yarn-site.xml didn't need to be to the machine. Meaning something like this was enough:
var/lib/ambari-agent/ambari-sudo.sh sed -i "s|jobTracker=localhost:8021|jobTracker=$JOBTRACKER|g" examples/apps/map-reduce/job.properties
/var/lib/ambari-agent/ambari-sudo.sh sed -i "s|jobTracker=localhost:9001|jobTracker=$JOBTRACKER|g" examples/apps/map-reduce/job.properties
/var/lib/ambari-agent/ambari-sudo.sh sed -i "s|jobTracker=localhost:8032|jobTracker=$JOBTRACKER|g" examples/apps/map-reduce/job.properties
before.
Did something changed upon that?
- Andrew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85500
-----------------------------------------------------------
On May 28, 2015, 2:58 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 28, 2015, 2:58 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85500
-----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
<https://reviews.apache.org/r/34750/#comment137072>
Unrelated change I wanted to fix.
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
<https://reviews.apache.org/r/34750/#comment137074>
This fixed the conf folder to not have conf/conf/
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py
<https://reviews.apache.org/r/34750/#comment137073>
This is needed because yarn-site.xml is used to retrieve yarn.resourcemanager.address in order to populate the JOBTRACKER value in the /usr/hdp/current/oozie-client/doc/examples/apps/map-reduce/job.properties file
- Alejandro Fernandez
On May 28, 2015, 2:58 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 28, 2015, 2:58 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85620
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On May 28, 2015, 5:11 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 28, 2015, 5:11 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85621
-----------------------------------------------------------
Ship it!
Ship It!
- Andrew Onischuk
On May 28, 2015, 9:11 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 28, 2015, 9:11 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/
-----------------------------------------------------------
(Updated May 28, 2015, 9:11 p.m.)
Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
Changes
-------
Addressed comments
Bugs: AMBARI-11476
https://issues.apache.org/jira/browse/AMBARI-11476
Repository: ambari
Description
-------
This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
Diff: https://reviews.apache.org/r/34750/diff/
Testing
-------
Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
----------------------------------------------------------------------
Total run:743
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On May 28, 2015, 2:53 a.m., Andrew Onischuk wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py, line 51
> > <https://reviews.apache.org/r/34750/diff/1/?file=973307#file973307line51>
> >
> > Why do we need to check is yarn-site is present. Seems like it's always present with Oozie installed.If it's really needed for Oozie service check, we are skipping here a ponential early failure, with good informative message that 'for some reason yarn-site is not present please check your configs' comparing to failing of oozie SC for unkwnown to user reason
Agreed; seems like it would always be in the configurations since we send all configs down regardless of the type of command or target service/component.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85523
-----------------------------------------------------------
On May 27, 2015, 10:58 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 27, 2015, 10:58 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On May 28, 2015, 6:53 a.m., Andrew Onischuk wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py, line 51
> > <https://reviews.apache.org/r/34750/diff/1/?file=973307#file973307line51>
> >
> > Why do we need to check is yarn-site is present. Seems like it's always present with Oozie installed.If it's really needed for Oozie service check, we are skipping here a ponential early failure, with good informative message that 'for some reason yarn-site is not present please check your configs' comparing to failing of oozie SC for unkwnown to user reason
>
> Jonathan Hurley wrote:
> Agreed; seems like it would always be in the configurations since we send all configs down regardless of the type of command or target service/component.
yarn-site *should* always be there based on the config dependencies, but I'll add an else clause to print a clear error if not present.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85523
-----------------------------------------------------------
On May 28, 2015, 2:58 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 28, 2015, 2:58 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85523
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py
<https://reviews.apache.org/r/34750/#comment137090>
Why do we need to check is yarn-site is present. Seems like it's always present with Oozie installed.If it's really needed for Oozie service check, we are skipping here a ponential early failure, with good informative message that 'for some reason yarn-site is not present please check your configs' comparing to failing of oozie SC for unkwnown to user reason
- Andrew Onischuk
On May 28, 2015, 2:58 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 28, 2015, 2:58 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On May 28, 2015, 3:18 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py, lines 55-56
> > <https://reviews.apache.org/r/34750/diff/1/?file=973307#file973307line55>
> >
> > Why does the oozie user have ownership and not hdfs user? What if it's there already? I suppose it should matter as long as it's world-readable.
I'll change it to hdfs:hadoop
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85550
-----------------------------------------------------------
On May 28, 2015, 2:58 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 28, 2015, 2:58 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 34750: RU from Dal to Dal+ has failures in First
Oozie Service Check, then Oozie Server Start,
because yarn-site.xml missing from hadoop conf and conf folder is incorrect
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34750/#review85550
-----------------------------------------------------------
Ship it!
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
<https://reviews.apache.org/r/34750/#comment137146>
% syntax is deprecated in favor of <string>.format() and the like
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py
<https://reviews.apache.org/r/34750/#comment137147>
Why does the oozie user have ownership and not hdfs user? What if it's there already? I suppose it should matter as long as it's world-readable.
- Nate Cole
On May 27, 2015, 10:58 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34750/
> -----------------------------------------------------------
>
> (Updated May 27, 2015, 10:58 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.
>
>
> Bugs: AMBARI-11476
> https://issues.apache.org/jira/browse/AMBARI-11476
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This occurs because when Core Masters are upgraded and the round of Service Checks are ran, the Oozie Service Check does not contain yarn-site.xml in its config folder.
> When Oozie Server is then upgraded, it fails because the tarball of the old conf folder actually zips the conf folder itself, so when it is unpacked, it creates /usr/hdp/2.3.0.0-2130/oozie/conf/conf/
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py acb1e9f
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py 47361c4
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py 11f4431
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 219c874
>
> Diff: https://reviews.apache.org/r/34750/diff/
>
>
> Testing
> -------
>
> Performed an RU from Dal to Dal+ with Oozie, and all of the service checks and Oozie Server restart passed.
>
> ----------------------------------------------------------------------
> Total run:743
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>