You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Ivan Kozlov <ik...@hortonworks.com> on 2015/08/26 20:03:49 UTC
Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
Repository: ambari
Description
-------
PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
Diffs
-----
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
Diff: https://reviews.apache.org/r/37803/diff/
Testing
-------
Thanks,
Ivan Kozlov
Re: Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
Posted by Robert Levas <rl...@hortonworks.com>.
> On Aug. 27, 2015, 6:06 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py, line 121
> > <https://reviews.apache.org/r/37803/diff/1-2/?file=1053327#file1053327line121>
> >
> > It seems odd that this file is created in /var/lib/hadoop-yarn when it is really an Ambari-related file. Are we sure we want to go in this direction? I thought it was more appropriate to have it in the /var/lib/ambari-agent directory tree.
>
> Ivan Kozlov wrote:
> We had similar file location for NN format marker:
> /var/lib/hdfs/namenode/formatted
> But if we think we'd better use /var/lib/ambari-agent/ I will fix this
>
> Andrew Onischuk wrote:
> @Ivan, I don't think /var/lib/ambari-agent should be trashed with it, we should find more precise path inside it maybe.
> @Robert, also why do you think using yarn folder is bad, this actually indicates something related to yarn itself, no?
@Andrew, I just thought that the /var/lib/hadoop-yarn directory was for Yarn-specific data, not to store Ambari-related data in. I am fine if we want to go this route - as Ivan pointed out we have done something simplar in the past.
@Ivan, I didn't mean to place the files /var/lib/ambari-agent, but maybe something like /var/lib/ambari-agent/markers or /var/lib/ambari-agent/data/<something>. Feel free to drop this issue since apparelty we have used this scheme in the past.
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/#review96773
-----------------------------------------------------------
On Aug. 27, 2015, 1:25 p.m., Ivan Kozlov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37803/
> -----------------------------------------------------------
>
> (Updated Aug. 27, 2015, 1:25 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
>
> Diff: https://reviews.apache.org/r/37803/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ivan Kozlov
>
>
Re: Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
Posted by Ivan Kozlov <ik...@hortonworks.com>.
> On Сер. 27, 2015, 10:06 після полудня, Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py, line 121
> > <https://reviews.apache.org/r/37803/diff/1-2/?file=1053327#file1053327line121>
> >
> > It seems odd that this file is created in /var/lib/hadoop-yarn when it is really an Ambari-related file. Are we sure we want to go in this direction? I thought it was more appropriate to have it in the /var/lib/ambari-agent directory tree.
We had similar file location for NN format marker:
/var/lib/hdfs/namenode/formatted
But if we think we'd better use /var/lib/ambari-agent/ I will fix this
- Ivan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/#review96773
-----------------------------------------------------------
On Сер. 27, 2015, 5:25 після полудня, Ivan Kozlov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37803/
> -----------------------------------------------------------
>
> (Updated Сер. 27, 2015, 5:25 після полудня)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
>
> Diff: https://reviews.apache.org/r/37803/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ivan Kozlov
>
>
Re: Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
Posted by Andrew Onischuk <ao...@hortonworks.com>.
> On Aug. 27, 2015, 10:06 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py, line 121
> > <https://reviews.apache.org/r/37803/diff/1-2/?file=1053327#file1053327line121>
> >
> > It seems odd that this file is created in /var/lib/hadoop-yarn when it is really an Ambari-related file. Are we sure we want to go in this direction? I thought it was more appropriate to have it in the /var/lib/ambari-agent directory tree.
>
> Ivan Kozlov wrote:
> We had similar file location for NN format marker:
> /var/lib/hdfs/namenode/formatted
> But if we think we'd better use /var/lib/ambari-agent/ I will fix this
@Ivan, I don't think /var/lib/ambari-agent should be trashed with it, we should find more precise path inside it maybe.
@Robert, also why do you think using yarn folder is bad, this actually indicates something related to yarn itself, no?
- Andrew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/#review96773
-----------------------------------------------------------
On Aug. 27, 2015, 5:25 p.m., Ivan Kozlov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37803/
> -----------------------------------------------------------
>
> (Updated Aug. 27, 2015, 5:25 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
>
> Diff: https://reviews.apache.org/r/37803/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ivan Kozlov
>
>
Re: Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/#review96773
-----------------------------------------------------------
Ship it!
Ship It!
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py (line 121)
<https://reviews.apache.org/r/37803/#comment152419>
It seems odd that this file is created in /var/lib/hadoop-yarn when it is really an Ambari-related file. Are we sure we want to go in this direction? I thought it was more appropriate to have it in the /var/lib/ambari-agent directory tree.
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py (line 124)
<https://reviews.apache.org/r/37803/#comment152420>
It seems odd that this file is created in /var/lib/hadoop-yarn when it is really an Ambari-related file. Are we sure we want to go in this direction? I thought it was more appropriate to have it in the /var/lib/ambari-agent directory tree.
- Robert Levas
On Aug. 27, 2015, 1:25 p.m., Ivan Kozlov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37803/
> -----------------------------------------------------------
>
> (Updated Aug. 27, 2015, 1:25 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
>
> Diff: https://reviews.apache.org/r/37803/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ivan Kozlov
>
>
Re: Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/#review97070
-----------------------------------------------------------
Ship it!
Ship It!
- Vitalyi Brodetskyi
On Сер. 28, 2015, 2:54 після полудня, Ivan Kozlov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37803/
> -----------------------------------------------------------
>
> (Updated Сер. 28, 2015, 2:54 після полудня)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
>
> Diff: https://reviews.apache.org/r/37803/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ivan Kozlov
>
>
Re: Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/#review96867
-----------------------------------------------------------
Ship it!
Ship It!
- Andrew Onischuk
On Aug. 28, 2015, 2:54 p.m., Ivan Kozlov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37803/
> -----------------------------------------------------------
>
> (Updated Aug. 28, 2015, 2:54 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
>
> Diff: https://reviews.apache.org/r/37803/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ivan Kozlov
>
>
Re: Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
Posted by Ivan Kozlov <ik...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/
-----------------------------------------------------------
(Updated Сер. 28, 2015, 2:54 після полудня)
Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
Repository: ambari
Description
-------
PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
Diff: https://reviews.apache.org/r/37803/diff/
Testing
-------
Thanks,
Ivan Kozlov
Re: Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/#review96851
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py (line 110)
<https://reviews.apache.org/r/37803/#comment152531>
I guess more proper way to do this is
InlineTemplate(...).get_content() rathen than using str
- Andrew Onischuk
On Aug. 27, 2015, 5:25 p.m., Ivan Kozlov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37803/
> -----------------------------------------------------------
>
> (Updated Aug. 27, 2015, 5:25 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
>
> Diff: https://reviews.apache.org/r/37803/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ivan Kozlov
>
>
Re: Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
Posted by Ivan Kozlov <ik...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/
-----------------------------------------------------------
(Updated Сер. 27, 2015, 5:25 після полудня)
Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
Repository: ambari
Description
-------
PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
Diff: https://reviews.apache.org/r/37803/diff/
Testing
-------
Thanks,
Ivan Kozlov
Re: Review Request 37803: Ambari is overwriting ownership of
yarn.local.dir with yarn:hadoop in Kerberized cluster
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37803/#review96604
-----------------------------------------------------------
Ship it!
Ship It!
- Robert Levas
On Aug. 26, 2015, 2:03 p.m., Ivan Kozlov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37803/
> -----------------------------------------------------------
>
> (Updated Aug. 26, 2015, 2:03 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, Robert Levas, and Vitalyi Brodetskyi.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PROBLEM: Ambari is overwriting ownership of yarn.local.dir with yarn:hadoop in Kerberized cluster when it should be owned by running user . This happens at restart of a nodemanager through Ambari. Only solution was to recreate yarn.local.dir at which point it makes correct ownership of files. Restart via Ambari then overwrites
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 9857d03
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py 30629be
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 613ab69
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 428655b
>
> Diff: https://reviews.apache.org/r/37803/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ivan Kozlov
>
>