You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Dmitro Lisnichenko <dl...@hortonworks.com> on 2017/07/20 10:37:56 UTC
Review Request 60995: Namenode restart - PID file delete happens
before the call to check status
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60995/
-----------------------------------------------------------
Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
Bugs: AMBARI-21532
https://issues.apache.org/jira/browse/AMBARI-21532
Repository: ambari
Description
-------
PID file delete happens before the call to check status.
{code}
...
2017-07-06 00:03:21,004 - File['/var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid'] {'action': ['delete']}
2017-07-06 00:05:21,103 - Waiting for actual component stop
2017-07-06 00:05:21,104 - Pid file /var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid is empty or does not exist
{code}
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py 7961f008ca
ambari-common/src/main/python/resource_management/libraries/script/script.py 2b374c5cff
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py d861ba93a5
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py 53774c661a
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/package/scripts/utils.py 7dcbca8d5b
Diff: https://reviews.apache.org/r/60995/diff/1/
Testing
-------
mvn clean test
Thanks,
Dmitro Lisnichenko
Re: Review Request 60995: Namenode restart - PID file delete happens
before the call to check status
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60995/#review181209
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On July 21, 2017, 11:23 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60995/
> -----------------------------------------------------------
>
> (Updated July 21, 2017, 11:23 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
>
>
> Bugs: AMBARI-21532
> https://issues.apache.org/jira/browse/AMBARI-21532
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PID file delete happens before the call to check status.
>
> {code}
> ...
> 2017-07-06 00:03:21,004 - File['/var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid'] {'action': ['delete']}
> 2017-07-06 00:05:21,103 - Waiting for actual component stop
> 2017-07-06 00:05:21,104 - Pid file /var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid is empty or does not exist
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py 7961f008ca
> ambari-common/src/main/python/resource_management/libraries/script/script.py 2b374c5cff
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py d861ba93a5
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py 53774c661a
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/package/scripts/utils.py 7dcbca8d5b
>
>
> Diff: https://reviews.apache.org/r/60995/diff/2/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 60995: Namenode restart - PID file delete happens
before the call to check status
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60995/
-----------------------------------------------------------
(Updated July 21, 2017, 6:23 p.m.)
Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
Changes
-------
increased sleep between retries
Bugs: AMBARI-21532
https://issues.apache.org/jira/browse/AMBARI-21532
Repository: ambari
Description
-------
PID file delete happens before the call to check status.
{code}
...
2017-07-06 00:03:21,004 - File['/var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid'] {'action': ['delete']}
2017-07-06 00:05:21,103 - Waiting for actual component stop
2017-07-06 00:05:21,104 - Pid file /var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid is empty or does not exist
{code}
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py 7961f008ca
ambari-common/src/main/python/resource_management/libraries/script/script.py 2b374c5cff
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py d861ba93a5
ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py 53774c661a
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/package/scripts/utils.py 7dcbca8d5b
Diff: https://reviews.apache.org/r/60995/diff/2/
Changes: https://reviews.apache.org/r/60995/diff/1-2/
Testing
-------
mvn clean test
Thanks,
Dmitro Lisnichenko
Re: Review Request 60995: Namenode restart - PID file delete happens
before the call to check status
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On July 20, 2017, 4:40 p.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py
> > Lines 74-77 (patched)
> > <https://reviews.apache.org/r/60995/diff/1/?file=1780253#file1780253line74>
> >
> > Do we have a reusable timer class which repeats? This might be a good time to write one. Nothing wrong with the sleep/while loop, but we might be able to write something more reusable. Your call.
> >
> > If we keep this, then is 0.1s too aggresive? Could we make it 1s instead?
we have a @retry annotation, it fits most cases. In current case, we need a custom exception handling during retries. It's used only in 2 places. Also, imho moving both retry loop body and exception handlers to method signature as lambdas would make calling code too complicated. So I'd go with current simpler variant.
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60995/#review181039
-----------------------------------------------------------
On July 20, 2017, 1:37 p.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60995/
> -----------------------------------------------------------
>
> (Updated July 20, 2017, 1:37 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
>
>
> Bugs: AMBARI-21532
> https://issues.apache.org/jira/browse/AMBARI-21532
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PID file delete happens before the call to check status.
>
> {code}
> ...
> 2017-07-06 00:03:21,004 - File['/var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid'] {'action': ['delete']}
> 2017-07-06 00:05:21,103 - Waiting for actual component stop
> 2017-07-06 00:05:21,104 - Pid file /var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid is empty or does not exist
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py 7961f008ca
> ambari-common/src/main/python/resource_management/libraries/script/script.py 2b374c5cff
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py d861ba93a5
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py 53774c661a
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/package/scripts/utils.py 7dcbca8d5b
>
>
> Diff: https://reviews.apache.org/r/60995/diff/1/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 60995: Namenode restart - PID file delete happens
before the call to check status
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On July 20, 2017, 9:40 a.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py
> > Lines 74-77 (patched)
> > <https://reviews.apache.org/r/60995/diff/1/?file=1780253#file1780253line74>
> >
> > Do we have a reusable timer class which repeats? This might be a good time to write one. Nothing wrong with the sleep/while loop, but we might be able to write something more reusable. Your call.
> >
> > If we keep this, then is 0.1s too aggresive? Could we make it 1s instead?
>
> Dmitro Lisnichenko wrote:
> we have a @retry annotation, it fits most cases. In current case, we need a custom exception handling during retries. It's used only in 2 places. Also, imho moving both retry loop body and exception handlers to method signature as lambdas would make calling code too complicated. So I'd go with current simpler variant.
Makes sense ... Thanks Dmitro
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60995/#review181039
-----------------------------------------------------------
On July 21, 2017, 11:23 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60995/
> -----------------------------------------------------------
>
> (Updated July 21, 2017, 11:23 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
>
>
> Bugs: AMBARI-21532
> https://issues.apache.org/jira/browse/AMBARI-21532
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PID file delete happens before the call to check status.
>
> {code}
> ...
> 2017-07-06 00:03:21,004 - File['/var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid'] {'action': ['delete']}
> 2017-07-06 00:05:21,103 - Waiting for actual component stop
> 2017-07-06 00:05:21,104 - Pid file /var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid is empty or does not exist
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py 7961f008ca
> ambari-common/src/main/python/resource_management/libraries/script/script.py 2b374c5cff
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py d861ba93a5
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py 53774c661a
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/package/scripts/utils.py 7dcbca8d5b
>
>
> Diff: https://reviews.apache.org/r/60995/diff/2/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 60995: Namenode restart - PID file delete happens
before the call to check status
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60995/#review181039
-----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py
Lines 74-77 (patched)
<https://reviews.apache.org/r/60995/#comment256476>
Do we have a reusable timer class which repeats? This might be a good time to write one. Nothing wrong with the sleep/while loop, but we might be able to write something more reusable. Your call.
If we keep this, then is 0.1s too aggresive? Could we make it 1s instead?
- Jonathan Hurley
On July 20, 2017, 6:37 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60995/
> -----------------------------------------------------------
>
> (Updated July 20, 2017, 6:37 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
>
>
> Bugs: AMBARI-21532
> https://issues.apache.org/jira/browse/AMBARI-21532
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PID file delete happens before the call to check status.
>
> {code}
> ...
> 2017-07-06 00:03:21,004 - File['/var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid'] {'action': ['delete']}
> 2017-07-06 00:05:21,103 - Waiting for actual component stop
> 2017-07-06 00:05:21,104 - Pid file /var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid is empty or does not exist
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py 7961f008ca
> ambari-common/src/main/python/resource_management/libraries/script/script.py 2b374c5cff
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py d861ba93a5
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py 53774c661a
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/package/scripts/utils.py 7dcbca8d5b
>
>
> Diff: https://reviews.apache.org/r/60995/diff/1/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 60995: Namenode restart - PID file delete happens
before the call to check status
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60995/#review181067
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On July 20, 2017, 10:37 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60995/
> -----------------------------------------------------------
>
> (Updated July 20, 2017, 10:37 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
>
>
> Bugs: AMBARI-21532
> https://issues.apache.org/jira/browse/AMBARI-21532
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PID file delete happens before the call to check status.
>
> {code}
> ...
> 2017-07-06 00:03:21,004 - File['/var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid'] {'action': ['delete']}
> 2017-07-06 00:05:21,103 - Waiting for actual component stop
> 2017-07-06 00:05:21,104 - Pid file /var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid is empty or does not exist
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py 7961f008ca
> ambari-common/src/main/python/resource_management/libraries/script/script.py 2b374c5cff
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py d861ba93a5
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py 53774c661a
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/package/scripts/utils.py 7dcbca8d5b
>
>
> Diff: https://reviews.apache.org/r/60995/diff/1/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>
Re: Review Request 60995: Namenode restart - PID file delete happens
before the call to check status
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60995/#review181040
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On July 20, 2017, 6:37 a.m., Dmitro Lisnichenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60995/
> -----------------------------------------------------------
>
> (Updated July 20, 2017, 6:37 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
>
>
> Bugs: AMBARI-21532
> https://issues.apache.org/jira/browse/AMBARI-21532
>
>
> Repository: ambari
>
>
> Description
> -------
>
> PID file delete happens before the call to check status.
>
> {code}
> ...
> 2017-07-06 00:03:21,004 - File['/var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid'] {'action': ['delete']}
> 2017-07-06 00:05:21,103 - Waiting for actual component stop
> 2017-07-06 00:05:21,104 - Pid file /var/run/hadoop/hdfs/hadoop-hdfs-namenode.pid is empty or does not exist
> {code}
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/check_process_status.py 7961f008ca
> ambari-common/src/main/python/resource_management/libraries/script/script.py 2b374c5cff
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py d861ba93a5
> ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/utils.py 53774c661a
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/package/scripts/utils.py 7dcbca8d5b
>
>
> Diff: https://reviews.apache.org/r/60995/diff/1/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Dmitro Lisnichenko
>
>