You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmytro Shkvyra <ds...@hortonworks.com> on 2014/01/17 19:16:11 UTC
Review Request 17052: set core file size on hosts to get core dump when JVM
crashes
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17052/
-----------------------------------------------------------
Review request for Ambari, Dmitro Lisnichenko and Mahadev Konar.
Bugs: AMBARI-4055, JVM, core, core, crashes, dump, file, get, hosts, on, set, size, to, and when
https://issues.apache.org/jira/browse/AMBARI-4055
https://issues.apache.org/jira/browse/JVM
https://issues.apache.org/jira/browse/core
https://issues.apache.org/jira/browse/core
https://issues.apache.org/jira/browse/crashes
https://issues.apache.org/jira/browse/dump
https://issues.apache.org/jira/browse/file
https://issues.apache.org/jira/browse/get
https://issues.apache.org/jira/browse/hosts
https://issues.apache.org/jira/browse/on
https://issues.apache.org/jira/browse/set
https://issues.apache.org/jira/browse/size
https://issues.apache.org/jira/browse/to
https://issues.apache.org/jira/browse/when
Repository: ambari
Description
-------
I have add set ulimit for HDFS for all Os'es except Oracle Linux because Oracle Linux security http://www.akadia.com/services/ora_enable_core.html
Also, I have improved checking of Linux platfom.
Diffs
-----
ambari-agent/src/main/python/resource_management/core/system.py b5857cb
ambari-server/src/main/resources/stacks/HDP/1.3.4/services/HDFS/package/scripts/params.py 3e0e65b
ambari-server/src/main/resources/stacks/HDP/1.3.4/services/HDFS/package/scripts/utils.py e28d0e9
ambari-server/src/main/resources/stacks/HDP/2.0.8/services/HDFS/package/scripts/params.py 685e25f
ambari-server/src/main/resources/stacks/HDP/2.0.8/services/HDFS/package/scripts/utils.py 225cd2e
Diff: https://reviews.apache.org/r/17052/diff/
Testing
-------
A have tested my patch with Centos6/HDP-2.0.8 and HDP-1.3.4
Thanks,
Dmytro Shkvyra
Re: Review Request 17052: set core file size on hosts to get core dump when
JVM crashes
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17052/#review32300
-----------------------------------------------------------
Ship it!
Ship It!
- Dmitro Lisnichenko
On Jan. 20, 2014, 1:42 p.m., Dmytro Shkvyra wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17052/
> -----------------------------------------------------------
>
> (Updated Jan. 20, 2014, 1:42 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Mahadev Konar.
>
>
> Bugs: AMBARI-4055
> https://issues.apache.org/jira/browse/AMBARI-4055
>
>
> Repository: ambari
>
>
> Description
> -------
>
> I have add set ulimit for HDFS for all Os'es except Oracle Linux because Oracle Linux security http://www.akadia.com/services/ora_enable_core.html
> Also, I have improved checking of Linux platfom.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/resource_management/core/system.py b5857cb
> ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HDFS/package/scripts/params.py 3e0e65b
> ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HDFS/package/scripts/utils.py e28d0e9
> ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HDFS/package/scripts/params.py 685e25f
> ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HDFS/package/scripts/utils.py 225cd2e
>
> Diff: https://reviews.apache.org/r/17052/diff/
>
>
> Testing
> -------
>
> A have tested my patch with Centos6/HDP-2.0.8 and HDP-1.3.4
>
>
> Thanks,
>
> Dmytro Shkvyra
>
>
Re: Review Request 17052: set core file size on hosts to get core dump when
JVM crashes
Posted by Dmytro Shkvyra <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17052/
-----------------------------------------------------------
(Updated Jan. 20, 2014, 1:42 p.m.)
Review request for Ambari, Dmitro Lisnichenko and Mahadev Konar.
Changes
-------
I have add exit code 77 and made patch for moved (from 2.0.8 to 2.1.1 and moved from 1.3.4 to 1.3.3) stacks
Bugs: AMBARI-4055
https://issues.apache.org/jira/browse/AMBARI-4055
Repository: ambari
Description
-------
I have add set ulimit for HDFS for all Os'es except Oracle Linux because Oracle Linux security http://www.akadia.com/services/ora_enable_core.html
Also, I have improved checking of Linux platfom.
Diffs (updated)
-----
ambari-agent/src/main/python/resource_management/core/system.py b5857cb
ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HDFS/package/scripts/params.py 3e0e65b
ambari-server/src/main/resources/stacks/HDP/1.3.3/services/HDFS/package/scripts/utils.py e28d0e9
ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HDFS/package/scripts/params.py 685e25f
ambari-server/src/main/resources/stacks/HDP/2.1.1/services/HDFS/package/scripts/utils.py 225cd2e
Diff: https://reviews.apache.org/r/17052/diff/
Testing
-------
A have tested my patch with Centos6/HDP-2.0.8 and HDP-1.3.4
Thanks,
Dmytro Shkvyra
Re: Review Request 17052: set core file size on hosts to get core dump when
JVM crashes
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17052/#review32188
-----------------------------------------------------------
1. Regarding
ulimit_cmd = "ulimit -c unlimited && if [ `ulimit -c` != 'unlimited' ]; then exit 1; fi && "
Let's add some special exit code like 77 that will be returned if there is a problem with ulimit. That will allow us to distinguish between ulimit fail and Hadoop fail when reading logs. And/or we may add some statement like
'echo "ulimit status: `ulimit -c`"'
for this purpose.
2. In current implementation, we don't touch ulimit settings when running under Oracle Linux. That is probably ok, as enabling them requires too much customizations and affects entire customer system, as described here http://www.akadia.com/services/ora_enable_core.html . But this topic might be worth mentioning at documentation.
- Dmitro Lisnichenko
On Jan. 17, 2014, 6:16 p.m., Dmytro Shkvyra wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17052/
> -----------------------------------------------------------
>
> (Updated Jan. 17, 2014, 6:16 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Mahadev Konar.
>
>
> Bugs: AMBARI-4055
> https://issues.apache.org/jira/browse/AMBARI-4055
>
>
> Repository: ambari
>
>
> Description
> -------
>
> I have add set ulimit for HDFS for all Os'es except Oracle Linux because Oracle Linux security http://www.akadia.com/services/ora_enable_core.html
> Also, I have improved checking of Linux platfom.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/resource_management/core/system.py b5857cb
> ambari-server/src/main/resources/stacks/HDP/1.3.4/services/HDFS/package/scripts/params.py 3e0e65b
> ambari-server/src/main/resources/stacks/HDP/1.3.4/services/HDFS/package/scripts/utils.py e28d0e9
> ambari-server/src/main/resources/stacks/HDP/2.0.8/services/HDFS/package/scripts/params.py 685e25f
> ambari-server/src/main/resources/stacks/HDP/2.0.8/services/HDFS/package/scripts/utils.py 225cd2e
>
> Diff: https://reviews.apache.org/r/17052/diff/
>
>
> Testing
> -------
>
> A have tested my patch with Centos6/HDP-2.0.8 and HDP-1.3.4
>
>
> Thanks,
>
> Dmytro Shkvyra
>
>
Re: Review Request 17052: set core file size on hosts to get core dump when
JVM crashes
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17052/#review32188
-----------------------------------------------------------
1. Regarding
ulimit_cmd = "ulimit -c unlimited && if [ `ulimit -c` != 'unlimited' ]; then exit 1; fi && "
Let's add some special exit code like 77 that will be returned if there is a problem with ulimit. That will allow us to distinguish between ulimit fail and Hadoop fail when reading logs. And/or we may add some statement like
'echo "ulimit status: `ulimit -c`"'
for this purpose.
2. In current implementation, we don't touch ulimit settings when running under Oracle Linux. That is probably ok, as enabling them requires too much customizations and affects entire customer system, as described here http://www.akadia.com/services/ora_enable_core.html . But this topic might be worth mentioning at documentation.
- Dmitro Lisnichenko
On Jan. 17, 2014, 6:16 p.m., Dmytro Shkvyra wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17052/
> -----------------------------------------------------------
>
> (Updated Jan. 17, 2014, 6:16 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Mahadev Konar.
>
>
> Bugs: AMBARI-4055
> https://issues.apache.org/jira/browse/AMBARI-4055
>
>
> Repository: ambari
>
>
> Description
> -------
>
> I have add set ulimit for HDFS for all Os'es except Oracle Linux because Oracle Linux security http://www.akadia.com/services/ora_enable_core.html
> Also, I have improved checking of Linux platfom.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/resource_management/core/system.py b5857cb
> ambari-server/src/main/resources/stacks/HDP/1.3.4/services/HDFS/package/scripts/params.py 3e0e65b
> ambari-server/src/main/resources/stacks/HDP/1.3.4/services/HDFS/package/scripts/utils.py e28d0e9
> ambari-server/src/main/resources/stacks/HDP/2.0.8/services/HDFS/package/scripts/params.py 685e25f
> ambari-server/src/main/resources/stacks/HDP/2.0.8/services/HDFS/package/scripts/utils.py 225cd2e
>
> Diff: https://reviews.apache.org/r/17052/diff/
>
>
> Testing
> -------
>
> A have tested my patch with Centos6/HDP-2.0.8 and HDP-1.3.4
>
>
> Thanks,
>
> Dmytro Shkvyra
>
>
Re: Review Request 17052: set core file size on hosts to get core dump when
JVM crashes
Posted by Dmytro Shkvyra <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17052/
-----------------------------------------------------------
(Updated Jan. 17, 2014, 6:16 p.m.)
Review request for Ambari, Dmitro Lisnichenko and Mahadev Konar.
Bugs: AMBARI-4055
https://issues.apache.org/jira/browse/AMBARI-4055
Repository: ambari
Description
-------
I have add set ulimit for HDFS for all Os'es except Oracle Linux because Oracle Linux security http://www.akadia.com/services/ora_enable_core.html
Also, I have improved checking of Linux platfom.
Diffs
-----
ambari-agent/src/main/python/resource_management/core/system.py b5857cb
ambari-server/src/main/resources/stacks/HDP/1.3.4/services/HDFS/package/scripts/params.py 3e0e65b
ambari-server/src/main/resources/stacks/HDP/1.3.4/services/HDFS/package/scripts/utils.py e28d0e9
ambari-server/src/main/resources/stacks/HDP/2.0.8/services/HDFS/package/scripts/params.py 685e25f
ambari-server/src/main/resources/stacks/HDP/2.0.8/services/HDFS/package/scripts/utils.py 225cd2e
Diff: https://reviews.apache.org/r/17052/diff/
Testing
-------
A have tested my patch with Centos6/HDP-2.0.8 and HDP-1.3.4
Thanks,
Dmytro Shkvyra