You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Sid Wagle <sw...@hortonworks.com> on 2015/10/22 03:46:31 UTC
Review Request 39536: Ambari Server JVM crashed after several clicks
in Web UI to navigate graph timerange
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39536/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty.
Bugs: AMBARI-13517
https://issues.apache.org/jira/browse/AMBARI-13517
Repository: ambari
Description
-------
- Max Heap restriction problem: Ehcache library that we use has a limit of how far in the object graph it traverses to find size of an Cached reference, this is to make it performant. Default limit = 1000.
- User gets following Warning which are harmless unless the data cannot fit in memory and eviction does not kick in:
WARN [qtp-client-70] ObjectGraphWalker:209 - The configured limit of 1,000 object references was reached while attempting to calculate the size of the object graph. Severe performance degradation could occur
if the sizing operation continues. This can be avoided by setting the CacheManger or Cache <sizeOfPolicy> elements maxDepthExceededBehavior to "abort" or adding stop points with @IgnoreSizeOf annotations. If performance degradation
is NOT an issue at the configured limit, raise the limit value using the CacheManager or Cache <sizeOfPolicy> elements maxDepth attribute. For more information, see the Ehcache configuration documentation.
_Objective of the patch_:
- Provide a custom Sizing engine for Ehcache that will provide a close approximation of the data in the cache with significant perf gain. Premise: Since we know the data structures we can make better estimates.
- Expectation from sizing engine : Discrepancy of less than 10K for a DS of size 10 MB (Proven with a unit test)
Diffs
-----
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java 8b8df06
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java 224b490
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java 4a13d63
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 9c0b94d
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 9488316
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java 573e09d
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java b7b1737
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java a3640d0
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java 26771d7
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPaddingMethod.java 522c0bd
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java 6d80687
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/MetricsPaddingMethodTest.java c30c5eb
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java 3432e1d
Diff: https://reviews.apache.org/r/39536/diff/
Testing
-------
Relevant unit test pass.
ambari-server:
AMSPropertyProviderTest
AMSReportPropertyProviderTest
StackDefinedPropertyProviderTest
TimelineMetricsCacheTest
TimleineMetricsCacheSizingTest
ambari-metrics:
All test passed
Thanks,
Sid Wagle
Re: Review Request 39536: Ambari Server JVM crashed after several
clicks in Web UI to navigate graph timerange
Posted by Aravindan Vijayan <av...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39536/#review103617
-----------------------------------------------------------
Ship it!
Ship It!
- Aravindan Vijayan
On Oct. 22, 2015, 1:46 a.m., Sid Wagle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39536/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2015, 1:46 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty.
>
>
> Bugs: AMBARI-13517
> https://issues.apache.org/jira/browse/AMBARI-13517
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - Max Heap restriction problem: Ehcache library that we use has a limit of how far in the object graph it traverses to find size of an Cached reference, this is to make it performant. Default limit = 1000.
> - User gets following Warning which are harmless unless the data cannot fit in memory and eviction does not kick in:
>
> WARN [qtp-client-70] ObjectGraphWalker:209 - The configured limit of 1,000 object references was reached while attempting to calculate the size of the object graph. Severe performance degradation could occur
> if the sizing operation continues. This can be avoided by setting the CacheManger or Cache <sizeOfPolicy> elements maxDepthExceededBehavior to "abort" or adding stop points with @IgnoreSizeOf annotations. If performance degradation
> is NOT an issue at the configured limit, raise the limit value using the CacheManager or Cache <sizeOfPolicy> elements maxDepth attribute. For more information, see the Ehcache configuration documentation.
>
> _Objective of the patch_:
> - Provide a custom Sizing engine for Ehcache that will provide a close approximation of the data in the cache with significant perf gain. Premise: Since we know the data structures we can make better estimates.
> - Expectation from sizing engine : Discrepancy of less than 10K for a DS of size 10 MB (Proven with a unit test)
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java 8b8df06
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java 224b490
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java 4a13d63
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 9c0b94d
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 9488316
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java 573e09d
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java b7b1737
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java a3640d0
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java 26771d7
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPaddingMethod.java 522c0bd
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java 6d80687
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/MetricsPaddingMethodTest.java c30c5eb
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java 3432e1d
>
> Diff: https://reviews.apache.org/r/39536/diff/
>
>
> Testing
> -------
>
> Relevant unit test pass.
> ambari-server:
> AMSPropertyProviderTest
> AMSReportPropertyProviderTest
> StackDefinedPropertyProviderTest
> TimelineMetricsCacheTest
> TimleineMetricsCacheSizingTest
> ambari-metrics:
> All test passed
>
>
> Thanks,
>
> Sid Wagle
>
>
Re: Review Request 39536: Ambari Server JVM crashed after several
clicks in Web UI to navigate graph timerange
Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39536/#review103554
-----------------------------------------------------------
Ship it!
Ship It!
- Dmytro Sen
On Окт. 22, 2015, 1:46 д.п., Sid Wagle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39536/
> -----------------------------------------------------------
>
> (Updated Окт. 22, 2015, 1:46 д.п.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty.
>
>
> Bugs: AMBARI-13517
> https://issues.apache.org/jira/browse/AMBARI-13517
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - Max Heap restriction problem: Ehcache library that we use has a limit of how far in the object graph it traverses to find size of an Cached reference, this is to make it performant. Default limit = 1000.
> - User gets following Warning which are harmless unless the data cannot fit in memory and eviction does not kick in:
>
> WARN [qtp-client-70] ObjectGraphWalker:209 - The configured limit of 1,000 object references was reached while attempting to calculate the size of the object graph. Severe performance degradation could occur
> if the sizing operation continues. This can be avoided by setting the CacheManger or Cache <sizeOfPolicy> elements maxDepthExceededBehavior to "abort" or adding stop points with @IgnoreSizeOf annotations. If performance degradation
> is NOT an issue at the configured limit, raise the limit value using the CacheManager or Cache <sizeOfPolicy> elements maxDepth attribute. For more information, see the Ehcache configuration documentation.
>
> _Objective of the patch_:
> - Provide a custom Sizing engine for Ehcache that will provide a close approximation of the data in the cache with significant perf gain. Premise: Since we know the data structures we can make better estimates.
> - Expectation from sizing engine : Discrepancy of less than 10K for a DS of size 10 MB (Proven with a unit test)
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java 8b8df06
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java 224b490
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java 4a13d63
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 9c0b94d
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 9488316
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java 573e09d
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java b7b1737
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java a3640d0
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java 26771d7
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPaddingMethod.java 522c0bd
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java 6d80687
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/MetricsPaddingMethodTest.java c30c5eb
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java 3432e1d
>
> Diff: https://reviews.apache.org/r/39536/diff/
>
>
> Testing
> -------
>
> Relevant unit test pass.
> ambari-server:
> AMSPropertyProviderTest
> AMSReportPropertyProviderTest
> StackDefinedPropertyProviderTest
> TimelineMetricsCacheTest
> TimleineMetricsCacheSizingTest
> ambari-metrics:
> All test passed
>
>
> Thanks,
>
> Sid Wagle
>
>
Re: Review Request 39536: Ambari Server JVM crashed after several
clicks in Web UI to navigate graph timerange
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39536/#review103514
-----------------------------------------------------------
Ship it!
Ship It!
- Sumit Mohanty
On Oct. 22, 2015, 1:46 a.m., Sid Wagle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39536/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2015, 1:46 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty.
>
>
> Bugs: AMBARI-13517
> https://issues.apache.org/jira/browse/AMBARI-13517
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - Max Heap restriction problem: Ehcache library that we use has a limit of how far in the object graph it traverses to find size of an Cached reference, this is to make it performant. Default limit = 1000.
> - User gets following Warning which are harmless unless the data cannot fit in memory and eviction does not kick in:
>
> WARN [qtp-client-70] ObjectGraphWalker:209 - The configured limit of 1,000 object references was reached while attempting to calculate the size of the object graph. Severe performance degradation could occur
> if the sizing operation continues. This can be avoided by setting the CacheManger or Cache <sizeOfPolicy> elements maxDepthExceededBehavior to "abort" or adding stop points with @IgnoreSizeOf annotations. If performance degradation
> is NOT an issue at the configured limit, raise the limit value using the CacheManager or Cache <sizeOfPolicy> elements maxDepth attribute. For more information, see the Ehcache configuration documentation.
>
> _Objective of the patch_:
> - Provide a custom Sizing engine for Ehcache that will provide a close approximation of the data in the cache with significant perf gain. Premise: Since we know the data structures we can make better estimates.
> - Expectation from sizing engine : Discrepancy of less than 10K for a DS of size 10 MB (Proven with a unit test)
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java 8b8df06
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java 224b490
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java 4a13d63
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 9c0b94d
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 9488316
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java 573e09d
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java b7b1737
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java a3640d0
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java 26771d7
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPaddingMethod.java 522c0bd
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java 6d80687
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/MetricsPaddingMethodTest.java c30c5eb
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java 3432e1d
>
> Diff: https://reviews.apache.org/r/39536/diff/
>
>
> Testing
> -------
>
> Relevant unit test pass.
> ambari-server:
> AMSPropertyProviderTest
> AMSReportPropertyProviderTest
> StackDefinedPropertyProviderTest
> TimelineMetricsCacheTest
> TimleineMetricsCacheSizingTest
> ambari-metrics:
> All test passed
>
>
> Thanks,
>
> Sid Wagle
>
>
Re: Review Request 39536: Ambari Server JVM crashed after several
clicks in Web UI to navigate graph timerange
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39536/#review103569
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On Oct. 21, 2015, 9:46 p.m., Sid Wagle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39536/
> -----------------------------------------------------------
>
> (Updated Oct. 21, 2015, 9:46 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty.
>
>
> Bugs: AMBARI-13517
> https://issues.apache.org/jira/browse/AMBARI-13517
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - Max Heap restriction problem: Ehcache library that we use has a limit of how far in the object graph it traverses to find size of an Cached reference, this is to make it performant. Default limit = 1000.
> - User gets following Warning which are harmless unless the data cannot fit in memory and eviction does not kick in:
>
> WARN [qtp-client-70] ObjectGraphWalker:209 - The configured limit of 1,000 object references was reached while attempting to calculate the size of the object graph. Severe performance degradation could occur
> if the sizing operation continues. This can be avoided by setting the CacheManger or Cache <sizeOfPolicy> elements maxDepthExceededBehavior to "abort" or adding stop points with @IgnoreSizeOf annotations. If performance degradation
> is NOT an issue at the configured limit, raise the limit value using the CacheManager or Cache <sizeOfPolicy> elements maxDepth attribute. For more information, see the Ehcache configuration documentation.
>
> _Objective of the patch_:
> - Provide a custom Sizing engine for Ehcache that will provide a close approximation of the data in the cache with significant perf gain. Premise: Since we know the data structures we can make better estimates.
> - Expectation from sizing engine : Discrepancy of less than 10K for a DS of size 10 MB (Proven with a unit test)
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java 8b8df06
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java 224b490
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java 4a13d63
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 9c0b94d
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 9488316
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java 573e09d
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java b7b1737
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java a3640d0
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java 26771d7
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPaddingMethod.java 522c0bd
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java 6d80687
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/MetricsPaddingMethodTest.java c30c5eb
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java 3432e1d
>
> Diff: https://reviews.apache.org/r/39536/diff/
>
>
> Testing
> -------
>
> Relevant unit test pass.
> ambari-server:
> AMSPropertyProviderTest
> AMSReportPropertyProviderTest
> StackDefinedPropertyProviderTest
> TimelineMetricsCacheTest
> TimleineMetricsCacheSizingTest
> ambari-metrics:
> All test passed
>
>
> Thanks,
>
> Sid Wagle
>
>
Re: Review Request 39536: Ambari Server JVM crashed after several
clicks in Web UI to navigate graph timerange
Posted by Sid Wagle <sw...@hortonworks.com>.
> On Oct. 22, 2015, 2:47 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java, line 37
> > <https://reviews.apache.org/r/39536/diff/1/?file=1102879#file1102879line37>
> >
> > Probably a non-issue, but should it be 10000 as in initializeCache()
The SizeOfPolicyConfiguration overrides this, so real value is 10k. And yes, it doesn't really matter.
- Sid
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39536/#review103513
-----------------------------------------------------------
On Oct. 22, 2015, 1:46 a.m., Sid Wagle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39536/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2015, 1:46 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty.
>
>
> Bugs: AMBARI-13517
> https://issues.apache.org/jira/browse/AMBARI-13517
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - Max Heap restriction problem: Ehcache library that we use has a limit of how far in the object graph it traverses to find size of an Cached reference, this is to make it performant. Default limit = 1000.
> - User gets following Warning which are harmless unless the data cannot fit in memory and eviction does not kick in:
>
> WARN [qtp-client-70] ObjectGraphWalker:209 - The configured limit of 1,000 object references was reached while attempting to calculate the size of the object graph. Severe performance degradation could occur
> if the sizing operation continues. This can be avoided by setting the CacheManger or Cache <sizeOfPolicy> elements maxDepthExceededBehavior to "abort" or adding stop points with @IgnoreSizeOf annotations. If performance degradation
> is NOT an issue at the configured limit, raise the limit value using the CacheManager or Cache <sizeOfPolicy> elements maxDepth attribute. For more information, see the Ehcache configuration documentation.
>
> _Objective of the patch_:
> - Provide a custom Sizing engine for Ehcache that will provide a close approximation of the data in the cache with significant perf gain. Premise: Since we know the data structures we can make better estimates.
> - Expectation from sizing engine : Discrepancy of less than 10K for a DS of size 10 MB (Proven with a unit test)
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java 8b8df06
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java 224b490
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java 4a13d63
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 9c0b94d
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 9488316
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java 573e09d
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java b7b1737
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java a3640d0
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java 26771d7
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPaddingMethod.java 522c0bd
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java 6d80687
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/MetricsPaddingMethodTest.java c30c5eb
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java 3432e1d
>
> Diff: https://reviews.apache.org/r/39536/diff/
>
>
> Testing
> -------
>
> Relevant unit test pass.
> ambari-server:
> AMSPropertyProviderTest
> AMSReportPropertyProviderTest
> StackDefinedPropertyProviderTest
> TimelineMetricsCacheTest
> TimleineMetricsCacheSizingTest
> ambari-metrics:
> All test passed
>
>
> Thanks,
>
> Sid Wagle
>
>
Re: Review Request 39536: Ambari Server JVM crashed after several
clicks in Web UI to navigate graph timerange
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39536/#review103513
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java (line 37)
<https://reviews.apache.org/r/39536/#comment161562>
Probably a non-issue, but should it be 10000 as in initializeCache()
- Sumit Mohanty
On Oct. 22, 2015, 1:46 a.m., Sid Wagle wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39536/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2015, 1:46 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Jonathan Hurley, and Sumit Mohanty.
>
>
> Bugs: AMBARI-13517
> https://issues.apache.org/jira/browse/AMBARI-13517
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - Max Heap restriction problem: Ehcache library that we use has a limit of how far in the object graph it traverses to find size of an Cached reference, this is to make it performant. Default limit = 1000.
> - User gets following Warning which are harmless unless the data cannot fit in memory and eviction does not kick in:
>
> WARN [qtp-client-70] ObjectGraphWalker:209 - The configured limit of 1,000 object references was reached while attempting to calculate the size of the object graph. Severe performance degradation could occur
> if the sizing operation continues. This can be avoided by setting the CacheManger or Cache <sizeOfPolicy> elements maxDepthExceededBehavior to "abort" or adding stop points with @IgnoreSizeOf annotations. If performance degradation
> is NOT an issue at the configured limit, raise the limit value using the CacheManager or Cache <sizeOfPolicy> elements maxDepth attribute. For more information, see the Ehcache configuration documentation.
>
> _Objective of the patch_:
> - Provide a custom Sizing engine for Ehcache that will provide a close approximation of the data in the cache with significant perf gain. Premise: Since we know the data structures we can make better estimates.
> - Expectation from sizing engine : Discrepancy of less than 10K for a DS of size 10 MB (Proven with a unit test)
>
>
> Diffs
> -----
>
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TimelineMetric.java 8b8df06
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCache.java 224b490
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/TimelineMetricsCacheTest.java 4a13d63
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java 9c0b94d
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java 9488316
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TimelineMetricReadHelper.java 573e09d
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITClusterAggregator.java b7b1737
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITMetricAggregator.java a3640d0
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricTestHelper.java 26771d7
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsPaddingMethod.java 522c0bd
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheProvider.java 6d80687
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricsCacheSizeOfEngine.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/MetricsPaddingMethodTest.java c30c5eb
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheSizingTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCacheTest.java 3432e1d
>
> Diff: https://reviews.apache.org/r/39536/diff/
>
>
> Testing
> -------
>
> Relevant unit test pass.
> ambari-server:
> AMSPropertyProviderTest
> AMSReportPropertyProviderTest
> StackDefinedPropertyProviderTest
> TimelineMetricsCacheTest
> TimleineMetricsCacheSizingTest
> ambari-metrics:
> All test passed
>
>
> Thanks,
>
> Sid Wagle
>
>