You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by GitBox <gi...@apache.org> on 2020/09/02 05:51:14 UTC
[GitHub] [carbondata] kunal642 commented on a change in pull request #3893: [CARBONDATA-3959] Added new property to set the value of executor LRU cache size to 70% of the total executor memory in IndexServer, if executor LRU cache size is not configured.
kunal642 commented on a change in pull request #3893:
URL: https://github.com/apache/carbondata/pull/3893#discussion_r481745121
##########
File path: core/src/main/java/org/apache/carbondata/core/cache/CacheProvider.java
##########
@@ -148,19 +148,39 @@ private void createLRULevelCacheInstance() {
carbonLRUCache = new CarbonLRUCache(CarbonCommonConstants.CARBON_MAX_DRIVER_LRU_CACHE_SIZE,
CarbonCommonConstants.CARBON_MAX_LRU_CACHE_SIZE_DEFAULT);
} else {
- // if executor cache size is not configured then driver cache conf will be used
String executorCacheSize = CarbonProperties.getInstance()
.getProperty(CarbonCommonConstants.CARBON_MAX_EXECUTOR_LRU_CACHE_SIZE);
- if (null != executorCacheSize) {
- carbonLRUCache =
- new CarbonLRUCache(CarbonCommonConstants.CARBON_MAX_EXECUTOR_LRU_CACHE_SIZE,
- CarbonCommonConstants.CARBON_MAX_LRU_CACHE_SIZE_DEFAULT);
- } else {
- LOGGER.info(
- "Executor LRU cache size not configured. Initializing with driver LRU cache size.");
- carbonLRUCache = new CarbonLRUCache(CarbonCommonConstants.CARBON_MAX_DRIVER_LRU_CACHE_SIZE,
- CarbonCommonConstants.CARBON_MAX_LRU_CACHE_SIZE_DEFAULT);
+ if (null == executorCacheSize) {
+ String executorCachePercent = CarbonProperties.getInstance()
+ .getProperty(CarbonCommonConstants.CARBON_EXECUTOR_LRU_CACHE_PERCENT);
+ long mSizeMB = Runtime.getRuntime().maxMemory() / (1024 * 1024);
+ long executorLruCache;
+ if (null != executorCachePercent) {
+ int percentValue = Integer.parseInt(executorCachePercent);
+ if (percentValue >= 5 && percentValue <= 95) {
+ double mPercent = (double) percentValue / 100;
+ executorLruCache = (long) (mSizeMB * mPercent);
+ }
+ else {
Review comment:
please fix the indentation
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org