You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by GitBox <gi...@apache.org> on 2020/05/18 12:38:39 UTC
[GitHub] [ambari] ivan-martynov-dell opened a new pull request #3195: OneFS mpack fix for performance (new version 1.0.3)
ivan-martynov-dell opened a new pull request #3195:
URL: https://github.com/apache/ambari/pull/3195
## What changes were proposed in this pull request?
(see. https://issues.apache.org/jira/browse/AMBARI-25505)
The problem is that a feature of HBase 2 AsyncFSWAL causes worse performance with OneFS. We need to disable this in the OneFS Ambari management pack by setting the property name: "hbase.wal.provider" with value: "filesystem" for HBASE. The way to change the property has been already introduced at:
https://jira.apache.org/jira/browse/HBASE-15536
We are going to issue a new version of isilon-onefs-mpack (1.0.3) where we will have some code lines which add this property to HBASE settings. Change of property is applied automatically after installation of HBASE service and installing a new version of mpack.
Business Justification: The problem we are working on is that a new feature of HBase 2 (AsyncFSWAL) causes worse performance with OneFS.
Some Isilon customers which use HBASE with Ambari with OneFS suffer from this problem. The fix should help these customers.
## How was this patch tested?
Performed unit testing and manual testing for 4 main use cases.
As they are textually long, please look at: https://jira.apache.org/jira/browse/HBASE-15536
for detailed explanations.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] zeroflag merged pull request #3195: OneFS mpack fix for performance (new version 1.0.3)
Posted by GitBox <gi...@apache.org>.
zeroflag merged pull request #3195:
URL: https://github.com/apache/ambari/pull/3195
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ambari] hapylestat commented on a change in pull request #3195: OneFS mpack fix for performance (new version 1.0.3)
Posted by GitBox <gi...@apache.org>.
hapylestat commented on a change in pull request #3195:
URL: https://github.com/apache/ambari/pull/3195#discussion_r428537513
##########
File path: contrib/management-packs/isilon-onefs-mpack/src/main/resources/addon-services/ONEFS/1.0.0/service_advisor.py
##########
@@ -116,9 +116,15 @@ def getServiceConfigurationRecommendations(self, configs, clusterData, services,
putHdfsSiteProperty("dfs.namenode.http-address", Uri.http_namenode(services).fix_host(onefs_host))
putHdfsSiteProperty("dfs.namenode.https-address", Uri.https_namenode(services).fix_host(onefs_host))
# self.updateYarnConfig(configs, services) TODO doesn't work possibly due to a UI bug (Couldn't retrieve 'capacity-scheduler' from services)
+ self.updateHbaseConfig(configs, services)
except KeyError as e:
self.logger.info('Cannot get OneFS properties from config. KeyError: %s' % e)
+ def updateHbaseConfig(self, configs, services):
+ if not 'HBASE' in self.installedServices(services): return
Review comment:
no one-liners allowed, it is pain to read
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org