You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ambari.apache.org by nc...@apache.org on 2016/11/16 14:39:31 UTC
[01/31] ambari git commit: AMBARI-18827. Allow acceptor / seclector
configuration for API and agent connectors. (swagle)
Repository: ambari
Updated Branches:
refs/heads/branch-feature-AMBARI-18634 b395c5df5 -> f4c2dbff3
AMBARI-18827. Allow acceptor / seclector configuration for API and agent connectors. (swagle)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/4d7d3cdc
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/4d7d3cdc
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/4d7d3cdc
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 4d7d3cdc6380db63bf9b1b221f548cd4470e2a51
Parents: 95233eb
Author: Siddharth Wagle <sw...@hortonworks.com>
Authored: Mon Nov 14 10:33:10 2016 -0800
Committer: Siddharth Wagle <sw...@hortonworks.com>
Committed: Mon Nov 14 10:33:10 2016 -0800
----------------------------------------------------------------------
ambari-server/docs/configuration/index.md | 35 +--
.../server/configuration/Configuration.java | 115 ++++++----
.../ambari/server/controller/AmbariServer.java | 219 ++++++++++---------
.../security/unsecured/rest/ConnectionInfo.java | 2 +-
.../server/configuration/ConfigurationTest.java | 6 +-
5 files changed, 212 insertions(+), 165 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/4d7d3cdc/ambari-server/docs/configuration/index.md
----------------------------------------------------------------------
diff --git a/ambari-server/docs/configuration/index.md b/ambari-server/docs/configuration/index.md
index 2fc6b26..77d3a4d 100644
--- a/ambari-server/docs/configuration/index.md
+++ b/ambari-server/docs/configuration/index.md
@@ -37,6 +37,7 @@ The following are the properties which can be used to configure Ambari.
| Property Name | Description | Default |
| --- | --- | --- |
| active.instance | Indicates whether the current ambari server instance is active or not. |`true` |
+| agent.api.acceptor.count | Count of acceptors to configure for the jetty connector used for Ambari agent. | |
| agent.api.gzip.compression.enabled | Determiens whether communication with the Ambari Agents should have the JSON payloads compressed with GZIP. |`true` |
| agent.auto.cache.update | Determines whether the agents will automatically attempt to download updates to stack resources from the Ambari Server. |`true` |
| agent.check.mounts.timeout | The timeout, used by the `timeout` command in linux, when checking mounts for free capacity. |`0` |
@@ -104,11 +105,12 @@ The following are the properties which can be used to configure Ambari.
| authentication.ldap.usernameAttribute | The attribute used for determining the user name, such as `uid`. |`uid` |
| authorization.ldap.adminGroupMappingRules | A comma-separate list of groups which would give a user administrative access to Ambari when syncing from LDAP. This is only used when `authorization.ldap.groupSearchFilter` is blank.<br/><br/>The following are examples of valid values:<ul><li>`administrators`<li>`Hadoop Admins,Hadoop Admins.*,DC Admins,.*Hadoop Operators`</ul> |`Ambari Administrators` |
| authorization.ldap.groupSearchFilter | The DN to use when searching for LDAP groups. | |
-| bootstrap.dir | The directory on the Ambari Server file system used for storing Ambari Agent bootstrap information such as request responses. |`/home/crashtua/dev/ambari-work/var/run/ambari-server/bootstrap` |
+| bootstrap.dir | The directory on the Ambari Server file system used for storing Ambari Agent bootstrap information such as request responses. |`/var/run/ambari-server/bootstrap` |
| bootstrap.master_host_name | The host name of the Ambari Server which will be used by the Ambari Agents for communication. | |
-| bootstrap.script | The location and name of the Python script used to bootstrap new Ambari Agent hosts. |`/home/crashtua/dev/ambari-work/usr/lib/python2.6/site-packages/ambari_server/bootstrap.py` |
+| bootstrap.script | The location and name of the Python script used to bootstrap new Ambari Agent hosts. |`/usr/lib/python2.6/site-packages/ambari_server/bootstrap.py` |
| bootstrap.setup_agent.password | The password to set on the `AMBARI_PASSPHRASE` environment variable before invoking the bootstrap script. |`password` |
-| bootstrap.setup_agent.script | The location and name of the Python script executed on the Ambari Agent host during the bootstrap process. |`/home/crashtua/dev/ambari-work/usr/lib/python2.6/site-packages/ambari_server/setupAgent.py` |
+| bootstrap.setup_agent.script | The location and name of the Python script executed on the Ambari Agent host during the bootstrap process. |`/usr/lib/python2.6/site-packages/ambari_server/setupAgent.py` |
+| client.api.acceptor.count | Count of acceptors to configure for the jetty connector used for Ambari API. | |
| client.api.port | The port that client connections will use with the REST API. The Ambari Web client runs on this port. |`8080` |
| client.api.ssl.cert_pass_file | The filename which contains the password for the keystores, truststores, and certificates for the REST API when it's protected by SSL. |`https.pass.txt` |
| client.api.ssl.crt_pass | The password for the keystores, truststores, and certificates for the REST API when it's protected by SSL. If not specified, then `client.api.ssl.cert_pass_file` should be used. | |
@@ -121,7 +123,7 @@ The following are the properties which can be used to configure Ambari.
| client.security | The type of authentication mechanism used by Ambari.<br/><br/>The following are examples of valid values:<ul><li>`local`<li>`ldap`</ul> | |
| client.threadpool.size.max | The size of the Jetty connection pool used for handling incoming REST API requests. This should be large enough to handle requests from both web browsers and embedded Views. |`25` |
| common.services.path | The location on the Ambari Server where common service resources exist. Stack services share the common service files.<br/><br/>The following are examples of valid values:<ul><li>`/var/lib/ambari-server/resources/common-services`</ul> | |
-| custom.action.definitions | The location on the Ambari Server where custom actions are defined. |`/home/crashtua/dev/ambari-work/var/lib/ambari-server/resources/custom_action_definitions` |
+| custom.action.definitions | The location on the Ambari Server where custom actions are defined. |`/var/lib/ambari-server/resources/custom_action_definitions` |
| db.mysql.jdbc.name | The name of the MySQL JDBC JAR connector. |`mysql-connector-java.jar` |
| db.oracle.jdbc.name | The name of the Oracle JDBC JAR connector. |`ojdbc6.jar` |
| default.kdcserver.port | The port used to communicate with the Kerberos Key Distribution Center. |`88` |
@@ -134,9 +136,10 @@ The following are the properties which can be used to configure Ambari.
| jdk.name | The name of the JDK installation binary.<br/><br/>The following are examples of valid values:<ul><li>`jdk-7u45-linux-x64.tar.gz`</ul> | |
| kdcserver.connection.check.timeout | The timeout, in milliseconds, to wait when communicating with a Kerberos Key Distribution Center. |`10000` |
| kerberos.check.jaas.configuration | Determines whether Kerberos-enabled Ambari deployments should use JAAS to validate login credentials. |`false` |
-| kerberos.keytab.cache.dir | The location on the Ambari Server where Kerberos keytabs are cached. |`/home/crashtua/dev/ambari-work/var/lib/ambari-server/data/cache` |
+| kerberos.keytab.cache.dir | The location on the Ambari Server where Kerberos keytabs are cached. |`/var/lib/ambari-server/data/cache` |
| kerberos.operation.retries | The number of times failed kerberos operations should be retried to execute. |`3` |
| ldap.sync.username.collision.behavior | Determines how to handle username collision while updating from LDAP.<br/><br/>The following are examples of valid values:<ul><li>`skip`<li>`convert`</ul> |`convert` |
+| log4j.monitor.delay | Indicates the delay, in milliseconds, for the log4j monitor to check for changes |`300000` |
| metadata.path | The location on the Ambari Server where the stack resources exist.<br/><br/>The following are examples of valid values:<ul><li>`/var/lib/ambari-server/resources/stacks`</ul> | |
| metrics.retrieval-service.cache.timeout | The amount of time, in minutes, that JMX and REST metrics retrieved directly can remain in the cache. |`30` |
| metrics.retrieval-service.request.ttl | The number of seconds to wait between issuing JMX or REST metric requests to the same endpoint. This property is used to throttle requests to the same URL being made too close together<br/><br/> This property is related to `metrics.retrieval-service.request.ttl.enabled`. |`5` |
@@ -145,7 +148,7 @@ The following are the properties which can be used to configure Ambari.
| packages.pre.installed | Determines whether Ambari Agent instances have already have the necessary stack software installed |`false` |
| proxy.allowed.hostports | A comma-separated whitelist of host and port values which Ambari Server can use to determine if a proxy value is valid. |`*:*` |
| recommendations.artifacts.lifetime | The amount of time that Recommendation API data is kept on the Ambari Server file system. This is specified using a `hdwmy` syntax for pairing the value with a time unit (hours, days, weeks, months, years)<br/><br/>The following are examples of valid values:<ul><li>`8h`<li>`2w`<li>`1m`</ul> |`1w` |
-| recommendations.dir | The directory on the Ambari Server file system used for storing Recommendation API artifacts. |`/home/crashtua/dev/ambari-work/var/run/ambari-server/stack-recommendations` |
+| recommendations.dir | The directory on the Ambari Server file system used for storing Recommendation API artifacts. |`/var/run/ambari-server/stack-recommendations` |
| recovery.disabled_components | A comma-separated list of component names which are not included in automatic recovery attempts.<br/><br/>The following are examples of valid values:<ul><li>`NAMENODE,ZOOKEEPER_SERVER`</ul> | |
| recovery.enabled_components | A comma-separated list of component names which are included in automatic recovery attempts.<br/><br/>The following are examples of valid values:<ul><li>`NAMENODE,ZOOKEEPER_SERVER`</ul> | |
| recovery.lifetime_max_count | The maximum number of recovery attempts of a failed component during the lifetime of an Ambari Agent instance. This is reset when the Ambari Agent is restarted. | |
@@ -155,7 +158,7 @@ The following are the properties which can be used to configure Ambari.
| recovery.window_in_minutes | The length of a recovery window, in minutes, in which recovery attempts can be retried.<br/><br/> This property is related to `recovery.max_count`. | |
| repo.validation.suffixes.default | The suffixes to use when validating most types of repositories. |`/repodata/repomd.xml` |
| repo.validation.suffixes.ubuntu | The suffixes to use when validating Ubuntu repositories. |`/dists/%s/Release` |
-| resources.dir | The location on the Ambari Server where all resources exist, including common services, stacks, and scripts. |`/home/crashtua/dev/ambari-work/var/lib/ambari-server/resources/` |
+| resources.dir | The location on the Ambari Server where all resources exist, including common services, stacks, and scripts. |`/var/lib/ambari-server/resources/` |
| rolling.upgrade.skip.packages.prefixes | A comma-separated list of packages which will be skipped during a stack upgrade. | |
| security.agent.hostname.validate | Determines whether the Ambari Agent host names should be validated against a regular expression to ensure that they are well-formed. |`true` |
| security.master.key.location | The location on the Ambari Server of the master key file. This is the key to the master keystore. | |
@@ -219,16 +222,16 @@ The following are the properties which can be used to configure Ambari.
| server.jdbc.user.passwd | The password for the user when logging into the database. |`bigdata` |
| server.locks.profiling | Enable the profiling of internal locks. |`false` |
| server.metrics.retrieval-service.thread.priority | The priority of threads used by the service which retrieves JMX and REST metrics directly from their respective endpoints. |`5` |
-| server.metrics.retrieval-service.threadpool.size.core | The core number of threads used to retrieve JMX and REST metrics directly from their respective endpoints. |`8` |
-| server.metrics.retrieval-service.threadpool.size.max | The maximum number of threads used to retrieve JMX and REST metrics directly from their respective endpoints. |`16` |
-| server.metrics.retrieval-service.threadpool.worker.size | The number of queued requests allowed for JMX and REST metrics before discarding old requests which have not been fullfilled. |`160` |
+| server.metrics.retrieval-service.threadpool.size.core | The core number of threads used to retrieve JMX and REST metrics directly from their respective endpoints. |`16` |
+| server.metrics.retrieval-service.threadpool.size.max | The maximum number of threads used to retrieve JMX and REST metrics directly from their respective endpoints. |`32` |
+| server.metrics.retrieval-service.threadpool.worker.size | The number of queued requests allowed for JMX and REST metrics before discarding old requests which have not been fullfilled. |`320` |
| server.operations.retry-attempts | The number of retry attempts for failed API and blueprint operations. |`0` |
| server.os_family | The operating system family for all hosts in the cluster. This is used when bootstrapping agents and when enabling Kerberos.<br/><br/>The following are examples of valid values:<ul><li>`redhat`<li>`ubuntu`</ul> | |
| server.os_type | The operating system version for all hosts in the cluster. This is used when bootstrapping agents and when enabling Kerberos.<br/><br/>The following are examples of valid values:<ul><li>`6`<li>`7`</ul> | |
| server.persistence.type | The type of database connection being used. Unless using an embedded PostgresSQL server, then this should be `remote`.<br/><br/>The following are examples of valid values:<ul><li>`local`<li>`remote`</ul> |`local` |
| server.property-provider.threadpool.completion.timeout | The maximum time, in milliseconds, that federated requests for data can execute before being terminated. Increasing this value could result in degraded performanc from the REST APIs. |`5000` |
-| server.property-provider.threadpool.size.core | The core number of threads that will be used to retrieve data from federated datasources, such as remote JMX endpoints. |`8` |
-| server.property-provider.threadpool.size.max | The maximum number of threads that will be used to retrieve data from federated datasources, such as remote JMX endpoints. |`16` |
+| server.property-provider.threadpool.size.core | The core number of threads that will be used to retrieve data from federated datasources, such as remote JMX endpoints. |`16` |
+| server.property-provider.threadpool.size.max | The maximum number of threads that will be used to retrieve data from federated datasources, such as remote JMX endpoints. |`32` |
| server.property-provider.threadpool.worker.size | The maximum size of pending federated datasource requests, such as those to JMX endpoints, which can be queued before rejecting new requests. |`2147483647` |
| server.script.timeout | The time, in milliseconds, until an external script is killed. |`5000` |
| server.stage.command.execution_type | How to execute commands in one stage |`STAGE` |
@@ -244,11 +247,11 @@ The following are the properties which can be used to configure Ambari.
| server.timeline.metrics.cache.read.timeout.millis | The time, in milliseconds, that initial requests to populate metric data will wait while reading from Ambari Metrics.<br/><br/> This property is related to `server.timeline.metrics.cache.disabled`. |`10000` |
| server.timeline.metrics.cache.use.custom.sizing.engine | Determines if a custom engine should be used to increase performance of calculating the current size of the cache for Ambari Metric data.<br/><br/> This property is related to `server.timeline.metrics.cache.disabled`. |`true` |
| server.timeline.metrics.https.enabled | Determines whether to use to SSL to connect to Ambari Metrics when retrieving metric data. |`false` |
-| server.tmp.dir | The location on the Ambari Server where temporary artifacts can be created. |`/home/crashtua/dev/ambari-work/var/lib/ambari-server/tmp` |
+| server.tmp.dir | The location on the Ambari Server where temporary artifacts can be created. |`/var/lib/ambari-server/tmp` |
| server.version.file | The full path to the file which contains the Ambari Server version. This is used to ensure that there is not a version mismatch between Ambari Agents and Ambari Server.<br/><br/>The following are examples of valid values:<ul><li>`/var/lib/ambari-server/resources/version`</ul> | |
| server.version_definition.connect.timeout.millis | The time, in milliseconds, that requests to connect to a URL to retrieve Version Definition Files (VDF) will wait before being terminated. |`5000` |
| server.version_definition.read.timeout.millis | The time, in milliseconds, that requests to read from a connected URL to retrieve Version Definition Files (VDF) will wait before being terminated. |`5000` |
-| shared.resources.dir | The location on the Ambari Server where resources are stored. This is exposed via HTTP in order for Ambari Agents to access them. |`/home/crashtua/dev/ambari-work/usr/lib/ambari-server/lib/ambari_commons/resources` |
+| shared.resources.dir | The location on the Ambari Server where resources are stored. This is exposed via HTTP in order for Ambari Agents to access them. |`/usr/lib/ambari-server/lib/ambari_commons/resources` |
| ssl.trustStore.password | The password to use when setting the `javax.net.ssl.trustStorePassword` property | |
| ssl.trustStore.path | The location of the truststore to use when setting the `javax.net.ssl.trustStore` property. | |
| ssl.trustStore.type | The type of truststore used by the `javax.net.ssl.trustStoreType` property. | |
@@ -257,7 +260,7 @@ The following are the properties which can be used to configure Ambari.
| stack.upgrade.auto.retry.command.names.to.ignore | A comma-separate list of upgrade tasks names to skip when retrying failed commands automatically. |`"ComponentVersionCheckAction","FinalizeUpgradeAction"` |
| stack.upgrade.auto.retry.timeout.mins | The amount of time to wait in order to retry a command during a stack upgrade when an agent loses communication. This value must be greater than the `agent.task.timeout` value. |`0` |
| stack.upgrade.bypass.prechecks | Determines whether pre-upgrade checks will be skipped when performing a rolling or express stack upgrade. |`false` |
-| stackadvisor.script | The location and name of the Python stack advisor script executed when configuring services. |`/home/crashtua/dev/ambari-work/var/lib/ambari-server/resources/scripts/stack_advisor.py` |
+| stackadvisor.script | The location and name of the Python stack advisor script executed when configuring services. |`/var/lib/ambari-server/resources/scripts/stack_advisor.py` |
| task.query.parameterlist.size | The maximum number of tasks which can be queried by ID from the database. |`999` |
| view.extraction.threadpool.size.core | The number of threads used to extract Ambari Views when Ambari Server is starting up. |`10` |
| view.extraction.threadpool.size.max | The maximum number of threads used to extract Ambari Views when Ambari Server is starting up. |`20` |
@@ -266,7 +269,7 @@ The following are the properties which can be used to configure Ambari.
| view.request.threadpool.timeout | The time, milliseconds, that REST API requests from embedded views can wait if there are no threads available to service the view's request. Setting this too low can cause views to timeout. |`2000` |
| views.ambari.request.connect.timeout.millis | The amount of time, in milliseconds, that a view will wait when trying to connect on HTTP(S) operations to the Ambari REST API. |`30000` |
| views.ambari.request.read.timeout.millis | The amount of time, in milliseconds, that a view will wait before terminating an HTTP(S) read request to the Ambari REST API. |`45000` |
-| views.dir | The directory on the Ambari Server file system used for expanding Views and storing webapp work. |`/home/crashtua/dev/ambari-work/var/lib/ambari-server/resources/views` |
+| views.dir | The directory on the Ambari Server file system used for expanding Views and storing webapp work. |`/var/lib/ambari-server/resources/views` |
| views.http.strict-transport-security | The value that will be used to set the `Strict-Transport-Security` HTTP response header for Ambari View requests. |`max-age=31536000` |
| views.http.x-frame-options | The value that will be used to set the `X-Frame-Options` HTTP response header for Ambari View requests. |`SAMEORIGIN` |
| views.http.x-xss-protection | The value that will be used to set the `X-XSS-Protection` HTTP response header for Ambari View requests. |`1; mode=block` |
http://git-wip-us.apache.org/repos/asf/ambari/blob/4d7d3cdc/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java b/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
index b5a29b3..0b8e195 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
@@ -17,42 +17,23 @@
*/
package org.apache.ambari.server.configuration;
-import java.io.File;
-import java.io.FileInputStream;
-import java.io.FileNotFoundException;
-import java.io.FileReader;
-import java.io.FileWriter;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.Writer;
-import java.lang.annotation.ElementType;
-import java.lang.annotation.Retention;
-import java.lang.annotation.RetentionPolicy;
-import java.lang.annotation.Target;
-import java.lang.reflect.Field;
-import java.security.cert.CertificateException;
-import java.security.interfaces.RSAPublicKey;
-import java.util.ArrayList;
-import java.util.EnumSet;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import java.util.Map.Entry;
-import java.util.Properties;
-import java.util.Set;
-import java.util.SortedMap;
-import java.util.TreeMap;
-import java.util.concurrent.Callable;
-import java.util.concurrent.ThreadPoolExecutor;
-import java.util.concurrent.TimeUnit;
-
+import com.google.common.collect.Multimap;
+import com.google.common.collect.Sets;
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+import com.google.gson.JsonElement;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import com.google.gson.JsonPrimitive;
+import com.google.inject.Inject;
+import com.google.inject.Singleton;
import org.apache.ambari.annotations.Experimental;
import org.apache.ambari.annotations.ExperimentalFeature;
import org.apache.ambari.annotations.Markdown;
import org.apache.ambari.server.AmbariException;
+import org.apache.ambari.server.actionmanager.CommandExecutionType;
import org.apache.ambari.server.actionmanager.HostRoleCommand;
import org.apache.ambari.server.actionmanager.Stage;
-import org.apache.ambari.server.actionmanager.CommandExecutionType;
import org.apache.ambari.server.controller.spi.PropertyProvider;
import org.apache.ambari.server.events.listeners.alerts.AlertReceivedListener;
import org.apache.ambari.server.orm.JPATableGenerationStrategy;
@@ -89,16 +70,34 @@ import org.apache.commons.lang.math.NumberUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import com.google.common.collect.Multimap;
-import com.google.common.collect.Sets;
-import com.google.gson.Gson;
-import com.google.gson.GsonBuilder;
-import com.google.gson.JsonElement;
-import com.google.gson.JsonObject;
-import com.google.gson.JsonParser;
-import com.google.gson.JsonPrimitive;
-import com.google.inject.Inject;
-import com.google.inject.Singleton;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.Writer;
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+import java.lang.reflect.Field;
+import java.security.cert.CertificateException;
+import java.security.interfaces.RSAPublicKey;
+import java.util.ArrayList;
+import java.util.EnumSet;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.concurrent.Callable;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
/**
* The {@link Configuration} class is used to read from the
@@ -2489,6 +2488,7 @@ public class Configuration {
"log4j.monitor.delay", TimeUnit.MINUTES.toMillis(5));
/**
+<<<<<<< a5fdae802210ae1f8d4fed2234f1651cbe61c2b5
* Indicates whether parallel topology task creation is enabled for blueprint cluster provisioning.
* Defaults to <code>false</code>.
* @see #TOPOLOGY_TASK_PARALLEL_CREATION_THREAD_COUNT
@@ -2504,6 +2504,20 @@ public class Configuration {
@Markdown(description = "The number of threads to use for parallel topology task creation if enabled")
public static final ConfigurationProperty<Integer> TOPOLOGY_TASK_PARALLEL_CREATION_THREAD_COUNT = new ConfigurationProperty<>("topology.task.creation.parallel.threads", 10);
+ /**
+ * The number of acceptor threads for the agent jetty connector.
+ */
+ @Markdown(description = "Count of acceptors to configure for the jetty connector used for Ambari agent.")
+ public static final ConfigurationProperty<Integer> SRVR_AGENT_ACCEPTOR_THREAD_COUNT = new ConfigurationProperty<>(
+ "agent.api.acceptor.count", null);
+
+ /**
+ * The number of acceptor threads for the api jetty connector.
+ */
+ @Markdown(description = "Count of acceptors to configure for the jetty connector used for Ambari API.")
+ public static final ConfigurationProperty<Integer> SRVR_API_ACCEPTOR_THREAD_COUNT = new ConfigurationProperty<>(
+ "client.api.acceptor.count", null);
+
private static final Logger LOG = LoggerFactory.getLogger(
Configuration.class);
@@ -3600,7 +3614,7 @@ public class Configuration {
*
* @return true two-way SSL authentication is enabled
*/
- public boolean getTwoWaySsl() {
+ public boolean isTwoWaySsl() {
return Boolean.parseBoolean(getProperty(SRVR_TWO_WAY_SSL));
}
@@ -5353,7 +5367,6 @@ public class Configuration {
InputStream inputStream = null;
try {
if (System.getProperties().containsKey(AMBARI_CONFIGURATION_MD_TEMPLATE_PROPERTY)) {
- // for using from IDEA or other tools without whole compilation
inputStream = new FileInputStream(System.getProperties().getProperty(AMBARI_CONFIGURATION_MD_TEMPLATE_PROPERTY));
} else {
inputStream = Configuration.class.getResourceAsStream(MARKDOWN_TEMPLATE_FILE);
@@ -5715,7 +5728,23 @@ public class Configuration {
return kerberosAuthProperties;
}
- public int getKerberosOperationRetries(){
+ public int getKerberosOperationRetries() {
return Integer.valueOf(getProperty(KERBEROS_OPERATION_RETRIES));
}
+
+ /**
+ * Return configured acceptors for agent api connector. Default = null
+ */
+ public Integer getAgentApiAcceptors() {
+ String acceptors = getProperty(SRVR_AGENT_ACCEPTOR_THREAD_COUNT);
+ return StringUtils.isEmpty(acceptors) ? null : Integer.parseInt(acceptors);
+ }
+
+ /**
+ * Return configured acceptors for server api connector. Default = null
+ */
+ public Integer getClientApiAcceptors() {
+ String acceptors = getProperty(SRVR_API_ACCEPTOR_THREAD_COUNT);
+ return StringUtils.isEmpty(acceptors) ? null : Integer.parseInt(acceptors);
+ }
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/4d7d3cdc/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java b/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
index 6351720..e54d54e 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
@@ -175,7 +175,7 @@ public class AmbariServer {
/**
* The thread name prefix for threads handling agent requests.
*/
- private static final String AGENT_THREAD_POOL_NAME = "ambari-agent-thread";
+ private static final String AGENT_THREAD_POOL_NAME = "qtp-ambari-agent";
/**
* The thread name prefix for threads handling REST API requests.
@@ -414,73 +414,27 @@ public class AmbariServer {
Map<String, String> configsMap = configs.getConfigsMap();
- if (configs.getAgentSSLAuthentication()) {
- //Secured connector for 2-way auth
- SslContextFactory contextFactoryTwoWay = new SslContextFactory();
- disableInsecureProtocols(contextFactoryTwoWay);
-
- SslSelectChannelConnector sslConnectorTwoWay = new SslSelectChannelConnector(contextFactoryTwoWay);
- sslConnectorTwoWay.setPort(configs.getTwoWayAuthPort());
-
- String keystore = configsMap.get(Configuration.SRVR_KSTR_DIR.getKey()) + File.separator
- + configsMap.get(Configuration.KSTR_NAME.getKey());
-
- String truststore = configsMap.get(Configuration.SRVR_KSTR_DIR.getKey()) + File.separator
- + configsMap.get(Configuration.TSTR_NAME.getKey());
-
- String srvrCrtPass = configsMap.get(Configuration.SRVR_CRT_PASS.getKey());
-
- sslConnectorTwoWay.setKeystore(keystore);
- sslConnectorTwoWay.setTruststore(truststore);
- sslConnectorTwoWay.setPassword(srvrCrtPass);
- sslConnectorTwoWay.setKeyPassword(srvrCrtPass);
- sslConnectorTwoWay.setTrustPassword(srvrCrtPass);
- sslConnectorTwoWay.setKeystoreType(configsMap.get(Configuration.KSTR_TYPE.getKey()));
- sslConnectorTwoWay.setTruststoreType(configsMap.get(Configuration.TSTR_TYPE.getKey()));
- sslConnectorTwoWay.setNeedClientAuth(configs.getTwoWaySsl());
- sslConnectorTwoWay.setRequestHeaderSize(configs.getHttpRequestHeaderSize());
- sslConnectorTwoWay.setResponseHeaderSize(configs.getHttpResponseHeaderSize());
-
- //SSL Context Factory
- SslContextFactory contextFactoryOneWay = new SslContextFactory(true);
- contextFactoryOneWay.setKeyStorePath(keystore);
- contextFactoryOneWay.setTrustStore(truststore);
- contextFactoryOneWay.setKeyStorePassword(srvrCrtPass);
- contextFactoryOneWay.setKeyManagerPassword(srvrCrtPass);
- contextFactoryOneWay.setTrustStorePassword(srvrCrtPass);
- contextFactoryOneWay.setKeyStoreType(configsMap.get(Configuration.KSTR_TYPE.getKey()));
- contextFactoryOneWay.setTrustStoreType(configsMap.get(Configuration.TSTR_TYPE.getKey()));
- contextFactoryOneWay.setNeedClientAuth(false);
- disableInsecureProtocols(contextFactoryOneWay);
-
- //Secured connector for 1-way auth
- SslSelectChannelConnector sslConnectorOneWay = new SslSelectChannelConnector(contextFactoryOneWay);
- sslConnectorOneWay.setPort(configs.getOneWayAuthPort());
- sslConnectorOneWay.setRequestHeaderSize(configs.getHttpRequestHeaderSize());
- sslConnectorOneWay.setResponseHeaderSize(configs.getHttpResponseHeaderSize());
-
- // because there are two connectors sharing the same pool, cut each's
- // acceptors in half
- int sslAcceptors = sslConnectorOneWay.getAcceptors();
- sslConnectorOneWay.setAcceptors(Math.max(2, sslAcceptors / 2));
- sslConnectorTwoWay.setAcceptors(Math.max(2, sslAcceptors / 2));
-
- // Agent Jetty thread pool
- configureJettyThreadPool(serverForAgent, sslConnectorOneWay.getAcceptors(),
- "qtp-ambari-agent", configs.getAgentThreadPoolSize());
-
- serverForAgent.addConnector(sslConnectorOneWay);
- serverForAgent.addConnector(sslConnectorTwoWay);
- } else {
- SelectChannelConnector agentConnector = new SelectChannelConnector();
- agentConnector.setPort(configs.getOneWayAuthPort());
- agentConnector.setMaxIdleTime(configs.getConnectionMaxIdleTime());
-
- // Agent Jetty thread pool
- configureJettyThreadPool(serverForAgent, agentConnector.getAcceptors(), "qtp-ambari-agent",
- configs.getAgentThreadPoolSize());
-
- serverForAgent.addConnector(agentConnector);
+ SelectChannelConnector agentOneWayConnector = createSelectChannelConnectorForAgent(configs.getOneWayAuthPort(), false);
+ // Override acceptor defaults if configured or use Jetty defaults
+ Integer acceptors = configs.getAgentApiAcceptors() != null ?
+ configs.getAgentApiAcceptors() : agentOneWayConnector.getAcceptors();
+ agentOneWayConnector.setAcceptors(acceptors);
+
+ SelectChannelConnector agentTwoWayConnector = null;
+ if (configs.isTwoWaySsl()) {
+ agentTwoWayConnector = createSelectChannelConnectorForAgent(configs.getTwoWayAuthPort(), true);
+ agentTwoWayConnector.setAcceptors(acceptors);
+ }
+
+ // Account for both the connectors if configured
+ int totalAgentAcceptorCount = configs.isTwoWaySsl() ? acceptors * 2 : acceptors;
+ // Agent Jetty thread pool - widen the thread pool if needed !
+ configureJettyThreadPool(serverForAgent, totalAgentAcceptorCount,
+ AGENT_THREAD_POOL_NAME, configs.getAgentThreadPoolSize());
+
+ serverForAgent.addConnector(agentOneWayConnector);
+ if (agentTwoWayConnector != null) {
+ serverForAgent.addConnector(agentTwoWayConnector);
}
ServletHolder sh = new ServletHolder(ServletContainer.class);
@@ -546,41 +500,15 @@ public class AmbariServer {
}
/* Configure the API server to use the NIO connectors */
- SelectChannelConnector apiConnector;
-
- if (configs.getApiSSLAuthentication()) {
- String httpsKeystore = configsMap.get(Configuration.CLIENT_API_SSL_KSTR_DIR_NAME.getKey()) +
- File.separator + configsMap.get(Configuration.CLIENT_API_SSL_KSTR_NAME.getKey());
- String httpsTruststore = configsMap.get(Configuration.CLIENT_API_SSL_KSTR_DIR_NAME.getKey()) +
- File.separator + configsMap.get(Configuration.CLIENT_API_SSL_TSTR_NAME.getKey());
- LOG.info("API SSL Authentication is turned on. Keystore - " + httpsKeystore);
-
- String httpsCrtPass = configsMap.get(Configuration.CLIENT_API_SSL_CRT_PASS.getKey());
-
- SslContextFactory contextFactoryApi = new SslContextFactory();
- disableInsecureProtocols(contextFactoryApi);
- SslSelectChannelConnector sapiConnector = new SslSelectChannelConnector(contextFactoryApi);
- sapiConnector.setPort(configs.getClientSSLApiPort());
- sapiConnector.setKeystore(httpsKeystore);
- sapiConnector.setTruststore(httpsTruststore);
- sapiConnector.setPassword(httpsCrtPass);
- sapiConnector.setKeyPassword(httpsCrtPass);
- sapiConnector.setTrustPassword(httpsCrtPass);
- sapiConnector.setKeystoreType(configsMap.get(Configuration.CLIENT_API_SSL_KSTR_TYPE.getKey()));
- sapiConnector.setTruststoreType(configsMap.get(Configuration.CLIENT_API_SSL_KSTR_TYPE.getKey()));
- sapiConnector.setMaxIdleTime(configs.getConnectionMaxIdleTime());
- apiConnector = sapiConnector;
- } else {
- apiConnector = new SelectChannelConnector();
- apiConnector.setPort(configs.getClientApiPort());
- apiConnector.setMaxIdleTime(configs.getConnectionMaxIdleTime());
- }
+ SelectChannelConnector apiConnector = createSelectChannelConnectorForClient();
- apiConnector.setRequestHeaderSize(configs.getHttpRequestHeaderSize());
- apiConnector.setResponseHeaderSize(configs.getHttpResponseHeaderSize());
+ acceptors = configs.getClientApiAcceptors() != null ? configs
+ .getClientApiAcceptors() : apiConnector.getAcceptors();
+ apiConnector.setAcceptors(acceptors);
- // Client Jetty thread pool
- configureJettyThreadPool(server, apiConnector.getAcceptors(), CLIENT_THREAD_POOL_NAME, configs.getClientThreadPoolSize());
+ // Client Jetty thread pool - widen the thread pool if needed !
+ configureJettyThreadPool(server, acceptors, CLIENT_THREAD_POOL_NAME,
+ configs.getClientThreadPoolSize());
server.addConnector(apiConnector);
server.setStopAtShutdown(true);
@@ -662,6 +590,93 @@ public class AmbariServer {
}
/**
+ * Create org.eclipse.jetty.server.nio.SelectChannelConnector
+ * implementation for SSL or non-SSL based on configuration.
+ *
+ * @param port connector port
+ * @param needClientAuth one-way / two-way SSL
+ * @return org.eclipse.jetty.server.nio.SelectChannelConnector
+ */
+ @SuppressWarnings("deprecation")
+ private SelectChannelConnector createSelectChannelConnectorForAgent(int port, boolean needClientAuth) {
+ Map<String, String> configsMap = configs.getConfigsMap();
+ SelectChannelConnector agentConnector;
+
+ if (configs.getAgentSSLAuthentication()) {
+ String keystore = configsMap.get(Configuration.SRVR_KSTR_DIR.getKey()) + File.separator
+ + configsMap.get(Configuration.KSTR_NAME.getKey());
+
+ String truststore = configsMap.get(Configuration.SRVR_KSTR_DIR.getKey()) + File.separator
+ + configsMap.get(Configuration.TSTR_NAME.getKey());
+
+ String srvrCrtPass = configsMap.get(Configuration.SRVR_CRT_PASS.getKey());
+
+ // Secured connector - default constructor sets trustAll = true for certs
+ SslContextFactory contextFactory = new SslContextFactory();
+ disableInsecureProtocols(contextFactory);
+
+ SslSelectChannelConnector agentSslConnector = new SslSelectChannelConnector(contextFactory);
+ agentSslConnector.setKeystore(keystore);
+ agentSslConnector.setTruststore(truststore);
+ agentSslConnector.setPassword(srvrCrtPass);
+ agentSslConnector.setKeyPassword(srvrCrtPass);
+ agentSslConnector.setTrustPassword(srvrCrtPass);
+ agentSslConnector.setKeystoreType(configsMap.get(Configuration.KSTR_TYPE.getKey()));
+ agentSslConnector.setTruststoreType(configsMap.get(Configuration.TSTR_TYPE.getKey()));
+ agentSslConnector.setNeedClientAuth(needClientAuth);
+ agentSslConnector.setRequestHeaderSize(configs.getHttpRequestHeaderSize());
+ agentSslConnector.setResponseHeaderSize(configs.getHttpResponseHeaderSize());
+ agentConnector = agentSslConnector;
+ } else {
+ agentConnector = new SelectChannelConnector();
+ agentConnector.setMaxIdleTime(configs.getConnectionMaxIdleTime());
+ }
+
+ agentConnector.setPort(port);
+
+ return agentConnector;
+ }
+
+ @SuppressWarnings("deprecation")
+ private SelectChannelConnector createSelectChannelConnectorForClient() {
+ Map<String, String> configsMap = configs.getConfigsMap();
+ SelectChannelConnector apiConnector;
+
+ if (configs.getApiSSLAuthentication()) {
+ String httpsKeystore = configsMap.get(Configuration.CLIENT_API_SSL_KSTR_DIR_NAME.getKey()) +
+ File.separator + configsMap.get(Configuration.CLIENT_API_SSL_KSTR_NAME.getKey());
+ String httpsTruststore = configsMap.get(Configuration.CLIENT_API_SSL_KSTR_DIR_NAME.getKey()) +
+ File.separator + configsMap.get(Configuration.CLIENT_API_SSL_TSTR_NAME.getKey());
+ LOG.info("API SSL Authentication is turned on. Keystore - " + httpsKeystore);
+
+ String httpsCrtPass = configsMap.get(Configuration.CLIENT_API_SSL_CRT_PASS.getKey());
+
+ SslContextFactory contextFactoryApi = new SslContextFactory();
+ disableInsecureProtocols(contextFactoryApi);
+ SslSelectChannelConnector sapiConnector = new SslSelectChannelConnector(contextFactoryApi);
+ sapiConnector.setPort(configs.getClientSSLApiPort());
+ sapiConnector.setKeystore(httpsKeystore);
+ sapiConnector.setTruststore(httpsTruststore);
+ sapiConnector.setPassword(httpsCrtPass);
+ sapiConnector.setKeyPassword(httpsCrtPass);
+ sapiConnector.setTrustPassword(httpsCrtPass);
+ sapiConnector.setKeystoreType(configsMap.get(Configuration.CLIENT_API_SSL_KSTR_TYPE.getKey()));
+ sapiConnector.setTruststoreType(configsMap.get(Configuration.CLIENT_API_SSL_KSTR_TYPE.getKey()));
+ sapiConnector.setMaxIdleTime(configs.getConnectionMaxIdleTime());
+ apiConnector = sapiConnector;
+ } else {
+ apiConnector = new SelectChannelConnector();
+ apiConnector.setPort(configs.getClientApiPort());
+ apiConnector.setMaxIdleTime(configs.getConnectionMaxIdleTime());
+ }
+
+ apiConnector.setRequestHeaderSize(configs.getHttpRequestHeaderSize());
+ apiConnector.setResponseHeaderSize(configs.getHttpResponseHeaderSize());
+
+ return apiConnector;
+ }
+
+ /**
* this method executes database consistency check if skip option was not added
*/
protected void runDatabaseConsistencyCheck() throws Exception {
@@ -770,7 +785,7 @@ public class AmbariServer {
private void disableInsecureProtocols(SslContextFactory factory) {
// by default all protocols should be available
factory.setExcludeProtocols();
- factory.setIncludeProtocols(new String[] { "SSLv2Hello","TLSv1"});
+ factory.setIncludeProtocols(new String[] { "SSLv2Hello", "TLSv1" });
if (!configs.getSrvrDisabledCiphers().isEmpty()) {
String[] masks = configs.getSrvrDisabledCiphers().split(DISABLED_ENTRIES_SPLITTER);
http://git-wip-us.apache.org/repos/asf/ambari/blob/4d7d3cdc/ambari-server/src/main/java/org/apache/ambari/server/security/unsecured/rest/ConnectionInfo.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/security/unsecured/rest/ConnectionInfo.java b/ambari-server/src/main/java/org/apache/ambari/server/security/unsecured/rest/ConnectionInfo.java
index a1b5a6e..cb25f0b 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/security/unsecured/rest/ConnectionInfo.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/security/unsecured/rest/ConnectionInfo.java
@@ -42,7 +42,7 @@ public class ConnectionInfo {
@Inject
public static void init(Configuration instance){
conf = instance;
- response.put(Configuration.SRVR_TWO_WAY_SSL.getKey(),String.valueOf(conf.getTwoWaySsl()));
+ response.put(Configuration.SRVR_TWO_WAY_SSL.getKey(),String.valueOf(conf.isTwoWaySsl()));
}
@GET
http://git-wip-us.apache.org/repos/asf/ambari/blob/4d7d3cdc/ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java b/ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
index b0bcc58..d7cb8a3 100644
--- a/ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
+++ b/ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
@@ -119,7 +119,7 @@ public class ConfigurationTest {
*/
@Test
public void testDefaultTwoWayAuthNotSet() throws Exception {
- Assert.assertFalse(new Configuration().getTwoWaySsl());
+ Assert.assertFalse(new Configuration().isTwoWaySsl());
}
/**
@@ -131,7 +131,7 @@ public class ConfigurationTest {
Properties ambariProperties = new Properties();
ambariProperties.setProperty("security.server.two_way_ssl", "true");
Configuration conf = new Configuration(ambariProperties);
- Assert.assertTrue(conf.getTwoWaySsl());
+ Assert.assertTrue(conf.isTwoWaySsl());
}
/**
@@ -143,7 +143,7 @@ public class ConfigurationTest {
Properties ambariProperties = new Properties();
ambariProperties.setProperty("security.server.two_way_ssl", "false");
Configuration conf = new Configuration(ambariProperties);
- Assert.assertFalse(conf.getTwoWaySsl());
+ Assert.assertFalse(conf.isTwoWaySsl());
}
@Test
[08/31] ambari git commit: AMBARI-18893. Remove redundant css (part
2) (onechiporenko)
Posted by nc...@apache.org.
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/test/mixins/common/hosts/host_component_validation_mixin_test.js
----------------------------------------------------------------------
diff --git a/ambari-web/test/mixins/common/hosts/host_component_validation_mixin_test.js b/ambari-web/test/mixins/common/hosts/host_component_validation_mixin_test.js
index 7cbe232..9589f90 100644
--- a/ambari-web/test/mixins/common/hosts/host_component_validation_mixin_test.js
+++ b/ambari-web/test/mixins/common/hosts/host_component_validation_mixin_test.js
@@ -17,6 +17,7 @@
*/
var App = require('app');
+var helpers = require('test/helpers');
describe('App.HostComponentValidationMixin', function() {
var mixedObject;
@@ -25,7 +26,7 @@ describe('App.HostComponentValidationMixin', function() {
});
describe('#formatValidateComponents', function() {
- var cases = [
+ [
{
components: undefined,
hosts: [],
@@ -51,9 +52,7 @@ describe('App.HostComponentValidationMixin', function() {
Em.Object.create({ componentName: 'C2', hostName: 'h3'}),
]
}
- ];
-
- cases.forEach(function(test) {
+ ].forEach(function(test) {
it(test.m, function() {
expect(mixedObject.formatValidateComponents(test.components, test.hosts)).to.be.eql(test.e);
});
@@ -61,7 +60,6 @@ describe('App.HostComponentValidationMixin', function() {
});
describe('#validateSelectedHostComponents', function() {
- var cases;
beforeEach(function() {
sinon.stub(mixedObject, 'getHostComponentValidationRequest');
sinon.stub(mixedObject, 'getHostComponentValidationParams');
@@ -74,7 +72,7 @@ describe('App.HostComponentValidationMixin', function() {
mixedObject.formatValidateComponents.restore();
});
- cases = [
+ [
{
opts: null,
e: {
@@ -100,9 +98,7 @@ describe('App.HostComponentValidationMixin', function() {
},
m: 'should normaly merge defaults with passed options'
}
- ];
-
- cases.forEach(function(test) {
+ ].forEach(function(test) {
it(test.m, function() {
mixedObject.validateSelectedHostComponents(test.opts);
expect(mixedObject.getHostComponentValidationParams.args[0][0]).to.be.eql(test.e);
@@ -113,7 +109,8 @@ describe('App.HostComponentValidationMixin', function() {
describe('#getHostComponentValidationRequest', function() {
it('default request options checking', function() {
mixedObject.getHostComponentValidationRequest('someData');
- expect(App.ajax.send.args[0][0]).to.be.eql({
+ var args = helpers.findAjaxRequest('name', 'config.validations');
+ expect(args[0]).to.be.eql({
name: 'config.validations',
sender: mixedObject,
data: 'someData',
@@ -124,14 +121,13 @@ describe('App.HostComponentValidationMixin', function() {
});
describe('#getHostComponentValidationParams', function() {
- var cases;
beforeEach(function() {
sinon.stub(App, 'get').withArgs('stackVersionURL').returns('/stack/url');
});
afterEach(function() {
App.get.restore();
});
- cases = [
+ [
{
options: {
hosts: ['h1'],
@@ -177,9 +173,7 @@ describe('App.HostComponentValidationMixin', function() {
},
m: 'when blueprint passed it should be used instead of generated blueprint'
}
- ];
-
- cases.forEach(function(test) {
+ ].forEach(function(test) {
it(test.m, function() {
expect(mixedObject.getHostComponentValidationParams(test.options)).to.be.eql(test.e);
});
[24/31] ambari git commit: AMBARI-18876. Remove old UpgradeCatalog
and tests (aonishuk)
Posted by nc...@apache.org.
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog151Test.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog151Test.java b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog151Test.java
deleted file mode 100644
index 3b82625..0000000
--- a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog151Test.java
+++ /dev/null
@@ -1,120 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.ambari.server.upgrade;
-
-import com.google.inject.Binder;
-import com.google.inject.Guice;
-import com.google.inject.Injector;
-import com.google.inject.Module;
-
-import org.apache.ambari.server.configuration.Configuration;
-import org.apache.ambari.server.orm.DBAccessor;
-import org.apache.ambari.server.orm.dao.MetainfoDAO;
-import org.apache.ambari.server.state.stack.OsFamily;
-import org.easymock.EasyMock;
-import org.junit.Assert;
-import org.junit.Ignore;
-import org.junit.Test;
-
-import java.lang.reflect.Field;
-import java.util.List;
-
-import static org.easymock.EasyMock.eq;
-import static org.easymock.EasyMock.createNiceMock;
-import static org.easymock.EasyMock.expect;
-import static org.easymock.EasyMock.replay;
-import static org.easymock.EasyMock.verify;
-
-/**
- * UpgradeCatalog151 tests.
- * @Deprecated
- */
-@Ignore
-public class UpgradeCatalog151Test {
-
-
- @Test
- public void testExecuteDDLUpdates() throws Exception {
-
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
-
- Configuration configuration = createNiceMock(Configuration.class);
- expect(configuration.getDatabaseUrl()).andReturn(Configuration.JDBC_IN_MEMORY_URL).anyTimes();
-
- dbAccessor.createTable(eq("viewmain"), EasyMock.<List<DBAccessor.DBColumnInfo>>anyObject(), eq("view_name"));
- dbAccessor.createTable(eq("viewinstancedata"), EasyMock.<List<DBAccessor.DBColumnInfo>>anyObject(),
- eq("view_name"), eq("view_instance_name"), eq("name"));
- dbAccessor.createTable(eq("viewinstance"), EasyMock.<List<DBAccessor.DBColumnInfo>>anyObject(),
- eq("view_name"), eq("name"));
- dbAccessor.createTable(eq("viewinstanceproperty"), EasyMock.<List<DBAccessor.DBColumnInfo>>anyObject(),
- eq("view_name"), eq("view_instance_name"), eq("name"));
- dbAccessor.createTable(eq("viewparameter"), EasyMock.<List<DBAccessor.DBColumnInfo>>anyObject(),
- eq("view_name"), eq("name"));
- dbAccessor.createTable(eq("viewresource"), EasyMock.<List<DBAccessor.DBColumnInfo>>anyObject(),
- eq("view_name"), eq("name"));
-
- dbAccessor.addFKConstraint("viewparameter", "FK_viewparam_view_name", "view_name", "viewmain", "view_name", true);
- dbAccessor.addFKConstraint("viewresource", "FK_viewres_view_name", "view_name", "viewmain", "view_name", true);
- dbAccessor.addFKConstraint("viewinstance", "FK_viewinst_view_name", "view_name", "viewmain", "view_name", true);
-
- replay(dbAccessor, configuration);
- AbstractUpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
- Class<?> c = AbstractUpgradeCatalog.class;
- Field f = c.getDeclaredField("configuration");
- f.setAccessible(true);
- f.set(upgradeCatalog, configuration);
-
- upgradeCatalog.executeDDLUpdates();
- verify(dbAccessor, configuration);
- }
-
- @Test
- public void testExecuteDMLUpdates() throws Exception {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- UpgradeCatalog151 upgradeCatalog = (UpgradeCatalog151) getUpgradeCatalog(dbAccessor);
-
- upgradeCatalog.executeDMLUpdates();
- }
-
- @Test
- public void testGetTargetVersion() throws Exception {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- UpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
-
- Assert.assertEquals("1.5.1", upgradeCatalog.getTargetVersion());
- }
-
- private AbstractUpgradeCatalog getUpgradeCatalog(final DBAccessor dbAccessor) {
- Module module = new Module() {
- @Override
- public void configure(Binder binder) {
- binder.bind(DBAccessor.class).toInstance(dbAccessor);
- binder.bind(OsFamily.class).toInstance(createNiceMock(OsFamily.class));
- }
- };
- Injector injector = Guice.createInjector(module);
- return injector.getInstance(UpgradeCatalog151.class);
- }
-
- @Test
- public void testGetSourceVersion() {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- UpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
- Assert.assertEquals("1.5.0", upgradeCatalog.getSourceVersion());
- }
-}
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog160Test.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog160Test.java b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog160Test.java
deleted file mode 100644
index 754999b..0000000
--- a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog160Test.java
+++ /dev/null
@@ -1,264 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ambari.server.upgrade;
-
-import static junit.framework.Assert.assertEquals;
-import static junit.framework.Assert.assertFalse;
-import static junit.framework.Assert.assertNull;
-import static junit.framework.Assert.assertTrue;
-import static org.easymock.EasyMock.aryEq;
-import static org.easymock.EasyMock.capture;
-import static org.easymock.EasyMock.createMockBuilder;
-import static org.easymock.EasyMock.createNiceMock;
-import static org.easymock.EasyMock.eq;
-import static org.easymock.EasyMock.expect;
-import static org.easymock.EasyMock.expectLastCall;
-import static org.easymock.EasyMock.replay;
-import static org.easymock.EasyMock.verify;
-
-import java.lang.reflect.Field;
-import java.lang.reflect.Method;
-import java.sql.SQLException;
-import java.util.Collections;
-import java.util.List;
-import java.util.Map;
-
-import org.apache.ambari.server.configuration.Configuration;
-import org.apache.ambari.server.orm.DBAccessor;
-import org.apache.ambari.server.state.stack.OsFamily;
-import org.easymock.Capture;
-import org.junit.Assert;
-import org.junit.Ignore;
-import org.junit.Test;
-
-import com.google.inject.Binder;
-import com.google.inject.Guice;
-import com.google.inject.Injector;
-import com.google.inject.Module;
-
-/**
- * UpgradeCatalog160 unit tests.
- * @Deprecated
- */
-@Ignore
-public class UpgradeCatalog160Test {
-
- @Test
- public void testExecuteDDLUpdates() throws Exception {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- Configuration configuration = createNiceMock(Configuration.class);
- Capture<List<DBAccessor.DBColumnInfo>> hgConfigcolumnCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> viewEntitycolumnCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<DBAccessor.DBColumnInfo> restartRequiredColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
-
- expect(configuration.getDatabaseUrl()).andReturn(Configuration.JDBC_IN_MEMORY_URL).anyTimes();
-
- setBPHostGroupConfigExpectations(dbAccessor, hgConfigcolumnCapture, restartRequiredColumnCapture);
- setViewEntityConfigExpectations(dbAccessor, viewEntitycolumnCapture);
-
- replay(dbAccessor, configuration);
- AbstractUpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
- Class<?> c = AbstractUpgradeCatalog.class;
- Field f = c.getDeclaredField("configuration");
- f.setAccessible(true);
- f.set(upgradeCatalog, configuration);
-
- upgradeCatalog.executeDDLUpdates();
- verify(dbAccessor, configuration);
-
- assertHGConfigColumns(hgConfigcolumnCapture);
- assertViewEntityColumns(viewEntitycolumnCapture);
- assertRestartRequiredColumn(restartRequiredColumnCapture);
- }
-
- @Test
- public void testExecuteDMLUpdates() throws Exception {
- Configuration configuration = createNiceMock(Configuration.class);
- DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
-
- Method m = AbstractUpgradeCatalog.class.getDeclaredMethod
- ("updateConfigurationProperties", String.class, Map.class, boolean.class, boolean.class);
-
- UpgradeCatalog160 upgradeCatalog = createMockBuilder(UpgradeCatalog160.class)
- .addMockedMethod(m).createMock();
-
- expect(configuration.getDatabaseUrl()).andReturn(Configuration.JDBC_IN_MEMORY_URL).anyTimes();
-
- upgradeCatalog.updateConfigurationProperties("global",
- Collections.singletonMap("jobhistory_heapsize", "900"), false, false);
-
- expectLastCall();
-
- replay(upgradeCatalog, dbAccessor, configuration);
-
- Class<?> c = AbstractUpgradeCatalog.class;
- Field f = c.getDeclaredField("configuration");
- f.setAccessible(true);
- f.set(upgradeCatalog, configuration);
- f = c.getDeclaredField("dbAccessor");
- f.setAccessible(true);
- f.set(upgradeCatalog, dbAccessor);
-
- upgradeCatalog.executeDMLUpdates();
-
- verify(upgradeCatalog, dbAccessor, configuration);
- }
-
- @Test
- public void testGetTargetVersion() throws Exception {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- UpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
-
- Assert.assertEquals("1.6.0", upgradeCatalog.getTargetVersion());
- }
-
- private AbstractUpgradeCatalog getUpgradeCatalog(final DBAccessor dbAccessor) {
- Module module = new Module() {
- @Override
- public void configure(Binder binder) {
- binder.bind(DBAccessor.class).toInstance(dbAccessor);
- binder.bind(OsFamily.class).toInstance(createNiceMock(OsFamily.class));
- }
- };
- Injector injector = Guice.createInjector(module);
- return injector.getInstance(UpgradeCatalog160.class);
- }
-
- private void setBPHostGroupConfigExpectations(DBAccessor dbAccessor,
- Capture<List<DBAccessor.DBColumnInfo>> columnCapture,
- Capture<DBAccessor.DBColumnInfo> restartRequiredColumnCapture) throws SQLException {
-
- dbAccessor.createTable(eq("hostgroup_configuration"), capture(columnCapture),
- eq("blueprint_name"), eq("hostgroup_name"), eq("type_name"));
-
- dbAccessor.addColumn(eq("hostcomponentdesiredstate"),
- capture(restartRequiredColumnCapture));
-
- dbAccessor.addFKConstraint(eq("hostgroup_configuration"), eq("FK_hg_config_blueprint_name"),
- aryEq(new String[]{"blueprint_name", "hostgroup_name"}), eq("hostgroup"), aryEq(new String[]{"blueprint_name", "name"}), eq(true));
- }
-
- private void setViewEntityConfigExpectations(DBAccessor dbAccessor,
- Capture<List<DBAccessor.DBColumnInfo>> columnCapture) throws SQLException {
-
- dbAccessor.createTable(eq("viewentity"), capture(columnCapture), eq("id"));
- }
-
- private void assertHGConfigColumns(Capture<List<DBAccessor.DBColumnInfo>> hgConfigcolumnCapture) {
- List<DBAccessor.DBColumnInfo> columns = hgConfigcolumnCapture.getValue();
- assertEquals(4, columns.size());
- DBAccessor.DBColumnInfo column = columns.get(0);
- assertEquals("blueprint_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertFalse(column.isNullable());
-
- column = columns.get(1);
- assertEquals("hostgroup_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertFalse(column.isNullable());
-
- column = columns.get(2);
- assertEquals("type_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertFalse(column.isNullable());
-
- column = columns.get(3);
- assertEquals("config_data", column.getName());
- assertEquals(null, column.getLength());
- assertEquals(char[].class, column.getType());
- assertNull(column.getDefaultValue());
- assertFalse(column.isNullable());
- }
-
- private void assertViewEntityColumns(Capture<List<DBAccessor.DBColumnInfo>> hgConfigcolumnCapture) {
- List<DBAccessor.DBColumnInfo> columns = hgConfigcolumnCapture.getValue();
- assertEquals(5, columns.size());
-
-
- columns.add(new DBAccessor.DBColumnInfo("id", Long.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("view_instance_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("class_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("id_property", String.class, 255, null, true));
-
-
- DBAccessor.DBColumnInfo column = columns.get(0);
- assertEquals("id", column.getName());
- assertNull(column.getLength());
- assertEquals(Long.class, column.getType());
- assertNull(column.getDefaultValue());
- assertFalse(column.isNullable());
-
- column = columns.get(1);
- assertEquals("view_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertFalse(column.isNullable());
-
- column = columns.get(2);
- assertEquals("view_instance_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertFalse(column.isNullable());
-
- column = columns.get(3);
- assertEquals("class_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertFalse(column.isNullable());
-
- column = columns.get(4);
- assertEquals("id_property", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
- }
-
- @Test
- public void testGetSourceVersion() {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- UpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
- Assert.assertEquals("1.5.1", upgradeCatalog.getSourceVersion());
- }
- /**
- * Checks that the restart_require column was created correct when using a
- * non-Postgres DB (MySQL, Oracle, etc).
- *
- * @param restartRequiredColumnCapture
- */
- private void assertRestartRequiredColumn(
- Capture<DBAccessor.DBColumnInfo> restartRequiredColumnCapture) {
- DBAccessor.DBColumnInfo column = restartRequiredColumnCapture.getValue();
- assertEquals("restart_required", column.getName());
- assertEquals(1, (int) column.getLength());
- assertEquals(Integer.class, column.getType());
- assertEquals(0, column.getDefaultValue());
- assertFalse(column.isNullable());
- }
-}
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog161Test.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog161Test.java b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog161Test.java
deleted file mode 100644
index a87a376..0000000
--- a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog161Test.java
+++ /dev/null
@@ -1,418 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ambari.server.upgrade;
-
-import static junit.framework.Assert.assertEquals;
-import static junit.framework.Assert.assertFalse;
-import static junit.framework.Assert.assertNull;
-import static junit.framework.Assert.assertTrue;
-import static org.easymock.EasyMock.anyObject;
-import static org.easymock.EasyMock.capture;
-import static org.easymock.EasyMock.createMock;
-import static org.easymock.EasyMock.createMockBuilder;
-import static org.easymock.EasyMock.createNiceMock;
-import static org.easymock.EasyMock.createStrictMock;
-import static org.easymock.EasyMock.eq;
-import static org.easymock.EasyMock.expect;
-import static org.easymock.EasyMock.expectLastCall;
-import static org.easymock.EasyMock.replay;
-import static org.easymock.EasyMock.verify;
-
-import java.lang.reflect.Field;
-import java.lang.reflect.Method;
-import java.sql.Connection;
-import java.sql.ResultSet;
-import java.sql.SQLException;
-import java.sql.Statement;
-import java.util.Collections;
-import java.util.List;
-import java.util.Map;
-
-import javax.persistence.EntityManager;
-import javax.persistence.EntityTransaction;
-import javax.persistence.TypedQuery;
-
-import org.apache.ambari.server.configuration.Configuration;
-import org.apache.ambari.server.orm.DBAccessor;
-import org.apache.ambari.server.orm.entities.ClusterEntity;
-import org.apache.ambari.server.state.State;
-import org.apache.ambari.server.state.stack.OsFamily;
-import org.easymock.Capture;
-import org.junit.Assert;
-import org.junit.Ignore;
-import org.junit.Test;
-
-import com.google.inject.Binder;
-import com.google.inject.Guice;
-import com.google.inject.Injector;
-import com.google.inject.Module;
-import com.google.inject.Provider;
-
-/**
- * UpgradeCatalog161 unit tests.
- * @Deprecated
- */
-@Ignore
-public class UpgradeCatalog161Test {
-
- @Test
- public void testExecuteDDLUpdates() throws Exception {
-
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- Connection connection = createNiceMock(Connection.class);
- Statement statement = createNiceMock(Statement.class);
- Configuration configuration = createNiceMock(Configuration.class);
- ResultSet resultSet = createNiceMock(ResultSet.class);
- expect(configuration.getDatabaseUrl()).andReturn(Configuration.JDBC_IN_MEMORY_URL).anyTimes();
-
- Capture<DBAccessor.DBColumnInfo> provisioningStateColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<List<DBAccessor.DBColumnInfo>> operationLevelEntityColumnCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<DBAccessor.DBColumnInfo> labelColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> descriptionColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> visibleColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> viewIconColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> viewIcon64ColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> instanceIconColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> instanceIcon64ColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
-
- setClustersConfigExpectations(dbAccessor, provisioningStateColumnCapture);
- setOperationLevelEntityConfigExpectations(dbAccessor, operationLevelEntityColumnCapture);
- setViewExpectations(dbAccessor, viewIconColumnCapture, viewIcon64ColumnCapture);
- dbAccessor.addColumn(eq("viewinstance"),
- anyObject(DBAccessor.DBColumnInfo.class));
- setViewInstanceExpectations(dbAccessor, labelColumnCapture, descriptionColumnCapture, visibleColumnCapture, instanceIconColumnCapture, instanceIcon64ColumnCapture);
- dbAccessor.getConnection();
- expectLastCall().andReturn(connection).anyTimes();
- connection.createStatement();
- expectLastCall().andReturn(statement).anyTimes();
- statement.executeQuery(anyObject(String.class));
- expectLastCall().andReturn(resultSet).anyTimes();
- resultSet.next();
- expectLastCall().andReturn(false).anyTimes();
- resultSet.close();
- expectLastCall().anyTimes();
-
- replay(dbAccessor, configuration, resultSet, statement, connection);
- AbstractUpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
- Class<?> c = AbstractUpgradeCatalog.class;
- Field f = c.getDeclaredField("configuration");
- f.setAccessible(true);
- f.set(upgradeCatalog, configuration);
-
- upgradeCatalog.executeDDLUpdates();
- verify(dbAccessor, configuration, resultSet, statement, connection);
-
- assertClusterColumns(provisioningStateColumnCapture);
- assertOperationLevelEntityColumns(operationLevelEntityColumnCapture);
- assertViewColumns(viewIconColumnCapture, viewIcon64ColumnCapture);
- assertViewInstanceColumns(labelColumnCapture, descriptionColumnCapture, visibleColumnCapture, instanceIconColumnCapture, instanceIcon64ColumnCapture);
- }
-
- @SuppressWarnings("unchecked")
- @Test
- public void testExecuteDMLUpdates() throws Exception {
- Configuration configuration = createNiceMock(Configuration.class);
- DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- Injector injector = createStrictMock(Injector.class);
- Provider provider = createStrictMock(Provider.class);
- EntityManager em = createStrictMock(EntityManager.class);
- EntityTransaction et = createMock(EntityTransaction.class);
- TypedQuery query = createMock(TypedQuery.class);
-
- Method m = AbstractUpgradeCatalog.class.getDeclaredMethod
- ("updateConfigurationProperties", String.class, Map.class, boolean.class, boolean.class);
-
- UpgradeCatalog161 upgradeCatalog = createMockBuilder(UpgradeCatalog161.class)
- .addMockedMethod(m).createMock();
-
- expect(configuration.getDatabaseUrl()).andReturn(Configuration.JDBC_IN_MEMORY_URL).anyTimes();
- expect(injector.getProvider(EntityManager.class)).andReturn(provider).anyTimes();
- expect(provider.get()).andReturn(em).anyTimes();
- expect(em.getTransaction()).andReturn(et);
- expect(et.isActive()).andReturn(true);
- expect(em.createQuery("UPDATE ClusterEntity SET provisioningState = " +
- ":provisioningState", ClusterEntity.class)).andReturn(query);
- expect(query.setParameter("provisioningState", State.INSTALLED)).andReturn(null);
- expect(query.executeUpdate()).andReturn(0);
-
- upgradeCatalog.updateConfigurationProperties("hbase-site",
- Collections.singletonMap("hbase.regionserver.info.port", "60030"), false, false);
- expectLastCall();
-
- upgradeCatalog.updateConfigurationProperties("hbase-site",
- Collections.singletonMap("hbase.master.info.port", "60010"), false, false);
- expectLastCall();
-
- upgradeCatalog.updateConfigurationProperties("hive-site",
- Collections.singletonMap("hive.heapsize", "1024"), false, false);
- expectLastCall();
-
- upgradeCatalog.updateConfigurationProperties("pig-properties", Collections.singletonMap("pig-content", "\n# Licensed to the Apache " +
- "Software Foundation (ASF) under one\n# or more contributor license agreements. See the NOTICE file\n# " +
- "distributed with this work for additional information\n# regarding copyright ownership. The ASF " +
- "licenses this file\n# to you under the Apache License, Version 2.0 (the\n# \"License\"); you may " +
- "not use this file except in compliance\n# with the License. You may obtain a copy of the License " +
- "at\n#\n#http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to " +
- "in writing,\n# software distributed under the License is distributed on an\n# \"AS IS\" BASIS, WITHOUT " +
- "WARRANTIES OR CONDITIONS OF ANY\n# KIND, either express or implied. See the License for the\n# " +
- "specific language governing permissions and limitations\n# under the License.\n\n# Pig default " +
- "configuration file. All values can be overwritten by pig.properties and command line arguments.\n# " +
- "see bin/pig -help\n\n# brief logging (no timestamps)\nbrief=false\n\n# debug level, INFO is default" +
- "\ndebug=INFO\n\n# verbose print all log messages to screen (default to print only INFO and above to " +
- "screen)\nverbose=false\n\n# exectype local|mapreduce, mapreduce is default\nexectype=mapreduce\n\n# " +
- "Enable insertion of information about script into hadoop job conf \npig.script.info.enabled=true\n\n# " +
- "Do not spill temp files smaller than this size (bytes)\npig.spill.size.threshold=5000000\n\n# " +
- "EXPERIMENT: Activate garbage collection when spilling a file bigger than this size (bytes)\n# " +
- "This should help reduce the number of files being spilled.\npig.spill.gc.activation.size=40000000\n\n# " +
- "the following two parameters are to help estimate the reducer number\npig.exec.reducers.bytes.per." +
- "reducer=1000000000\npig.exec.reducers.max=999\n\n# Temporary location to store the intermediate " +
- "data.\npig.temp.dir=/tmp/\n\n# Threshold for merging FRJoin fragment files\npig.files.concatenation." +
- "threshold=100\npig.optimistic.files.concatenation=false;\n\npig.disable.counter=false\n\n" +
- "hcat.bin=/usr/bin/hcat"), true, false);
- expectLastCall();
-
- replay(upgradeCatalog, dbAccessor, configuration, injector, provider, em,
- et, query);
-
- Class<?> c = AbstractUpgradeCatalog.class;
- Field f = c.getDeclaredField("configuration");
- f.setAccessible(true);
- f.set(upgradeCatalog, configuration);
- f = c.getDeclaredField("dbAccessor");
- f.setAccessible(true);
- f.set(upgradeCatalog, dbAccessor);
- f = c.getDeclaredField("injector");
- f.setAccessible(true);
- f.set(upgradeCatalog, injector);
-
- upgradeCatalog.executeDMLUpdates();
-
- verify(upgradeCatalog, dbAccessor, configuration, injector, provider, em,
- et, query);
- }
-
-
- @Test
- public void testGetTargetVersion() throws Exception {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- UpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
-
- Assert.assertEquals("1.6.1", upgradeCatalog.getTargetVersion());
- }
-
- private AbstractUpgradeCatalog getUpgradeCatalog(final DBAccessor dbAccessor) {
- Module module = new Module() {
- @Override
- public void configure(Binder binder) {
- binder.bind(DBAccessor.class).toInstance(dbAccessor);
- binder.bind(OsFamily.class).toInstance(createNiceMock(OsFamily.class));
- }
- };
- Injector injector = Guice.createInjector(module);
- return injector.getInstance(UpgradeCatalog161.class);
- }
-
-
- private void setOperationLevelEntityConfigExpectations(DBAccessor dbAccessor,
- Capture<List<DBAccessor.DBColumnInfo>> operationLevelEntitycolumnCapture)
- throws SQLException {
-
- dbAccessor.createTable(eq("requestoperationlevel"),
- capture(operationLevelEntitycolumnCapture), eq("operation_level_id"));
-
- dbAccessor.addFKConstraint("requestoperationlevel", "FK_req_op_level_req_id",
- "request_id", "request", "request_id", true);
- }
-
-
- private void assertOperationLevelEntityColumns(Capture<List<DBAccessor.DBColumnInfo>> operationLevelEntitycolumnCapture) {
- List<DBAccessor.DBColumnInfo> columns = operationLevelEntitycolumnCapture.getValue();
- assertEquals(7, columns.size());
-
- DBAccessor.DBColumnInfo column = columns.get(0);
- assertEquals("operation_level_id", column.getName());
- assertNull(column.getLength());
- assertEquals(Long.class, column.getType());
- assertNull(column.getDefaultValue());
- assertFalse(column.isNullable());
-
- column = columns.get(1);
- assertEquals("request_id", column.getName());
- assertNull(column.getLength());
- assertEquals(Long.class, column.getType());
- assertNull(column.getDefaultValue());
- assertFalse(column.isNullable());
-
- column = columns.get(2);
- assertEquals("level_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = columns.get(3);
- assertEquals("cluster_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = columns.get(4);
- assertEquals("service_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = columns.get(5);
- assertEquals("host_component_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = columns.get(6);
- assertEquals("host_name", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
- }
-
- private void setClustersConfigExpectations(DBAccessor dbAccessor,
- Capture<DBAccessor.DBColumnInfo> provisioningStateColumnCapture) throws SQLException {
-
- dbAccessor.addColumn(eq("clusters"),
- capture(provisioningStateColumnCapture));
- }
-
- private void assertClusterColumns(
- Capture<DBAccessor.DBColumnInfo> provisiontStateColumnCapture) {
- DBAccessor.DBColumnInfo column = provisiontStateColumnCapture.getValue();
- assertEquals("provisioning_state", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertEquals(State.INIT.name(), column.getDefaultValue());
- assertFalse(column.isNullable());
- }
-
-
- private void setViewExpectations(DBAccessor dbAccessor,
- Capture<DBAccessor.DBColumnInfo> viewIconColumnCapture,
- Capture<DBAccessor.DBColumnInfo> viewIcon64ColumnCapture)
- throws SQLException {
-
- dbAccessor.addColumn(eq("viewmain"),
- capture(viewIconColumnCapture));
-
- dbAccessor.addColumn(eq("viewmain"),
- capture(viewIcon64ColumnCapture));
- }
-
- private void setViewInstanceExpectations(DBAccessor dbAccessor,
- Capture<DBAccessor.DBColumnInfo> labelColumnCapture,
- Capture<DBAccessor.DBColumnInfo> descriptionColumnCapture,
- Capture<DBAccessor.DBColumnInfo> visibleColumnCapture,
- Capture<DBAccessor.DBColumnInfo> instanceIconColumnCapture,
- Capture<DBAccessor.DBColumnInfo> instanceIcon64ColumnCapture)
- throws SQLException {
-
- dbAccessor.addColumn(eq("viewinstance"),
- capture(labelColumnCapture));
-
- dbAccessor.addColumn(eq("viewinstance"),
- capture(descriptionColumnCapture));
-
- dbAccessor.addColumn(eq("viewinstance"),
- capture(visibleColumnCapture));
-
- dbAccessor.addColumn(eq("viewinstance"),
- capture(instanceIconColumnCapture));
-
- dbAccessor.addColumn(eq("viewinstance"),
- capture(instanceIcon64ColumnCapture));
- }
-
- private void assertViewColumns(
- Capture<DBAccessor.DBColumnInfo> viewIconColumnCapture,
- Capture<DBAccessor.DBColumnInfo> viewIcon64ColumnCapture) {
- DBAccessor.DBColumnInfo column = viewIconColumnCapture.getValue();
- assertEquals("icon", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = viewIcon64ColumnCapture.getValue();
- assertEquals("icon64", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
- }
-
- private void assertViewInstanceColumns(
- Capture<DBAccessor.DBColumnInfo> labelColumnCapture,
- Capture<DBAccessor.DBColumnInfo> descriptionColumnCapture,
- Capture<DBAccessor.DBColumnInfo> visibleColumnCapture,
- Capture<DBAccessor.DBColumnInfo> instanceIconColumnCapture,
- Capture<DBAccessor.DBColumnInfo> instanceIcon64ColumnCapture) {
- DBAccessor.DBColumnInfo column = labelColumnCapture.getValue();
- assertEquals("label", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = descriptionColumnCapture.getValue();
- assertEquals("description", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = visibleColumnCapture.getValue();
- assertEquals("visible", column.getName());
- assertEquals(1, (int) column.getLength());
- assertEquals(Character.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = instanceIconColumnCapture.getValue();
- assertEquals("icon", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = instanceIcon64ColumnCapture.getValue();
- assertEquals("icon64", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
- }
-
- @Test
- public void testGetSourceVersion() {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- UpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
- Assert.assertEquals("1.6.0", upgradeCatalog.getSourceVersion());
- }
-}
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog170Test.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog170Test.java b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog170Test.java
deleted file mode 100644
index d13b40e..0000000
--- a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog170Test.java
+++ /dev/null
@@ -1,791 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ambari.server.upgrade;
-
-import static junit.framework.Assert.assertEquals;
-import static junit.framework.Assert.assertNotNull;
-import static junit.framework.Assert.assertNull;
-import static junit.framework.Assert.assertTrue;
-import static org.apache.ambari.server.upgrade.UpgradeCatalog170.CLUSTER_STATE_STACK_HDP_2_1;
-import static org.apache.ambari.server.upgrade.UpgradeCatalog170.JOBS_VIEW_NAME;
-import static org.apache.ambari.server.upgrade.UpgradeCatalog170.SHOW_JOBS_FOR_NON_ADMIN_KEY;
-import static org.apache.ambari.server.upgrade.UpgradeCatalog170.YARN_RESOURCEMANAGER_WEBAPP_ADDRESS_PROPERTY;
-import static org.apache.ambari.server.upgrade.UpgradeCatalog170.YARN_SITE;
-import static org.apache.ambari.server.upgrade.UpgradeCatalog170.YARN_TIMELINE_SERVICE_WEBAPP_ADDRESS_PROPERTY;
-import static org.easymock.EasyMock.anyObject;
-import static org.easymock.EasyMock.capture;
-import static org.easymock.EasyMock.createMockBuilder;
-import static org.easymock.EasyMock.createNiceMock;
-import static org.easymock.EasyMock.createStrictMock;
-import static org.easymock.EasyMock.eq;
-import static org.easymock.EasyMock.expect;
-import static org.easymock.EasyMock.expectLastCall;
-import static org.easymock.EasyMock.getCurrentArguments;
-import static org.easymock.EasyMock.isA;
-import static org.easymock.EasyMock.replay;
-import static org.easymock.EasyMock.reset;
-import static org.easymock.EasyMock.verify;
-
-import java.lang.reflect.Field;
-import java.lang.reflect.Method;
-import java.sql.Connection;
-import java.sql.PreparedStatement;
-import java.sql.ResultSet;
-import java.sql.SQLException;
-import java.sql.Statement;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-
-import javax.persistence.EntityManager;
-import javax.persistence.EntityTransaction;
-import javax.persistence.TypedQuery;
-import javax.persistence.criteria.CriteriaBuilder;
-import javax.persistence.criteria.CriteriaQuery;
-import javax.persistence.criteria.Order;
-import javax.persistence.criteria.Path;
-import javax.persistence.criteria.Predicate;
-import javax.persistence.criteria.Root;
-import javax.persistence.metamodel.SingularAttribute;
-
-import org.apache.ambari.server.AmbariException;
-import org.apache.ambari.server.api.services.AmbariMetaInfo;
-import org.apache.ambari.server.configuration.Configuration;
-import org.apache.ambari.server.controller.AmbariManagementController;
-import org.apache.ambari.server.orm.DBAccessor;
-import org.apache.ambari.server.orm.GuiceJpaInitializer;
-import org.apache.ambari.server.orm.InMemoryDefaultTestModule;
-import org.apache.ambari.server.orm.dao.ClusterDAO;
-import org.apache.ambari.server.orm.dao.ConfigGroupConfigMappingDAO;
-import org.apache.ambari.server.orm.dao.HostComponentDesiredStateDAO;
-import org.apache.ambari.server.orm.dao.HostComponentStateDAO;
-import org.apache.ambari.server.orm.dao.KeyValueDAO;
-import org.apache.ambari.server.orm.dao.PermissionDAO;
-import org.apache.ambari.server.orm.dao.PrincipalDAO;
-import org.apache.ambari.server.orm.dao.PrincipalTypeDAO;
-import org.apache.ambari.server.orm.dao.PrivilegeDAO;
-import org.apache.ambari.server.orm.dao.ResourceDAO;
-import org.apache.ambari.server.orm.dao.ResourceTypeDAO;
-import org.apache.ambari.server.orm.dao.ServiceComponentDesiredStateDAO;
-import org.apache.ambari.server.orm.dao.StackDAO;
-import org.apache.ambari.server.orm.dao.UserDAO;
-import org.apache.ambari.server.orm.dao.ViewDAO;
-import org.apache.ambari.server.orm.dao.ViewInstanceDAO;
-import org.apache.ambari.server.orm.entities.ClusterConfigEntity;
-import org.apache.ambari.server.orm.entities.ClusterConfigMappingEntity;
-import org.apache.ambari.server.orm.entities.ClusterEntity;
-import org.apache.ambari.server.orm.entities.ClusterServiceEntity;
-import org.apache.ambari.server.orm.entities.ClusterStateEntity;
-import org.apache.ambari.server.orm.entities.ConfigGroupConfigMappingEntity;
-import org.apache.ambari.server.orm.entities.HostComponentDesiredStateEntity;
-import org.apache.ambari.server.orm.entities.HostComponentDesiredStateEntityPK;
-import org.apache.ambari.server.orm.entities.HostEntity;
-import org.apache.ambari.server.orm.entities.HostRoleCommandEntity;
-import org.apache.ambari.server.orm.entities.KeyValueEntity;
-import org.apache.ambari.server.orm.entities.PermissionEntity;
-import org.apache.ambari.server.orm.entities.PrincipalEntity;
-import org.apache.ambari.server.orm.entities.PrivilegeEntity;
-import org.apache.ambari.server.orm.entities.ResourceEntity;
-import org.apache.ambari.server.orm.entities.ServiceComponentDesiredStateEntity;
-import org.apache.ambari.server.orm.entities.StackEntity;
-import org.apache.ambari.server.orm.entities.UserEntity;
-import org.apache.ambari.server.orm.entities.ViewEntity;
-import org.apache.ambari.server.orm.entities.ViewInstanceEntity;
-import org.apache.ambari.server.state.Cluster;
-import org.apache.ambari.server.state.Clusters;
-import org.apache.ambari.server.state.Config;
-import org.apache.ambari.server.state.ConfigHelper;
-import org.apache.ambari.server.state.StackId;
-import org.apache.ambari.server.state.stack.OsFamily;
-import org.apache.ambari.server.view.ViewRegistry;
-import org.easymock.Capture;
-import org.easymock.IAnswer;
-import org.junit.After;
-import org.junit.Assert;
-import org.junit.Before;
-import org.junit.Ignore;
-import org.junit.Test;
-
-import com.google.inject.Binder;
-import com.google.inject.Guice;
-import com.google.inject.Injector;
-import com.google.inject.Module;
-import com.google.inject.Provider;
-import com.google.inject.persist.PersistService;
-
-/**
- * UpgradeCatalog170 unit tests.
- * @Deprecated
- */
-@Ignore
-public class UpgradeCatalog170Test {
-
- private Injector injector;
- private final String CLUSTER_NAME = "c1";
- private final String SERVICE_NAME = "HDFS";
- private final String HOST_NAME = "h1";
-
- public static final StackId DESIRED_STACK = new StackId("HDP", "2.0.6");
-
- Provider<EntityManager> entityManagerProvider = createStrictMock(Provider.class);
- EntityManager entityManager = createNiceMock(EntityManager.class);
- private UpgradeCatalogHelper upgradeCatalogHelper;
-
- @Before
- public void init() {
- reset(entityManagerProvider);
- expect(entityManagerProvider.get()).andReturn(entityManager).anyTimes();
- replay(entityManagerProvider);
- injector = Guice.createInjector(new InMemoryDefaultTestModule());
- injector.getInstance(GuiceJpaInitializer.class);
- upgradeCatalogHelper = injector.getInstance(UpgradeCatalogHelper.class);
- }
-
- @After
- public void tearDown() {
- injector.getInstance(PersistService.class).stop();
- }
-
- @Test
- public void testExecuteDDLUpdates_DBAccessor() throws Exception {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- Statement statement = createNiceMock(Statement.class);
- Connection connection = createNiceMock(Connection.class);
- PreparedStatement stmt = createNiceMock(PreparedStatement.class);
- Configuration configuration = createNiceMock(Configuration.class);
- ResultSet resultSet = createNiceMock(ResultSet.class);
- expect(configuration.getDatabaseUrl()).andReturn(Configuration.JDBC_IN_MEMORY_URL).anyTimes();
- expect(dbAccessor.getNewConnection()).andReturn(connection);
- expect(connection.prepareStatement("SELECT config_id FROM clusterconfig " +
- "WHERE type_name = ? ORDER BY create_timestamp")).andReturn(stmt);
- expect(connection.prepareStatement("UPDATE clusterconfig SET version = ? " +
- "WHERE config_id = ?")).andReturn(stmt);
- stmt.close();
- expectLastCall().times(2);
- connection.close();
- expectLastCall();
-
- Capture<DBAccessor.DBColumnInfo> clusterConfigAttributesColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> hostgroupConfigAttributesColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> blueprintConfigAttributesColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> maskColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> systemColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> maskedColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> stageCommandParamsColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> stageHostParamsColumnCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<List<DBAccessor.DBColumnInfo>> groupsCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> alertDefinitionColumnCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> alertHistoryColumnCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> alertCurrentColumnCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> alertGroupColumnCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> alertTargetCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> alertGroupTargetCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> alertGroupingCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> alertNoticeCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> serviceConfigCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<List<DBAccessor.DBColumnInfo>> serviceConfigMappingCapture = new Capture<List<DBAccessor.DBColumnInfo>>();
- Capture<DBAccessor.DBColumnInfo> configDataClusterConfigCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> configDataBlueprintConfigurationCapture = new Capture<DBAccessor.DBColumnInfo>();
- Capture<DBAccessor.DBColumnInfo> configDataHostGroupConfigurationCapture = new Capture<DBAccessor.DBColumnInfo>();
-
- dbAccessor.createTable(eq("groups"),
- capture(groupsCapture), eq("group_id"));
-
- setViewExpectations(dbAccessor, maskColumnCapture, systemColumnCapture);
- setViewParameterExpectations(dbAccessor, maskedColumnCapture);
- setConfigAttributesColumnExpectations(dbAccessor, clusterConfigAttributesColumnCapture, "clusterconfig");
- setConfigAttributesColumnExpectations(dbAccessor, hostgroupConfigAttributesColumnCapture, "hostgroup_configuration");
- setConfigAttributesColumnExpectations(dbAccessor, blueprintConfigAttributesColumnCapture, "blueprint_configuration");
- setStageExpectations(dbAccessor, stageCommandParamsColumnCapture, stageHostParamsColumnCapture);
-
- dbAccessor.createTable(eq("alert_definition"),
- capture(alertDefinitionColumnCapture), eq("definition_id"));
-
- dbAccessor.createTable(eq("alert_history"),
- capture(alertHistoryColumnCapture), eq("alert_id"));
-
- dbAccessor.createTable(eq("alert_current"),
- capture(alertCurrentColumnCapture), eq("alert_id"));
-
- dbAccessor.createTable(eq("alert_group"), capture(alertGroupColumnCapture),
- eq("group_id"));
-
- dbAccessor.createTable(eq("alert_target"), capture(alertTargetCapture),
- eq("target_id"));
-
- dbAccessor.createTable(eq("alert_group_target"),
- capture(alertGroupTargetCapture), eq("group_id"), eq("target_id"));
-
- dbAccessor.createTable(eq("alert_grouping"), capture(alertGroupingCapture),
- eq("group_id"), eq("definition_id"));
-
- dbAccessor.createTable(eq("alert_notice"), capture(alertNoticeCapture),
- eq("notification_id"));
-
- dbAccessor.alterColumn(eq("clusterconfig"), capture(configDataClusterConfigCapture));
- dbAccessor.alterColumn(eq("blueprint_configuration"), capture(configDataBlueprintConfigurationCapture));
- dbAccessor.alterColumn(eq("hostgroup_configuration"), capture(configDataHostGroupConfigurationCapture));
-
- dbAccessor.createTable(eq("serviceconfig"), capture(serviceConfigCapture),
- eq("service_config_id"));
-
- dbAccessor.createTable(eq("serviceconfigmapping"),
- capture(serviceConfigMappingCapture), eq("service_config_id"),
- eq("config_id"));
- dbAccessor.getConnection();
- expectLastCall().andReturn(connection).anyTimes();
- connection.createStatement();
- expectLastCall().andReturn(statement).anyTimes();
- statement.executeQuery(anyObject(String.class));
- expectLastCall().andReturn(resultSet).anyTimes();
- resultSet.next();
- expectLastCall().andReturn(false).anyTimes();
- resultSet.close();
- expectLastCall().anyTimes();
-
- replay(dbAccessor, configuration, resultSet, connection, stmt, statement);
- AbstractUpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
- Class<?> c = AbstractUpgradeCatalog.class;
- Field f = c.getDeclaredField("configuration");
- f.setAccessible(true);
- f.set(upgradeCatalog, configuration);
-
- upgradeCatalog.executeDDLUpdates();
- verify(dbAccessor, configuration, resultSet, connection, stmt, statement);
-
- assertClusterConfigColumns(clusterConfigAttributesColumnCapture);
- assertHostgroupConfigColumns(hostgroupConfigAttributesColumnCapture);
- assertBlueprintConfigColumns(blueprintConfigAttributesColumnCapture);
- assertViewColumns(maskColumnCapture, systemColumnCapture);
- assertViewParameterColumns(maskedColumnCapture);
- assertStageColumns(stageCommandParamsColumnCapture, stageHostParamsColumnCapture);
-
- assertEquals(4, groupsCapture.getValue().size());
- List<DBAccessor.DBColumnInfo> columnInfoList = groupsCapture.getValue();
- for (DBAccessor.DBColumnInfo info : columnInfoList) {
- if (info.getName().equals("group_name")) {
- assertEquals(Integer.valueOf(255), info.getLength());
- break;
- }
- }
-
- assertEquals(12, alertDefinitionColumnCapture.getValue().size());
- assertEquals(11, alertHistoryColumnCapture.getValue().size());
- assertEquals(7, alertCurrentColumnCapture.getValue().size());
- assertEquals(5, alertGroupColumnCapture.getValue().size());
- assertEquals(5, alertTargetCapture.getValue().size());
- assertEquals(2, alertGroupTargetCapture.getValue().size());
- assertEquals(2, alertGroupingCapture.getValue().size());
- assertEquals(5, alertNoticeCapture.getValue().size());
- assertEquals(2, serviceConfigCapture.getValue().size());
- assertEquals(2, serviceConfigMappingCapture.getValue().size());
- }
-
- @Test
- public void testExecuteDMLUpdates() throws Exception {
- Configuration configuration = createNiceMock(Configuration.class);
- Connection connection = createNiceMock(Connection.class);
- Statement statement = createNiceMock(Statement.class);
- DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- Injector injector = createNiceMock(Injector.class);
- ConfigHelper configHelper = createNiceMock(ConfigHelper.class);
- AmbariManagementController amc = createNiceMock(AmbariManagementController.class);
- Cluster cluster = createNiceMock(Cluster.class);
- Clusters clusters = createStrictMock(Clusters.class);
- Config config = createStrictMock(Config.class);
- Config pigConfig = createStrictMock(Config.class);
- ViewRegistry viewRegistry = createNiceMock(ViewRegistry.class);
- PermissionEntity adminPermission = createNiceMock(PermissionEntity.class);
- PermissionEntity viewUsePermission = createNiceMock(PermissionEntity.class);
-
- ClusterConfigEntity clusterConfigEntity = createNiceMock(ClusterConfigEntity.class);
- ConfigGroupConfigMappingDAO configGroupConfigMappingDAO = createNiceMock(ConfigGroupConfigMappingDAO.class);
- UserDAO userDAO = createNiceMock(UserDAO.class);
- PrincipalDAO principalDAO = createNiceMock(PrincipalDAO.class);
- PrincipalTypeDAO principalTypeDAO = createNiceMock(PrincipalTypeDAO.class);
- ClusterDAO clusterDAO = createNiceMock(ClusterDAO.class);
- ResourceTypeDAO resourceTypeDAO = createNiceMock(ResourceTypeDAO.class);
- ResourceDAO resourceDAO = createNiceMock(ResourceDAO.class);
- ViewDAO viewDAO = createNiceMock(ViewDAO.class);
- ViewInstanceDAO viewInstanceDAO = createNiceMock(ViewInstanceDAO.class);
- PermissionDAO permissionDAO = createNiceMock(PermissionDAO.class);
- PrivilegeDAO privilegeDAO = createNiceMock(PrivilegeDAO.class);
- KeyValueDAO keyValueDAO = createNiceMock(KeyValueDAO.class);
-
- EntityTransaction trans = createNiceMock(EntityTransaction.class);
- CriteriaBuilder cb = createNiceMock(CriteriaBuilder.class);
- CriteriaQuery<HostRoleCommandEntity> cq = createNiceMock(CriteriaQuery.class);
- Root<HostRoleCommandEntity> hrc = createNiceMock(Root.class);
- Path<Long> taskId = null;
- Path<String> outputLog = null;
- Path<String> errorLog = null;
- Order o = createNiceMock(Order.class);
- TypedQuery<HostRoleCommandEntity> q = createNiceMock(TypedQuery.class);
- List<HostRoleCommandEntity> r = new ArrayList<HostRoleCommandEntity>();
- ResultSet userRolesResultSet = createNiceMock(ResultSet.class);
- ClusterEntity clusterEntity = createNiceMock(ClusterEntity.class);
- ClusterConfigEntity configEntity = createNiceMock(ClusterConfigEntity.class);
-
- Method m = AbstractUpgradeCatalog.class.getDeclaredMethod
- ("updateConfigurationProperties", String.class, Map.class, boolean.class, boolean.class);
- Method n = AbstractUpgradeCatalog.class.getDeclaredMethod("getEntityManagerProvider");
- Method l = AbstractUpgradeCatalog.class.getDeclaredMethod
- ("addNewConfigurationsFromXml");
- Method newPropMap = UpgradeCatalog170.class.getDeclaredMethod("getNewPropertyName");
-
- UpgradeCatalog170 upgradeCatalog = createMockBuilder(UpgradeCatalog170.class)
- .addMockedMethod(m).addMockedMethod(n).addMockedMethod(l).addMockedMethod(newPropMap)
- .createMock();
-
- List<ConfigGroupConfigMappingEntity> configGroupConfigMappingEntities =
- new ArrayList<ConfigGroupConfigMappingEntity>();
- ConfigGroupConfigMappingEntity configGroupConfigMappingEntity = new ConfigGroupConfigMappingEntity();
- configGroupConfigMappingEntity.setConfigType(Configuration.GLOBAL_CONFIG_TAG);
- configGroupConfigMappingEntity.setClusterConfigEntity(clusterConfigEntity);
- configGroupConfigMappingEntity.setClusterId(1L);
- configGroupConfigMappingEntities.add(configGroupConfigMappingEntity);
-
- Map<String, Cluster> clustersMap = new HashMap<String, Cluster>();
- clustersMap.put("c1", cluster);
-
- Map<String, String> globalConfigs = new HashMap<String, String>();
- globalConfigs.put("prop1", "val1");
- globalConfigs.put("smokeuser_keytab", "val2");
-
- Map<String, String> pigSettings = new HashMap<String, String>();
- pigSettings.put("pig-content", "foo");
-
- Set<String> envDicts = new HashSet<String>();
- envDicts.add("hadoop-env");
- envDicts.add("global");
-
- Set<String> configTypes = new HashSet<String>();
- configTypes.add("hadoop-env");
-
- Map<String, String> contentOfHadoopEnv = new HashMap<String, String>();
- contentOfHadoopEnv.put("content", "env file contents");
-
- upgradeCatalog.updateConfigurationProperties("hadoop-env",
- globalConfigs, false, true);
- expectLastCall();
-
- upgradeCatalog.addNewConfigurationsFromXml();
- expectLastCall();
-
- dbAccessor.getConnection();
- expectLastCall().andReturn(connection).anyTimes();
- connection.createStatement();
- expectLastCall().andReturn(statement).anyTimes();
- statement.executeQuery("SELECT role_name, user_id FROM user_roles");
- expectLastCall().andReturn(userRolesResultSet).once();
-
- expect(entityManager.getTransaction()).andReturn(trans).anyTimes();
- expect(entityManager.getCriteriaBuilder()).andReturn(cb).anyTimes();
- expect(entityManager.createQuery(cq)).andReturn(q).anyTimes();
- expect(trans.isActive()).andReturn(true).anyTimes();
- expect(upgradeCatalog.getEntityManagerProvider()).andReturn(entityManagerProvider).anyTimes();
- expect(upgradeCatalog.getNewPropertyName()).andReturn(new HashMap<String, String>()).anyTimes();
- expect(cb.createQuery(HostRoleCommandEntity.class)).andReturn(cq).anyTimes();
- expect(cb.desc(taskId)).andReturn(o).anyTimes();
- expect(cq.from(HostRoleCommandEntity.class)).andReturn(hrc).anyTimes();
- expect(cq.select(hrc)).andReturn(cq).anyTimes();
- expect(cq.where(anyObject(Predicate.class))).andReturn(cq).anyTimes();
- expect(hrc.get(isA(SingularAttribute.class))).andReturn(taskId).times(2);
- expect(hrc.get(isA(SingularAttribute.class))).andReturn(outputLog).once();
- expect(hrc.get(isA(SingularAttribute.class))).andReturn(errorLog).once();
- expect(q.setMaxResults(1000)).andReturn(q).anyTimes();
- expect(q.getResultList()).andReturn(r).anyTimes();
- expect(clusterConfigEntity.getData()).andReturn("{\"dtnode_heapsize\":\"1028m\"}");
-
- expect(configuration.getDatabaseUrl()).andReturn(Configuration.JDBC_IN_MEMORY_URL).anyTimes();
- expect(injector.getInstance(ConfigHelper.class)).andReturn(configHelper).anyTimes();
- expect(injector.getInstance(AmbariManagementController.class)).andReturn(amc).anyTimes();
- expect(amc.getClusters()).andReturn(clusters).anyTimes();
- expect(clusters.getClusters()).andReturn(clustersMap).anyTimes();
- expect(clusters.getClusterById(1L)).andReturn(clustersMap.values().iterator().next()).anyTimes();
- expect(clusters.getClusters()).andReturn(clustersMap).times(1);
- expect(cluster.getDesiredConfigByType("global")).andReturn(config).anyTimes();
- expect(cluster.getDesiredConfigByType("oozie-log4j")).andReturn(config).anyTimes();
- expect(cluster.getDesiredConfigByType("oozie-env")).andReturn(config).anyTimes();
- expect(cluster.getClusterId()).andReturn(1L);
- expect(cluster.getNextConfigVersion("hadoop-env")).andReturn(3L);
- expect(config.getProperties()).andReturn(globalConfigs).anyTimes();
- expect(cluster.getCurrentStackVersion()).andReturn(new StackId("HDP", "2.1")).anyTimes();
- expect(cluster.getClusterName()).andReturn("c1").anyTimes();
- expect(configHelper.findConfigTypesByPropertyName(new StackId("HDP", "2.1"), "prop1", "c1")).andReturn(envDicts).once();
- expect(configHelper.findConfigTypesByPropertyName(new StackId("HDP", "2.1"), "smokeuser_keytab", "c1")).andReturn(new HashSet<String>()).once();
- expect(configHelper.findConfigTypesByPropertyName(new StackId("HDP", "2.1"), "content", "c1")).andReturn(envDicts).once();
- expect(configHelper.findConfigTypesByPropertyName(new StackId("HDP", "2.1"), "dtnode_heapsize", "c1")).andReturn(configTypes).once();
-
- expect(injector.getInstance(ConfigGroupConfigMappingDAO.class)).andReturn(configGroupConfigMappingDAO).anyTimes();
- expect(injector.getInstance(UserDAO.class)).andReturn(userDAO).anyTimes();
- expect(injector.getInstance(PrincipalDAO.class)).andReturn(principalDAO).anyTimes();
- expect(injector.getInstance(PrincipalTypeDAO.class)).andReturn(principalTypeDAO).anyTimes();
- expect(injector.getInstance(ClusterDAO.class)).andReturn(clusterDAO).anyTimes();
- expect(injector.getInstance(ResourceTypeDAO.class)).andReturn(resourceTypeDAO).anyTimes();
- expect(injector.getInstance(ResourceDAO.class)).andReturn(resourceDAO).anyTimes();
- expect(injector.getInstance(ViewDAO.class)).andReturn(viewDAO).anyTimes();
- expect(injector.getInstance(ViewInstanceDAO.class)).andReturn(viewInstanceDAO).anyTimes();
- expect(injector.getInstance(PermissionDAO.class)).andReturn(permissionDAO).anyTimes();
- expect(injector.getInstance(PrivilegeDAO.class)).andReturn(privilegeDAO).anyTimes();
- expect(injector.getInstance(KeyValueDAO.class)).andReturn(keyValueDAO).anyTimes();
- expect(injector.getInstance(ViewRegistry.class)).andReturn(viewRegistry).anyTimes();
-
- expect(userRolesResultSet.next()).andReturn(true).times(3);
- expect(userRolesResultSet.next()).andReturn(false).times(1);
- expect(userRolesResultSet.getString(1)).andReturn("admin").times(1);
- expect(userRolesResultSet.getString(1)).andReturn("user").times(2);
- expect(userRolesResultSet.getInt(2)).andReturn(1).times(2);
- expect(userRolesResultSet.getInt(2)).andReturn(2).times(1);
-
- UserEntity userEntity1 = createNiceMock(UserEntity.class);
- UserEntity userEntity2 = createNiceMock(UserEntity.class);
- PrincipalEntity userPrincipal1 = createNiceMock(PrincipalEntity.class);
- PrincipalEntity userPrincipal2 = createNiceMock(PrincipalEntity.class);
- Set<PrivilegeEntity> userPrivileges1 = createNiceMock(Set.class);
- Set<PrivilegeEntity> userPrivileges2 = createNiceMock(Set.class);
- expect(userEntity1.getPrincipal()).andReturn(userPrincipal1).anyTimes();
- expect(userEntity2.getPrincipal()).andReturn(userPrincipal2).anyTimes();
- expect(userPrincipal1.getPrivileges()).andReturn(userPrivileges1).anyTimes();
- expect(userPrincipal2.getPrivileges()).andReturn(userPrivileges2).anyTimes();
- expect(userPrivileges1.add(anyObject(PrivilegeEntity.class))).andReturn(true).once();
- expect(userDAO.findByPK(1)).andReturn(userEntity1).times(2);
- expect(userDAO.findByPK(2)).andReturn(userEntity2).once();
- expect(userDAO.merge(userEntity1)).andReturn(userEntity1).once();
- expect(userDAO.merge(userEntity2)).andReturn(userEntity2).once();
-
- expect(configGroupConfigMappingDAO.findAll()).andReturn(configGroupConfigMappingEntities).once();
- expect(userDAO.findAll()).andReturn(Collections.<UserEntity> emptyList()).times(1);
- expect(userDAO.findAll()).andReturn(Arrays.asList(userEntity1, userEntity2)).times(1);
- expect(clusterDAO.findAll()).andReturn(Collections.<ClusterEntity> emptyList()).times(2);
-
- String yarnConfig = String.format("{'%s':'%s', '%s':'%s'}",
- YARN_TIMELINE_SERVICE_WEBAPP_ADDRESS_PROPERTY, "timeline:8081",
- YARN_RESOURCEMANAGER_WEBAPP_ADDRESS_PROPERTY, "resource_man:8081");
- expect(configGroupConfigMappingDAO.findAll()).andReturn(configGroupConfigMappingEntities).once();
- expect(clusterDAO.findAll()).andReturn(Collections.singletonList(clusterEntity)).anyTimes();
- expect(configEntity.getData()).andReturn(yarnConfig);
- expect(clusterDAO.findConfig(1L, YARN_SITE, "version1")).andReturn(configEntity).anyTimes();
- expect(viewDAO.findAll()).andReturn(Collections.<ViewEntity> emptyList()).anyTimes();
- expect(viewInstanceDAO.findAll()).andReturn(Collections.<ViewInstanceEntity> emptyList()).anyTimes();
- expect(permissionDAO.findAmbariAdminPermission()).andReturn(adminPermission).anyTimes();
- expect(permissionDAO.findViewUsePermission()).andReturn(viewUsePermission).anyTimes();
- expect(permissionDAO.findClusterOperatePermission()).andReturn(null);
- expect(permissionDAO.findClusterReadPermission()).andReturn(null);
-
- expect(cluster.getDesiredConfigByType("pig-properties")).andReturn(pigConfig).anyTimes();
- expect(pigConfig.getProperties()).andReturn(pigSettings).anyTimes();
-
- ViewEntity jobsView = createNiceMock(ViewEntity.class);
- KeyValueEntity showJobsKeyValue = createNiceMock(KeyValueEntity.class);
- UserEntity user = createNiceMock(UserEntity.class);
- ClusterConfigMappingEntity configMappingEntity = createNiceMock(ClusterConfigMappingEntity.class);
- ClusterStateEntity clusterStateEntity = createNiceMock(ClusterStateEntity.class);
-
- StackEntity stackEntity = createNiceMock(StackEntity.class);
- expect(stackEntity.getStackName()).andReturn(
- CLUSTER_STATE_STACK_HDP_2_1.getStackName());
-
- expect(stackEntity.getStackVersion()).andReturn(
- CLUSTER_STATE_STACK_HDP_2_1.getStackVersion());
-
- expect(clusterEntity.getClusterId()).andReturn(1L).anyTimes();
- expect(clusterEntity.getConfigMappingEntities()).andReturn(Collections.singleton(configMappingEntity)).times(2);
- expect(clusterEntity.getClusterStateEntity()).andReturn(clusterStateEntity).anyTimes();
- expect(clusterStateEntity.getCurrentStack()).andReturn(stackEntity);
- expect(configMappingEntity.getType()).andReturn(YARN_SITE).anyTimes();
- expect(configMappingEntity.isSelected()).andReturn(1).anyTimes();
- expect(configMappingEntity.getTag()).andReturn("version1");
- expect(userDAO.findAll()).andReturn(Collections.singletonList(user));
- expect(jobsView.getCommonName()).andReturn(JOBS_VIEW_NAME);
- expect(jobsView.getVersion()).andReturn("1.0.0");
- expect(viewDAO.findByCommonName(JOBS_VIEW_NAME)).andReturn(jobsView).once();
- expect(showJobsKeyValue.getValue()).andReturn("true");
- expect(keyValueDAO.findByKey(SHOW_JOBS_FOR_NON_ADMIN_KEY)).andReturn(showJobsKeyValue);
- expect(privilegeDAO.findAllByPrincipal(anyObject(List.class))).andReturn(Collections.<PrivilegeEntity>emptyList());
- expect(viewDAO.merge(jobsView)).andReturn(jobsView);
-
- resourceDAO.create(anyObject(ResourceEntity.class));
- expect(adminPermission.getId()).andReturn(3);
- expect(viewUsePermission.getId()).andReturn(4);
- viewInstanceDAO.create(anyObject(ViewInstanceEntity.class));
- expectLastCall().andAnswer(new IAnswer<Object>() {
- @Override
- public Object answer() throws Throwable {
- ((ViewInstanceEntity) getCurrentArguments()[0]).getResource().setId(1L);
- return null;
- }
- });
- keyValueDAO.remove(showJobsKeyValue);
- privilegeDAO.create(anyObject(PrivilegeEntity.class));
-
- replay(entityManager, trans, upgradeCatalog, cb, cq, hrc, q, connection, statement, userRolesResultSet);
-
- replay(dbAccessor, configuration, injector, cluster, clusters, amc, config, configHelper, pigConfig);
- replay(userDAO, clusterDAO, viewDAO, viewInstanceDAO, permissionDAO, configGroupConfigMappingDAO);
- replay(resourceTypeDAO, resourceDAO, keyValueDAO, privilegeDAO, clusterConfigEntity);
- replay(jobsView, showJobsKeyValue, user);
- replay(userEntity1, userEntity2, userPrincipal1, userPrincipal2, userPrivileges1, userPrivileges2);
- replay(viewRegistry, viewUsePermission, adminPermission);
- replay(clusterEntity, configEntity, configMappingEntity, clusterStateEntity, stackEntity);
-
- Class<?> c = AbstractUpgradeCatalog.class;
- Field f = c.getDeclaredField("configuration");
- f.setAccessible(true);
- f.set(upgradeCatalog, configuration);
- f = c.getDeclaredField("dbAccessor");
- f.setAccessible(true);
- f.set(upgradeCatalog, dbAccessor);
- f = c.getDeclaredField("injector");
- f.setAccessible(true);
- f.set(upgradeCatalog, injector);
-
- upgradeCatalog.executeDMLUpdates();
-
- verify(upgradeCatalog, dbAccessor, configuration, injector, cluster,
- clusters, amc, config, configHelper,jobsView, showJobsKeyValue,
- privilegeDAO, viewDAO, viewInstanceDAO, resourceDAO, keyValueDAO,
- connection, statement, userRolesResultSet, userEntity1, userEntity2,
- userPrincipal1, userPrincipal2, userPrivileges1, userPrivileges2,
- viewRegistry, clusterEntity, configEntity, configMappingEntity,
- clusterStateEntity);
- }
-
- @Test
- public void testMoveHcatalogIntoHiveService() throws AmbariException {
- UpgradeCatalog170 upgradeCatalog170 = injector.getInstance(UpgradeCatalog170.class);
- ServiceComponentDesiredStateDAO serviceComponentDesiredStateDAO = injector.getInstance(ServiceComponentDesiredStateDAO.class);
- HostComponentDesiredStateDAO hostComponentDesiredStateDAO = injector.getInstance(HostComponentDesiredStateDAO.class);
- HostComponentStateDAO hostComponentStateDAO = injector.getInstance(HostComponentStateDAO.class);
- StackDAO stackDAO = injector.getInstance(StackDAO.class);
-
- // inject AmbariMetaInfo to ensure that stacks get populated in the DB
- injector.getInstance(AmbariMetaInfo.class);
-
- StackEntity desiredStackEntity = stackDAO.find(
- DESIRED_STACK.getStackName(),
- DESIRED_STACK.getStackVersion());
-
- assertNotNull(desiredStackEntity);
-
- final ClusterEntity clusterEntity = upgradeCatalogHelper.createCluster(
- injector, CLUSTER_NAME, desiredStackEntity);
-
- final ClusterServiceEntity clusterServiceEntityHDFS = upgradeCatalogHelper.addService(
- injector, clusterEntity, "HDFS", desiredStackEntity);
-
- final ClusterServiceEntity clusterServiceEntityHIVE = upgradeCatalogHelper.addService(
- injector, clusterEntity, "HIVE", desiredStackEntity);
-
- final ClusterServiceEntity clusterServiceEntityHCATALOG = upgradeCatalogHelper.addService(
- injector, clusterEntity, "HCATALOG", desiredStackEntity);
-
- final ClusterServiceEntity clusterServiceEntityWEBHCAT = upgradeCatalogHelper.addService(
- injector, clusterEntity, "WEBHCAT", desiredStackEntity);
-
- final HostEntity hostEntity = upgradeCatalogHelper.createHost(injector,
- clusterEntity, HOST_NAME);
-
- upgradeCatalogHelper.addComponent(injector, clusterEntity,
- clusterServiceEntityHDFS, hostEntity, "NAMENODE", desiredStackEntity);
-
- upgradeCatalogHelper.addComponent(injector, clusterEntity,
- clusterServiceEntityHIVE, hostEntity, "HIVE_SERVER", desiredStackEntity);
-
- upgradeCatalogHelper.addComponent(injector, clusterEntity,
- clusterServiceEntityHCATALOG, hostEntity, "HCAT", desiredStackEntity);
-
- upgradeCatalogHelper.addComponent(injector, clusterEntity,
- clusterServiceEntityWEBHCAT, hostEntity, "WEBHCAT_SERVER",
- desiredStackEntity);
-
- upgradeCatalog170.moveHcatalogIntoHiveService();
-
- ServiceComponentDesiredStateEntity serviceComponentDesiredStateEntity = serviceComponentDesiredStateDAO.findByName(
- clusterEntity.getClusterId(), "HIVE", "HCAT");
-
- assertNotNull(serviceComponentDesiredStateEntity);
-
- HostComponentDesiredStateEntityPK hcDesiredStateEntityPk = new HostComponentDesiredStateEntityPK();
- hcDesiredStateEntityPk.setServiceName("HIVE");
- hcDesiredStateEntityPk.setClusterId(clusterEntity.getClusterId());
- hcDesiredStateEntityPk.setComponentName("HCAT");
- hcDesiredStateEntityPk.setHostId(hostEntity.getHostId());
- HostComponentDesiredStateEntity hcDesiredStateEntity = hostComponentDesiredStateDAO.findByPK(hcDesiredStateEntityPk);
- assertNotNull(hcDesiredStateEntity);
- }
-
- @Test
- public void updateClusterProvisionState() throws AmbariException {
- StackDAO stackDAO = injector.getInstance(StackDAO.class);
-
- // inject AmbariMetaInfo to ensure that stacks get populated in the DB
- injector.getInstance(AmbariMetaInfo.class);
-
- StackEntity desiredStackEntity = stackDAO.find(
- DESIRED_STACK.getStackName(), DESIRED_STACK.getStackVersion());
-
- assertNotNull(desiredStackEntity);
-
- ClusterEntity clusterEntity = upgradeCatalogHelper.createCluster(injector,
- CLUSTER_NAME, desiredStackEntity);
-
- UpgradeCatalog170 upgradeCatalog170 = injector.getInstance(UpgradeCatalog170.class);
- upgradeCatalog170.updateClusterProvisionState(); //action
-
- ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- String assertMsg = "updated provisioning state should be installed";
- String expectedState = "INSTALLED";
- String actualState = clusterDAO.findById(1L).getProvisioningState().name();
- assertEquals(assertMsg,expectedState, actualState); //validation
- }
-
-
- @Test
- public void testGetTargetVersion() throws Exception {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- UpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
-
- Assert.assertEquals("1.7.0", upgradeCatalog.getTargetVersion());
- }
-
- private AbstractUpgradeCatalog getUpgradeCatalog(final DBAccessor dbAccessor) {
- Module module = new Module() {
- @Override
- public void configure(Binder binder) {
- binder.bind(DBAccessor.class).toInstance(dbAccessor);
- binder.bind(OsFamily.class).toInstance(createNiceMock(OsFamily.class));
- }
- };
- Injector injector = Guice.createInjector(module);
- return injector.getInstance(UpgradeCatalog170.class);
- }
-
- private void assertClusterConfigColumns(Capture<DBAccessor.DBColumnInfo> clusterConfigAttributesColumnCapture) {
- DBAccessor.DBColumnInfo column = clusterConfigAttributesColumnCapture.getValue();
- assertConfigAttriburesColumn(column);
- }
-
- private void assertHostgroupConfigColumns(Capture<DBAccessor.DBColumnInfo> hostgroupConfigAttributesColumnCapture) {
- DBAccessor.DBColumnInfo column = hostgroupConfigAttributesColumnCapture.getValue();
- assertConfigAttriburesColumn(column);
- }
-
- private void assertBlueprintConfigColumns(Capture<DBAccessor.DBColumnInfo> blueprintConfigAttributesColumnCapture) {
- DBAccessor.DBColumnInfo column = blueprintConfigAttributesColumnCapture.getValue();
- assertConfigAttriburesColumn(column);
- }
-
- private void assertConfigAttriburesColumn(DBAccessor.DBColumnInfo column) {
- assertEquals("config_attributes", column.getName());
- assertEquals(Character[].class, column.getType());
- assertEquals(null, column.getDefaultValue());
- assertTrue(column.isNullable());
- }
-
- private void setConfigAttributesColumnExpectations(DBAccessor dbAccessor,
- Capture<DBAccessor.DBColumnInfo> configAttributesColumnCapture, String tableName)
- throws SQLException {
- dbAccessor.addColumn(eq(tableName),
- capture(configAttributesColumnCapture));
- }
-
- private void setStageExpectations(DBAccessor dbAccessor,
- Capture<DBAccessor.DBColumnInfo> stageCommandParamsColumnCapture,
- Capture<DBAccessor.DBColumnInfo> stageHostParamsColumnCapture)
- throws SQLException {
- dbAccessor.addColumn(eq("stage"),
- capture(stageCommandParamsColumnCapture));
-
- dbAccessor.addColumn(eq("stage"),
- capture(stageHostParamsColumnCapture));
- }
-
- @Test
- public void testGetSourceVersion() {
- final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
- UpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
- Assert.assertEquals("1.6.1", upgradeCatalog.getSourceVersion());
- }
-
- private void setViewExpectations(DBAccessor dbAccessor,
- Capture<DBAccessor.DBColumnInfo> maskColumnCapture,
- Capture<DBAccessor.DBColumnInfo> systemColumnCapture)
- throws SQLException {
-
- dbAccessor.addColumn(eq("viewmain"), capture(maskColumnCapture));
- dbAccessor.addColumn(eq("viewmain"), capture(systemColumnCapture));
- }
-
- private void setViewParameterExpectations(DBAccessor dbAccessor,
- Capture<DBAccessor.DBColumnInfo> maskedColumnCapture)
- throws SQLException {
-
- dbAccessor.addColumn(eq("viewparameter"), capture(maskedColumnCapture));
- }
-
- private void assertViewColumns(
- Capture<DBAccessor.DBColumnInfo> maskColumnCapture,
- Capture<DBAccessor.DBColumnInfo> systemColumnCapture) {
-
- DBAccessor.DBColumnInfo column = maskColumnCapture.getValue();
- assertEquals("mask", column.getName());
- assertEquals(255, (int) column.getLength());
- assertEquals(String.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = systemColumnCapture.getValue();
- assertEquals("system_view", column.getName());
- assertEquals(1, (int) column.getLength());
- assertEquals(Character.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
- }
-
- private void assertViewParameterColumns(
- Capture<DBAccessor.DBColumnInfo> maskedColumnCapture) {
- DBAccessor.DBColumnInfo column = maskedColumnCapture.getValue();
- assertEquals("masked", column.getName());
- assertEquals(1, (int) column.getLength());
- assertEquals(Character.class, column.getType());
- assertNull(column.getDefaultValue());
- assertTrue(column.isNullable());
- }
-
- private void assertStageColumns(Capture<DBAccessor.DBColumnInfo> stageCommandParamsColumnCapture,
- Capture<DBAccessor.DBColumnInfo> stageHostParamsColumnCapture) {
- DBAccessor.DBColumnInfo column = stageCommandParamsColumnCapture.getValue();
- assertEquals("command_params", column.getName());
- assertEquals(byte[].class, column.getType());
- assertEquals(null, column.getDefaultValue());
- assertTrue(column.isNullable());
-
- column = stageHostParamsColumnCapture.getValue();
- assertEquals("host_params", column.getName());
- assertEquals(byte[].class, column.getType());
- assertEquals(null, column.getDefaultValue());
- assertTrue(column.isNullable());
- }
-}
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog200Test.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog200Test.java b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog200Test.java
index a090718..1513209 100644
--- a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog200Test.java
+++ b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog200Test.java
@@ -805,7 +805,7 @@ public class UpgradeCatalog200Test {
public void testGetSourceVersion() {
final DBAccessor dbAccessor = createNiceMock(DBAccessor.class);
UpgradeCatalog upgradeCatalog = getUpgradeCatalog(dbAccessor);
- Assert.assertEquals("1.7.0", upgradeCatalog.getSourceVersion());
+ Assert.assertEquals(null, upgradeCatalog.getSourceVersion());
}
@Test
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogTest.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogTest.java b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogTest.java
index 1bd1395..554cd99 100644
--- a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogTest.java
+++ b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogTest.java
@@ -53,10 +53,10 @@ public class UpgradeCatalogTest {
private final String DESIRED_STACK_VERSION = "{\"stackName\":\"HDP\"," +
"\"stackVersion\":\"1.2.0\"}";
- private static class UpgradeCatalog149 extends AbstractUpgradeCatalog {
+ private static class UpgradeCatalog201 extends AbstractUpgradeCatalog {
@Inject
- public UpgradeCatalog149(Injector injector) {
+ public UpgradeCatalog201(Injector injector) {
super(injector);
}
@@ -77,7 +77,7 @@ public class UpgradeCatalogTest {
@Override
public String getTargetVersion() {
- return "1.4.9";
+ return "2.0.1";
}
}
@@ -89,9 +89,7 @@ public class UpgradeCatalogTest {
// Add binding to each newly created catalog
Multibinder<UpgradeCatalog> catalogBinder =
Multibinder.newSetBinder(binder(), UpgradeCatalog.class);
- catalogBinder.addBinding().to(UpgradeCatalog149.class);
- catalogBinder.addBinding().to(UpgradeCatalog150.class);
- catalogBinder.addBinding().to(UpgradeCatalog170.class);
+ catalogBinder.addBinding().to(UpgradeCatalog201.class);
catalogBinder.addBinding().to(UpgradeCatalog200.class);
catalogBinder.addBinding().to(UpgradeCatalog210.class);
}
@@ -123,80 +121,13 @@ public class UpgradeCatalogTest {
Set<UpgradeCatalog> upgradeCatalogSet = schemaUpgradeHelper.getAllUpgradeCatalogs();
Assert.assertNotNull(upgradeCatalogSet);
- Assert.assertEquals(5, upgradeCatalogSet.size());
+ Assert.assertEquals(3, upgradeCatalogSet.size());
- List<UpgradeCatalog> upgradeCatalogs = schemaUpgradeHelper.getUpgradePath(null, "1.5.1");
+ List<UpgradeCatalog> upgradeCatalogs = schemaUpgradeHelper.getUpgradePath(null, "2.0.1");
Assert.assertNotNull(upgradeCatalogs);
Assert.assertEquals(2, upgradeCatalogs.size());
- Assert.assertEquals("1.4.9", upgradeCatalogs.get(0).getTargetVersion());
- Assert.assertEquals("1.5.0", upgradeCatalogs.get(1).getTargetVersion());
- }
-
- @Test
- public void testUpdateConfigurationProperties() throws Exception {
- AmbariManagementController controller = injector.getInstance(AmbariManagementController.class);
- ClusterRequest r = new ClusterRequest(null, "c1", "HDP-0.1", null);
- controller.createCluster(r);
- Clusters clusters = injector.getInstance(Clusters.class);
- Cluster cluster = clusters.getCluster("c1");
-
- Assert.assertNotNull(cluster);
-
- Map<String, String> properties = new HashMap<String, String>() {{
- put("a", "b");
- }};
-
- final ClusterRequest cl = new ClusterRequest(
- cluster.getClusterId(),
- cluster.getClusterName(),
- cluster.getDesiredStackVersion().getStackVersion(),
- null
- );
-
- ConfigurationRequest cr = new ConfigurationRequest();
- cr.setClusterName(cluster.getClusterName());
- cr.setType("global");
- cr.setVersionTag("version1");
- cr.setProperties(properties);
-
- cl.setDesiredConfig(Collections.singletonList(cr));
-
- controller.updateClusters(new HashSet<ClusterRequest>() {{ add(cl); }}, null);
-
- Config config = cluster.getConfig("global", "version1");
-
- Assert.assertNotNull(config);
- Assert.assertEquals(properties, config.getProperties());
-
- // Add new
- UpgradeCatalog149 testCatalog = injector.getInstance(UpgradeCatalog149.class);
- testCatalog.updateConfigurationProperties("global",
- Collections.singletonMap("x", "y"), false, false);
- config = cluster.getDesiredConfigByType("global");
- String version = config.getTag();
- Assert.assertNotNull(config);
- Assert.assertNotSame("version1", version);
- Assert.assertTrue(config.getProperties().containsKey("x"));
- Assert.assertEquals("y", config.getProperties().get("x"));
-
- // Override value
- testCatalog.updateConfigurationProperties("global",
- Collections.singletonMap("x", "z"), true, false);
- config = cluster.getDesiredConfigByType("global");
- Assert.assertNotNull(config);
- Assert.assertNotSame(version, config.getTag());
- Assert.assertTrue(config.getProperties().containsKey("x"));
- Assert.assertEquals("z", config.getProperties().get("x"));
- version = config.getTag();
-
- // Retain original
- testCatalog.updateConfigurationProperties("global",
- Collections.singletonMap("x", "y"), false, false);
- config = cluster.getDesiredConfigByType("global");
- Assert.assertNotNull(config);
- Assert.assertSame(version, config.getTag());
- Assert.assertTrue(config.getProperties().containsKey("x"));
- Assert.assertEquals("z", config.getProperties().get("x"));
+ Assert.assertEquals("2.0.0", upgradeCatalogs.get(0).getTargetVersion());
+ Assert.assertEquals("2.0.1", upgradeCatalogs.get(1).getTargetVersion());
}
}
[14/31] ambari git commit: AMBARI-18891. Oozie config not shown as
stale upon changing DefaultFS. (stoader)
Posted by nc...@apache.org.
AMBARI-18891. Oozie config not shown as stale upon changing DefaultFS. (stoader)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/7df30b17
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/7df30b17
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/7df30b17
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 7df30b172d88271ffa5300149a73e86bf330a912
Parents: fcd0d2a
Author: Toader, Sebastian <st...@hortonworks.com>
Authored: Tue Nov 15 13:56:12 2016 +0100
Committer: Toader, Sebastian <st...@hortonworks.com>
Committed: Tue Nov 15 13:56:12 2016 +0100
----------------------------------------------------------------------
.../src/main/resources/common-services/OOZIE/4.0.0.2.0/metainfo.xml | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/7df30b17/ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/metainfo.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/metainfo.xml b/ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/metainfo.xml
index 66d272a..8282240 100644
--- a/ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/metainfo.xml
+++ b/ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/metainfo.xml
@@ -164,6 +164,7 @@
<config-type>yarn-site</config-type>
<config-type>hive-site</config-type>
<config-type>tez-site</config-type>
+ <config-type>core-site</config-type>
</configuration-dependencies>
</service>
</services>
[31/31] ambari git commit: Merge branch 'trunk' into
branch-feature-AMBARI-18634
Posted by nc...@apache.org.
Merge branch 'trunk' into branch-feature-AMBARI-18634
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/f4c2dbff
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/f4c2dbff
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/f4c2dbff
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: f4c2dbff3117255ecdeebea0694aaf820823d7b6
Parents: b395c5d 58ef394
Author: Nate Cole <nc...@hortonworks.com>
Authored: Wed Nov 16 09:39:16 2016 -0500
Committer: Nate Cole <nc...@hortonworks.com>
Committed: Wed Nov 16 09:39:16 2016 -0500
----------------------------------------------------------------------
ambari-agent/conf/unix/ambari-agent.ini | 3 +
ambari-agent/conf/windows/ambari-agent.ini | 2 +
.../src/main/python/ambari_agent/Controller.py | 8 +-
.../src/main/python/ambari_agent/HostCleanup.py | 87 +-
.../src/main/python/ambari_agent/NetUtil.py | 10 +-
.../test/python/ambari_agent/TestHostCleanup.py | 18 +-
ambari-metrics/ambari-metrics-assembly/pom.xml | 1 -
.../conf/unix/metric_monitor.ini | 2 -
.../src/main/python/core/config_reader.py | 10 +-
.../src/main/python/core/emitter.py | 57 +-
.../src/main/python/core/krberr.py | 42 -
.../main/python/core/spnego_kerberos_auth.py | 163 --
.../src/test/python/core/TestEmitter.py | 26 -
.../metrics/timeline/TimelineMetricStore.java | 2 +-
ambari-server/docs/configuration/index.md | 35 +-
ambari-server/pom.xml | 10 +
ambari-server/sbin/ambari-server | 4 +
.../stackadvisor/StackAdvisorHelper.java | 18 +-
.../server/configuration/Configuration.java | 162 +-
.../AmbariManagementControllerImpl.java | 49 +-
.../ambari/server/controller/AmbariServer.java | 222 +--
.../ambari/server/controller/GroupResponse.java | 14 +
.../server/controller/KerberosHelperImpl.java | 6 -
.../internal/GroupResourceProvider.java | 4 +
.../internal/UserPrivilegeResourceProvider.java | 3 +
.../metrics/MetricsCollectorHAClusterState.java | 10 +
.../apache/ambari/server/orm/dao/GroupDAO.java | 19 +-
.../ambari/server/orm/dao/ResourceDAO.java | 21 +
.../ambari/server/orm/entities/GroupEntity.java | 18 +
.../server/security/ClientSecurityType.java | 3 +-
.../AmbariPamAuthenticationProvider.java | 252 +++
.../server/security/authorization/Group.java | 6 +
.../security/authorization/GroupType.java | 25 +
.../PamAuthenticationException.java | 36 +
.../server/security/authorization/UserType.java | 3 +-
.../server/security/authorization/Users.java | 54 +-
.../security/unsecured/rest/ConnectionInfo.java | 2 +-
.../kerberos/MITKerberosOperationHandler.java | 2 +-
.../ambari/server/state/ConfigHelper.java | 85 -
.../server/state/DependencyConditionInfo.java | 104 ++
.../ambari/server/state/DependencyInfo.java | 37 +-
.../ambari/server/topology/AmbariContext.java | 20 -
.../server/topology/BlueprintValidatorImpl.java | 18 +-
.../server/upgrade/SchemaUpgradeHelper.java | 5 -
.../server/upgrade/UpgradeCatalog150.java | 910 -----------
.../server/upgrade/UpgradeCatalog151.java | 152 --
.../server/upgrade/UpgradeCatalog160.java | 205 ---
.../server/upgrade/UpgradeCatalog161.java | 343 ----
.../server/upgrade/UpgradeCatalog170.java | 1530 ------------------
.../server/upgrade/UpgradeCatalog200.java | 8 -
.../server/upgrade/UpgradeCatalog250.java | 11 +
ambari-server/src/main/python/ambari-server.py | 7 +-
.../python/ambari_server/resourceFilesKeeper.py | 4 +-
.../main/python/ambari_server/setupActions.py | 1 +
.../main/python/ambari_server/setupMpacks.py | 2 +
.../main/python/ambari_server/setupSecurity.py | 53 +-
.../main/resources/Ambari-DDL-MySQL-CREATE.sql | 1 +
.../main/resources/Ambari-DDL-Oracle-CREATE.sql | 1 +
.../resources/Ambari-DDL-Postgres-CREATE.sql | 1 +
.../resources/Ambari-DDL-SQLAnywhere-CREATE.sql | 1 +
.../resources/Ambari-DDL-SQLServer-CREATE.sql | 1 +
.../AMBARI_METRICS/0.1.0/kerberos.json | 26 -
.../AMBARI_METRICS/0.1.0/package/scripts/ams.py | 13 +-
.../0.1.0/package/scripts/params.py | 7 +-
.../0.1.0/package/scripts/split_points.py | 43 +-
.../package/templates/metric_monitor.ini.j2 | 2 -
.../configuration/application-properties.xml | 11 +
.../HDFS/2.1.0.2.0/configuration/hadoop-env.xml | 2 +-
.../common-services/HDFS/2.1.0.2.0/metainfo.xml | 43 +-
.../0.12.0.2.0/package/scripts/params_linux.py | 2 -
.../0.12.0.2.0/package/scripts/service_check.py | 10 +-
.../configuration/kafka_client_jaas_conf.xml | 41 +
.../0.8.1/configuration/kafka_jaas_conf.xml | 59 +
.../common-services/KAFKA/0.8.1/metainfo.xml | 2 +
.../KAFKA/0.8.1/package/scripts/kafka.py | 12 +
.../KAFKA/0.8.1/package/scripts/params.py | 2 +
.../OOZIE/4.0.0.2.0/metainfo.xml | 1 +
.../src/main/resources/properties.json | 1 +
.../stacks/HDP/2.0.6/services/stack_advisor.py | 29 +-
.../services/HDFS/configuration/core-site.xml | 2 +-
.../services/HDFS/configuration/hdfs-site.xml | 2 +-
.../configuration/application-properties.xml | 95 ++
.../ATLAS/configuration/atlas-log4j.xml | 118 ++
.../stacks/HDP/2.6/services/ATLAS/metainfo.xml | 10 +-
.../services/ATLAS/themes/theme_version_2.json | 845 ++++++++++
.../stacks/HDP/2.6/services/stack_advisor.py | 381 ++---
.../services/HDFS/configuration/core-site.xml | 2 +-
.../services/HDFS/configuration/hdfs-site.xml | 2 +-
.../webapp/WEB-INF/spring-security.xml | 1 +
.../stackadvisor/StackAdvisorHelperTest.java | 10 +-
.../server/configuration/ConfigurationTest.java | 6 +-
.../AmbariPamAuthenticationProviderTest.java | 97 ++
.../security/authorization/TestUsers.java | 10 +-
.../server/topology/AmbariContextTest.java | 2 -
.../server/topology/BlueprintImplTest.java | 41 -
.../topology/BlueprintValidatorImplTest.java | 75 +-
.../server/upgrade/UpgradeCatalog150Test.java | 258 ---
.../server/upgrade/UpgradeCatalog151Test.java | 120 --
.../server/upgrade/UpgradeCatalog160Test.java | 264 ---
.../server/upgrade/UpgradeCatalog161Test.java | 418 -----
.../server/upgrade/UpgradeCatalog170Test.java | 791 ---------
.../server/upgrade/UpgradeCatalog200Test.java | 2 +-
.../server/upgrade/UpgradeCatalog250Test.java | 13 +
.../server/upgrade/UpgradeCatalogTest.java | 85 +-
.../src/test/python/TestResourceFilesKeeper.py | 45 +-
.../2.0.6/HIVE/test_hive_service_check.py | 43 +-
.../stacks/2.0.6/common/test_stack_advisor.py | 4 +-
.../stacks/2.6/common/test_stack_advisor.py | 97 ++
ambari-web/app/config.js | 2 +-
.../main/service/configs/config_overridable.js | 1 +
ambari-web/app/styles/alerts.less | 55 -
ambari-web/app/styles/application.less | 997 +++++-------
ambari-web/app/styles/bootstrap_overrides.less | 14 +
ambari-web/app/styles/common.less | 298 +---
ambari-web/app/styles/config_history_flow.less | 6 -
ambari-web/app/styles/modal_popups.less | 3 -
ambari-web/app/styles/stack_versions.less | 24 +-
ambari-web/app/styles/widgets.less | 4 +-
.../common/configs/selectCreateConfigGroup.hbs | 4 +-
.../templates/common/host_progress_popup.hbs | 4 +-
.../common/modal_popups/log_tail_popup.hbs | 2 +-
ambari-web/app/templates/login.hbs | 2 +-
.../main/alerts/definition_details.hbs | 8 +-
.../main/dashboard/plus_button_filter.hbs | 8 +-
.../main/dashboard/widgets/yarn_links.hbs | 2 +-
ambari-web/app/templates/main/host/summary.hbs | 2 +-
.../manage_configuration_groups_popup.hbs | 8 +-
.../templates/main/service/services/flume.hbs | 4 +-
ambari-web/app/templates/main/views.hbs | 5 +-
ambari-web/app/templates/wizard/step3.hbs | 4 +-
ambari-web/app/views/main/dashboard/widgets.js | 2 +
.../host_component_recommendation_mixin_test.js | 24 +-
.../host_component_validation_mixin_test.js | 24 +-
133 files changed, 3461 insertions(+), 7100 deletions(-)
----------------------------------------------------------------------
[20/31] ambari git commit: AMBARI-18890 - Enable "Auto Start" feature
for services & components by default (rzang)
Posted by nc...@apache.org.
AMBARI-18890 - Enable "Auto Start" feature for services & components by default (rzang)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/b7ee658d
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/b7ee658d
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/b7ee658d
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: b7ee658d0a7802eba00805658fd50f5e9d0023b3
Parents: 0e8474f
Author: Richard Zang <rz...@apache.org>
Authored: Tue Nov 15 16:03:26 2016 -0800
Committer: Richard Zang <rz...@apache.org>
Committed: Tue Nov 15 16:04:18 2016 -0800
----------------------------------------------------------------------
ambari-web/app/config.js | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/b7ee658d/ambari-web/app/config.js
----------------------------------------------------------------------
diff --git a/ambari-web/app/config.js b/ambari-web/app/config.js
index a14f38e..130cb39 100644
--- a/ambari-web/app/config.js
+++ b/ambari-web/app/config.js
@@ -82,7 +82,7 @@ App.supports = {
skipComponentStartAfterInstall: false,
preInstallChecks: false,
hostComboSearchBox: true,
- serviceAutoStart: false,
+ serviceAutoStart: true,
logSearch: true,
redhatSatellite: false,
enableIpa: false,
[27/31] ambari git commit: AMBARI-18876. Remove old UpgradeCatalog
and tests (aonishuk)
Posted by nc...@apache.org.
AMBARI-18876. Remove old UpgradeCatalog and tests (aonishuk)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/7cf4d866
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/7cf4d866
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/7cf4d866
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 7cf4d8662386475eddc405df7c84df7ea337686f
Parents: 56b4d57
Author: Andrew Onishuk <ao...@hortonworks.com>
Authored: Wed Nov 16 12:57:46 2016 +0200
Committer: Andrew Onishuk <ao...@hortonworks.com>
Committed: Wed Nov 16 12:57:46 2016 +0200
----------------------------------------------------------------------
.../server/configuration/Configuration.java | 13 -
.../AmbariManagementControllerImpl.java | 42 -
.../server/controller/KerberosHelperImpl.java | 6 -
.../ambari/server/state/ConfigHelper.java | 85 -
.../ambari/server/topology/AmbariContext.java | 20 -
.../server/upgrade/SchemaUpgradeHelper.java | 5 -
.../server/upgrade/UpgradeCatalog150.java | 910 -----------
.../server/upgrade/UpgradeCatalog151.java | 152 --
.../server/upgrade/UpgradeCatalog160.java | 205 ---
.../server/upgrade/UpgradeCatalog161.java | 343 ----
.../server/upgrade/UpgradeCatalog170.java | 1530 ------------------
.../server/upgrade/UpgradeCatalog200.java | 8 -
.../server/topology/AmbariContextTest.java | 2 -
.../server/upgrade/UpgradeCatalog150Test.java | 258 ---
.../server/upgrade/UpgradeCatalog151Test.java | 120 --
.../server/upgrade/UpgradeCatalog160Test.java | 264 ---
.../server/upgrade/UpgradeCatalog161Test.java | 418 -----
.../server/upgrade/UpgradeCatalog170Test.java | 791 ---------
.../server/upgrade/UpgradeCatalog200Test.java | 2 +-
.../server/upgrade/UpgradeCatalogTest.java | 85 +-
20 files changed, 9 insertions(+), 5250 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java b/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
index ff91b36..863c948 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
@@ -258,24 +258,11 @@ public class Configuration {
public static final String JAVAX_SSL_TRUSTSTORE_TYPE = "javax.net.ssl.trustStoreType";
/**
- * The configuration tag used for "global" properties affecting the entire
- * cluster or the Ambari server itself.
- */
- @Deprecated
- public static final String GLOBAL_CONFIG_TAG = "global";
-
- /**
* The configuration tag for {@code mapreduce2-log4j}
*/
public static final String MAPREDUCE2_LOG4J_CONFIG_TAG = "mapreduce2-log4j";
/**
- * The configuration property for determining whether RCA is enabled.
- */
- @Deprecated
- public static final String RCA_ENABLED_PROPERTY = "rca_enabled";
-
- /**
* Threadpool sizing based on the number of available processors multiplied by
* 2.
*/
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java b/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
index 8e2fe74..ac9e405 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
@@ -885,54 +885,12 @@ public class AmbariManagementControllerImpl implements AmbariManagementControlle
request.getType()));
}
- handleGlobalsBackwardsCompability(request, propertiesAttributes);
-
Config config = createConfig(cluster, request.getType(), requestProperties,
request.getVersionTag(), propertiesAttributes);
return new ConfigurationResponse(cluster.getClusterName(), config);
}
- private void handleGlobalsBackwardsCompability(ConfigurationRequest request,
- Map<String, Map<String, String>> propertiesAttributes) throws AmbariException {
- Cluster cluster = clusters.getCluster(request.getClusterName());
- if(request.getType().equals(Configuration.GLOBAL_CONFIG_TAG)) {
- Map<String, Map<String, String>> configTypes = new HashMap<String, Map<String, String>>();
- configTypes.put(Configuration.GLOBAL_CONFIG_TAG, request.getProperties());
- configHelper.moveDeprecatedGlobals(cluster.getCurrentStackVersion(), configTypes, cluster.getClusterName());
-
- for(Map.Entry<String, Map<String, String>> configType : configTypes.entrySet()) {
- String configTypeName = configType.getKey();
- Map<String, String> properties = configType.getValue();
-
- if(configTypeName.equals(Configuration.GLOBAL_CONFIG_TAG)) {
- continue;
- }
-
- String tag;
- if(cluster.getConfigsByType(configTypeName) == null) {
- tag = "version1";
- } else {
- tag = "version" + System.currentTimeMillis();
- }
-
- Config config = createConfig(cluster, configTypeName, properties, tag, propertiesAttributes);
-
- if (config != null) {
- String authName = getAuthName();
-
- if (cluster.addDesiredConfig(authName, Collections.singleton(config)) != null) {
- LOG.info("cluster '" + cluster.getClusterName() + "' "
- + "changed by: '" + authName + "'; "
- + "type='" + config.getType() + "' "
- + "tag='" + config.getTag());
- }
- }
-
- }
- }
- }
-
@Override
public Config createConfig(Cluster cluster, String type, Map<String, String> properties,
String versionTag, Map<String, Map<String, String>> propertiesAttributes) {
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java b/ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
index fb87495..810c7d5 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
@@ -2455,12 +2455,6 @@ public class KerberosHelperImpl implements KerberosHelper {
Map<String, Map<String, String>> configurations = new HashMap<String, Map<String, String>>();
Map<String, Map<String, String>> configurationTags = ambariManagementController.findConfigurationTagsWithOverrides(cluster, hostname);
- if (configurationTags.get(Configuration.GLOBAL_CONFIG_TAG) != null) {
- configHelper.applyCustomConfig(
- configurations, Configuration.GLOBAL_CONFIG_TAG,
- Configuration.RCA_ENABLED_PROPERTY, "false", false);
- }
-
Map<String, Map<String, String>> configProperties = configHelper.getEffectiveConfigProperties(cluster, configurationTags);
// Apply the configurations saved with the Execution Cmd on top of
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java b/ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
index 197ffdd..f975053 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
@@ -40,7 +40,6 @@ import org.apache.ambari.server.orm.dao.ClusterDAO;
import org.apache.ambari.server.orm.entities.ClusterConfigEntity;
import org.apache.ambari.server.state.PropertyInfo.PropertyType;
import org.apache.ambari.server.state.configgroup.ConfigGroup;
-import org.apache.ambari.server.upgrade.UpgradeCatalog170;
import org.apache.ambari.server.utils.SecretReference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -868,73 +867,6 @@ public class ConfigHelper {
return controller.createConfig(cluster, type, properties, tag, propertyAttributes);
}
-
-
- /**
- * Since global configs are deprecated since 1.7.0, but still supported.
- * We should automatically map any globals used, to *-env dictionaries.
- *
- * @param configurations map of configurations keyed by type
- */
- public void moveDeprecatedGlobals(StackId stackId, Map<String, Map<String, String>> configurations, String clusterName) {
- Map<String, String> globalConfigurations = new HashMap<String, String>();
-
- if (configurations.get(Configuration.GLOBAL_CONFIG_TAG) == null ||
- configurations.get(Configuration.GLOBAL_CONFIG_TAG).size() == 0) {
- return;
- }
-
- globalConfigurations.putAll(configurations.get(Configuration.GLOBAL_CONFIG_TAG));
-
- if (globalConfigurations != null && globalConfigurations.size() != 0) {
- LOG.warn("Global configurations are deprecated, "
- + "please use *-env");
- }
-
- for (Map.Entry<String, String> property : globalConfigurations.entrySet()) {
- String propertyName = property.getKey();
- String propertyValue = property.getValue();
-
- Set<String> newConfigTypes = null;
- try {
- newConfigTypes = findConfigTypesByPropertyName(stackId, propertyName, clusterName);
- } catch (AmbariException e) {
- LOG.error("Exception while getting configurations from the stacks", e);
- return;
- }
-
- newConfigTypes.remove(Configuration.GLOBAL_CONFIG_TAG);
-
- String newConfigType = null;
- if (newConfigTypes.size() > 0) {
- newConfigType = newConfigTypes.iterator().next();
- } else {
- newConfigType = UpgradeCatalog170.getAdditionalMappingGlobalToEnv().get(propertyName);
- }
-
- if (newConfigType == null) {
- LOG.warn("Cannot find where to map " + propertyName + " from " + Configuration.GLOBAL_CONFIG_TAG +
- " (value=" + propertyValue + ")");
- continue;
- }
-
- LOG.info("Mapping config " + propertyName + " from " + Configuration.GLOBAL_CONFIG_TAG +
- " to " + newConfigType +
- " (value=" + propertyValue + ")");
-
- configurations.get(Configuration.GLOBAL_CONFIG_TAG).remove(propertyName);
-
- if (!configurations.containsKey(newConfigType)) {
- configurations.put(newConfigType, new HashMap<String, String>());
- }
- configurations.get(newConfigType).put(propertyName, propertyValue);
- }
-
- if (configurations.get(Configuration.GLOBAL_CONFIG_TAG).size() == 0) {
- configurations.remove(Configuration.GLOBAL_CONFIG_TAG);
- }
- }
-
/**
* Gets the default properties from the specified stack and services when a
* cluster is first installed.
@@ -1066,14 +998,6 @@ public class ConfigHelper {
// desired is set, but actual is not
if (!serviceInfo.hasConfigDependency(type)) {
stale = componentInfo != null && componentInfo.hasConfigType(type);
- } else if (type.equals(Configuration.GLOBAL_CONFIG_TAG)) {
- // find out if the keys are stale by first checking the target service,
- // then all services
- Collection<String> keys = mergeKeyNames(cluster, type, tags.values());
-
- if (serviceInfo.hasDependencyAndPropertyFor(type, keys) || !hasPropertyFor(stackId, type, keys)) {
- stale = true;
- }
} else {
stale = true;
}
@@ -1084,15 +1008,6 @@ public class ConfigHelper {
if (!isTagChanged(tags, actualTags, hasGroupSpecificConfigsForType(cluster, sch.getHostName(), type))) {
stale = false;
- } else if (type.equals(Configuration.GLOBAL_CONFIG_TAG)) {
- // tags are changed, need to find out what has changed,
- // and if it applies
- // to the service
- Collection<String> changed = findChangedKeys(cluster, type,
- tags.values(), actualTags.values());
- if (serviceInfo.hasDependencyAndPropertyFor(type, changed)) {
- stale = true;
- }
} else {
stale = serviceInfo.hasConfigDependency(type) || componentInfo.hasConfigType(type);
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java b/ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
index 83f8470..e1f5cd2 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
@@ -290,22 +290,6 @@ public class AmbariContext {
}
}
- /**
- * Since global configs are deprecated since 1.7.0, but still supported.
- * We should automatically map any globals used, to *-env dictionaries.
- *
- * @param blueprintConfigurations map of blueprint configurations keyed by type
- */
- //todo: do once for all configs
- public void convertGlobalProperties(ClusterTopology topology,
- Map<String, Map<String, String>> blueprintConfigurations) throws AmbariException {
-
- Stack stack = topology.getBlueprint().getStack();
- StackId stackId = new StackId(stack.getName(), stack.getVersion());
- getController().getConfigHelper().moveDeprecatedGlobals(
- stackId, blueprintConfigurations, getClusterName(topology.getClusterId()));
- }
-
public Long getNextRequestId() {
return getController().getActionManager().getNextRequestId();
}
@@ -597,10 +581,6 @@ public class AmbariContext {
Map<String, Map<String, String>> userProvidedGroupProperties =
topologyHostGroupConfig.getFullProperties(1);
- //todo: doesn't belong here.
- //handling backwards compatibility for group configs
- convertGlobalProperties(topology, userProvidedGroupProperties);
-
// iterate over topo host group configs which were defined in
for (Map.Entry<String, Map<String, String>> entry : userProvidedGroupProperties.entrySet()) {
String type = entry.getKey();
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
index 28e374b..5bf7f21 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java
@@ -177,11 +177,6 @@ public class SchemaUpgradeHelper {
// Add binding to each newly created catalog
Multibinder<UpgradeCatalog> catalogBinder =
Multibinder.newSetBinder(binder(), UpgradeCatalog.class);
- catalogBinder.addBinding().to(UpgradeCatalog150.class);
- catalogBinder.addBinding().to(UpgradeCatalog151.class);
- catalogBinder.addBinding().to(UpgradeCatalog160.class);
- catalogBinder.addBinding().to(UpgradeCatalog161.class);
- catalogBinder.addBinding().to(UpgradeCatalog170.class);
catalogBinder.addBinding().to(UpgradeCatalog200.class);
catalogBinder.addBinding().to(UpgradeCatalog210.class);
catalogBinder.addBinding().to(UpgradeCatalog211.class);
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java
deleted file mode 100644
index ff94795..0000000
--- a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java
+++ /dev/null
@@ -1,910 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.ambari.server.upgrade;
-
-import java.io.File;
-import java.io.IOException;
-import java.sql.ResultSet;
-import java.sql.SQLException;
-import java.sql.Statement;
-import java.util.ArrayList;
-import java.util.Collection;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
-import javax.persistence.EntityManager;
-import javax.persistence.Query;
-import javax.persistence.TypedQuery;
-import javax.persistence.criteria.CriteriaBuilder;
-import javax.persistence.criteria.CriteriaQuery;
-import javax.persistence.criteria.Expression;
-import javax.persistence.criteria.Predicate;
-import javax.persistence.criteria.Root;
-
-import org.apache.ambari.server.AmbariException;
-import org.apache.ambari.server.actionmanager.HostRoleStatus;
-import org.apache.ambari.server.api.services.AmbariMetaInfo;
-import org.apache.ambari.server.configuration.Configuration.DatabaseType;
-import org.apache.ambari.server.orm.DBAccessor.DBColumnInfo;
-import org.apache.ambari.server.orm.dao.ClusterDAO;
-import org.apache.ambari.server.orm.dao.ClusterServiceDAO;
-import org.apache.ambari.server.orm.dao.ClusterStateDAO;
-import org.apache.ambari.server.orm.dao.HostComponentDesiredStateDAO;
-import org.apache.ambari.server.orm.dao.HostComponentStateDAO;
-import org.apache.ambari.server.orm.dao.HostDAO;
-import org.apache.ambari.server.orm.dao.HostRoleCommandDAO;
-import org.apache.ambari.server.orm.dao.KeyValueDAO;
-import org.apache.ambari.server.orm.dao.ServiceComponentDesiredStateDAO;
-import org.apache.ambari.server.orm.entities.ClusterConfigEntity;
-import org.apache.ambari.server.orm.entities.ClusterConfigMappingEntity;
-import org.apache.ambari.server.orm.entities.ClusterEntity;
-import org.apache.ambari.server.orm.entities.ClusterServiceEntity;
-import org.apache.ambari.server.orm.entities.ClusterServiceEntityPK;
-import org.apache.ambari.server.orm.entities.ClusterStateEntity;
-import org.apache.ambari.server.orm.entities.HostComponentDesiredStateEntity;
-import org.apache.ambari.server.orm.entities.HostComponentDesiredStateEntityPK;
-import org.apache.ambari.server.orm.entities.HostComponentStateEntity;
-import org.apache.ambari.server.orm.entities.HostEntity;
-import org.apache.ambari.server.orm.entities.HostRoleCommandEntity;
-import org.apache.ambari.server.orm.entities.KeyValueEntity;
-import org.apache.ambari.server.orm.entities.ServiceComponentDesiredStateEntity;
-import org.apache.ambari.server.orm.entities.StackEntity;
-import org.apache.ambari.server.state.HostComponentAdminState;
-import org.apache.ambari.server.state.PropertyInfo;
-import org.apache.ambari.server.state.ServiceInfo;
-import org.apache.ambari.server.state.State;
-import org.eclipse.persistence.jpa.JpaEntityManager;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import com.google.gson.Gson;
-import com.google.inject.Inject;
-import com.google.inject.Injector;
-
-public class UpgradeCatalog150 extends AbstractUpgradeCatalog {
- private static final Logger LOG = LoggerFactory.getLogger(UpgradeCatalog150.class);
- private static final String quartzScriptFilePattern = "quartz.%s.sql";
- private Injector injector;
-
- @Inject
- public UpgradeCatalog150(Injector injector) {
- super(injector);
- this.injector = injector;
- }
-
- @Override
- public String getTargetVersion() {
- return "1.5.0";
- }
-
- @Override
- public void executeDDLUpdates() throws AmbariException, SQLException {
- LOG.debug("Upgrading schema...");
- DatabaseType databaseType = configuration.getDatabaseType();
- List<DBColumnInfo> columns = new ArrayList<DBColumnInfo>();
-
- // ========================================================================
- // Create tables
-
- // ClusterConfigMapping
- columns.add(new DBColumnInfo("cluster_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("type_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("version_tag", String.class, 255, null, false));
- columns.add(new DBColumnInfo("create_timestamp", Long.class, null, null, false));
- columns.add(new DBColumnInfo("selected", Integer.class, 0, null, false));
- columns.add(new DBColumnInfo("user_name", String.class, 255, "_db", false));
-
- dbAccessor.createTable("clusterconfigmapping", columns, "cluster_id", "type_name", "create_timestamp");
-
- // Request
- columns.clear();
- columns.add(new DBColumnInfo("request_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("cluster_id", Long.class, null, null, true));
- columns.add(new DBColumnInfo("request_schedule_id", Long.class, null, null, true));
- columns.add(new DBColumnInfo("command_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("create_time", Long.class, null, null, true));
- columns.add(new DBColumnInfo("end_time", Long.class, null, null, true));
- columns.add(new DBColumnInfo("inputs", byte[].class, null, null, true));
- columns.add(new DBColumnInfo("request_context", String.class, 255, null, true));
- columns.add(new DBColumnInfo("request_type", String.class, 255, null, true));
- columns.add(new DBColumnInfo("start_time", Long.class, null, null, false));
- columns.add(new DBColumnInfo("status", String.class, 255));
-
- dbAccessor.createTable("request", columns, "request_id");
-
- // RequestSchedule
- columns.clear();
- columns.add(new DBColumnInfo("schedule_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("cluster_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("description", String.class, 255, null, true));
- columns.add(new DBColumnInfo("status", String.class, 255, null, true));
- columns.add(new DBColumnInfo("batch_separation_seconds", Integer.class, null, null, true));
- columns.add(new DBColumnInfo("batch_toleration_limit", Integer.class, null, null, true));
- columns.add(new DBColumnInfo("create_user", String.class, 255, null, true));
- columns.add(new DBColumnInfo("create_timestamp", Long.class, null, null, true));
- columns.add(new DBColumnInfo("update_user", String.class, 255, null, true));
- columns.add(new DBColumnInfo("update_timestamp", Long.class, null, null, true));
- columns.add(new DBColumnInfo("minutes", String.class, 10, null, true));
- columns.add(new DBColumnInfo("hours", String.class, 10, null, true));
- columns.add(new DBColumnInfo("days_of_month", String.class, 10, null, true));
- columns.add(new DBColumnInfo("month", String.class, 10, null, true));
- columns.add(new DBColumnInfo("day_of_week", String.class, 10, null, true));
- columns.add(new DBColumnInfo("yearToSchedule", String.class, 10, null, true));
- columns.add(new DBColumnInfo("startTime", String.class, 50, null, true));
- columns.add(new DBColumnInfo("endTime", String.class, 50, null, true));
- columns.add(new DBColumnInfo("last_execution_status", String.class, 255, null, true));
-
- dbAccessor.createTable("requestschedule", columns, "schedule_id");
-
- // RequestScheduleBatchRequest
- columns.clear();
- columns.add(new DBColumnInfo("schedule_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("batch_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("request_id", Long.class, null, null, true));
- columns.add(new DBColumnInfo("request_type", String.class, 255, null, true));
- columns.add(new DBColumnInfo("request_uri", String.class, 1024, null, true));
- columns.add(new DBColumnInfo("request_body", byte[].class, null, null, true));
- columns.add(new DBColumnInfo("request_status", String.class, 255, null, true));
- columns.add(new DBColumnInfo("return_code", Integer.class, null, null, true));
- columns.add(new DBColumnInfo("return_message", String.class, 2000, null, true));
-
- dbAccessor.createTable("requestschedulebatchrequest", columns, "schedule_id", "batch_id");
-
- // HostConfigMapping
- columns.clear();
- columns.add(new DBColumnInfo("cluster_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("host_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("type_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("version_tag", String.class, 255, null, true));
- columns.add(new DBColumnInfo("service_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("create_timestamp", Long.class, null, null, false));
- columns.add(new DBColumnInfo("selected", Integer.class, 0, null, false));
-
- dbAccessor.createTable("hostconfigmapping", columns, "cluster_id", "host_name", "type_name", "create_timestamp");
-
- // Sequences
- columns.clear();
- columns.add(new DBColumnInfo("sequence_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("value", Long.class, null, null, false));
-
- dbAccessor.createTable("ambari_sequences", columns, "sequence_name");
-
- // Metainfo
-
- columns.clear();
- columns.add(new DBColumnInfo("metainfo_key", String.class, 255, null, false));
- columns.add(new DBColumnInfo("metainfo_value", String.class, 255, null, false));
-
- dbAccessor.createTable("metainfo", columns, "metainfo_key");
-
- // ConfigGroup
- columns.clear();
- columns.add(new DBColumnInfo("group_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("cluster_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("group_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("tag", String.class, 1024, null, false));
- columns.add(new DBColumnInfo("description", String.class, 1024, null, true));
- columns.add(new DBColumnInfo("create_timestamp", Long.class, null, null, false));
-
- dbAccessor.createTable("configgroup", columns, "group_id");
-
- // ConfigGroupClusterConfigMapping
- columns.clear();
- columns.add(new DBColumnInfo("config_group_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("cluster_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("config_type", String.class, 255, null, false));
- columns.add(new DBColumnInfo("version_tag", String.class, 255, null, false));
- columns.add(new DBColumnInfo("user_name", String.class, 255, "_db", true));
- columns.add(new DBColumnInfo("create_timestamp", Long.class, null, null, false));
-
- dbAccessor.createTable("confgroupclusterconfigmapping", columns, "config_group_id", "cluster_id", "config_type");
-
- // ConfigGroupHostMapping
- columns.clear();
- columns.add(new DBColumnInfo("config_group_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("host_name", String.class, 255, null, false));
-
- dbAccessor.createTable("configgrouphostmapping", columns, "config_group_id", "host_name");
-
- // Blueprint
- columns.clear();
- columns.add(new DBColumnInfo("blueprint_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("stack_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("stack_version", String.class, 255, null, false));
-
- dbAccessor.createTable("blueprint", columns, "blueprint_name");
-
- // Blueprint Config
- columns.clear();
- columns.add(new DBColumnInfo("blueprint_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("type_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("config_data", byte[].class, null, null, false));
-
- dbAccessor.createTable("blueprint_configuration", columns, "blueprint_name", "type_name");
-
- // HostGroup
- columns.clear();
- columns.add(new DBColumnInfo("blueprint_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("cardinality", String.class, 255, null, false));
-
- dbAccessor.createTable("hostgroup", columns, "blueprint_name", "name");
-
- // HostGroupComponent
- columns.clear();
- columns.add(new DBColumnInfo("blueprint_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("hostgroup_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("name", String.class, 255, null, false));
-
- dbAccessor.createTable("hostgroup_component", columns, "blueprint_name", "hostgroup_name", "name");
-
- // RequestResourceFilter
- columns.clear();
- columns.add(new DBColumnInfo("filter_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("request_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("service_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("component_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("hosts", byte[].class, null, null, true));
-
- dbAccessor.createTable("requestresourcefilter", columns, "filter_id");
-
- createQuartzTables();
-
- // ========================================================================
- // Add columns
-
- dbAccessor.addColumn("hostcomponentdesiredstate", new DBColumnInfo("maintenance_state", String.class, 32, "OFF", false));
- dbAccessor.addColumn("servicedesiredstate", new DBColumnInfo("maintenance_state", String.class, 32, "OFF", false));
- dbAccessor.addColumn("hoststate", new DBColumnInfo("maintenance_state", String.class, 512, null, true));
- dbAccessor.addColumn("hostcomponentdesiredstate", new DBColumnInfo("admin_state", String.class, 32, null, true));
- dbAccessor.addColumn("hosts", new DBColumnInfo("ph_cpu_count", Integer.class, 32, null, true));
- dbAccessor.addColumn("clusterstate", new DBColumnInfo("current_stack_version", String.class, 255, null, false));
- dbAccessor.addColumn("hostconfigmapping", new DBColumnInfo("user_name", String.class, 255, "_db", false));
- dbAccessor.addColumn("stage", new DBColumnInfo("request_context", String.class, 255, null, true));
- dbAccessor.addColumn("stage", new DBColumnInfo("cluster_host_info", byte[].class, null, null, true));
- dbAccessor.addColumn("clusterconfigmapping", new DBColumnInfo("user_name", String.class, 255, "_db", false));
- dbAccessor.addColumn("host_role_command", new DBColumnInfo("end_time", Long.class, null, null, true));
- dbAccessor.addColumn("host_role_command", new DBColumnInfo("structured_out", byte[].class, null, null, true));
- dbAccessor.addColumn("host_role_command", new DBColumnInfo("command_detail", String.class, 255, null, true));
- dbAccessor.addColumn("host_role_command", new DBColumnInfo("custom_command_name", String.class, 255, null, true));
-
- // Alter columns
-
- if (databaseType == DatabaseType.POSTGRES) {
- if (dbAccessor.tableExists("hostcomponentdesiredconfigmapping")) {
- dbAccessor.executeQuery("ALTER TABLE hostcomponentdesiredconfigmapping rename to hcdesiredconfigmapping", true);
- }
- dbAccessor.executeQuery("ALTER TABLE users ALTER column user_id DROP DEFAULT", true);
- dbAccessor.executeQuery("ALTER TABLE users ALTER column ldap_user TYPE INTEGER USING CASE WHEN ldap_user=true THEN 1 ELSE 0 END", true);
- }
-
- if (databaseType == DatabaseType.ORACLE || databaseType == DatabaseType.POSTGRES) {
- if (dbAccessor.tableHasColumn("hosts", "disks_info")) {
- dbAccessor.executeQuery("ALTER TABLE hosts DROP COLUMN disks_info", true);
- }
- }
-
-
- //Move tables from ambarirca db to ambari db; drop ambarirca; Mysql
- if (databaseType == DatabaseType.MYSQL) {
- String dbName = configuration.getServerJDBCPostgresSchemaName();
- moveRCATableInMySQL("workflow", dbName);
- moveRCATableInMySQL("job", dbName);
- moveRCATableInMySQL("task", dbName);
- moveRCATableInMySQL("taskAttempt", dbName);
- moveRCATableInMySQL("hdfsEvent", dbName);
- moveRCATableInMySQL("mapreduceEvent", dbName);
- moveRCATableInMySQL("clusterEvent", dbName);
- dbAccessor.executeQuery("DROP DATABASE IF EXISTS ambarirca");
- }
-
- //Newly created tables should be filled before creating FKs
- // Request Entries
- String tableName = "request";
- if (!dbAccessor.tableExists(tableName)) {
- String msg = String.format("Table \"%s\" was not created during schema upgrade", tableName);
- LOG.error(msg);
- throw new AmbariException(msg);
- } else if (!dbAccessor.tableHasData(tableName)) {
- String query = null;
- if (databaseType == DatabaseType.POSTGRES) {
- query = getPostgresRequestUpgradeQuery();
- } else if (databaseType == DatabaseType.ORACLE) {
- query = getOracleRequestUpgradeQuery();
- } else if (databaseType == DatabaseType.MYSQL) {
- query = getMysqlRequestUpgradeQuery();
- }
-
- if (query != null) {
- dbAccessor.executeQuery(query);
- }
- } else {
- LOG.info("Table {} already filled", tableName);
- }
-
- // Drop old constraints
- // ========================================================================
- if (databaseType == DatabaseType.POSTGRES
- || databaseType == DatabaseType.MYSQL
- || databaseType == DatabaseType.DERBY) {
-
- //recreate old constraints to sync with oracle
- dbAccessor.dropFKConstraint("clusterconfigmapping", "FK_clusterconfigmapping_cluster_id");
- dbAccessor.dropFKConstraint("hostcomponentdesiredstate", "FK_hostcomponentdesiredstate_host_name");
- dbAccessor.dropFKConstraint("hostcomponentdesiredstate", "FK_hostcomponentdesiredstate_component_name");
- dbAccessor.dropFKConstraint("hostcomponentstate", "FK_hostcomponentstate_component_name");
- dbAccessor.dropFKConstraint("hostcomponentstate", "FK_hostcomponentstate_host_name");
- dbAccessor.dropFKConstraint("servicecomponentdesiredstate", "FK_servicecomponentdesiredstate_service_name");
- dbAccessor.dropFKConstraint("servicedesiredstate", "FK_servicedesiredstate_service_name");
- dbAccessor.dropFKConstraint("role_success_criteria", "FK_role_success_criteria_stage_id");
- dbAccessor.dropFKConstraint("ClusterHostMapping", "FK_ClusterHostMapping_host_name");
- dbAccessor.dropFKConstraint("ClusterHostMapping", "FK_ClusterHostMapping_cluster_id");
-
- dbAccessor.addFKConstraint("clusterconfigmapping", "clusterconfigmappingcluster_id", "cluster_id", "clusters", "cluster_id", false);
- dbAccessor.addFKConstraint("hostcomponentdesiredstate", "hstcmponentdesiredstatehstname", "host_name", "hosts", "host_name", false);
- dbAccessor.addFKConstraint("hostcomponentdesiredstate", "hstcmpnntdesiredstatecmpnntnme",
- new String[] {"component_name", "cluster_id", "service_name"}, "servicecomponentdesiredstate",
- new String[] {"component_name", "cluster_id", "service_name"}, false);
- dbAccessor.addFKConstraint("hostcomponentstate", "hstcomponentstatecomponentname",
- new String[] {"component_name", "cluster_id", "service_name"}, "servicecomponentdesiredstate",
- new String[] {"component_name", "cluster_id", "service_name"}, false);
- dbAccessor.addFKConstraint("hostcomponentstate", "hostcomponentstate_host_name", "host_name", "hosts", "host_name", false);
- dbAccessor.addFKConstraint("servicecomponentdesiredstate", "srvccmponentdesiredstatesrvcnm",
- new String[] {"service_name", "cluster_id"}, "clusterservices",
- new String[] {"service_name", "cluster_id"}, false);
- dbAccessor.addFKConstraint("servicedesiredstate", "servicedesiredstateservicename",
- new String[] {"service_name", "cluster_id"}, "clusterservices",
- new String[] {"service_name", "cluster_id"}, false);
- dbAccessor.addFKConstraint("role_success_criteria", "role_success_criteria_stage_id",
- new String[] {"stage_id", "request_id"}, "stage",
- new String[] {"stage_id", "request_id"}, false);
- dbAccessor.addFKConstraint("ClusterHostMapping", "ClusterHostMapping_cluster_id", "cluster_id", "clusters", "cluster_id", false);
- dbAccessor.addFKConstraint("ClusterHostMapping", "ClusterHostMapping_host_name", "host_name", "hosts", "host_name", false);
-
-
- //drop new constraints with to sync with oracle
- dbAccessor.dropFKConstraint("confgroupclusterconfigmapping", "FK_confgroupclusterconfigmapping_config_tag", true);
- dbAccessor.dropFKConstraint("confgroupclusterconfigmapping", "FK_confgroupclusterconfigmapping_group_id", true);
- dbAccessor.dropFKConstraint("configgrouphostmapping", "FK_configgrouphostmapping_configgroup_id", true);
- dbAccessor.dropFKConstraint("configgrouphostmapping", "FK_configgrouphostmapping_host_name", true);
-
-
- }
-
- // ========================================================================
- // Add constraints
-
- dbAccessor.addFKConstraint("stage", "FK_stage_request_id", "request_id", "request", "request_id", true);
- dbAccessor.addFKConstraint("request", "FK_request_cluster_id", "cluster_id", "clusters", "cluster_id", true);
- dbAccessor.addFKConstraint("request", "FK_request_schedule_id", "request_schedule_id", "requestschedule", "schedule_id", true);
- dbAccessor.addFKConstraint("requestschedulebatchrequest", "FK_rsbatchrequest_schedule_id", "schedule_id", "requestschedule", "schedule_id", true);
- dbAccessor.addFKConstraint("hostconfigmapping", "FK_hostconfmapping_cluster_id", "cluster_id", "clusters", "cluster_id", true);
- dbAccessor.addFKConstraint("hostconfigmapping", "FK_hostconfmapping_host_name", "host_name", "hosts", "host_name", true);
- dbAccessor.addFKConstraint("configgroup", "FK_configgroup_cluster_id", "cluster_id", "clusters", "cluster_id", true);
- dbAccessor.addFKConstraint("confgroupclusterconfigmapping", "FK_confg", new String[] {"version_tag", "config_type", "cluster_id"}, "clusterconfig", new String[] {"version_tag", "type_name", "cluster_id"}, true);
- dbAccessor.addFKConstraint("confgroupclusterconfigmapping", "FK_cgccm_gid", "config_group_id", "configgroup", "group_id", true);
- dbAccessor.addFKConstraint("configgrouphostmapping", "FK_cghm_cgid", "config_group_id", "configgroup", "group_id", true);
- dbAccessor.addFKConstraint("configgrouphostmapping", "FK_cghm_hname", "host_name", "hosts", "host_name", true);
- dbAccessor.addFKConstraint("clusterconfigmapping", "FK_clustercfgmap_cluster_id", "cluster_id", "clusters", "cluster_id", true);
- dbAccessor.addFKConstraint("requestresourcefilter", "FK_reqresfilter_req_id", "request_id", "request", "request_id", true);
- dbAccessor.addFKConstraint("hostgroup", "FK_hostgroup_blueprint_name", "blueprint_name", "blueprint", "blueprint_name", true);
- dbAccessor.addFKConstraint("hostgroup_component", "FK_hg_blueprint_name", "blueprint_name", "hostgroup", "blueprint_name", true);
- dbAccessor.addFKConstraint("hostgroup_component", "FK_hgc_blueprint_name", "hostgroup_name", "hostgroup", "name", true);
- dbAccessor.addFKConstraint("blueprint_configuration", "FK_cfg_blueprint_name", "blueprint_name", "blueprint", "blueprint_name", true);
- }
-
- private void moveRCATableInMySQL(String tableName, String dbName) throws SQLException {
- if (!dbAccessor.tableExists(tableName)) {
- dbAccessor.executeQuery(String.format("RENAME TABLE ambarirca.%s TO %s.%s", tableName, dbName, tableName), true);
- }
- }
-
- /**
- * {@inheritDoc}
- */
- @Override
- public void executePreDMLUpdates() {
- ;
- }
-
- @Override
- public void executeDMLUpdates() throws AmbariException, SQLException {
- // Service Config mapping
- String tableName = "serviceconfigmapping";
- DatabaseType databaseType = configuration.getDatabaseType();
-
- EntityManager em = getEntityManagerProvider().get();
-
- //unable to do via dao, as they were dropped
- //TODO should we move this to ddl and drop unused tables then?
- if (dbAccessor.tableExists(tableName)
- && dbAccessor.tableHasData(tableName)
- && dbAccessor.tableExists("clusterconfigmapping")) {
-
- if (databaseType == DatabaseType.POSTGRES) {
- // Service config mapping entity object will be deleted so need to
- // proceed with executing as query
-
- dbAccessor.executeQuery(getPostgresServiceConfigMappingQuery());
-
- dbAccessor.truncateTable(tableName);
-
- } else {
- LOG.warn("Unsupported database for service config mapping query. " +
- "database = " + databaseType);
- }
- }
-
- // Sequences
- if (dbAccessor.tableExists("ambari_sequences")) {
- if (databaseType == DatabaseType.POSTGRES) {
- Statement statement = null;
- ResultSet rs = null;
- try {
- statement = dbAccessor.getConnection().createStatement();
- if (statement != null) {
- rs = statement.executeQuery("select * from ambari_sequences where sequence_name in " +
- "('cluster_id_seq','user_id_seq','host_role_command_id_seq')");
- if (rs != null) {
- if (!rs.next()) {
- dbAccessor.executeQuery(getPostgresSequenceUpgradeQuery(), true);
- // Deletes
- dbAccessor.dropSequence("host_role_command_task_id_seq");
- dbAccessor.dropSequence("users_user_id_seq");
- dbAccessor.dropSequence("clusters_cluster_id_seq");
- }
- }
- }
- } finally {
- if (rs != null) {
- rs.close();
- }
- if (statement != null) {
- statement.close();
- }
- }
- }
- }
-
- //add new sequences for config groups
- String valueColumnName = "sequence_value";
-
- dbAccessor.executeQuery("INSERT INTO ambari_sequences(sequence_name, " + valueColumnName + ") " +
- "VALUES('configgroup_id_seq', 1)", true);
- dbAccessor.executeQuery("INSERT INTO ambari_sequences(sequence_name, " + valueColumnName + ") " +
- "VALUES('requestschedule_id_seq', 1)", true);
- dbAccessor.executeQuery("INSERT INTO ambari_sequences(sequence_name, " + valueColumnName + ") " +
- "VALUES('resourcefilter_id_seq', 1)", true);
-
- //clear cache due to direct table manipulation
- ((JpaEntityManager)em.getDelegate()).getServerSession().getIdentityMapAccessor().invalidateAll();
-
- // Updates
-
- // HostComponentState - reverted to native query due to incorrect criteria api usage
- // (it forces us to use enums not strings, which were deleted)
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- EntityManager em = getEntityManagerProvider().get();
- Query nativeQuery = em.createNativeQuery("UPDATE hostcomponentstate SET current_state=?1 WHERE current_state in (?2, ?3)");
- nativeQuery.setParameter(1, "INSTALLED");
- nativeQuery.setParameter(2, "STOP_FAILED");
- nativeQuery.setParameter(3, "START_FAILED");
- nativeQuery.executeUpdate();
- }
- });
-
- // HostRoleCommand
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- EntityManager em = getEntityManagerProvider().get();
- CriteriaBuilder cb = em.getCriteriaBuilder();
- CriteriaQuery<HostRoleCommandEntity> c2 = cb.createQuery(HostRoleCommandEntity.class);
- Root<HostRoleCommandEntity> hrc = c2.from(HostRoleCommandEntity.class);
- List<HostRoleStatus> statuses = new ArrayList<HostRoleStatus>() {{
- add(HostRoleStatus.PENDING);
- add(HostRoleStatus.QUEUED);
- add(HostRoleStatus.IN_PROGRESS);
- }};
- Expression<String> exp = hrc.get("status");
- Predicate predicate = exp.in(statuses);
- c2.select(hrc).where(predicate);
-
- TypedQuery<HostRoleCommandEntity> q2 = em.createQuery(c2);
- List<HostRoleCommandEntity> r2 = q2.getResultList();
-
- HostRoleCommandDAO hostRoleCommandDAO = injector.getInstance(HostRoleCommandDAO.class);
- if (r2 != null && !r2.isEmpty()) {
- for (HostRoleCommandEntity entity : r2) {
- entity.setStatus(HostRoleStatus.ABORTED);
- hostRoleCommandDAO.merge(entity);
- }
- }
- }
- });
-
- // Stack version changes from HDPLocal to HDP
- stackUpgradeUtil.updateStackDetails("HDP", null);
-
- //create cluster state entities if not present
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- ClusterStateDAO clusterStateDAO = injector.getInstance(ClusterStateDAO.class);
- List<ClusterEntity> clusterEntities = clusterDAO.findAll();
- for (ClusterEntity clusterEntity : clusterEntities) {
- if (clusterStateDAO.findByPK(clusterEntity.getClusterId()) == null) {
- ClusterStateEntity clusterStateEntity = new ClusterStateEntity();
- clusterStateEntity.setClusterEntity(clusterEntity);
- clusterStateEntity.setCurrentStack(clusterEntity.getDesiredStack());
-
- clusterStateDAO.create(clusterStateEntity);
-
- clusterEntity.setClusterStateEntity(clusterStateEntity);
-
- clusterDAO.merge(clusterEntity);
- }
- }
- }
- });
-
- // add history server on the host where jobtracker is
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- addHistoryServer();
- }
- });
-
- // Add default log4j configs if they are absent
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- addMissingLog4jConfigs();
- }
- });
-
- // Move decommissioned datanode data to new table
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- try {
- processDecommissionedDatanodes();
- } catch (Exception e) {
- LOG.warn("Updating decommissioned datanodes to new format threw " +
- "exception. ", e);
- }
- }
- });
- }
-
- protected void addHistoryServer() {
- ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- ClusterServiceDAO clusterServiceDAO = injector.getInstance(ClusterServiceDAO.class);
- ServiceComponentDesiredStateDAO serviceComponentDesiredStateDAO = injector.getInstance(ServiceComponentDesiredStateDAO.class);
- HostDAO hostDao = injector.getInstance(HostDAO.class);
-
- List<ClusterEntity> clusterEntities = clusterDAO.findAll();
- for (final ClusterEntity clusterEntity : clusterEntities) {
- ServiceComponentDesiredStateEntity serviceComponentDesiredStateEntityHS = serviceComponentDesiredStateDAO.findByName(
- clusterEntity.getClusterId(), "MAPREDUCE", "HISTORYSERVER");
-
- // already have historyserver
- if(serviceComponentDesiredStateEntityHS != null) {
- continue;
- }
-
- ServiceComponentDesiredStateEntity serviceComponentDesiredStateEntityJT = serviceComponentDesiredStateDAO.findByName(
- clusterEntity.getClusterId(), "MAPREDUCE", "JOBTRACKER");
-
- // no jobtracker present probably mapreduce is not installed
- if(serviceComponentDesiredStateEntityJT == null) {
- continue;
- }
-
-
- HostComponentStateEntity jtHostComponentStateEntity = serviceComponentDesiredStateEntityJT.getHostComponentStateEntities().iterator().next();
- HostComponentDesiredStateEntity jtHostComponentDesiredStateEntity = serviceComponentDesiredStateEntityJT.getHostComponentDesiredStateEntities().iterator().next();
- String jtHostname = jtHostComponentStateEntity.getHostName();
- State jtCurrState = jtHostComponentStateEntity.getCurrentState();
- State jtHostComponentDesiredState = jtHostComponentDesiredStateEntity.getDesiredState();
- State jtServiceComponentDesiredState = serviceComponentDesiredStateEntityJT.getDesiredState();
-
- ClusterServiceEntityPK pk = new ClusterServiceEntityPK();
- pk.setClusterId(clusterEntity.getClusterId());
- pk.setServiceName("MAPREDUCE");
-
- ClusterServiceEntity clusterServiceEntity = clusterServiceDAO.findByPK(pk);
-
- final ServiceComponentDesiredStateEntity serviceComponentDesiredStateEntity = new ServiceComponentDesiredStateEntity();
- serviceComponentDesiredStateEntity.setClusterId(clusterEntity.getClusterId());
- serviceComponentDesiredStateEntity.setComponentName("HISTORYSERVER");
- serviceComponentDesiredStateEntity.setDesiredStack(clusterEntity.getDesiredStack());
- serviceComponentDesiredStateEntity.setDesiredState(jtServiceComponentDesiredState);
- serviceComponentDesiredStateEntity.setClusterServiceEntity(clusterServiceEntity);
- serviceComponentDesiredStateEntity.setHostComponentDesiredStateEntities(new ArrayList<HostComponentDesiredStateEntity>());
- serviceComponentDesiredStateEntity.setHostComponentStateEntities(new ArrayList<HostComponentStateEntity>());
-
- serviceComponentDesiredStateDAO.create(serviceComponentDesiredStateEntity);
-
- final HostEntity host = hostDao.findByName(jtHostname);
- if (host == null) {
- continue;
- }
-
- final HostComponentStateEntity stateEntity = new HostComponentStateEntity();
- stateEntity.setHostEntity(host);
- stateEntity.setCurrentState(jtCurrState);
- stateEntity.setCurrentStack(clusterEntity.getDesiredStack());
- stateEntity.setClusterId(clusterEntity.getClusterId());
-
- final HostComponentDesiredStateEntity desiredStateEntity = new HostComponentDesiredStateEntity();
- desiredStateEntity.setDesiredState(jtHostComponentDesiredState);
- desiredStateEntity.setDesiredStack(clusterEntity.getDesiredStack());
- desiredStateEntity.setClusterId(clusterEntity.getClusterId());
-
- persistComponentEntities(stateEntity, desiredStateEntity, serviceComponentDesiredStateEntity);
- }
- }
-
- private void persistComponentEntities(HostComponentStateEntity stateEntity,
- HostComponentDesiredStateEntity desiredStateEntity,
- ServiceComponentDesiredStateEntity serviceComponentDesiredStateEntity) {
- ServiceComponentDesiredStateDAO serviceComponentDesiredStateDAO = injector.getInstance(ServiceComponentDesiredStateDAO.class);
- HostComponentStateDAO hostComponentStateDAO = injector.getInstance(HostComponentStateDAO.class);
- HostComponentDesiredStateDAO hostComponentDesiredStateDAO = injector.getInstance(HostComponentDesiredStateDAO.class);
- HostDAO hostDAO = injector.getInstance(HostDAO.class);
-
- HostEntity hostEntity = stateEntity.getHostEntity();
-
- desiredStateEntity.setHostEntity(hostEntity);
- desiredStateEntity.setServiceComponentDesiredStateEntity(serviceComponentDesiredStateEntity);
- serviceComponentDesiredStateEntity.getHostComponentDesiredStateEntities().add(desiredStateEntity);
- hostComponentDesiredStateDAO.create(desiredStateEntity);
-
- stateEntity.setHostEntity(hostEntity);
- stateEntity.setServiceComponentDesiredStateEntity(serviceComponentDesiredStateEntity);
- serviceComponentDesiredStateEntity.getHostComponentStateEntities().add(stateEntity);
- hostComponentStateDAO.create(stateEntity);
-
- serviceComponentDesiredStateDAO.merge(serviceComponentDesiredStateEntity);
-
- hostEntity.addHostComponentDesiredStateEntity(desiredStateEntity);
- hostEntity.addHostComponentStateEntity(stateEntity);
-
- hostDAO.merge(hostEntity);
- }
-
- protected void addMissingLog4jConfigs() {
-
- final String log4jConfigTypeContains = "log4j";
- final String defaultVersionTag = "version1";
- final String defaultUser = "admin";
-
- LOG.debug("Adding missing configs into Ambari DB.");
- ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- ClusterServiceDAO clusterServiceDAO = injector.getInstance(ClusterServiceDAO.class);
-
- AmbariMetaInfo ambariMetaInfo = injector.getInstance(AmbariMetaInfo.class);
- Gson gson = injector.getInstance(Gson.class);
-
- List <ClusterEntity> clusterEntities = clusterDAO.findAll();
- for (final ClusterEntity clusterEntity : clusterEntities) {
- Long clusterId = clusterEntity.getClusterId();
- StackEntity stackEntity = clusterEntity.getDesiredStack();
- String stackName = stackEntity.getStackName();
- String stackVersion = stackEntity.getStackVersion();
-
- List<ClusterServiceEntity> clusterServiceEntities = clusterServiceDAO.findAll();
- for (final ClusterServiceEntity clusterServiceEntity : clusterServiceEntities) {
- String serviceName = clusterServiceEntity.getServiceName();
- ServiceInfo serviceInfo = null;
- try {
- serviceInfo = ambariMetaInfo.getService(stackName, stackVersion, serviceName);
- } catch (AmbariException e) {
- LOG.error("Service " + serviceName + " not found for " + stackName + stackVersion);
- continue;
- }
- List<String> configTypes = serviceInfo.getConfigDependencies();
- if (configTypes != null) {
- for (String configType : configTypes) {
- if (configType.contains(log4jConfigTypeContains)) {
- ClusterConfigEntity configEntity = clusterDAO.findConfig(clusterId, configType, defaultVersionTag);
-
- if (configEntity == null) {
- String filename = configType + ".xml";
- Map<String, String> properties = new HashMap<String, String>();
- for (PropertyInfo propertyInfo : serviceInfo.getProperties()) {
- if (filename.equals(propertyInfo.getFilename())) {
- properties.put(propertyInfo.getName(), propertyInfo.getValue());
- }
- }
- if (!properties.isEmpty()) {
- String configData = gson.toJson(properties);
- configEntity = new ClusterConfigEntity();
- configEntity.setClusterId(clusterId);
- configEntity.setType(configType);
- configEntity.setTag(defaultVersionTag);
- configEntity.setData(configData);
- configEntity.setVersion(1L);
- configEntity.setTimestamp(System.currentTimeMillis());
- configEntity.setClusterEntity(clusterEntity);
- configEntity.setStack(stackEntity);
-
- LOG.debug("Creating new " + configType + " config...");
- clusterDAO.createConfig(configEntity);
-
- Collection<ClusterConfigMappingEntity> entities =
- clusterEntity.getConfigMappingEntities();
-
- ClusterConfigMappingEntity clusterConfigMappingEntity =
- new ClusterConfigMappingEntity();
- clusterConfigMappingEntity.setClusterEntity(clusterEntity);
- clusterConfigMappingEntity.setClusterId(clusterId);
- clusterConfigMappingEntity.setType(configType);
- clusterConfigMappingEntity.setCreateTimestamp(
- Long.valueOf(System.currentTimeMillis()));
- clusterConfigMappingEntity.setSelected(1);
- clusterConfigMappingEntity.setUser(defaultUser);
- clusterConfigMappingEntity.setTag(configEntity.getTag());
- entities.add(clusterConfigMappingEntity);
- clusterDAO.persistConfigMapping(clusterConfigMappingEntity);
- clusterDAO.merge(clusterEntity);
- }
- }
- }
-
- }
-
- }
- }
- }
- LOG.debug("Missing configs have been successfully added into Ambari DB.");
- }
-
- protected void processDecommissionedDatanodes() {
- KeyValueDAO keyValueDAO = injector.getInstance(KeyValueDAO.class);
- ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- HostDAO hostDAO = injector.getInstance(HostDAO.class);
- Gson gson = injector.getInstance(Gson.class);
- HostComponentDesiredStateDAO desiredStateDAO = injector.getInstance
- (HostComponentDesiredStateDAO.class);
-
- KeyValueEntity keyValueEntity = keyValueDAO.findByKey("decommissionDataNodesTag");
- String value = null;
- if (keyValueEntity != null) {
- value = keyValueEntity.getValue();
- if (value != null && !value.isEmpty()) {
- List<ClusterEntity> clusterEntities = clusterDAO.findAll();
- for (ClusterEntity clusterEntity : clusterEntities) {
- Long clusterId = clusterEntity.getClusterId();
- ClusterConfigEntity configEntity = clusterDAO.findConfig(clusterId, "hdfs-exclude-file", value.trim());
- if (configEntity != null) {
- String configData = configEntity.getData();
- if (configData != null) {
- Map<String, String> properties = gson.<Map<String, String>>fromJson(configData, Map.class);
- if (properties != null && !properties.isEmpty()) {
- String decommissionedNodes = properties.get("datanodes");
- if (decommissionedNodes != null) {
- String[] nodes = decommissionedNodes.split(",");
- if (nodes.length > 0) {
- for (String node : nodes) {
- HostEntity hostEntity = hostDAO.findByName(node.trim());
- HostComponentDesiredStateEntityPK entityPK =
- new HostComponentDesiredStateEntityPK();
- entityPK.setClusterId(clusterId);
- entityPK.setServiceName("HDFS");
- entityPK.setComponentName("DATANODE");
- entityPK.setHostId(hostEntity.getHostId());
- HostComponentDesiredStateEntity desiredStateEntity =
- desiredStateDAO.findByPK(entityPK);
- desiredStateEntity.setAdminState(HostComponentAdminState.DECOMMISSIONED);
- desiredStateDAO.merge(desiredStateEntity);
- }
- }
- }
- }
- }
- }
- }
- }
- // Rename saved key value entity so that the move is finalized
- KeyValueEntity newEntity = new KeyValueEntity();
- newEntity.setKey("decommissionDataNodesTag-Moved");
- newEntity.setValue(value);
- keyValueDAO.create(newEntity);
- keyValueDAO.remove(keyValueEntity);
- }
- }
-
- private String getPostgresServiceConfigMappingQuery() {
- return "INSERT INTO clusterconfigmapping " +
- "(cluster_id, type_name, version_tag, create_timestamp, selected) " +
- "(SELECT DISTINCT cluster_id, config_type, config_tag, " +
- "cast(date_part('epoch', now()) as bigint), 1 " +
- "FROM serviceconfigmapping scm " +
- "WHERE timestamp = (SELECT max(timestamp) FROM serviceconfigmapping " +
- "WHERE cluster_id = scm.cluster_id AND config_type = scm.config_type))";
- }
-
- private String getPostgresSequenceUpgradeQuery() {
- return "INSERT INTO ambari_sequences(sequence_name, sequence_value) " +
- "SELECT 'cluster_id_seq', nextval('clusters_cluster_id_seq') " +
- "UNION ALL " +
- "SELECT 'user_id_seq', nextval('users_user_id_seq') " +
- "UNION ALL " +
- "SELECT 'host_role_command_id_seq', COALESCE((SELECT max(task_id) FROM host_role_command), 1) + 50 ";
- }
-
- private String getPostgresRequestUpgradeQuery() {
- return "insert into ambari.request(request_id, cluster_id, request_context, start_time, end_time, create_time) (\n" +
- " select distinct s.request_id, s.cluster_id, s.request_context, coalesce (cmd.start_time, -1), coalesce (cmd.end_time, -1), -1\n" +
- " from\n" +
- " (select distinct request_id, cluster_id, request_context from ambari.stage ) s\n" +
- " left join\n" +
- " (select request_id, min(start_time) as start_time, max(end_time) as end_time from ambari.host_role_command group by request_id) cmd\n" +
- " on s.request_id=cmd.request_id\n" +
- ")";
- }
-
- private String getOracleRequestUpgradeQuery() {
- return "INSERT INTO request" +
- "(request_id, cluster_id, request_context, start_time, end_time, create_time) " +
- "SELECT DISTINCT s.request_id, s.cluster_id, s.request_context, " +
- "nvl(cmd.start_time, -1), nvl(cmd.end_time, -1), -1 " +
- "FROM " +
- "(SELECT DISTINCT request_id, cluster_id, request_context FROM stage ) s " +
- "LEFT JOIN " +
- "(SELECT request_id, min(start_time) as start_time, max(end_time) " +
- "as end_time FROM host_role_command GROUP BY request_id) cmd " +
- "ON s.request_id=cmd.request_id";
- }
-
- private String getMysqlRequestUpgradeQuery() {
- return "insert into request" +
- "(request_id, cluster_id, request_context, start_time, end_time, create_time) " +
- "select distinct s.request_id, s.cluster_id, s.request_context, " +
- "coalesce(cmd.start_time, -1), coalesce(cmd.end_time, -1), -1 " +
- "from " +
- "(select distinct request_id, cluster_id, request_context from stage ) s " +
- "left join " +
- "(select request_id, min(start_time) as start_time, max(end_time) " +
- "as end_time from host_role_command group by request_id) cmd " +
- "on s.request_id=cmd.request_id";
- }
-
- private void createQuartzTables() throws SQLException {
- DatabaseType databaseType = configuration.getDatabaseType();
-
- // Run script to create quartz tables
- String scriptPath = configuration.getResourceDirPath() +
- File.separator + "upgrade" + File.separator + "ddl" +
- File.separator + String.format(quartzScriptFilePattern, databaseType.getName());
-
- try {
- dbAccessor.executeScript(scriptPath);
- } catch (IOException e) {
- LOG.error("Error reading file.", e);
- }
-
- }
-}
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog151.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog151.java b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog151.java
deleted file mode 100644
index 2ab296d..0000000
--- a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog151.java
+++ /dev/null
@@ -1,152 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.ambari.server.upgrade;
-
-import java.sql.SQLException;
-import java.util.ArrayList;
-import java.util.List;
-
-import org.apache.ambari.server.AmbariException;
-import org.apache.ambari.server.configuration.Configuration.DatabaseType;
-import org.apache.ambari.server.orm.DBAccessor;
-
-import com.google.inject.Inject;
-import com.google.inject.Injector;
-
-/**
- * Upgrade catalog for version 1.5.1.
- */
-public class UpgradeCatalog151 extends AbstractUpgradeCatalog {
-
- //SourceVersion is only for book-keeping purpose
- @Override
- public String getSourceVersion() {
- return "1.5.0";
- }
-
- @Override
- public String getTargetVersion() {
- return "1.5.1";
- }
-
- // ----- Constructors ------------------------------------------------------
-
- @Inject
- public UpgradeCatalog151(Injector injector) {
- super(injector);
- }
-
-
- // ----- AbstractUpgradeCatalog --------------------------------------------
-
- @Override
- protected void executeDDLUpdates() throws AmbariException, SQLException {
- List<DBAccessor.DBColumnInfo> columns = new ArrayList<DBAccessor.DBColumnInfo>();
- DatabaseType databaseType = configuration.getDatabaseType();
-
- // ========================================================================
- // Create tables
-
- // View
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("label", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("version", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("archive", String.class, 255, null, true));
-
- dbAccessor.createTable("viewmain", columns, "view_name");
-
- // View Instance Data
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("view_instance_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("value", String.class, 255, null, true));
-
- dbAccessor.createTable("viewinstancedata", columns, "view_name", "view_instance_name", "name");
-
- // View Instance
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("name", String.class, 255, null, false));
-
- dbAccessor.createTable("viewinstance", columns, "view_name", "name");
-
- // View Instance Property
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("view_instance_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("value", String.class, 255, null, true));
-
- dbAccessor.createTable("viewinstanceproperty", columns, "view_name", "view_instance_name", "name");
-
- // View Parameter
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("description", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("required", Character.class, 1, null, true));
-
- dbAccessor.createTable("viewparameter", columns, "view_name", "name");
-
- // View Resource
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("plural_name", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("id_property", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("subResource_names", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("provider", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("service", String.class, 255, null, true));
- if (databaseType == DatabaseType.MYSQL) {
- columns.add(new DBAccessor.DBColumnInfo("`resource`", String.class, 255, null, true));
- //TODO incorrect name for MySQL
-
- } else{
- columns.add(new DBAccessor.DBColumnInfo("\"resource\"", String.class, 255, null, true));
- //TODO incorrect name for oracle
- }
- dbAccessor.createTable("viewresource", columns, "view_name", "name");
-
- // ========================================================================
- // Add constraints
- dbAccessor.addFKConstraint("viewparameter", "FK_viewparam_view_name", "view_name", "viewmain", "view_name", true);
- dbAccessor.addFKConstraint("viewresource", "FK_viewres_view_name", "view_name", "viewmain", "view_name", true);
- dbAccessor.addFKConstraint("viewinstance", "FK_viewinst_view_name", "view_name", "viewmain", "view_name", true);
- dbAccessor.addFKConstraint("viewinstanceproperty", "FK_viewinstprop_view_name",
- new String[]{"view_name", "view_instance_name"}, "viewinstance", new String[]{"view_name", "name"}, true);
- dbAccessor.addFKConstraint("viewinstancedata", "FK_viewinstdata_view_name",
- new String[]{"view_name", "view_instance_name"}, "viewinstance", new String[]{"view_name", "name"}, true);
-
- }
-
-
- // ----- UpgradeCatalog ----------------------------------------------------
- /**
- * {@inheritDoc}
- */
- @Override
- public void executePreDMLUpdates() {
- ;
- }
-
- @Override
- public void executeDMLUpdates() throws AmbariException, SQLException {
- }
-}
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog160.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog160.java b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog160.java
deleted file mode 100644
index 2f7c84a..0000000
--- a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog160.java
+++ /dev/null
@@ -1,205 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ambari.server.upgrade;
-
-import java.sql.SQLException;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-import org.apache.ambari.server.AmbariException;
-import org.apache.ambari.server.configuration.Configuration.DatabaseType;
-import org.apache.ambari.server.orm.DBAccessor;
-
-import com.google.inject.Inject;
-import com.google.inject.Injector;
-
-/**
- * Upgrade catalog for version 1.6.0.
- */
-public class UpgradeCatalog160 extends AbstractUpgradeCatalog {
-
- //SourceVersion is only for book-keeping purpos
- @Override
- public String getSourceVersion() {
- return "1.5.1";
- }
-
- @Override
- public String getTargetVersion() {
- return "1.6.0";
- }
-
- // ----- Constructors ------------------------------------------------------
-
- @Inject
- public UpgradeCatalog160(Injector injector) {
- super(injector);
- }
-
-
- // ----- AbstractUpgradeCatalog --------------------------------------------
-
- @Override
- protected void executeDDLUpdates() throws AmbariException, SQLException {
- fixViewTablesForMysql();
-
- List<DBAccessor.DBColumnInfo> columns = new ArrayList<DBAccessor.DBColumnInfo>();
-
- // BP host group configuration
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("blueprint_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("hostgroup_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("type_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("config_data", char[].class, null, null, false));
-
- dbAccessor.createTable("hostgroup_configuration", columns, "blueprint_name",
- "hostgroup_name", "type_name");
-
- // View entity
- columns = new ArrayList<DBAccessor.DBColumnInfo>();
- columns.add(new DBAccessor.DBColumnInfo("id", Long.class, null, null, false));
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("view_instance_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("class_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("id_property", String.class, 255, null, true));
-
- dbAccessor.createTable("viewentity", columns, "id");
-
- //=========================================================================
- // Add columns
-
- // treat restart_required as an integer to maintain consistency with other
- // tables that have booleans
- DBAccessor.DBColumnInfo restartRequiredColumn = new DBAccessor.DBColumnInfo(
- "restart_required", Integer.class, 1, 0, false);
-
- dbAccessor.addColumn("hostcomponentdesiredstate", restartRequiredColumn);
-
- // ========================================================================
- // Add constraints
- dbAccessor.addFKConstraint("hostgroup_configuration", "FK_hg_config_blueprint_name",
- new String[] {"blueprint_name", "hostgroup_name"}, "hostgroup", new String[] {"blueprint_name", "name"}, true);
-
- dbAccessor.addFKConstraint("viewentity", "FK_viewentity_view_name",
- new String[]{"view_name", "view_instance_name"}, "viewinstance", new String[]{"view_name", "name"}, true);
- }
-
-
- protected void fixViewTablesForMysql() throws SQLException {
- //fixes 1.5.1 issue for mysql with non-default db name
- //view tables were not created
- DatabaseType databaseType = configuration.getDatabaseType();
- if (!(databaseType == DatabaseType.MYSQL)
- || "ambari".equals(configuration.getServerDBName())) {
- //no need to run for non-mysql dbms or default db name
- return;
- }
-
- List<DBAccessor.DBColumnInfo> columns = new ArrayList<DBAccessor.DBColumnInfo>();
-
- // ========================================================================
- // Create tables
-
- // View
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("label", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("version", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("archive", String.class, 255, null, true));
-
- dbAccessor.createTable("viewmain", columns, "view_name");
-
- // View Instance Data
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("view_instance_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("value", String.class, 255, null, true));
-
- dbAccessor.createTable("viewinstancedata", columns, "view_name", "view_instance_name", "name");
-
- // View Instance
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("name", String.class, 255, null, false));
-
- dbAccessor.createTable("viewinstance", columns, "view_name", "name");
-
- // View Instance Property
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("view_instance_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("value", String.class, 255, null, true));
-
- dbAccessor.createTable("viewinstanceproperty", columns, "view_name", "view_instance_name", "name");
-
- // View Parameter
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("description", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("required", Character.class, 1, null, true));
-
- dbAccessor.createTable("viewparameter", columns, "view_name", "name");
-
- // View Resource
- columns.clear();
- columns.add(new DBAccessor.DBColumnInfo("view_name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("name", String.class, 255, null, false));
- columns.add(new DBAccessor.DBColumnInfo("plural_name", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("id_property", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("subResource_names", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("provider", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("service", String.class, 255, null, true));
- columns.add(new DBAccessor.DBColumnInfo("`resource`", String.class, 255, null, true));
- dbAccessor.createTable("viewresource", columns, "view_name", "name");
-
- // ========================================================================
- // Add constraints
- dbAccessor.addFKConstraint("viewparameter", "FK_viewparam_view_name", "view_name", "viewmain", "view_name", true);
- dbAccessor.addFKConstraint("viewresource", "FK_viewres_view_name", "view_name", "viewmain", "view_name", true);
- dbAccessor.addFKConstraint("viewinstance", "FK_viewinst_view_name", "view_name", "viewmain", "view_name", true);
- dbAccessor.addFKConstraint("viewinstanceproperty", "FK_viewinstprop_view_name",
- new String[]{"view_name", "view_instance_name"}, "viewinstance", new String[]{"view_name", "name"}, true);
- dbAccessor.addFKConstraint("viewinstancedata", "FK_viewinstdata_view_name",
- new String[]{"view_name", "view_instance_name"}, "viewinstance", new String[]{"view_name", "name"}, true);
- }
-
-
- // ----- UpgradeCatalog ----------------------------------------------------
- /**
- * {@inheritDoc}
- */
- @Override
- public void executePreDMLUpdates() {
- ;
- }
-
- @Override
- protected void executeDMLUpdates() throws AmbariException, SQLException {
- //add new sequences for view entity
- dbAccessor.executeQuery("INSERT INTO ambari_sequences(sequence_name, sequence_value) " +
- "VALUES('viewentity_id_seq', 0)", true);
-
- // Add missing property for YARN
- updateConfigurationProperties("global", Collections.singletonMap("jobhistory_heapsize", "900"), false, false);
- }
-}
[18/31] ambari git commit: AMBARI-18846 - Custom services should be
able to easily specify their own dashboards
Posted by nc...@apache.org.
AMBARI-18846 - Custom services should be able to easily specify their own dashboards
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/b413d921
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/b413d921
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/b413d921
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: b413d921cdeca51ad8d36088100ee13e853bb925
Parents: d2118f5
Author: Tim Thorpe <tt...@apache.org>
Authored: Tue Nov 15 13:03:39 2016 -0800
Committer: Tim Thorpe <tt...@apache.org>
Committed: Tue Nov 15 13:03:39 2016 -0800
----------------------------------------------------------------------
.../ambari_agent/CustomServiceOrchestrator.py | 3 +-
.../src/main/python/ambari_agent/FileCache.py | 10 ++++
.../TestCustomServiceOrchestrator.py | 11 +++-
.../test/python/ambari_agent/TestFileCache.py | 12 ++++
.../python/ambari_server/resourceFilesKeeper.py | 12 ++--
.../python/ambari_server/serverConfiguration.py | 11 ++++
.../package/scripts/metrics_grafana_util.py | 8 +--
.../0.1.0/package/scripts/params.py | 3 +
.../0.1.0/package/scripts/split_points.py | 29 ++++++----
.../stacks/HDP/2.0.6/services/stack_advisor.py | 3 +-
.../src/test/python/TestResourceFilesKeeper.py | 61 +++++++++++++++++---
11 files changed, 131 insertions(+), 32 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py b/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
index 57416a4..fbc1509 100644
--- a/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
+++ b/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
@@ -153,7 +153,8 @@ class CustomServiceOrchestrator():
self.file_cache.get_host_scripts_base_dir(server_url_prefix)
hook_dir = self.file_cache.get_hook_base_dir(command, server_url_prefix)
base_dir = self.file_cache.get_service_base_dir(command, server_url_prefix)
-
+ self.file_cache.get_dashboard_base_dir(server_url_prefix)
+
script_path = self.resolve_script_path(base_dir, script)
script_tuple = (script_path, base_dir)
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-agent/src/main/python/ambari_agent/FileCache.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/FileCache.py b/ambari-agent/src/main/python/ambari_agent/FileCache.py
index b7c5dee..83ac373 100644
--- a/ambari-agent/src/main/python/ambari_agent/FileCache.py
+++ b/ambari-agent/src/main/python/ambari_agent/FileCache.py
@@ -45,6 +45,7 @@ class FileCache():
STACKS_CACHE_DIRECTORY="stacks"
COMMON_SERVICES_DIRECTORY="common-services"
CUSTOM_ACTIONS_CACHE_DIRECTORY="custom_actions"
+ DASHBOARD_DIRECTORY="dashboards"
HOST_SCRIPTS_CACHE_DIRECTORY="host_scripts"
HASH_SUM_FILE=".hash"
ARCHIVE_NAME="archive.zip"
@@ -99,6 +100,15 @@ class FileCache():
server_url_prefix)
+ def get_dashboard_base_dir(self, server_url_prefix):
+ """
+ Returns a base directory for dashboards
+ """
+ return self.provide_directory(self.cache_dir,
+ self.DASHBOARD_DIRECTORY,
+ server_url_prefix)
+
+
def get_host_scripts_base_dir(self, server_url_prefix):
"""
Returns a base directory for host scripts (host alerts, etc) which
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py b/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
index c9724b7..9f1241e 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
@@ -220,7 +220,7 @@ class TestCustomServiceOrchestrator(TestCase):
except AgentException:
pass # Expected
-
+ @patch.object(FileCache, "get_dashboard_base_dir")
@patch.object(CustomServiceOrchestrator, "resolve_script_path")
@patch.object(CustomServiceOrchestrator, "resolve_hook_script_path")
@patch.object(FileCache, "get_host_scripts_base_dir")
@@ -234,7 +234,8 @@ class TestCustomServiceOrchestrator(TestCase):
get_hook_base_dir_mock, get_service_base_dir_mock,
get_host_scripts_base_dir_mock,
resolve_hook_script_path_mock,
- resolve_script_path_mock):
+ resolve_script_path_mock,
+ get_dashboard_base_dir_mock):
FileCache_mock.return_value = None
command = {
@@ -265,6 +266,7 @@ class TestCustomServiceOrchestrator(TestCase):
unix_process_id = 111
orchestrator.commands_in_progress = {command['taskId']: unix_process_id}
get_hook_base_dir_mock.return_value = "/hooks/"
+ get_dashboard_base_dir_mock.return_value = "/dashboards/"
# normal run case
run_file_mock.return_value = {
'stdout' : 'sss',
@@ -309,6 +311,7 @@ class TestCustomServiceOrchestrator(TestCase):
pass
+ @patch.object(FileCache, "get_dashboard_base_dir")
@patch("ambari_commons.shell.kill_process_with_children")
@patch.object(CustomServiceOrchestrator, "resolve_script_path")
@patch.object(CustomServiceOrchestrator, "resolve_hook_script_path")
@@ -323,7 +326,8 @@ class TestCustomServiceOrchestrator(TestCase):
get_hook_base_dir_mock, get_service_base_dir_mock,
get_host_scripts_base_dir_mock,
resolve_hook_script_path_mock, resolve_script_path_mock,
- kill_process_with_children_mock):
+ kill_process_with_children_mock,
+ get_dashboard_base_dir_mock):
FileCache_mock.return_value = None
command = {
'role' : 'REGION_SERVER',
@@ -353,6 +357,7 @@ class TestCustomServiceOrchestrator(TestCase):
unix_process_id = 111
orchestrator.commands_in_progress = {command['taskId']: unix_process_id}
get_hook_base_dir_mock.return_value = "/hooks/"
+ get_dashboard_base_dir_mock.return_value = "/dashboards/"
run_file_mock_return_value = {
'stdout' : 'killed',
'stderr' : 'killed',
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/test/python/ambari_agent/TestFileCache.py b/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
index 5933daa..fbefc2b 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
@@ -117,6 +117,18 @@ class TestFileCache(TestCase):
"('/var/lib/ambari-agent/cache', 'custom_actions', 'server_url_pref')")
self.assertEquals(res, "dummy value")
+
+ @patch.object(FileCache, "provide_directory")
+ def test_get_dashboard_base_dir(self, provide_directory_mock):
+ provide_directory_mock.return_value = "dummy value"
+ fileCache = FileCache(self.config)
+ res = fileCache.get_dashboard_base_dir("server_url_pref")
+ self.assertEquals(
+ pprint.pformat(provide_directory_mock.call_args_list[0][0]),
+ "('/var/lib/ambari-agent/cache', 'dashboards', 'server_url_pref')")
+ self.assertEquals(res, "dummy value")
+
+
@patch.object(FileCache, "build_download_url")
def test_provide_directory_no_update(self, build_download_url_mock):
try:
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py b/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
index f2a3ebd..8047e7d 100644
--- a/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
+++ b/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
@@ -39,6 +39,7 @@ class ResourceFilesKeeper():
COMMON_SERVICES_DIR="common-services"
CUSTOM_ACTIONS_DIR="custom_actions"
HOST_SCRIPTS_DIR="host_scripts"
+ DASHBOARDS_DIR="dashboards"
# For these directories archives are created
ARCHIVABLE_DIRS = [HOOKS_DIR, PACKAGE_DIR]
@@ -68,7 +69,7 @@ class ResourceFilesKeeper():
"""
Performs housekeeping operations on resource files
"""
- self.update_directory_archieves()
+ self.update_directory_archives()
# probably, later we will need some additional operations
@@ -87,7 +88,7 @@ class ResourceFilesKeeper():
# update the directories so that the .hash is generated
self.update_directory_archive(archive_root)
- def update_directory_archieves(self):
+ def update_directory_archives(self):
"""
Please see AMBARI-4481 for more details
"""
@@ -112,6 +113,9 @@ class ResourceFilesKeeper():
# agent host scripts
self._update_resources_subdir_archive(self.HOST_SCRIPTS_DIR)
+ # custom service dashboards
+ self._update_resources_subdir_archive(self.DASHBOARDS_DIR)
+
def _list_metainfo_dirs(self, root_dir):
valid_items = [] # Format: <stack_dir, ignore(True|False)>
@@ -153,7 +157,7 @@ class ResourceFilesKeeper():
if not self.nozip:
self.zip_directory(directory, skip_empty_directory)
# Skip generation of .hash file is directory is empty
- if (skip_empty_directory and not os.listdir(directory)):
+ if (skip_empty_directory and (not os.path.exists(directory) or not os.listdir(directory))):
self.dbg_out("Empty directory. Skipping generation of hash file for {0}".format(directory))
else:
self.write_hash_sum(directory, cur_hash)
@@ -228,7 +232,7 @@ class ResourceFilesKeeper():
self.dbg_out("creating archive for directory {0}".format(directory))
try:
if skip_if_empty:
- if not os.listdir(directory):
+ if not os.path.exists(directory) or not os.listdir(directory):
self.dbg_out("Empty directory. Skipping archive creation for {0}".format(directory))
return
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-server/src/main/python/ambari_server/serverConfiguration.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/serverConfiguration.py b/ambari-server/src/main/python/ambari_server/serverConfiguration.py
index 04509cf..3086003 100644
--- a/ambari-server/src/main/python/ambari_server/serverConfiguration.py
+++ b/ambari-server/src/main/python/ambari_server/serverConfiguration.py
@@ -397,6 +397,7 @@ class ServerConfigDefaults(object):
self.EXTENSION_LOCATION_DEFAULT = ""
self.COMMON_SERVICES_LOCATION_DEFAULT = ""
self.MPACKS_STAGING_LOCATION_DEFAULT = ""
+ self.DASHBOARD_LOCATION_DEFAULT = ""
self.SERVER_TMP_DIR_DEFAULT = ""
self.DEFAULT_VIEWS_DIR = ""
@@ -468,6 +469,7 @@ class ServerConfigDefaultsWindows(ServerConfigDefaults):
self.EXTENSION_LOCATION_DEFAULT = "resources\\extensions"
self.COMMON_SERVICES_LOCATION_DEFAULT = "resources\\common-services"
self.MPACKS_STAGING_LOCATION_DEFAULT = "resources\\mpacks"
+ self.DASHBOARD_LOCATION_DEFAULT = "resources\\dashboards"
self.SERVER_TMP_DIR_DEFAULT = "data\\tmp"
self.DEFAULT_VIEWS_DIR = "resources\\views"
@@ -554,6 +556,7 @@ class ServerConfigDefaultsLinux(ServerConfigDefaults):
self.EXTENSION_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/extensions")
self.COMMON_SERVICES_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/common-services")
self.MPACKS_STAGING_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/mpacks")
+ self.DASHBOARD_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/dashboards")
self.SERVER_TMP_DIR_DEFAULT = AmbariPath.get("/var/lib/ambari-server/data/tmp")
self.DEFAULT_VIEWS_DIR = AmbariPath.get("/var/lib/ambari-server/resources/views")
@@ -1439,6 +1442,14 @@ def get_mpacks_staging_location(properties):
mpacks_staging_location = configDefaults.MPACKS_STAGING_LOCATION_DEFAULT
return mpacks_staging_location
+
+#
+# Dashboard location
+#
+def get_dashboard_location(properties):
+ dashboard_location = configDefaults.DASHBOARD_LOCATION_DEFAULT
+ return dashboard_location
+
#
# Server temp location
#
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
index b98dc1d..6030fc4 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
@@ -36,7 +36,7 @@ import os
GRAFANA_CONNECT_TRIES = 5
GRAFANA_CONNECT_TIMEOUT = 10
-GRAFANA_SEARCH_BULTIN_DASHBOARDS = "/api/search?tag=builtin"
+GRAFANA_SEARCH_BUILTIN_DASHBOARDS = "/api/search?tag=builtin"
GRAFANA_DATASOURCE_URL = "/api/datasources"
GRAFANA_DASHBOARDS_URL = "/api/dashboards/db"
METRICS_GRAFANA_DATASOURCE_NAME = "AMBARI_METRICS"
@@ -313,14 +313,14 @@ def create_ams_dashboards():
Dashboard = namedtuple('Dashboard', ['uri', 'id', 'title', 'tags'])
existing_dashboards = []
- response = perform_grafana_get_call(GRAFANA_SEARCH_BULTIN_DASHBOARDS, server)
+ response = perform_grafana_get_call(GRAFANA_SEARCH_BUILTIN_DASHBOARDS, server)
if response and response.status == 200:
data = response.read()
try:
dashboards = json.loads(data)
except:
Logger.error("Unable to parse JSON response from grafana request: %s" %
- GRAFANA_SEARCH_BULTIN_DASHBOARDS)
+ GRAFANA_SEARCH_BUILTIN_DASHBOARDS)
Logger.info(data)
return
@@ -336,7 +336,7 @@ def create_ams_dashboards():
else:
Logger.error("Failed to execute search query on Grafana dashboards. "
"query = %s\n statuscode = %s\n reason = %s\n data = %s\n" %
- (GRAFANA_SEARCH_BULTIN_DASHBOARDS, response.status, response.reason, response.read()))
+ (GRAFANA_SEARCH_BUILTIN_DASHBOARDS, response.status, response.reason, response.read()))
return
Logger.debug('Dashboard definitions found = %s' % str(dashboard_files))
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
index f04f5c2..acc3763 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
@@ -83,6 +83,9 @@ dashboards_dirs.append(os.path.join(agent_cache_dir, service_package_folder,
dashboards_dirs.append(os.path.join(agent_cache_dir, service_package_folder,
'files', 'grafana-dashboards', 'default'))
+# Custom services
+dashboards_dirs.append(os.path.join(agent_cache_dir, 'dashboards', 'grafana-dashboards'))
+
def get_grafana_dashboard_defs():
dashboard_defs = []
for dashboards_dir in dashboards_dirs:
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
index aa03d197..ffe10b5 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
@@ -67,11 +67,12 @@ def format_Xmx_size_to_bytes(value, default='b'):
# pre-splits based on selected services also passed as a parameter to the class.
class FindSplitPointsForAMSRegions():
- def __init__(self, ams_hbase_site, ams_hbase_env, serviceMetricsDir,
+ def __init__(self, ams_hbase_site, ams_hbase_env, serviceMetricsDir, customServiceMetricsDir,
operation_mode = 'embedded', services = None):
self.ams_hbase_site = ams_hbase_site
self.ams_hbase_env = ams_hbase_env
self.serviceMetricsDir = serviceMetricsDir
+ self.customServiceMetricsDir = customServiceMetricsDir
self.services = services
self.mode = operation_mode
# Add host metrics if not present as input
@@ -117,18 +118,28 @@ class FindSplitPointsForAMSRegions():
pass
def initialize_ordered_set_of_metrics(self):
- onlyServicefiles = [ f for f in os.listdir(self.serviceMetricsDir) if
- os.path.isfile(os.path.join(self.serviceMetricsDir, f)) ]
-
metrics = set()
+ self.gatherMetrics(metrics, self.serviceMetricsDir)
+ self.gatherMetrics(metrics, self.customServiceMetricsDir)
+
+ self.metrics = sorted(metrics)
+ print 'metrics length: %s' % len(self.metrics)
- for file in onlyServicefiles:
- # Process for services selected at deploy time or all services if
+
+ def gatherMetrics(self, metrics, dir):
+ if os.path.exists(dir):
+ files = [ f for f in os.listdir(dir) if
+ os.path.isfile(os.path.join(dir, f)) ]
+ else:
+ return
+
+ for file in files:
+ # Process for stack services selected at deploy time or all stack services if
# services arg is not passed
if self.services is None or file.rstrip(metric_filename_ext) in self.services:
- print 'Processing file: %s' % os.path.join(self.serviceMetricsDir, file)
+ print 'Processing file: %s' % os.path.join(dir, file)
service_metrics = set()
- with open(os.path.join(self.serviceMetricsDir, file), 'r') as f:
+ with open(os.path.join(dir, file), 'r') as f:
for metric in f:
service_metrics.add(metric.strip())
pass
@@ -137,8 +148,6 @@ class FindSplitPointsForAMSRegions():
pass
pass
- self.metrics = sorted(metrics)
- print 'metrics length: %s' % len(self.metrics)
# Pick 50 metric points for each service that are equidistant from
# each other for a service
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py b/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
index c7c3b2b..9b5ff68 100644
--- a/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
+++ b/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
@@ -832,6 +832,7 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
scriptDir = os.path.dirname(os.path.abspath(__file__))
metricsDir = os.path.join(scriptDir, '../../../../common-services/AMBARI_METRICS/0.1.0/package')
serviceMetricsDir = os.path.join(metricsDir, 'files', 'service-metrics')
+ customServiceMetricsDir = os.path.join(scriptDir, '../../../../dashboards/service-metrics')
sys.path.append(os.path.join(metricsDir, 'scripts'))
servicesList = [service["StackServices"]["service_name"] for service in services["services"]]
@@ -853,7 +854,7 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
ams_hbase_env = configurations["ams-hbase-env"]["properties"]
split_point_finder = FindSplitPointsForAMSRegions(
- ams_hbase_site, ams_hbase_env, serviceMetricsDir, operatingMode, servicesList)
+ ams_hbase_site, ams_hbase_env, serviceMetricsDir, customServiceMetricsDir, operatingMode, servicesList)
result = split_point_finder.get_split_points()
precision_splits = ' '
http://git-wip-us.apache.org/repos/asf/ambari/blob/b413d921/ambari-server/src/test/python/TestResourceFilesKeeper.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/python/TestResourceFilesKeeper.py b/ambari-server/src/test/python/TestResourceFilesKeeper.py
index 966d3b1..4801fad 100644
--- a/ambari-server/src/test/python/TestResourceFilesKeeper.py
+++ b/ambari-server/src/test/python/TestResourceFilesKeeper.py
@@ -82,7 +82,8 @@ class TestResourceFilesKeeper(TestCase):
"call('../resources/TestAmbaryServer.samples/" \
"dummy_common_services/HIVE/0.11.0.2.0.5.0/package'),\n " \
"call('../resources/custom_actions'),\n " \
- "call('../resources/host_scripts')]"
+ "call('../resources/host_scripts'),\n " \
+ "call('../resources/dashboards')]"
else:
UPDATE_DIRECTORY_ARCHIVE_CALL_LIST = \
"[call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_stack\\\\HIVE\\\\package'),\n " \
@@ -91,17 +92,18 @@ class TestResourceFilesKeeper(TestCase):
"call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_common_services\\\\HIVE\\\\0.11.0.2.0.5.0\\\\package'),\n " \
"call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_common_services\\\\HIVE\\\\0.11.0.2.0.5.0\\\\package'),\n " \
"call('..\\\\resources\\\\custom_actions'),\n " \
- "call('..\\\\resources\\\\host_scripts')]"
+ "call('..\\\\resources\\\\host_scripts'),\n " \
+ "call('..\\\\resources\\\\dashboards')]"
def setUp(self):
logging.basicConfig(level=logging.ERROR)
- @patch.object(ResourceFilesKeeper, "update_directory_archieves")
- def test_perform_housekeeping(self, update_directory_archieves_mock):
+ @patch.object(ResourceFilesKeeper, "update_directory_archives")
+ def test_perform_housekeeping(self, update_directory_archives_mock):
resource_files_keeper = ResourceFilesKeeper(os.sep + "dummy-resources", os.sep + "dummy-path")
resource_files_keeper.perform_housekeeping()
- update_directory_archieves_mock.assertCalled()
+ update_directory_archives_mock.assertCalled()
pass
@@ -109,7 +111,7 @@ class TestResourceFilesKeeper(TestCase):
@patch.object(ResourceFilesKeeper, "list_common_services")
@patch.object(ResourceFilesKeeper, "list_stacks")
@patch("os.path.abspath")
- def test_update_directory_archieves(self, abspath_mock,
+ def test_update_directory_archives(self, abspath_mock,
list_active_stacks_mock,
list_common_services_mock,
update_directory_archive_mock):
@@ -120,7 +122,7 @@ class TestResourceFilesKeeper(TestCase):
self.DUMMY_UNCHANGEABLE_COMMON_SERVICES]
abspath_mock.side_effect = lambda s : s
resource_files_keeper = ResourceFilesKeeper(self.TEST_RESOURCES_DIR, self.TEST_STACKS_DIR)
- resource_files_keeper.update_directory_archieves()
+ resource_files_keeper.update_directory_archives()
self.assertEquals(pprint.pformat(
update_directory_archive_mock.call_args_list),
self.UPDATE_DIRECTORY_ARCHIVE_CALL_LIST)
@@ -168,6 +170,7 @@ class TestResourceFilesKeeper(TestCase):
except Exception, e:
self.fail('Unexpected exception thrown:' + str(e))
+ @patch("os.path.exists")
@patch("os.listdir")
@patch.object(ResourceFilesKeeper, "count_hash_sum")
@patch.object(ResourceFilesKeeper, "read_hash_sum")
@@ -176,11 +179,12 @@ class TestResourceFilesKeeper(TestCase):
def test_update_directory_archive(self, write_hash_sum_mock,
zip_directory_mock, read_hash_sum_mock,
count_hash_sum_mock,
- os_listdir_mock):
+ os_listdir_mock, os_path_exists_mock):
os_listdir_mock.return_value = ['file1', 'dir1']
# Test situation when there is no saved directory hash
read_hash_sum_mock.return_value = None
count_hash_sum_mock.return_value = self.YA_HASH
+ os_path_exists_mock.return_value = True
resource_files_keeper = ResourceFilesKeeper(self.TEST_RESOURCES_DIR, self.SOME_PATH)
resource_files_keeper.update_directory_archive(self.SOME_PATH)
self.assertTrue(read_hash_sum_mock.called)
@@ -196,6 +200,7 @@ class TestResourceFilesKeeper(TestCase):
# Test situation when saved directory hash == current hash
read_hash_sum_mock.return_value = self.DUMMY_HASH
count_hash_sum_mock.return_value = self.YA_HASH
+ os_path_exists_mock.return_value = True
resource_files_keeper.update_directory_archive(self.SOME_PATH)
self.assertTrue(read_hash_sum_mock.called)
self.assertTrue(count_hash_sum_mock.called)
@@ -210,6 +215,7 @@ class TestResourceFilesKeeper(TestCase):
# Test situation when saved directory hash == current hash
read_hash_sum_mock.return_value = self.DUMMY_HASH
count_hash_sum_mock.return_value = self.DUMMY_HASH
+ os_path_exists_mock.return_value = True
resource_files_keeper.update_directory_archive(self.SOME_PATH)
self.assertTrue(read_hash_sum_mock.called)
self.assertTrue(count_hash_sum_mock.called)
@@ -225,6 +231,7 @@ class TestResourceFilesKeeper(TestCase):
zip_directory_mock.side_effect = self.keeper_exc_side_effect
read_hash_sum_mock.return_value = self.DUMMY_HASH
count_hash_sum_mock.return_value = self.YA_HASH
+ os_path_exists_mock.return_value = True
try:
resource_files_keeper.update_directory_archive(self.SOME_PATH)
self.fail('KeeperException not thrown')
@@ -245,6 +252,7 @@ class TestResourceFilesKeeper(TestCase):
# Test nozip option
read_hash_sum_mock.return_value = None
count_hash_sum_mock.return_value = self.YA_HASH
+ os_path_exists_mock.return_value = True
resource_files_keeper = ResourceFilesKeeper(self.TEST_RESOURCES_DIR, self.SOME_PATH, nozip=True)
resource_files_keeper.update_directory_archive(self.SOME_PATH)
self.assertTrue(read_hash_sum_mock.called)
@@ -262,6 +270,26 @@ class TestResourceFilesKeeper(TestCase):
os_listdir_mock.return_value = [] # Empty dir
zip_directory_mock.side_effect = None
read_hash_sum_mock.return_value = None # hash read from .hash file
+ os_path_exists_mock.return_value = True
+ resource_files_keeper = ResourceFilesKeeper(self.TEST_RESOURCES_DIR, self.SOME_PATH)
+ resource_files_keeper.update_directory_archive(self.SOME_PATH)
+
+ self.assertTrue(read_hash_sum_mock.called)
+ self.assertTrue(count_hash_sum_mock.called)
+ self.assertTrue(zip_directory_mock.called)
+ self.assertFalse(write_hash_sum_mock.called)
+ pass
+
+ # Test no directory
+ read_hash_sum_mock.reset_mock()
+ count_hash_sum_mock.reset_mock()
+ zip_directory_mock.reset_mock()
+ write_hash_sum_mock.reset_mock()
+
+ # If the input directory doesn't exist, then write_hash_sum() should not be called
+ zip_directory_mock.side_effect = None
+ read_hash_sum_mock.return_value = None # hash read from .hash file
+ os_path_exists_mock.return_value = False
resource_files_keeper = ResourceFilesKeeper(self.TEST_RESOURCES_DIR, self.SOME_PATH)
resource_files_keeper.update_directory_archive(self.SOME_PATH)
@@ -347,8 +375,10 @@ class TestResourceFilesKeeper(TestCase):
self.fail('Unexpected exception thrown:' + str(e))
- def test_zip_directory(self):
+ @patch("os.path.exists")
+ def test_zip_directory(self, os_path_exists_mock):
# Test normal flow
+ os_path_exists_mock.return_value = True
resource_files_keeper = ResourceFilesKeeper(self.TEST_RESOURCES_DIR, self.DUMMY_UNCHANGEABLE_PACKAGE)
resource_files_keeper.zip_directory(self.DUMMY_UNCHANGEABLE_PACKAGE)
arc_file = os.path.join(self.DUMMY_UNCHANGEABLE_PACKAGE,
@@ -384,6 +414,19 @@ class TestResourceFilesKeeper(TestCase):
self.fail('Unexpected exception thrown: ' + str(e))
pass
+ # Test skip zipping of a missing directory
+ with patch("os.listdir") as os_listdir_mock:
+ os_path_exists_mock.return_value = False
+ os_listdir_mock.return_value = False # Empty dir
+ try:
+ skip_empty_directory = True
+ resource_files_keeper.zip_directory("empty-to-directory", skip_empty_directory)
+ self.assertTrue(os_path_exists_mock.called)
+ except Exception, e:
+ self.fail('Unexpected exception thrown: ' + str(e))
+ pass
+
+
def test_is_ignored(self):
resource_files_keeper = ResourceFilesKeeper(self.TEST_RESOURCES_DIR, self.DUMMY_UNCHANGEABLE_PACKAGE)
self.assertTrue(resource_files_keeper.is_ignored(".hash"))
[12/31] ambari git commit: Revert "AMBARI-18846 - Custom services
should be able to easily specify their own dashboards"
Posted by nc...@apache.org.
Revert "AMBARI-18846 - Custom services should be able to easily specify their own dashboards"
This reverts commit a1906001cc17070068cf529ad70f5f9e34d56ff0.
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/3809ebc1
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/3809ebc1
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/3809ebc1
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 3809ebc1f9413a12ed7ca6f67452208cd86f6331
Parents: 3c7d8bd
Author: Tim Thorpe <tt...@apache.org>
Authored: Tue Nov 15 04:44:24 2016 -0800
Committer: Tim Thorpe <tt...@apache.org>
Committed: Tue Nov 15 04:44:24 2016 -0800
----------------------------------------------------------------------
.../ambari_agent/CustomServiceOrchestrator.py | 3 +-
.../src/main/python/ambari_agent/FileCache.py | 10 -------
.../TestCustomServiceOrchestrator.py | 11 ++-----
.../test/python/ambari_agent/TestFileCache.py | 12 --------
.../python/ambari_server/resourceFilesKeeper.py | 10 ++-----
.../python/ambari_server/serverConfiguration.py | 11 -------
.../package/scripts/metrics_grafana_util.py | 8 +++---
.../0.1.0/package/scripts/params.py | 3 --
.../0.1.0/package/scripts/split_points.py | 30 +++-----------------
.../stacks/HDP/2.0.6/services/stack_advisor.py | 3 +-
.../src/test/python/TestResourceFilesKeeper.py | 16 +++++------
11 files changed, 23 insertions(+), 94 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py b/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
index fbc1509..57416a4 100644
--- a/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
+++ b/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
@@ -153,8 +153,7 @@ class CustomServiceOrchestrator():
self.file_cache.get_host_scripts_base_dir(server_url_prefix)
hook_dir = self.file_cache.get_hook_base_dir(command, server_url_prefix)
base_dir = self.file_cache.get_service_base_dir(command, server_url_prefix)
- self.file_cache.get_dashboard_base_dir(server_url_prefix)
-
+
script_path = self.resolve_script_path(base_dir, script)
script_tuple = (script_path, base_dir)
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-agent/src/main/python/ambari_agent/FileCache.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/FileCache.py b/ambari-agent/src/main/python/ambari_agent/FileCache.py
index 83ac373..b7c5dee 100644
--- a/ambari-agent/src/main/python/ambari_agent/FileCache.py
+++ b/ambari-agent/src/main/python/ambari_agent/FileCache.py
@@ -45,7 +45,6 @@ class FileCache():
STACKS_CACHE_DIRECTORY="stacks"
COMMON_SERVICES_DIRECTORY="common-services"
CUSTOM_ACTIONS_CACHE_DIRECTORY="custom_actions"
- DASHBOARD_DIRECTORY="dashboards"
HOST_SCRIPTS_CACHE_DIRECTORY="host_scripts"
HASH_SUM_FILE=".hash"
ARCHIVE_NAME="archive.zip"
@@ -100,15 +99,6 @@ class FileCache():
server_url_prefix)
- def get_dashboard_base_dir(self, server_url_prefix):
- """
- Returns a base directory for dashboards
- """
- return self.provide_directory(self.cache_dir,
- self.DASHBOARD_DIRECTORY,
- server_url_prefix)
-
-
def get_host_scripts_base_dir(self, server_url_prefix):
"""
Returns a base directory for host scripts (host alerts, etc) which
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py b/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
index 9f1241e..c9724b7 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
@@ -220,7 +220,7 @@ class TestCustomServiceOrchestrator(TestCase):
except AgentException:
pass # Expected
- @patch.object(FileCache, "get_dashboard_base_dir")
+
@patch.object(CustomServiceOrchestrator, "resolve_script_path")
@patch.object(CustomServiceOrchestrator, "resolve_hook_script_path")
@patch.object(FileCache, "get_host_scripts_base_dir")
@@ -234,8 +234,7 @@ class TestCustomServiceOrchestrator(TestCase):
get_hook_base_dir_mock, get_service_base_dir_mock,
get_host_scripts_base_dir_mock,
resolve_hook_script_path_mock,
- resolve_script_path_mock,
- get_dashboard_base_dir_mock):
+ resolve_script_path_mock):
FileCache_mock.return_value = None
command = {
@@ -266,7 +265,6 @@ class TestCustomServiceOrchestrator(TestCase):
unix_process_id = 111
orchestrator.commands_in_progress = {command['taskId']: unix_process_id}
get_hook_base_dir_mock.return_value = "/hooks/"
- get_dashboard_base_dir_mock.return_value = "/dashboards/"
# normal run case
run_file_mock.return_value = {
'stdout' : 'sss',
@@ -311,7 +309,6 @@ class TestCustomServiceOrchestrator(TestCase):
pass
- @patch.object(FileCache, "get_dashboard_base_dir")
@patch("ambari_commons.shell.kill_process_with_children")
@patch.object(CustomServiceOrchestrator, "resolve_script_path")
@patch.object(CustomServiceOrchestrator, "resolve_hook_script_path")
@@ -326,8 +323,7 @@ class TestCustomServiceOrchestrator(TestCase):
get_hook_base_dir_mock, get_service_base_dir_mock,
get_host_scripts_base_dir_mock,
resolve_hook_script_path_mock, resolve_script_path_mock,
- kill_process_with_children_mock,
- get_dashboard_base_dir_mock):
+ kill_process_with_children_mock):
FileCache_mock.return_value = None
command = {
'role' : 'REGION_SERVER',
@@ -357,7 +353,6 @@ class TestCustomServiceOrchestrator(TestCase):
unix_process_id = 111
orchestrator.commands_in_progress = {command['taskId']: unix_process_id}
get_hook_base_dir_mock.return_value = "/hooks/"
- get_dashboard_base_dir_mock.return_value = "/dashboards/"
run_file_mock_return_value = {
'stdout' : 'killed',
'stderr' : 'killed',
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/test/python/ambari_agent/TestFileCache.py b/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
index fbefc2b..5933daa 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
@@ -117,18 +117,6 @@ class TestFileCache(TestCase):
"('/var/lib/ambari-agent/cache', 'custom_actions', 'server_url_pref')")
self.assertEquals(res, "dummy value")
-
- @patch.object(FileCache, "provide_directory")
- def test_get_dashboard_base_dir(self, provide_directory_mock):
- provide_directory_mock.return_value = "dummy value"
- fileCache = FileCache(self.config)
- res = fileCache.get_dashboard_base_dir("server_url_pref")
- self.assertEquals(
- pprint.pformat(provide_directory_mock.call_args_list[0][0]),
- "('/var/lib/ambari-agent/cache', 'dashboards', 'server_url_pref')")
- self.assertEquals(res, "dummy value")
-
-
@patch.object(FileCache, "build_download_url")
def test_provide_directory_no_update(self, build_download_url_mock):
try:
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py b/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
index f10b05d..f2a3ebd 100644
--- a/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
+++ b/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
@@ -39,7 +39,6 @@ class ResourceFilesKeeper():
COMMON_SERVICES_DIR="common-services"
CUSTOM_ACTIONS_DIR="custom_actions"
HOST_SCRIPTS_DIR="host_scripts"
- DASHBOARDS_DIR="dashboards"
# For these directories archives are created
ARCHIVABLE_DIRS = [HOOKS_DIR, PACKAGE_DIR]
@@ -69,7 +68,7 @@ class ResourceFilesKeeper():
"""
Performs housekeeping operations on resource files
"""
- self.update_directory_archives()
+ self.update_directory_archieves()
# probably, later we will need some additional operations
@@ -88,7 +87,7 @@ class ResourceFilesKeeper():
# update the directories so that the .hash is generated
self.update_directory_archive(archive_root)
- def update_directory_archives(self):
+ def update_directory_archieves(self):
"""
Please see AMBARI-4481 for more details
"""
@@ -113,9 +112,6 @@ class ResourceFilesKeeper():
# agent host scripts
self._update_resources_subdir_archive(self.HOST_SCRIPTS_DIR)
- # custom service dashboards
- self._update_resources_subdir_archive(self.DASHBOARDS_DIR)
-
def _list_metainfo_dirs(self, root_dir):
valid_items = [] # Format: <stack_dir, ignore(True|False)>
@@ -157,7 +153,7 @@ class ResourceFilesKeeper():
if not self.nozip:
self.zip_directory(directory, skip_empty_directory)
# Skip generation of .hash file is directory is empty
- if (skip_empty_directory and (not os.path.exists(directory) or not os.listdir(directory))):
+ if (skip_empty_directory and not os.listdir(directory)):
self.dbg_out("Empty directory. Skipping generation of hash file for {0}".format(directory))
else:
self.write_hash_sum(directory, cur_hash)
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-server/src/main/python/ambari_server/serverConfiguration.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/serverConfiguration.py b/ambari-server/src/main/python/ambari_server/serverConfiguration.py
index 3086003..04509cf 100644
--- a/ambari-server/src/main/python/ambari_server/serverConfiguration.py
+++ b/ambari-server/src/main/python/ambari_server/serverConfiguration.py
@@ -397,7 +397,6 @@ class ServerConfigDefaults(object):
self.EXTENSION_LOCATION_DEFAULT = ""
self.COMMON_SERVICES_LOCATION_DEFAULT = ""
self.MPACKS_STAGING_LOCATION_DEFAULT = ""
- self.DASHBOARD_LOCATION_DEFAULT = ""
self.SERVER_TMP_DIR_DEFAULT = ""
self.DEFAULT_VIEWS_DIR = ""
@@ -469,7 +468,6 @@ class ServerConfigDefaultsWindows(ServerConfigDefaults):
self.EXTENSION_LOCATION_DEFAULT = "resources\\extensions"
self.COMMON_SERVICES_LOCATION_DEFAULT = "resources\\common-services"
self.MPACKS_STAGING_LOCATION_DEFAULT = "resources\\mpacks"
- self.DASHBOARD_LOCATION_DEFAULT = "resources\\dashboards"
self.SERVER_TMP_DIR_DEFAULT = "data\\tmp"
self.DEFAULT_VIEWS_DIR = "resources\\views"
@@ -556,7 +554,6 @@ class ServerConfigDefaultsLinux(ServerConfigDefaults):
self.EXTENSION_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/extensions")
self.COMMON_SERVICES_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/common-services")
self.MPACKS_STAGING_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/mpacks")
- self.DASHBOARD_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/dashboards")
self.SERVER_TMP_DIR_DEFAULT = AmbariPath.get("/var/lib/ambari-server/data/tmp")
self.DEFAULT_VIEWS_DIR = AmbariPath.get("/var/lib/ambari-server/resources/views")
@@ -1442,14 +1439,6 @@ def get_mpacks_staging_location(properties):
mpacks_staging_location = configDefaults.MPACKS_STAGING_LOCATION_DEFAULT
return mpacks_staging_location
-
-#
-# Dashboard location
-#
-def get_dashboard_location(properties):
- dashboard_location = configDefaults.DASHBOARD_LOCATION_DEFAULT
- return dashboard_location
-
#
# Server temp location
#
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
index 6030fc4..b98dc1d 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
@@ -36,7 +36,7 @@ import os
GRAFANA_CONNECT_TRIES = 5
GRAFANA_CONNECT_TIMEOUT = 10
-GRAFANA_SEARCH_BUILTIN_DASHBOARDS = "/api/search?tag=builtin"
+GRAFANA_SEARCH_BULTIN_DASHBOARDS = "/api/search?tag=builtin"
GRAFANA_DATASOURCE_URL = "/api/datasources"
GRAFANA_DASHBOARDS_URL = "/api/dashboards/db"
METRICS_GRAFANA_DATASOURCE_NAME = "AMBARI_METRICS"
@@ -313,14 +313,14 @@ def create_ams_dashboards():
Dashboard = namedtuple('Dashboard', ['uri', 'id', 'title', 'tags'])
existing_dashboards = []
- response = perform_grafana_get_call(GRAFANA_SEARCH_BUILTIN_DASHBOARDS, server)
+ response = perform_grafana_get_call(GRAFANA_SEARCH_BULTIN_DASHBOARDS, server)
if response and response.status == 200:
data = response.read()
try:
dashboards = json.loads(data)
except:
Logger.error("Unable to parse JSON response from grafana request: %s" %
- GRAFANA_SEARCH_BUILTIN_DASHBOARDS)
+ GRAFANA_SEARCH_BULTIN_DASHBOARDS)
Logger.info(data)
return
@@ -336,7 +336,7 @@ def create_ams_dashboards():
else:
Logger.error("Failed to execute search query on Grafana dashboards. "
"query = %s\n statuscode = %s\n reason = %s\n data = %s\n" %
- (GRAFANA_SEARCH_BUILTIN_DASHBOARDS, response.status, response.reason, response.read()))
+ (GRAFANA_SEARCH_BULTIN_DASHBOARDS, response.status, response.reason, response.read()))
return
Logger.debug('Dashboard definitions found = %s' % str(dashboard_files))
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
index 1a754f6..b052999 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
@@ -83,9 +83,6 @@ dashboards_dirs.append(os.path.join(agent_cache_dir, service_package_folder,
dashboards_dirs.append(os.path.join(agent_cache_dir, service_package_folder,
'files', 'grafana-dashboards', 'default'))
-# Custom services
-dashboards_dirs.append(os.path.join(agent_cache_dir, 'dashboards', 'grafana-dashboards'))
-
def get_grafana_dashboard_defs():
dashboard_defs = []
for dashboards_dir in dashboards_dirs:
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
index fda0199..aa03d197 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
@@ -67,12 +67,11 @@ def format_Xmx_size_to_bytes(value, default='b'):
# pre-splits based on selected services also passed as a parameter to the class.
class FindSplitPointsForAMSRegions():
- def __init__(self, ams_hbase_site, ams_hbase_env, serviceMetricsDir, customServiceMetricsDir,
+ def __init__(self, ams_hbase_site, ams_hbase_env, serviceMetricsDir,
operation_mode = 'embedded', services = None):
self.ams_hbase_site = ams_hbase_site
self.ams_hbase_env = ams_hbase_env
self.serviceMetricsDir = serviceMetricsDir
- self.customServiceMetricsDir = customServiceMetricsDir
self.services = services
self.mode = operation_mode
# Add host metrics if not present as input
@@ -118,19 +117,13 @@ class FindSplitPointsForAMSRegions():
pass
def initialize_ordered_set_of_metrics(self):
- stackServiceFiles = [ f for f in os.listdir(self.serviceMetricsDir) if
+ onlyServicefiles = [ f for f in os.listdir(self.serviceMetricsDir) if
os.path.isfile(os.path.join(self.serviceMetricsDir, f)) ]
- if os.path.exists(self.customServiceMetricsDir):
- customServiceFiles = [ f for f in os.listdir(self.customServiceMetricsDir) if
- os.path.isfile(os.path.join(self.customServiceMetricsDir, f)) ]
- else:
- customServiceFiles = []
-
metrics = set()
- for file in stackServiceFiles:
- # Process for stack services selected at deploy time or all stack services if
+ for file in onlyServicefiles:
+ # Process for services selected at deploy time or all services if
# services arg is not passed
if self.services is None or file.rstrip(metric_filename_ext) in self.services:
print 'Processing file: %s' % os.path.join(self.serviceMetricsDir, file)
@@ -144,21 +137,6 @@ class FindSplitPointsForAMSRegions():
pass
pass
- for file in customServiceFiles:
- # Process for custom services selected at deploy time or all custom services if
- # services arg is not passed
- if self.services is None or file.rstrip(metric_filename_ext) in self.services:
- print 'Processing file: %s' % os.path.join(self.customServiceMetricsDir, file)
- service_metrics = set()
- with open(os.path.join(self.customServiceMetricsDir, file), 'r') as f:
- for metric in f:
- service_metrics.add(metric.strip())
- pass
- pass
- metrics.update(self.find_equidistant_metrics(list(sorted(service_metrics))))
- pass
- pass
-
self.metrics = sorted(metrics)
print 'metrics length: %s' % len(self.metrics)
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py b/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
index 9b5ff68..c7c3b2b 100644
--- a/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
+++ b/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
@@ -832,7 +832,6 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
scriptDir = os.path.dirname(os.path.abspath(__file__))
metricsDir = os.path.join(scriptDir, '../../../../common-services/AMBARI_METRICS/0.1.0/package')
serviceMetricsDir = os.path.join(metricsDir, 'files', 'service-metrics')
- customServiceMetricsDir = os.path.join(scriptDir, '../../../../dashboards/service-metrics')
sys.path.append(os.path.join(metricsDir, 'scripts'))
servicesList = [service["StackServices"]["service_name"] for service in services["services"]]
@@ -854,7 +853,7 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
ams_hbase_env = configurations["ams-hbase-env"]["properties"]
split_point_finder = FindSplitPointsForAMSRegions(
- ams_hbase_site, ams_hbase_env, serviceMetricsDir, customServiceMetricsDir, operatingMode, servicesList)
+ ams_hbase_site, ams_hbase_env, serviceMetricsDir, operatingMode, servicesList)
result = split_point_finder.get_split_points()
precision_splits = ' '
http://git-wip-us.apache.org/repos/asf/ambari/blob/3809ebc1/ambari-server/src/test/python/TestResourceFilesKeeper.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/python/TestResourceFilesKeeper.py b/ambari-server/src/test/python/TestResourceFilesKeeper.py
index ba29354..966d3b1 100644
--- a/ambari-server/src/test/python/TestResourceFilesKeeper.py
+++ b/ambari-server/src/test/python/TestResourceFilesKeeper.py
@@ -82,8 +82,7 @@ class TestResourceFilesKeeper(TestCase):
"call('../resources/TestAmbaryServer.samples/" \
"dummy_common_services/HIVE/0.11.0.2.0.5.0/package'),\n " \
"call('../resources/custom_actions'),\n " \
- "call('../resources/host_scripts'),\n " \
- "call('../resources/dashboards')]"
+ "call('../resources/host_scripts')]"
else:
UPDATE_DIRECTORY_ARCHIVE_CALL_LIST = \
"[call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_stack\\\\HIVE\\\\package'),\n " \
@@ -92,18 +91,17 @@ class TestResourceFilesKeeper(TestCase):
"call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_common_services\\\\HIVE\\\\0.11.0.2.0.5.0\\\\package'),\n " \
"call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_common_services\\\\HIVE\\\\0.11.0.2.0.5.0\\\\package'),\n " \
"call('..\\\\resources\\\\custom_actions'),\n " \
- "call('..\\\\resources\\\\host_scripts'),\n " \
- "call('..\\\\resources\\\\dashboards')]"
+ "call('..\\\\resources\\\\host_scripts')]"
def setUp(self):
logging.basicConfig(level=logging.ERROR)
- @patch.object(ResourceFilesKeeper, "update_directory_archives")
- def test_perform_housekeeping(self, update_directory_archives_mock):
+ @patch.object(ResourceFilesKeeper, "update_directory_archieves")
+ def test_perform_housekeeping(self, update_directory_archieves_mock):
resource_files_keeper = ResourceFilesKeeper(os.sep + "dummy-resources", os.sep + "dummy-path")
resource_files_keeper.perform_housekeeping()
- update_directory_archives_mock.assertCalled()
+ update_directory_archieves_mock.assertCalled()
pass
@@ -111,7 +109,7 @@ class TestResourceFilesKeeper(TestCase):
@patch.object(ResourceFilesKeeper, "list_common_services")
@patch.object(ResourceFilesKeeper, "list_stacks")
@patch("os.path.abspath")
- def test_update_directory_archives(self, abspath_mock,
+ def test_update_directory_archieves(self, abspath_mock,
list_active_stacks_mock,
list_common_services_mock,
update_directory_archive_mock):
@@ -122,7 +120,7 @@ class TestResourceFilesKeeper(TestCase):
self.DUMMY_UNCHANGEABLE_COMMON_SERVICES]
abspath_mock.side_effect = lambda s : s
resource_files_keeper = ResourceFilesKeeper(self.TEST_RESOURCES_DIR, self.TEST_STACKS_DIR)
- resource_files_keeper.update_directory_archives()
+ resource_files_keeper.update_directory_archieves()
self.assertEquals(pprint.pformat(
update_directory_archive_mock.call_args_list),
self.UPDATE_DIRECTORY_ARCHIVE_CALL_LIST)
[03/31] ambari git commit: AMBARI-18846 - Custom services should be
able to easily specify their own dashboards
Posted by nc...@apache.org.
AMBARI-18846 - Custom services should be able to easily specify their own dashboards
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/a1906001
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/a1906001
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/a1906001
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: a1906001cc17070068cf529ad70f5f9e34d56ff0
Parents: 3868cbe
Author: Tim Thorpe <tt...@apache.org>
Authored: Mon Nov 14 11:52:14 2016 -0800
Committer: Tim Thorpe <tt...@apache.org>
Committed: Mon Nov 14 11:52:14 2016 -0800
----------------------------------------------------------------------
.../ambari_agent/CustomServiceOrchestrator.py | 3 +-
.../src/main/python/ambari_agent/FileCache.py | 10 +++++++
.../TestCustomServiceOrchestrator.py | 11 +++++--
.../test/python/ambari_agent/TestFileCache.py | 12 ++++++++
.../python/ambari_server/resourceFilesKeeper.py | 10 +++++--
.../python/ambari_server/serverConfiguration.py | 11 +++++++
.../package/scripts/metrics_grafana_util.py | 8 +++---
.../0.1.0/package/scripts/params.py | 3 ++
.../0.1.0/package/scripts/split_points.py | 30 +++++++++++++++++---
.../stacks/HDP/2.0.6/services/stack_advisor.py | 3 +-
.../src/test/python/TestResourceFilesKeeper.py | 16 ++++++-----
11 files changed, 94 insertions(+), 23 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py b/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
index 57416a4..fbc1509 100644
--- a/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
+++ b/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
@@ -153,7 +153,8 @@ class CustomServiceOrchestrator():
self.file_cache.get_host_scripts_base_dir(server_url_prefix)
hook_dir = self.file_cache.get_hook_base_dir(command, server_url_prefix)
base_dir = self.file_cache.get_service_base_dir(command, server_url_prefix)
-
+ self.file_cache.get_dashboard_base_dir(server_url_prefix)
+
script_path = self.resolve_script_path(base_dir, script)
script_tuple = (script_path, base_dir)
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-agent/src/main/python/ambari_agent/FileCache.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/FileCache.py b/ambari-agent/src/main/python/ambari_agent/FileCache.py
index b7c5dee..83ac373 100644
--- a/ambari-agent/src/main/python/ambari_agent/FileCache.py
+++ b/ambari-agent/src/main/python/ambari_agent/FileCache.py
@@ -45,6 +45,7 @@ class FileCache():
STACKS_CACHE_DIRECTORY="stacks"
COMMON_SERVICES_DIRECTORY="common-services"
CUSTOM_ACTIONS_CACHE_DIRECTORY="custom_actions"
+ DASHBOARD_DIRECTORY="dashboards"
HOST_SCRIPTS_CACHE_DIRECTORY="host_scripts"
HASH_SUM_FILE=".hash"
ARCHIVE_NAME="archive.zip"
@@ -99,6 +100,15 @@ class FileCache():
server_url_prefix)
+ def get_dashboard_base_dir(self, server_url_prefix):
+ """
+ Returns a base directory for dashboards
+ """
+ return self.provide_directory(self.cache_dir,
+ self.DASHBOARD_DIRECTORY,
+ server_url_prefix)
+
+
def get_host_scripts_base_dir(self, server_url_prefix):
"""
Returns a base directory for host scripts (host alerts, etc) which
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py b/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
index c9724b7..9f1241e 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
@@ -220,7 +220,7 @@ class TestCustomServiceOrchestrator(TestCase):
except AgentException:
pass # Expected
-
+ @patch.object(FileCache, "get_dashboard_base_dir")
@patch.object(CustomServiceOrchestrator, "resolve_script_path")
@patch.object(CustomServiceOrchestrator, "resolve_hook_script_path")
@patch.object(FileCache, "get_host_scripts_base_dir")
@@ -234,7 +234,8 @@ class TestCustomServiceOrchestrator(TestCase):
get_hook_base_dir_mock, get_service_base_dir_mock,
get_host_scripts_base_dir_mock,
resolve_hook_script_path_mock,
- resolve_script_path_mock):
+ resolve_script_path_mock,
+ get_dashboard_base_dir_mock):
FileCache_mock.return_value = None
command = {
@@ -265,6 +266,7 @@ class TestCustomServiceOrchestrator(TestCase):
unix_process_id = 111
orchestrator.commands_in_progress = {command['taskId']: unix_process_id}
get_hook_base_dir_mock.return_value = "/hooks/"
+ get_dashboard_base_dir_mock.return_value = "/dashboards/"
# normal run case
run_file_mock.return_value = {
'stdout' : 'sss',
@@ -309,6 +311,7 @@ class TestCustomServiceOrchestrator(TestCase):
pass
+ @patch.object(FileCache, "get_dashboard_base_dir")
@patch("ambari_commons.shell.kill_process_with_children")
@patch.object(CustomServiceOrchestrator, "resolve_script_path")
@patch.object(CustomServiceOrchestrator, "resolve_hook_script_path")
@@ -323,7 +326,8 @@ class TestCustomServiceOrchestrator(TestCase):
get_hook_base_dir_mock, get_service_base_dir_mock,
get_host_scripts_base_dir_mock,
resolve_hook_script_path_mock, resolve_script_path_mock,
- kill_process_with_children_mock):
+ kill_process_with_children_mock,
+ get_dashboard_base_dir_mock):
FileCache_mock.return_value = None
command = {
'role' : 'REGION_SERVER',
@@ -353,6 +357,7 @@ class TestCustomServiceOrchestrator(TestCase):
unix_process_id = 111
orchestrator.commands_in_progress = {command['taskId']: unix_process_id}
get_hook_base_dir_mock.return_value = "/hooks/"
+ get_dashboard_base_dir_mock.return_value = "/dashboards/"
run_file_mock_return_value = {
'stdout' : 'killed',
'stderr' : 'killed',
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/test/python/ambari_agent/TestFileCache.py b/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
index 5933daa..fbefc2b 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
@@ -117,6 +117,18 @@ class TestFileCache(TestCase):
"('/var/lib/ambari-agent/cache', 'custom_actions', 'server_url_pref')")
self.assertEquals(res, "dummy value")
+
+ @patch.object(FileCache, "provide_directory")
+ def test_get_dashboard_base_dir(self, provide_directory_mock):
+ provide_directory_mock.return_value = "dummy value"
+ fileCache = FileCache(self.config)
+ res = fileCache.get_dashboard_base_dir("server_url_pref")
+ self.assertEquals(
+ pprint.pformat(provide_directory_mock.call_args_list[0][0]),
+ "('/var/lib/ambari-agent/cache', 'dashboards', 'server_url_pref')")
+ self.assertEquals(res, "dummy value")
+
+
@patch.object(FileCache, "build_download_url")
def test_provide_directory_no_update(self, build_download_url_mock):
try:
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py b/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
index f2a3ebd..f10b05d 100644
--- a/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
+++ b/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
@@ -39,6 +39,7 @@ class ResourceFilesKeeper():
COMMON_SERVICES_DIR="common-services"
CUSTOM_ACTIONS_DIR="custom_actions"
HOST_SCRIPTS_DIR="host_scripts"
+ DASHBOARDS_DIR="dashboards"
# For these directories archives are created
ARCHIVABLE_DIRS = [HOOKS_DIR, PACKAGE_DIR]
@@ -68,7 +69,7 @@ class ResourceFilesKeeper():
"""
Performs housekeeping operations on resource files
"""
- self.update_directory_archieves()
+ self.update_directory_archives()
# probably, later we will need some additional operations
@@ -87,7 +88,7 @@ class ResourceFilesKeeper():
# update the directories so that the .hash is generated
self.update_directory_archive(archive_root)
- def update_directory_archieves(self):
+ def update_directory_archives(self):
"""
Please see AMBARI-4481 for more details
"""
@@ -112,6 +113,9 @@ class ResourceFilesKeeper():
# agent host scripts
self._update_resources_subdir_archive(self.HOST_SCRIPTS_DIR)
+ # custom service dashboards
+ self._update_resources_subdir_archive(self.DASHBOARDS_DIR)
+
def _list_metainfo_dirs(self, root_dir):
valid_items = [] # Format: <stack_dir, ignore(True|False)>
@@ -153,7 +157,7 @@ class ResourceFilesKeeper():
if not self.nozip:
self.zip_directory(directory, skip_empty_directory)
# Skip generation of .hash file is directory is empty
- if (skip_empty_directory and not os.listdir(directory)):
+ if (skip_empty_directory and (not os.path.exists(directory) or not os.listdir(directory))):
self.dbg_out("Empty directory. Skipping generation of hash file for {0}".format(directory))
else:
self.write_hash_sum(directory, cur_hash)
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-server/src/main/python/ambari_server/serverConfiguration.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/serverConfiguration.py b/ambari-server/src/main/python/ambari_server/serverConfiguration.py
index 04509cf..3086003 100644
--- a/ambari-server/src/main/python/ambari_server/serverConfiguration.py
+++ b/ambari-server/src/main/python/ambari_server/serverConfiguration.py
@@ -397,6 +397,7 @@ class ServerConfigDefaults(object):
self.EXTENSION_LOCATION_DEFAULT = ""
self.COMMON_SERVICES_LOCATION_DEFAULT = ""
self.MPACKS_STAGING_LOCATION_DEFAULT = ""
+ self.DASHBOARD_LOCATION_DEFAULT = ""
self.SERVER_TMP_DIR_DEFAULT = ""
self.DEFAULT_VIEWS_DIR = ""
@@ -468,6 +469,7 @@ class ServerConfigDefaultsWindows(ServerConfigDefaults):
self.EXTENSION_LOCATION_DEFAULT = "resources\\extensions"
self.COMMON_SERVICES_LOCATION_DEFAULT = "resources\\common-services"
self.MPACKS_STAGING_LOCATION_DEFAULT = "resources\\mpacks"
+ self.DASHBOARD_LOCATION_DEFAULT = "resources\\dashboards"
self.SERVER_TMP_DIR_DEFAULT = "data\\tmp"
self.DEFAULT_VIEWS_DIR = "resources\\views"
@@ -554,6 +556,7 @@ class ServerConfigDefaultsLinux(ServerConfigDefaults):
self.EXTENSION_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/extensions")
self.COMMON_SERVICES_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/common-services")
self.MPACKS_STAGING_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/mpacks")
+ self.DASHBOARD_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/dashboards")
self.SERVER_TMP_DIR_DEFAULT = AmbariPath.get("/var/lib/ambari-server/data/tmp")
self.DEFAULT_VIEWS_DIR = AmbariPath.get("/var/lib/ambari-server/resources/views")
@@ -1439,6 +1442,14 @@ def get_mpacks_staging_location(properties):
mpacks_staging_location = configDefaults.MPACKS_STAGING_LOCATION_DEFAULT
return mpacks_staging_location
+
+#
+# Dashboard location
+#
+def get_dashboard_location(properties):
+ dashboard_location = configDefaults.DASHBOARD_LOCATION_DEFAULT
+ return dashboard_location
+
#
# Server temp location
#
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
index b98dc1d..6030fc4 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
@@ -36,7 +36,7 @@ import os
GRAFANA_CONNECT_TRIES = 5
GRAFANA_CONNECT_TIMEOUT = 10
-GRAFANA_SEARCH_BULTIN_DASHBOARDS = "/api/search?tag=builtin"
+GRAFANA_SEARCH_BUILTIN_DASHBOARDS = "/api/search?tag=builtin"
GRAFANA_DATASOURCE_URL = "/api/datasources"
GRAFANA_DASHBOARDS_URL = "/api/dashboards/db"
METRICS_GRAFANA_DATASOURCE_NAME = "AMBARI_METRICS"
@@ -313,14 +313,14 @@ def create_ams_dashboards():
Dashboard = namedtuple('Dashboard', ['uri', 'id', 'title', 'tags'])
existing_dashboards = []
- response = perform_grafana_get_call(GRAFANA_SEARCH_BULTIN_DASHBOARDS, server)
+ response = perform_grafana_get_call(GRAFANA_SEARCH_BUILTIN_DASHBOARDS, server)
if response and response.status == 200:
data = response.read()
try:
dashboards = json.loads(data)
except:
Logger.error("Unable to parse JSON response from grafana request: %s" %
- GRAFANA_SEARCH_BULTIN_DASHBOARDS)
+ GRAFANA_SEARCH_BUILTIN_DASHBOARDS)
Logger.info(data)
return
@@ -336,7 +336,7 @@ def create_ams_dashboards():
else:
Logger.error("Failed to execute search query on Grafana dashboards. "
"query = %s\n statuscode = %s\n reason = %s\n data = %s\n" %
- (GRAFANA_SEARCH_BULTIN_DASHBOARDS, response.status, response.reason, response.read()))
+ (GRAFANA_SEARCH_BUILTIN_DASHBOARDS, response.status, response.reason, response.read()))
return
Logger.debug('Dashboard definitions found = %s' % str(dashboard_files))
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
index b052999..1a754f6 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
@@ -83,6 +83,9 @@ dashboards_dirs.append(os.path.join(agent_cache_dir, service_package_folder,
dashboards_dirs.append(os.path.join(agent_cache_dir, service_package_folder,
'files', 'grafana-dashboards', 'default'))
+# Custom services
+dashboards_dirs.append(os.path.join(agent_cache_dir, 'dashboards', 'grafana-dashboards'))
+
def get_grafana_dashboard_defs():
dashboard_defs = []
for dashboards_dir in dashboards_dirs:
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
index aa03d197..fda0199 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
@@ -67,11 +67,12 @@ def format_Xmx_size_to_bytes(value, default='b'):
# pre-splits based on selected services also passed as a parameter to the class.
class FindSplitPointsForAMSRegions():
- def __init__(self, ams_hbase_site, ams_hbase_env, serviceMetricsDir,
+ def __init__(self, ams_hbase_site, ams_hbase_env, serviceMetricsDir, customServiceMetricsDir,
operation_mode = 'embedded', services = None):
self.ams_hbase_site = ams_hbase_site
self.ams_hbase_env = ams_hbase_env
self.serviceMetricsDir = serviceMetricsDir
+ self.customServiceMetricsDir = customServiceMetricsDir
self.services = services
self.mode = operation_mode
# Add host metrics if not present as input
@@ -117,13 +118,19 @@ class FindSplitPointsForAMSRegions():
pass
def initialize_ordered_set_of_metrics(self):
- onlyServicefiles = [ f for f in os.listdir(self.serviceMetricsDir) if
+ stackServiceFiles = [ f for f in os.listdir(self.serviceMetricsDir) if
os.path.isfile(os.path.join(self.serviceMetricsDir, f)) ]
+ if os.path.exists(self.customServiceMetricsDir):
+ customServiceFiles = [ f for f in os.listdir(self.customServiceMetricsDir) if
+ os.path.isfile(os.path.join(self.customServiceMetricsDir, f)) ]
+ else:
+ customServiceFiles = []
+
metrics = set()
- for file in onlyServicefiles:
- # Process for services selected at deploy time or all services if
+ for file in stackServiceFiles:
+ # Process for stack services selected at deploy time or all stack services if
# services arg is not passed
if self.services is None or file.rstrip(metric_filename_ext) in self.services:
print 'Processing file: %s' % os.path.join(self.serviceMetricsDir, file)
@@ -137,6 +144,21 @@ class FindSplitPointsForAMSRegions():
pass
pass
+ for file in customServiceFiles:
+ # Process for custom services selected at deploy time or all custom services if
+ # services arg is not passed
+ if self.services is None or file.rstrip(metric_filename_ext) in self.services:
+ print 'Processing file: %s' % os.path.join(self.customServiceMetricsDir, file)
+ service_metrics = set()
+ with open(os.path.join(self.customServiceMetricsDir, file), 'r') as f:
+ for metric in f:
+ service_metrics.add(metric.strip())
+ pass
+ pass
+ metrics.update(self.find_equidistant_metrics(list(sorted(service_metrics))))
+ pass
+ pass
+
self.metrics = sorted(metrics)
print 'metrics length: %s' % len(self.metrics)
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py b/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
index ee63f1e..5e7698d 100644
--- a/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
+++ b/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
@@ -811,6 +811,7 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
scriptDir = os.path.dirname(os.path.abspath(__file__))
metricsDir = os.path.join(scriptDir, '../../../../common-services/AMBARI_METRICS/0.1.0/package')
serviceMetricsDir = os.path.join(metricsDir, 'files', 'service-metrics')
+ customServiceMetricsDir = os.path.join(scriptDir, '../../../../dashboards/service-metrics')
sys.path.append(os.path.join(metricsDir, 'scripts'))
servicesList = [service["StackServices"]["service_name"] for service in services["services"]]
@@ -832,7 +833,7 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
ams_hbase_env = configurations["ams-hbase-env"]["properties"]
split_point_finder = FindSplitPointsForAMSRegions(
- ams_hbase_site, ams_hbase_env, serviceMetricsDir, operatingMode, servicesList)
+ ams_hbase_site, ams_hbase_env, serviceMetricsDir, customServiceMetricsDir, operatingMode, servicesList)
result = split_point_finder.get_split_points()
precision_splits = ' '
http://git-wip-us.apache.org/repos/asf/ambari/blob/a1906001/ambari-server/src/test/python/TestResourceFilesKeeper.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/python/TestResourceFilesKeeper.py b/ambari-server/src/test/python/TestResourceFilesKeeper.py
index 966d3b1..ba29354 100644
--- a/ambari-server/src/test/python/TestResourceFilesKeeper.py
+++ b/ambari-server/src/test/python/TestResourceFilesKeeper.py
@@ -82,7 +82,8 @@ class TestResourceFilesKeeper(TestCase):
"call('../resources/TestAmbaryServer.samples/" \
"dummy_common_services/HIVE/0.11.0.2.0.5.0/package'),\n " \
"call('../resources/custom_actions'),\n " \
- "call('../resources/host_scripts')]"
+ "call('../resources/host_scripts'),\n " \
+ "call('../resources/dashboards')]"
else:
UPDATE_DIRECTORY_ARCHIVE_CALL_LIST = \
"[call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_stack\\\\HIVE\\\\package'),\n " \
@@ -91,17 +92,18 @@ class TestResourceFilesKeeper(TestCase):
"call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_common_services\\\\HIVE\\\\0.11.0.2.0.5.0\\\\package'),\n " \
"call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_common_services\\\\HIVE\\\\0.11.0.2.0.5.0\\\\package'),\n " \
"call('..\\\\resources\\\\custom_actions'),\n " \
- "call('..\\\\resources\\\\host_scripts')]"
+ "call('..\\\\resources\\\\host_scripts'),\n " \
+ "call('..\\\\resources\\\\dashboards')]"
def setUp(self):
logging.basicConfig(level=logging.ERROR)
- @patch.object(ResourceFilesKeeper, "update_directory_archieves")
- def test_perform_housekeeping(self, update_directory_archieves_mock):
+ @patch.object(ResourceFilesKeeper, "update_directory_archives")
+ def test_perform_housekeeping(self, update_directory_archives_mock):
resource_files_keeper = ResourceFilesKeeper(os.sep + "dummy-resources", os.sep + "dummy-path")
resource_files_keeper.perform_housekeeping()
- update_directory_archieves_mock.assertCalled()
+ update_directory_archives_mock.assertCalled()
pass
@@ -109,7 +111,7 @@ class TestResourceFilesKeeper(TestCase):
@patch.object(ResourceFilesKeeper, "list_common_services")
@patch.object(ResourceFilesKeeper, "list_stacks")
@patch("os.path.abspath")
- def test_update_directory_archieves(self, abspath_mock,
+ def test_update_directory_archives(self, abspath_mock,
list_active_stacks_mock,
list_common_services_mock,
update_directory_archive_mock):
@@ -120,7 +122,7 @@ class TestResourceFilesKeeper(TestCase):
self.DUMMY_UNCHANGEABLE_COMMON_SERVICES]
abspath_mock.side_effect = lambda s : s
resource_files_keeper = ResourceFilesKeeper(self.TEST_RESOURCES_DIR, self.TEST_STACKS_DIR)
- resource_files_keeper.update_directory_archieves()
+ resource_files_keeper.update_directory_archives()
self.assertEquals(pprint.pformat(
update_directory_archive_mock.call_args_list),
self.UPDATE_DIRECTORY_ARCHIVE_CALL_LIST)
[25/31] ambari git commit: AMBARI-18876. Remove old UpgradeCatalog
and tests (aonishuk)
Posted by nc...@apache.org.
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog170.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog170.java b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog170.java
deleted file mode 100644
index b3b1b70..0000000
--- a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog170.java
+++ /dev/null
@@ -1,1530 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ambari.server.upgrade;
-
-import java.lang.reflect.Type;
-import java.sql.Connection;
-import java.sql.PreparedStatement;
-import java.sql.ResultSet;
-import java.sql.SQLException;
-import java.sql.Statement;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.Collections;
-import java.util.Date;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.Iterator;
-import java.util.List;
-import java.util.Map;
-import java.util.Map.Entry;
-import java.util.Set;
-
-import javax.persistence.EntityManager;
-import javax.persistence.TypedQuery;
-import javax.persistence.criteria.CriteriaBuilder;
-import javax.persistence.criteria.CriteriaQuery;
-import javax.persistence.criteria.Expression;
-import javax.persistence.criteria.Predicate;
-import javax.persistence.criteria.Root;
-
-import org.apache.ambari.server.AmbariException;
-import org.apache.ambari.server.configuration.Configuration;
-import org.apache.ambari.server.configuration.Configuration.DatabaseType;
-import org.apache.ambari.server.controller.AmbariManagementController;
-import org.apache.ambari.server.orm.DBAccessor.DBColumnInfo;
-import org.apache.ambari.server.orm.dao.ClusterDAO;
-import org.apache.ambari.server.orm.dao.ClusterServiceDAO;
-import org.apache.ambari.server.orm.dao.ConfigGroupConfigMappingDAO;
-import org.apache.ambari.server.orm.dao.DaoUtils;
-import org.apache.ambari.server.orm.dao.HostDAO;
-import org.apache.ambari.server.orm.dao.HostRoleCommandDAO;
-import org.apache.ambari.server.orm.dao.KeyValueDAO;
-import org.apache.ambari.server.orm.dao.PermissionDAO;
-import org.apache.ambari.server.orm.dao.PrincipalDAO;
-import org.apache.ambari.server.orm.dao.PrincipalTypeDAO;
-import org.apache.ambari.server.orm.dao.PrivilegeDAO;
-import org.apache.ambari.server.orm.dao.ResourceDAO;
-import org.apache.ambari.server.orm.dao.ResourceTypeDAO;
-import org.apache.ambari.server.orm.dao.ServiceComponentDesiredStateDAO;
-import org.apache.ambari.server.orm.dao.ServiceDesiredStateDAO;
-import org.apache.ambari.server.orm.dao.UserDAO;
-import org.apache.ambari.server.orm.dao.ViewDAO;
-import org.apache.ambari.server.orm.dao.ViewInstanceDAO;
-import org.apache.ambari.server.orm.entities.ClusterConfigEntity;
-import org.apache.ambari.server.orm.entities.ClusterConfigMappingEntity;
-import org.apache.ambari.server.orm.entities.ClusterEntity;
-import org.apache.ambari.server.orm.entities.ClusterServiceEntity;
-import org.apache.ambari.server.orm.entities.ClusterServiceEntityPK;
-import org.apache.ambari.server.orm.entities.ConfigGroupConfigMappingEntity;
-import org.apache.ambari.server.orm.entities.HostComponentDesiredStateEntity;
-import org.apache.ambari.server.orm.entities.HostComponentStateEntity;
-import org.apache.ambari.server.orm.entities.HostEntity;
-import org.apache.ambari.server.orm.entities.HostRoleCommandEntity;
-import org.apache.ambari.server.orm.entities.HostRoleCommandEntity_;
-import org.apache.ambari.server.orm.entities.KeyValueEntity;
-import org.apache.ambari.server.orm.entities.PermissionEntity;
-import org.apache.ambari.server.orm.entities.PrincipalEntity;
-import org.apache.ambari.server.orm.entities.PrincipalTypeEntity;
-import org.apache.ambari.server.orm.entities.PrivilegeEntity;
-import org.apache.ambari.server.orm.entities.ResourceEntity;
-import org.apache.ambari.server.orm.entities.ResourceTypeEntity;
-import org.apache.ambari.server.orm.entities.ServiceComponentDesiredStateEntity;
-import org.apache.ambari.server.orm.entities.ServiceDesiredStateEntity;
-import org.apache.ambari.server.orm.entities.ServiceDesiredStateEntityPK;
-import org.apache.ambari.server.orm.entities.StackEntity;
-import org.apache.ambari.server.orm.entities.UserEntity;
-import org.apache.ambari.server.orm.entities.ViewEntity;
-import org.apache.ambari.server.orm.entities.ViewInstanceEntity;
-import org.apache.ambari.server.security.authorization.ResourceType;
-import org.apache.ambari.server.state.Cluster;
-import org.apache.ambari.server.state.Clusters;
-import org.apache.ambari.server.state.Config;
-import org.apache.ambari.server.state.ConfigHelper;
-import org.apache.ambari.server.state.StackId;
-import org.apache.ambari.server.state.State;
-import org.apache.ambari.server.state.alert.Scope;
-import org.apache.ambari.server.utils.StageUtils;
-import org.apache.ambari.server.view.ViewRegistry;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import com.google.common.reflect.TypeToken;
-import com.google.inject.Inject;
-import com.google.inject.Injector;
-
-/**
- * Upgrade catalog for version 1.7.0.
- */
-public class UpgradeCatalog170 extends AbstractUpgradeCatalog {
- private static final String CONTENT_FIELD_NAME = "content";
- private static final String PIG_CONTENT_FIELD_NAME = "pig-content";
- private static final String ENV_CONFIGS_POSTFIX = "-env";
-
- private static final String PIG_PROPERTIES_CONFIG_TYPE = "pig-properties";
-
- private static final String ALERT_TABLE_DEFINITION = "alert_definition";
- private static final String ALERT_TABLE_HISTORY = "alert_history";
- private static final String ALERT_TABLE_CURRENT = "alert_current";
- private static final String ALERT_TABLE_GROUP = "alert_group";
- private static final String ALERT_TABLE_TARGET = "alert_target";
- private static final String ALERT_TABLE_GROUP_TARGET = "alert_group_target";
- private static final String ALERT_TABLE_GROUPING = "alert_grouping";
- private static final String ALERT_TABLE_NOTICE = "alert_notice";
- public static final String JOBS_VIEW_NAME = "JOBS";
- public static final String VIEW_NAME_REG_EXP = JOBS_VIEW_NAME + "\\{.*\\}";
- public static final String JOBS_VIEW_INSTANCE_NAME = "JOBS_1";
- public static final String SHOW_JOBS_FOR_NON_ADMIN_KEY = "showJobsForNonAdmin";
- public static final String JOBS_VIEW_INSTANCE_LABEL = "Jobs";
- public static final String YARN_TIMELINE_SERVICE_WEBAPP_ADDRESS_PROPERTY = "yarn.timeline-service.webapp.address";
- public static final String YARN_RESOURCEMANAGER_WEBAPP_ADDRESS_PROPERTY = "yarn.resourcemanager.webapp.address";
- public static final String YARN_SITE = "yarn-site";
- public static final String YARN_ATS_URL_PROPERTY = "yarn.ats.url";
- public static final String YARN_RESOURCEMANAGER_URL_PROPERTY = "yarn.resourcemanager.url";
-
- public static final StackId CLUSTER_STATE_STACK_HDP_2_1 = new StackId("HDP",
- "2.1");
-
- //SourceVersion is only for book-keeping purpos
- @Override
- public String getSourceVersion() {
- return "1.6.1";
- }
-
- @Override
- public String getTargetVersion() {
- return "1.7.0";
- }
-
- /**
- * Logger.
- */
- private static final Logger LOG = LoggerFactory.getLogger
- (UpgradeCatalog170.class);
-
- // ----- Constructors ------------------------------------------------------
-
- @Inject
- public UpgradeCatalog170(Injector injector) {
- super(injector);
- this.injector = injector;
- }
-
- @Inject
- DaoUtils daoUtils;
-
- // ----- AbstractUpgradeCatalog --------------------------------------------
-
- @Override
- protected void executeDDLUpdates() throws AmbariException, SQLException {
- DatabaseType databaseType = configuration.getDatabaseType();
-
- // needs to be executed first
- renameSequenceValueColumnName();
-
-
- List<DBColumnInfo> columns;
-
- // add group and members tables
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("group_id", Integer.class, null, null, false));
- columns.add(new DBColumnInfo("principal_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("group_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("ldap_group", Integer.class, 1, 0, false));
- dbAccessor.createTable("groups", columns, "group_id");
-
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("member_id", Integer.class, null, null, false));
- columns.add(new DBColumnInfo("group_id", Integer.class, null, null, false));
- columns.add(new DBColumnInfo("user_id", Integer.class, null, null, false));
- dbAccessor.createTable("members", columns, "member_id");
-
- // add admin tables and initial values prior to adding referencing columns on existing tables
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("principal_type_id", Integer.class, null, null,
- false));
- columns.add(new DBColumnInfo("principal_type_name", String.class, null,
- null, false));
-
- dbAccessor.createTable("adminprincipaltype", columns, "principal_type_id");
-
- dbAccessor.insertRow("adminprincipaltype", new String[]{"principal_type_id", "principal_type_name"}, new String[]{"1", "'USER'"}, true);
- dbAccessor.insertRow("adminprincipaltype", new String[]{"principal_type_id", "principal_type_name"}, new String[]{"2", "'GROUP'"}, true);
-
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("principal_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("principal_type_id", Integer.class, null, null,
- false));
-
- dbAccessor.createTable("adminprincipal", columns, "principal_id");
-
- dbAccessor.insertRow("adminprincipal", new String[]{"principal_id", "principal_type_id"}, new String[]{"1", "1"}, true);
-
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("resource_type_id", Integer.class, null, null,
- false));
- columns.add(new DBColumnInfo("resource_type_name", String.class, null,
- null, false));
-
- dbAccessor.createTable("adminresourcetype", columns, "resource_type_id");
-
- dbAccessor.insertRow("adminresourcetype", new String[]{"resource_type_id", "resource_type_name"}, new String[]{"1", "'AMBARI'"}, true);
- dbAccessor.insertRow("adminresourcetype", new String[]{"resource_type_id", "resource_type_name"}, new String[]{"2", "'CLUSTER'"}, true);
- dbAccessor.insertRow("adminresourcetype", new String[]{"resource_type_id", "resource_type_name"}, new String[]{"3", "'VIEW'"}, true);
-
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("resource_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("resource_type_id", Integer.class, null, null,
- false));
-
- dbAccessor.createTable("adminresource", columns, "resource_id");
-
- dbAccessor.insertRow("adminresource", new String[]{"resource_id", "resource_type_id"}, new String[]{"1", "1"}, true);
-
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("permission_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("permission_name", String.class, null, null,
- false));
- columns.add(new DBColumnInfo("resource_type_id", Integer.class, null, null,
- false));
-
- dbAccessor.createTable("adminpermission", columns, "permission_id");
-
- dbAccessor.insertRow("adminpermission", new String[]{"permission_id", "permission_name", "resource_type_id"}, new String[]{"1", "'AMBARI.ADMIN'", "1"}, true);
- dbAccessor.insertRow("adminpermission", new String[]{"permission_id", "permission_name", "resource_type_id"}, new String[]{"2", "'CLUSTER.READ'", "2"}, true);
- dbAccessor.insertRow("adminpermission", new String[]{"permission_id", "permission_name", "resource_type_id"}, new String[]{"3", "'CLUSTER.OPERATE'", "2"}, true);
- dbAccessor.insertRow("adminpermission", new String[]{"permission_id", "permission_name", "resource_type_id"}, new String[]{"4", "'VIEW.USE'", "3"}, true);
-
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("privilege_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("permission_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("resource_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("principal_id", Long.class, null, null, false));
-
- dbAccessor.createTable("adminprivilege", columns, "privilege_id");
-
- dbAccessor.insertRow("adminprivilege", new String[]{"privilege_id", "permission_id", "resource_id", "principal_id"}, new String[]{"1", "1", "1", "1"}, true);
-
- String [] configAttributesTableNames = {"clusterconfig", "hostgroup_configuration", "blueprint_configuration"};
-
- for(String tableName : configAttributesTableNames) {
- addConfigAttributesColumn(tableName);
- }
-
- // Add columns
- dbAccessor.addColumn("viewmain", new DBColumnInfo("mask",
- String.class, 255, null, true));
- dbAccessor.addColumn("viewmain", new DBColumnInfo("system_view",
- Character.class, 1, null, true));
- dbAccessor.addColumn("viewmain", new DBColumnInfo("resource_type_id",
- Integer.class, null, 1, false));
- dbAccessor.addColumn("viewmain", new DBColumnInfo("description",
- String.class, 2048, null, true));
- dbAccessor.addColumn("viewparameter", new DBColumnInfo("masked",
- Character.class, 1, null, true));
- dbAccessor.addColumn("users", new DBColumnInfo("active",
- Integer.class, 1, 1, false));
- dbAccessor.addColumn("users", new DBColumnInfo("principal_id",
- Long.class, null, 1, false));
- dbAccessor.addColumn("viewinstance", new DBColumnInfo("resource_id",
- Long.class, null, 1, false));
- dbAccessor.addColumn("viewinstance", new DBColumnInfo("xml_driven",
- Character.class, 1, null, true));
- dbAccessor.addColumn("clusters", new DBColumnInfo("resource_id",
- Long.class, null, 1, false));
-
- dbAccessor.addColumn("host_role_command", new DBColumnInfo("output_log",
- String.class, 255, null, true));
-
- dbAccessor.addColumn("stage", new DBColumnInfo("command_params",
- byte[].class, null, null, true));
- dbAccessor.addColumn("stage", new DBColumnInfo("host_params",
- byte[].class, null, null, true));
-
- dbAccessor.addColumn("host_role_command", new DBColumnInfo("error_log",
- String.class, 255, null, true));
-
- addAlertingFrameworkDDL();
-
- // Exclusive requests changes
- dbAccessor.addColumn("request", new DBColumnInfo(
- "exclusive_execution", Integer.class, 1, 0, false));
-
- //service config versions changes
-
- //remove old artifacts (for versions <=1.4.1) which depend on tables changed
- //actually these had to be dropped in UC150, but some of tables were never used, and others were just cleared
- if (dbAccessor.tableExists("componentconfigmapping")) {
- dbAccessor.dropTable("componentconfigmapping");
- }
- if (dbAccessor.tableExists("hostcomponentconfigmapping")) {
- dbAccessor.dropTable("hostcomponentconfigmapping");
- }
- if (dbAccessor.tableExists("hcdesiredconfigmapping")) {
- dbAccessor.dropTable("hcdesiredconfigmapping");
- }
- if (dbAccessor.tableExists("serviceconfigmapping")) {
- dbAccessor.dropTable("serviceconfigmapping");
- }
-
- dbAccessor.dropFKConstraint("confgroupclusterconfigmapping", "FK_confg");
-
- if (databaseType == DatabaseType.ORACLE
- || databaseType == DatabaseType.MYSQL
- || databaseType == DatabaseType.DERBY) {
- dbAccessor.executeQuery("ALTER TABLE clusterconfig DROP PRIMARY KEY", true);
- } else if (databaseType == DatabaseType.POSTGRES) {
- dbAccessor.executeQuery("ALTER TABLE clusterconfig DROP CONSTRAINT clusterconfig_pkey CASCADE", true);
- }
-
- dbAccessor.addColumn("clusterconfig", new DBColumnInfo("config_id", Long.class, null, null, true));
-
- if (databaseType == DatabaseType.ORACLE) {
- //sequence looks to be simpler than rownum
- if (dbAccessor.tableHasData("clusterconfig")) {
- dbAccessor.executeQuery("CREATE SEQUENCE TEMP_SEQ " +
- " START WITH 1 " +
- " MAXVALUE 999999999999999999999999999 " +
- " MINVALUE 1 " +
- " NOCYCLE " +
- " NOCACHE " +
- " NOORDER");
- dbAccessor.executeQuery("UPDATE clusterconfig SET config_id = TEMP_SEQ.NEXTVAL");
- dbAccessor.dropSequence("TEMP_SEQ");
- }
- } else if (databaseType == DatabaseType.MYSQL) {
- if (dbAccessor.tableHasData("clusterconfig")) {
- dbAccessor.executeQuery("UPDATE clusterconfig " +
- "SET config_id = (SELECT @a := @a + 1 FROM (SELECT @a := 1) s)");
- }
- } else if (databaseType == DatabaseType.POSTGRES) {
- if (dbAccessor.tableHasData("clusterconfig")) {
- //window functions like row_number were added in 8.4, workaround for earlier versions (redhat/centos 5)
- dbAccessor.executeQuery("CREATE SEQUENCE temp_seq START WITH 1");
- dbAccessor.executeQuery("UPDATE clusterconfig SET config_id = nextval('temp_seq')");
- dbAccessor.dropSequence("temp_seq");
- }
- }
-
- // alter view tables description columns size
- if (databaseType == DatabaseType.ORACLE
- || databaseType == DatabaseType.MYSQL) {
- dbAccessor.executeQuery("ALTER TABLE viewinstance MODIFY description VARCHAR(2048)");
- dbAccessor.executeQuery("ALTER TABLE viewparameter MODIFY description VARCHAR(2048)");
- } else if (databaseType == DatabaseType.POSTGRES) {
- dbAccessor.executeQuery("ALTER TABLE viewinstance ALTER COLUMN description TYPE VARCHAR(2048)");
- dbAccessor.executeQuery("ALTER TABLE viewparameter ALTER COLUMN description TYPE VARCHAR(2048)");
- } else if (databaseType == DatabaseType.DERBY) {
- dbAccessor.executeQuery("ALTER TABLE viewinstance ALTER COLUMN description SET DATA TYPE VARCHAR(2048)");
- dbAccessor.executeQuery("ALTER TABLE viewparameter ALTER COLUMN description SET DATA TYPE VARCHAR(2048)");
- }
-
- //upgrade unit test workaround
- if (databaseType == DatabaseType.DERBY) {
- dbAccessor.executeQuery("ALTER TABLE clusterconfig ALTER COLUMN config_id DEFAULT 0");
- dbAccessor.executeQuery("ALTER TABLE clusterconfig ALTER COLUMN config_id NOT NULL");
- }
-
- dbAccessor.executeQuery("ALTER TABLE clusterconfig ADD PRIMARY KEY (config_id)");
-
- //fill version column
- dbAccessor.addColumn("clusterconfig", new DBColumnInfo("version", Long.class, null));
-
- populateConfigVersions();
-
- dbAccessor.setColumnNullable("clusterconfig", new DBColumnInfo("version", Long.class, null), false);
-
- dbAccessor.executeQuery("ALTER TABLE clusterconfig ADD CONSTRAINT UQ_config_type_tag UNIQUE (cluster_id, type_name, version_tag)", true);
- dbAccessor.executeQuery("ALTER TABLE clusterconfig ADD CONSTRAINT UQ_config_type_version UNIQUE (cluster_id, type_name, version)", true);
-
- if (databaseType != DatabaseType.ORACLE) {
- dbAccessor.alterColumn("clusterconfig", new DBColumnInfo("config_data", char[].class, null, null, false));
- dbAccessor.alterColumn("blueprint_configuration", new DBColumnInfo("config_data", char[].class, null, null, false));
- dbAccessor.alterColumn("hostgroup_configuration", new DBColumnInfo("config_data", char[].class, null, null, false));
- }
-
- columns.clear();
- columns.add(new DBColumnInfo("service_config_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("cluster_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("service_name", String.class, null, null, false));
- columns.add(new DBColumnInfo("version", Long.class, null, null, false));
- columns.add(new DBColumnInfo("create_timestamp", Long.class, null, null, false));
- columns.add(new DBColumnInfo("user_name", String.class, null, "_db", false));
- columns.add(new DBColumnInfo("note", char[].class, null, null, true));
- columns.add(new DBColumnInfo("group_id", Long.class, null, null, true));
- dbAccessor.createTable("serviceconfig", columns, "service_config_id");
-
- columns.clear();
- columns.add(new DBColumnInfo("service_config_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("hostname", String.class, 255, null, false));
- dbAccessor.createTable("serviceconfighosts", columns, "service_config_id", "hostname");
-
- dbAccessor.executeQuery("ALTER TABLE serviceconfig ADD CONSTRAINT UQ_scv_service_version UNIQUE (cluster_id, service_name, version)", true);
-
- columns.clear();
- columns.add(new DBColumnInfo("service_config_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("config_id", Long.class, null, null, false));
- dbAccessor.createTable("serviceconfigmapping", columns, "service_config_id", "config_id");
-
- dbAccessor.addFKConstraint("confgroupclusterconfigmapping", "FK_confg",
- new String[]{"cluster_id", "config_type", "version_tag"}, "clusterconfig",
- new String[]{"cluster_id", "type_name", "version_tag"}, true);
-
- dbAccessor.addFKConstraint("serviceconfighosts", "FK_scvhosts_scv",
- new String[]{"service_config_id"}, "serviceconfig",
- new String[]{"service_config_id"}, true);
-
- dbAccessor.addColumn("configgroup", new DBColumnInfo("service_name", String.class, 255));
-
- addSequences(Arrays.asList(
- "alert_definition_id_seq",
- "alert_group_id_seq",
- "alert_target_id_seq",
- "alert_history_id_seq",
- "alert_notice_id_seq",
- "alert_current_id_seq"
- ), 0L, false);
- addSequence("group_id_seq", 1L, false);
- addSequence("member_id_seq", 1L, false);
- addSequence("resource_type_id_seq", 4L, false);
- addSequence("resource_id_seq", 2L, false);
- addSequence("principal_type_id_seq", 3L, false);
- addSequence("principal_id_seq", 2L, false);
- addSequence("permission_id_seq", 5L, false);
- addSequence("privilege_id_seq", 1L, false);
- addSequence("service_config_id_seq", 1L, false);
- addSequence("service_config_application_id_seq", 1L, false);
-
- long count = 1;
-
- Statement statement = null;
- ResultSet rs = null;
- try {
- statement = dbAccessor.getConnection().createStatement();
- if (statement != null) {
- rs = statement.executeQuery("SELECT count(*) FROM clusterconfig");
- if (rs != null) {
- if (rs.next()) {
- count = rs.getLong(1) + 2;
- }
- }
- }
- } finally {
- if (rs != null) {
- rs.close();
- }
- if (statement != null) {
- statement.close();
- }
- }
-
- addSequence("config_id_seq", count, false);
-
- dbAccessor.addFKConstraint("users", "FK_users_principal_id", "principal_id", "adminprincipal", "principal_id", true);
- dbAccessor.addFKConstraint("clusters", "FK_clusters_resource_id", "resource_id", "adminresource", "resource_id", true);
- dbAccessor.addFKConstraint("viewinstance", "FK_viewinstance_resource_id", "resource_id", "adminresource", "resource_id", true);
- dbAccessor.addFKConstraint("adminresource", "FK_resource_resource_type_id", "resource_type_id", "adminresourcetype", "resource_type_id", true);
- dbAccessor.addFKConstraint("adminprincipal", "FK_principal_principal_type_id", "principal_type_id", "adminprincipaltype", "principal_type_id", true);
- dbAccessor.addFKConstraint("adminpermission", "FK_permission_resource_type_id", "resource_type_id", "adminresourcetype", "resource_type_id", true);
- dbAccessor.addFKConstraint("adminprivilege", "FK_privilege_permission_id", "permission_id", "adminpermission", "permission_id", true);
- dbAccessor.addFKConstraint("adminprivilege", "FK_privilege_resource_id", "resource_id", "adminresource", "resource_id", true);
-
- dbAccessor.addFKConstraint("groups", "FK_groups_principal_id", "principal_id", "adminprincipal", "principal_id", true);
- dbAccessor.addFKConstraint("members", "FK_members_user_id", "user_id", "users", "user_id", true);
- dbAccessor.addFKConstraint("members", "FK_members_group_id", "group_id", "groups", "group_id", true);
-
- dbAccessor.addUniqueConstraint("groups", "UNQ_groups_0", "group_name", "ldap_group");
- dbAccessor.addUniqueConstraint("members", "UNQ_members_0", "group_id", "user_id");
- dbAccessor.addUniqueConstraint("adminpermission", "UQ_perm_name_resource_type_id", "permission_name", "resource_type_id");
- }
-
- /**
- * @param tableName
- * @throws SQLException
- */
- private void addConfigAttributesColumn(String tableName) throws SQLException {
- final DatabaseType databaseType = configuration.getDatabaseType();
- if (databaseType == DatabaseType.ORACLE) {
- dbAccessor.addColumn(tableName, new DBColumnInfo("config_attributes", char[].class));
- } else {
- DBColumnInfo clusterConfigAttributesColumn = new DBColumnInfo(
- "config_attributes", Character[].class, null, null, true);
- dbAccessor.addColumn(tableName, clusterConfigAttributesColumn);
- }
- }
-
- /**
- * Note that you can't use dbAccessor.renameColumn(...) here as the column name is a reserved word and
- * thus requires custom approach for every database type.
- */
- private void renameSequenceValueColumnName() throws AmbariException, SQLException {
- final DatabaseType databaseType = configuration.getDatabaseType();
- if (dbAccessor.tableHasColumn("ambari_sequences", "sequence_value")) {
- return;
- }
- if (databaseType == DatabaseType.MYSQL) {
- dbAccessor.executeQuery("ALTER TABLE ambari_sequences CHANGE value sequence_value DECIMAL(38) NOT NULL");
- } else if (databaseType == DatabaseType.DERBY) {
- dbAccessor.executeQuery("RENAME COLUMN ambari_sequences.\"value\" to sequence_value");
- } else if (databaseType == DatabaseType.ORACLE) {
- dbAccessor.executeQuery("ALTER TABLE ambari_sequences RENAME COLUMN value to sequence_value");
- } else {
- // Postgres
- dbAccessor.executeQuery("ALTER TABLE ambari_sequences RENAME COLUMN \"value\" to sequence_value");
- }
- }
-
- private void populateConfigVersions() throws SQLException {
- ResultSet resultSet = null;
- Set<String> configTypes = new HashSet<String>();
-
- //use new connection to not affect state of internal one
- Connection connection = null;
- PreparedStatement orderedConfigsStatement = null;
- Map<String, List<Long>> configVersionMap = new HashMap<String, List<Long>>();
- try {
- connection = dbAccessor.getNewConnection();
-
- Statement statement = null;
- try {
- statement = connection.createStatement();
- if (statement != null) {
- resultSet = statement.executeQuery("SELECT DISTINCT type_name FROM clusterconfig ");
- if (resultSet != null) {
- while (resultSet.next()) {
- configTypes.add(resultSet.getString("type_name"));
- }
- }
- }
- } finally {
- if (statement != null) {
- statement.close();
- }
- }
-
- try {
- orderedConfigsStatement
- = connection.prepareStatement("SELECT config_id FROM clusterconfig WHERE type_name = ? ORDER BY create_timestamp");
-
- for (String configType : configTypes) {
- List<Long> configIds = new ArrayList<Long>();
- orderedConfigsStatement.setString(1, configType);
- resultSet = orderedConfigsStatement.executeQuery();
- if (resultSet != null) {
- try {
- while (resultSet.next()) {
- configIds.add(resultSet.getLong("config_id"));
- }
- } finally {
- resultSet.close();
- }
- }
- configVersionMap.put(configType, configIds);
- }
- } finally {
- if (orderedConfigsStatement != null) {
- orderedConfigsStatement.close();
- }
- }
-
- connection.setAutoCommit(false); //disable autocommit
- PreparedStatement configVersionStatement = null;
- try {
- configVersionStatement = connection.prepareStatement("UPDATE clusterconfig SET version = ? WHERE config_id = ?");
-
- for (Entry<String, List<Long>> entry : configVersionMap.entrySet()) {
- long version = 1L;
- for (Long configId : entry.getValue()) {
- configVersionStatement.setLong(1, version++);
- configVersionStatement.setLong(2, configId);
- configVersionStatement.addBatch();
- }
- configVersionStatement.executeBatch();
- }
- connection.commit(); //commit changes manually
- } catch (SQLException e) {
- connection.rollback();
- throw e;
- } finally {
- if (configVersionStatement != null){
- configVersionStatement.close();
- }
- }
- } finally {
- if (connection != null) {
- connection.close();
- }
- }
-
- }
-
-
- // ----- UpgradeCatalog ----------------------------------------------------
- /**
- * {@inheritDoc}
- */
- @Override
- public void executePreDMLUpdates() {
- ;
- }
-
- @Override
- protected void executeDMLUpdates() throws AmbariException, SQLException {
-
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- try {
- moveHcatalogIntoHiveService();
- moveWebHcatIntoHiveService();
- } catch (Exception e) {
- LOG.warn("Integrating HCatalog and WebHCat services into Hive threw " +
- "exception. ", e);
- }
- }
- });
-
- // Update historic records with the log paths, but only enough so as to not prolong the upgrade process
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- try {
- HostRoleCommandDAO hostRoleCommandDAO = injector.getInstance(HostRoleCommandDAO.class);
- EntityManager em = getEntityManagerProvider().get();
- CriteriaBuilder cb = em.getCriteriaBuilder();
- CriteriaQuery<HostRoleCommandEntity> cq1 = cb.createQuery(HostRoleCommandEntity.class);
- CriteriaQuery<HostRoleCommandEntity> cq2 = cb.createQuery(HostRoleCommandEntity.class);
- Root<HostRoleCommandEntity> hrc1 = cq1.from(HostRoleCommandEntity.class);
- Root<HostRoleCommandEntity> hrc2 = cq1.from(HostRoleCommandEntity.class);
-
- // Rather than using Java reflection, which is more susceptible to breaking, use the classname_.field canonical model
- // that is safer because it exposes the persistent attributes statically.
- Expression<Long> taskID1 = hrc1.get(HostRoleCommandEntity_.taskId);
- Expression<Long> taskID2 = hrc2.get(HostRoleCommandEntity_.taskId);
- Expression<String> outputLog = hrc1.get(HostRoleCommandEntity_.outputLog);
- Expression<String> errorLog = hrc2.get(HostRoleCommandEntity_.errorLog);
-
- Predicate p1 = cb.isNull(outputLog);
- Predicate p2 = cb.equal(outputLog, "");
- Predicate p1_or_2 = cb.or(p1, p2);
-
- Predicate p3 = cb.isNull(errorLog);
- Predicate p4 = cb.equal(errorLog, "");
- Predicate p3_or_4 = cb.or(p3, p4);
-
- if (daoUtils == null) {
- daoUtils = new DaoUtils();
- }
-
- // Update output_log
- cq1.select(hrc1).where(p1_or_2).orderBy(cb.desc(taskID1));
- TypedQuery<HostRoleCommandEntity> q1 = em.createQuery(cq1);
- q1.setMaxResults(1000);
- List<HostRoleCommandEntity> r1 = daoUtils.selectList(q1);
- for (HostRoleCommandEntity entity : r1) {
- entity.setOutputLog("/var/lib/ambari-agent/data/output-" + entity.getTaskId() + ".txt");
- hostRoleCommandDAO.merge(entity);
- }
-
- // Update error_log
- cq2.select(hrc2).where(p3_or_4).orderBy(cb.desc(taskID2));
- TypedQuery<HostRoleCommandEntity> q2 = em.createQuery(cq2);
- q2.setMaxResults(1000);
- List<HostRoleCommandEntity> r2 = daoUtils.selectList(q2);
- for (HostRoleCommandEntity entity : r2) {
- entity.setErrorLog("/var/lib/ambari-agent/data/errors-" + entity.getTaskId() + ".txt");
- hostRoleCommandDAO.merge(entity);
- }
- } catch (Exception e) {
- LOG.warn("Could not populate historic records with output_log and error_log in host_role_command table. ", e);
- }
- }
- });
-
- moveGlobalsToEnv();
- addEnvContentFields();
- renamePigProperties();
- upgradePermissionModel();
- addJobsViewPermissions();
- moveConfigGroupsGlobalToEnv();
- addMissingConfigs();
- updateClusterProvisionState();
- removeMapred2Log4jConfig();
- }
-
- private void removeMapred2Log4jConfig() {
- final ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
-
- List<ClusterEntity> clusters = clusterDAO.findAll();
- for (ClusterEntity cluster : clusters) {
- for (ClusterConfigMappingEntity configMapping : cluster.getConfigMappingEntities()) {
- if (configMapping.getType().equals(Configuration.MAPREDUCE2_LOG4J_CONFIG_TAG)) {
- configMapping.setSelected(0);
- configMapping = clusterDAO.mergeConfigMapping(configMapping);
- }
- }
- clusterDAO.merge(cluster);
- }
- }
-
- public void updateClusterProvisionState() {
- // Change the provisioning_state of the cluster to INIT state
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- try {
- final ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- EntityManager em = getEntityManagerProvider().get();
- List<ClusterEntity> clusterEntities = clusterDAO.findAll();
- for (ClusterEntity clusterEntity : clusterEntities) {
- clusterEntity.setProvisioningState(State.INSTALLED);
- em.merge(clusterEntity);
- }
- } catch (Exception e) {
- LOG.warn("Updating cluster provisioning_state to INSTALLED threw " +
- "exception. ", e);
- }
- }
- });
- }
-
- public void moveHcatalogIntoHiveService() throws AmbariException {
- final String serviceName = "HIVE";
- final String serviceNameToBeDeleted = "HCATALOG";
- final String componentName = "HCAT";
- moveComponentsIntoService(serviceName, serviceNameToBeDeleted, componentName);
- }
-
- private void moveWebHcatIntoHiveService() throws AmbariException {
- final String serviceName = "HIVE";
- final String serviceNameToBeDeleted = "WEBHCAT";
- final String componentName = "WEBHCAT_SERVER";
- moveComponentsIntoService(serviceName, serviceNameToBeDeleted, componentName);
- }
-
- private void moveComponentsIntoService(String serviceName, String serviceNameToBeDeleted, String componentName) throws AmbariException {
- EntityManager em = getEntityManagerProvider().get();
- ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- ClusterServiceDAO clusterServiceDAO = injector.getInstance(ClusterServiceDAO.class);
- ServiceDesiredStateDAO serviceDesiredStateDAO = injector.getInstance(ServiceDesiredStateDAO.class);
- ServiceComponentDesiredStateDAO serviceComponentDesiredStateDAO = injector.getInstance(ServiceComponentDesiredStateDAO.class);
- HostDAO hostDAO = injector.getInstance(HostDAO.class);
-
- List<ClusterEntity> clusterEntities = clusterDAO.findAll();
- for (final ClusterEntity clusterEntity : clusterEntities) {
- ServiceComponentDesiredStateEntity serviceComponentDesiredStateEntityToDelete = serviceComponentDesiredStateDAO.findByName(
- clusterEntity.getClusterId(), serviceNameToBeDeleted, componentName);
-
- if (serviceComponentDesiredStateEntityToDelete == null) {
- continue;
- }
-
- ServiceDesiredStateEntityPK serviceDesiredStateEntityPK = new ServiceDesiredStateEntityPK();
- serviceDesiredStateEntityPK.setClusterId(clusterEntity.getClusterId());
- serviceDesiredStateEntityPK.setServiceName(serviceNameToBeDeleted);
- ServiceDesiredStateEntity serviceDesiredStateEntity = serviceDesiredStateDAO.findByPK(serviceDesiredStateEntityPK);
-
- ClusterServiceEntityPK clusterServiceEntityToBeDeletedPK = new ClusterServiceEntityPK();
- clusterServiceEntityToBeDeletedPK.setClusterId(clusterEntity.getClusterId());
- clusterServiceEntityToBeDeletedPK.setServiceName(serviceNameToBeDeleted);
- ClusterServiceEntity clusterServiceEntityToBeDeleted = clusterServiceDAO.findByPK(clusterServiceEntityToBeDeletedPK);
-
- ClusterServiceEntityPK clusterServiceEntityPK = new ClusterServiceEntityPK();
- clusterServiceEntityPK.setClusterId(clusterEntity.getClusterId());
- clusterServiceEntityPK.setServiceName(serviceName);
-
-
- ClusterServiceEntity clusterServiceEntity = clusterServiceDAO.findByPK(clusterServiceEntityPK);
-
- ServiceComponentDesiredStateEntity serviceComponentDesiredStateEntity = new ServiceComponentDesiredStateEntity();
- serviceComponentDesiredStateEntity.setServiceName(serviceName);
- serviceComponentDesiredStateEntity.setComponentName(serviceComponentDesiredStateEntityToDelete.getComponentName());
- serviceComponentDesiredStateEntity.setClusterId(clusterEntity.getClusterId());
- serviceComponentDesiredStateEntity.setDesiredStack(serviceComponentDesiredStateEntityToDelete.getDesiredStack());
- serviceComponentDesiredStateEntity.setDesiredState(serviceComponentDesiredStateEntityToDelete.getDesiredState());
- serviceComponentDesiredStateEntity.setClusterServiceEntity(clusterServiceEntity);
- serviceComponentDesiredStateDAO.create(serviceComponentDesiredStateEntity);
-
- Iterator<HostComponentDesiredStateEntity> hostComponentDesiredStateIterator = serviceComponentDesiredStateEntityToDelete.getHostComponentDesiredStateEntities().iterator();
- Iterator<HostComponentStateEntity> hostComponentStateIterator = serviceComponentDesiredStateEntityToDelete.getHostComponentStateEntities().iterator();
-
- while (hostComponentDesiredStateIterator.hasNext()) {
- HostComponentDesiredStateEntity hcDesiredStateEntityToBeDeleted = hostComponentDesiredStateIterator.next();
- HostComponentDesiredStateEntity hostComponentDesiredStateEntity = new HostComponentDesiredStateEntity();
- hostComponentDesiredStateEntity.setClusterId(clusterEntity.getClusterId());
- hostComponentDesiredStateEntity.setComponentName(hcDesiredStateEntityToBeDeleted.getComponentName());
- hostComponentDesiredStateEntity.setDesiredStack(hcDesiredStateEntityToBeDeleted.getDesiredStack());
- hostComponentDesiredStateEntity.setDesiredState(hcDesiredStateEntityToBeDeleted.getDesiredState());
- hostComponentDesiredStateEntity.setHostEntity(hcDesiredStateEntityToBeDeleted.getHostEntity());
- hostComponentDesiredStateEntity.setAdminState(hcDesiredStateEntityToBeDeleted.getAdminState());
- hostComponentDesiredStateEntity.setMaintenanceState(hcDesiredStateEntityToBeDeleted.getMaintenanceState());
- hostComponentDesiredStateEntity.setRestartRequired(hcDesiredStateEntityToBeDeleted.isRestartRequired());
- hostComponentDesiredStateEntity.setServiceName(serviceName);
- hostComponentDesiredStateEntity.setServiceComponentDesiredStateEntity(serviceComponentDesiredStateEntity);
- em.merge(hostComponentDesiredStateEntity);
- em.remove(hcDesiredStateEntityToBeDeleted);
- }
-
- while (hostComponentStateIterator.hasNext()) {
- HostComponentStateEntity hcStateToBeDeleted = hostComponentStateIterator.next();
- HostEntity hostToBeDeleted = hostDAO.findByName(hcStateToBeDeleted.getHostName());
- if (hostToBeDeleted == null) {
- continue;
- }
-
- HostComponentStateEntity hostComponentStateEntity = new HostComponentStateEntity();
- hostComponentStateEntity.setClusterId(clusterEntity.getClusterId());
- hostComponentStateEntity.setComponentName(hcStateToBeDeleted.getComponentName());
- hostComponentStateEntity.setCurrentStack(hcStateToBeDeleted.getCurrentStack());
- hostComponentStateEntity.setCurrentState(hcStateToBeDeleted.getCurrentState());
- hostComponentStateEntity.setHostEntity(hcStateToBeDeleted.getHostEntity());
- hostComponentStateEntity.setServiceName(serviceName);
- hostComponentStateEntity.setServiceComponentDesiredStateEntity(serviceComponentDesiredStateEntity);
- em.merge(hostComponentStateEntity);
- em.remove(hcStateToBeDeleted);
- }
- serviceComponentDesiredStateEntity.setClusterServiceEntity(clusterServiceEntity);
- em.merge(serviceComponentDesiredStateEntity);
- em.remove(serviceComponentDesiredStateEntityToDelete);
- em.remove(serviceDesiredStateEntity);
- em.remove(clusterServiceEntityToBeDeleted);
- }
- }
-
-
- private void moveConfigGroupsGlobalToEnv() throws AmbariException {
- final ConfigGroupConfigMappingDAO confGroupConfMappingDAO = injector.getInstance(ConfigGroupConfigMappingDAO.class);
- ConfigHelper configHelper = injector.getInstance(ConfigHelper.class);
- final ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- AmbariManagementController ambariManagementController = injector.getInstance(AmbariManagementController.class);
- List<ConfigGroupConfigMappingEntity> configGroupConfigMappingEntities = confGroupConfMappingDAO.findAll();
- List<ConfigGroupConfigMappingEntity> configGroupsWithGlobalConfigs = new ArrayList<ConfigGroupConfigMappingEntity>();
- Type type = new TypeToken<Map<String, String>>() {}.getType();
-
- for (ConfigGroupConfigMappingEntity entity : configGroupConfigMappingEntities) {
- if (entity.getConfigType().equals(Configuration.GLOBAL_CONFIG_TAG)) {
- configGroupsWithGlobalConfigs.add(entity);
- }
- }
-
- for (ConfigGroupConfigMappingEntity entity : configGroupsWithGlobalConfigs) {
- String configData = entity.getClusterConfigEntity().getData();
- StackEntity stackEntity = entity.getClusterConfigEntity().getStack();
-
- Map<String, String> properties = StageUtils.getGson().fromJson(configData, type);
- Cluster cluster = ambariManagementController.getClusters().getClusterById(entity.getClusterId());
- HashMap<String, HashMap<String, String>> configs = new HashMap<String, HashMap<String, String>>();
-
- for (Entry<String, String> property : properties.entrySet()) {
- Set<String> configTypes = configHelper.findConfigTypesByPropertyName(cluster.getCurrentStackVersion(),
- property.getKey(), cluster.getClusterName());
- // i'm not sure, but i hope that every service property is unique
- if (configTypes != null && configTypes.size() > 0) {
- String configType = configTypes.iterator().next();
-
- if (configs.containsKey(configType)) {
- HashMap<String, String> config = configs.get(configType);
- config.put(property.getKey(), property.getValue());
- } else {
- HashMap<String, String> config = new HashMap<String, String>();
- config.put(property.getKey(), property.getValue());
- configs.put(configType, config);
- }
- }
- }
-
- for (Entry<String, HashMap<String, String>> config : configs.entrySet()) {
-
- String tag;
- if(cluster.getConfigsByType(config.getKey()) == null) {
- tag = "version1";
- } else {
- tag = "version" + System.currentTimeMillis();
- }
-
- ClusterConfigEntity clusterConfigEntity = new ClusterConfigEntity();
- clusterConfigEntity.setClusterEntity(entity.getClusterConfigEntity().getClusterEntity());
- clusterConfigEntity.setClusterId(cluster.getClusterId());
- clusterConfigEntity.setType(config.getKey());
- clusterConfigEntity.setVersion(cluster.getNextConfigVersion(config.getKey()));
- clusterConfigEntity.setTag(tag);
- clusterConfigEntity.setTimestamp(new Date().getTime());
- clusterConfigEntity.setData(StageUtils.getGson().toJson(config.getValue()));
- clusterConfigEntity.setStack(stackEntity);
-
- clusterDAO.createConfig(clusterConfigEntity);
-
- ConfigGroupConfigMappingEntity configGroupConfigMappingEntity = new ConfigGroupConfigMappingEntity();
- configGroupConfigMappingEntity.setTimestamp(System.currentTimeMillis());
- configGroupConfigMappingEntity.setClusterId(entity.getClusterId());
- configGroupConfigMappingEntity.setClusterConfigEntity(clusterConfigEntity);
- configGroupConfigMappingEntity.setConfigGroupEntity(entity.getConfigGroupEntity());
- configGroupConfigMappingEntity.setConfigGroupId(entity.getConfigGroupId());
- configGroupConfigMappingEntity.setConfigType(config.getKey());
- configGroupConfigMappingEntity.setVersionTag(clusterConfigEntity.getTag());
- confGroupConfMappingDAO.create(configGroupConfigMappingEntity);
- }
- }
-
- for (ConfigGroupConfigMappingEntity entity : configGroupsWithGlobalConfigs) {
- confGroupConfMappingDAO.remove(entity);
- }
- }
-
- /**
- * Adds the alert tables and constraints.
- */
- private void addAlertingFrameworkDDL() throws AmbariException, SQLException {
- // alert_definition
- ArrayList<DBColumnInfo> columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("definition_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("cluster_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("definition_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("service_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("component_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("scope", String.class, 255, Scope.ANY.name(), false));
- columns.add(new DBColumnInfo("label", String.class, 255, null, true));
- columns.add(new DBColumnInfo("enabled", Short.class, 1, 1, false));
- columns.add(new DBColumnInfo("schedule_interval", Integer.class, null, null, false));
- columns.add(new DBColumnInfo("source_type", String.class, 255, null, false));
- columns.add(new DBColumnInfo("alert_source", char[].class, 32672, null,
- false));
- columns.add(new DBColumnInfo("hash", String.class, 64, null, false));
- dbAccessor.createTable(ALERT_TABLE_DEFINITION, columns, "definition_id");
-
- dbAccessor.addFKConstraint(ALERT_TABLE_DEFINITION,
- "fk_alert_def_cluster_id",
- "cluster_id", "clusters", "cluster_id", false);
-
- dbAccessor.addUniqueConstraint(ALERT_TABLE_DEFINITION, "uni_alert_def_name", "cluster_id", "definition_name");
-
- // alert_history
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("alert_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("cluster_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("alert_definition_id", Long.class, null, null,
- false));
- columns.add(new DBColumnInfo("service_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("component_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("host_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("alert_instance", String.class, 255, null,
- true));
- columns.add(new DBColumnInfo("alert_timestamp", Long.class, null, null,
- false));
- columns.add(new DBColumnInfo("alert_label", String.class, 1024, null, true));
- columns.add(new DBColumnInfo("alert_state", String.class, 255, null, false));
- columns.add(new DBColumnInfo("alert_text", String.class, 4000, null, true));
- dbAccessor.createTable(ALERT_TABLE_HISTORY, columns, "alert_id");
-
- dbAccessor.addFKConstraint(ALERT_TABLE_HISTORY, "fk_alert_history_def_id",
- "alert_definition_id", ALERT_TABLE_DEFINITION, "definition_id", false);
-
- dbAccessor.addFKConstraint(ALERT_TABLE_HISTORY,
- "fk_alert_history_cluster_id",
- "cluster_id", "clusters", "cluster_id", false);
-
- // alert_current
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("alert_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("definition_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("history_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("maintenance_state", String.class, 255, null,
- true));
- columns.add(new DBColumnInfo("original_timestamp", Long.class, 0, null,
- false));
- columns.add(new DBColumnInfo("latest_timestamp", Long.class, 0, null, false));
- columns.add(new DBColumnInfo("latest_text", String.class, 4000, null, true));
- dbAccessor.createTable(ALERT_TABLE_CURRENT, columns, "alert_id");
-
- dbAccessor.addFKConstraint(ALERT_TABLE_CURRENT, "fk_alert_current_def_id",
- "definition_id", ALERT_TABLE_DEFINITION, "definition_id", false);
-
- dbAccessor.addFKConstraint(ALERT_TABLE_CURRENT,
- "fk_alert_current_history_id", "history_id", ALERT_TABLE_HISTORY,
- "alert_id", false);
-
- dbAccessor.addUniqueConstraint(ALERT_TABLE_CURRENT, "uni_alert_current_hist_id", "history_id");
-
- // alert_group
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("group_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("cluster_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("group_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("is_default", Short.class, 1, 1, false));
- columns.add(new DBColumnInfo("service_name", String.class, 255, null, true));
- dbAccessor.createTable(ALERT_TABLE_GROUP, columns, "group_id");
-
- dbAccessor.addUniqueConstraint(ALERT_TABLE_GROUP, "uni_alert_group_name", "cluster_id", "group_name");
-
- // alert_target
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("target_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("target_name", String.class, 255, null, false));
- columns.add(new DBColumnInfo("notification_type", String.class, 64, null, false));
- columns.add(new DBColumnInfo("properties", char[].class, 32672, null, true));
- columns.add(new DBColumnInfo("description", String.class, 1024, null, true));
- dbAccessor.createTable(ALERT_TABLE_TARGET, columns, "target_id");
-
- dbAccessor.addUniqueConstraint(ALERT_TABLE_TARGET, "uni_alert_target_name", "target_name");
-
- // alert_group_target
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("group_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("target_id", Long.class, null, null, false));
- dbAccessor.createTable(ALERT_TABLE_GROUP_TARGET, columns, "group_id",
- "target_id");
-
- dbAccessor.addFKConstraint(ALERT_TABLE_GROUP_TARGET,
- "fk_alert_gt_group_id", "group_id", ALERT_TABLE_GROUP, "group_id",
- false);
-
- dbAccessor.addFKConstraint(ALERT_TABLE_GROUP_TARGET,
- "fk_alert_gt_target_id", "target_id", ALERT_TABLE_TARGET, "target_id",
- false);
-
- // alert_grouping
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("definition_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("group_id", Long.class, null, null, false));
- dbAccessor.createTable(ALERT_TABLE_GROUPING, columns, "group_id",
- "definition_id");
-
- dbAccessor.addFKConstraint(ALERT_TABLE_GROUPING,
- "fk_alert_grouping_def_id", "definition_id", ALERT_TABLE_DEFINITION,
- "definition_id", false);
-
- dbAccessor.addFKConstraint(ALERT_TABLE_GROUPING,
- "fk_alert_grouping_group_id", "group_id", ALERT_TABLE_GROUP,
- "group_id", false);
-
- // alert_notice
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("notification_id", Long.class, null, null,
- false));
- columns.add(new DBColumnInfo("target_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("history_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("notify_state", String.class, 255, null, false));
- columns.add(new DBColumnInfo("uuid", String.class, 64, null, false));
- dbAccessor.createTable(ALERT_TABLE_NOTICE, columns, "notification_id");
-
- dbAccessor.addFKConstraint(ALERT_TABLE_NOTICE, "fk_alert_notice_target_id",
- "target_id", ALERT_TABLE_TARGET, "target_id", false);
-
- dbAccessor.addFKConstraint(ALERT_TABLE_NOTICE, "fk_alert_notice_hist_id",
- "history_id", ALERT_TABLE_HISTORY, "alert_id", false);
-
- dbAccessor.addUniqueConstraint(ALERT_TABLE_NOTICE, "uni_alert_notice_uuid", "uuid");
- // Indexes
- dbAccessor.createIndex("idx_alert_history_def_id", ALERT_TABLE_HISTORY,
- "alert_definition_id");
- dbAccessor.createIndex("idx_alert_history_service", ALERT_TABLE_HISTORY,
- "service_name");
- dbAccessor.createIndex("idx_alert_history_host", ALERT_TABLE_HISTORY,
- "host_name");
- dbAccessor.createIndex("idx_alert_history_time", ALERT_TABLE_HISTORY,
- "alert_timestamp");
- dbAccessor.createIndex("idx_alert_history_state", ALERT_TABLE_HISTORY,
- "alert_state");
- dbAccessor.createIndex("idx_alert_group_name", ALERT_TABLE_GROUP,
- "group_name");
- dbAccessor.createIndex("idx_alert_notice_state", ALERT_TABLE_NOTICE,
- "notify_state");
- }
-
- protected void addMissingConfigs() throws AmbariException {
- addNewConfigurationsFromXml();
- updateOozieConfigs();
- }
-
- protected void updateOozieConfigs() throws AmbariException {
- final String PROPERTY_NAME = "log4j.appender.oozie.layout.ConversionPattern=";
- final String PROPERTY_VALUE_OLD = "%d{ISO8601} %5p %c{1}:%L - %m%n";
- final String PROPERTY_VALUE_NEW = "%d{ISO8601} %5p %c{1}:%L - SERVER[${oozie.instance.id}] %m%n";
-
- AmbariManagementController ambariManagementController = injector.getInstance(
- AmbariManagementController.class);
- Clusters clusters = ambariManagementController.getClusters();
-
- if (clusters != null) {
- Map<String, Cluster> clusterMap = clusters.getClusters();
- Map<String, String> prop = new HashMap<String, String>();
- String content = null;
-
- if (clusterMap != null && !clusterMap.isEmpty()) {
- for (final Cluster cluster : clusterMap.values()) {
- content = null;
- if (cluster.getDesiredConfigByType("oozie-log4j") != null) {
- content = cluster.getDesiredConfigByType(
- "oozie-log4j").getProperties().get("content");
- }
-
- if (content != null) {
- content = content.replace(PROPERTY_NAME + PROPERTY_VALUE_OLD,
- PROPERTY_NAME + PROPERTY_VALUE_NEW);
-
- prop.put("content", content);
- updateConfigurationPropertiesForCluster(cluster, "oozie-log4j",
- prop, true, false);
- }
-
- //oozie_heapsize is added for HDP2, we should check if it exists and not add it for HDP1
- if(cluster.getDesiredConfigByType("oozie-env") != null &&
- cluster.getDesiredConfigByType("oozie-env").getProperties().containsKey("oozie_heapsize")) {
- Map<String, String> oozieProps = new HashMap<String, String>();
- oozieProps.put("oozie_heapsize","2048m");
- oozieProps.put("oozie_permsize","256m");
- updateConfigurationPropertiesForCluster(cluster, "oozie-env",
- oozieProps, true, false);
- }
- }
- }
- }
- }
-
- /**
- * Rename pig-content to content in pig-properties config
- * @throws AmbariException
- */
- protected void renamePigProperties() throws AmbariException {
- ConfigHelper configHelper = injector.getInstance(ConfigHelper.class);
- AmbariManagementController ambariManagementController = injector.getInstance(
- AmbariManagementController.class);
-
- Clusters clusters = ambariManagementController.getClusters();
- if (clusters == null) {
- return;
- }
-
- Map<String, Cluster> clusterMap = clusters.getClusters();
-
- if (clusterMap != null && !clusterMap.isEmpty()) {
- for (final Cluster cluster : clusterMap.values()) {
- Config oldConfig = cluster.getDesiredConfigByType(PIG_PROPERTIES_CONFIG_TYPE);
- if (oldConfig != null) {
- Map<String, String> properties = oldConfig.getProperties();
-
- if(!properties.containsKey(CONTENT_FIELD_NAME)) {
- String value = properties.remove(PIG_CONTENT_FIELD_NAME);
- properties.put(CONTENT_FIELD_NAME, value);
- configHelper.createConfigType(cluster, ambariManagementController,
- PIG_PROPERTIES_CONFIG_TYPE, properties, "ambari-upgrade", null);
- }
- }
- }
- }
- }
-
- protected void addEnvContentFields() throws AmbariException {
- ConfigHelper configHelper = injector.getInstance(ConfigHelper.class);
- AmbariManagementController ambariManagementController = injector.getInstance(
- AmbariManagementController.class);
-
- Clusters clusters = ambariManagementController.getClusters();
- if (clusters == null) {
- return;
- }
-
- Map<String, Cluster> clusterMap = clusters.getClusters();
-
- if (clusterMap != null && !clusterMap.isEmpty()) {
- for (final Cluster cluster : clusterMap.values()) {
- Set<String> configTypes = configHelper.findConfigTypesByPropertyName(cluster.getCurrentStackVersion(),
- CONTENT_FIELD_NAME, cluster.getClusterName());
-
- for(String configType:configTypes) {
- if(!configType.endsWith(ENV_CONFIGS_POSTFIX) && !configType.equals("pig-properties")) {
- continue;
- }
-
- updateConfigurationPropertiesWithValuesFromXml(configType, Collections.singleton(CONTENT_FIELD_NAME), false, true);
- }
- }
- }
- }
-
- protected void moveGlobalsToEnv() throws AmbariException {
- ConfigHelper configHelper = injector.getInstance(ConfigHelper.class);
-
- AmbariManagementController ambariManagementController = injector.getInstance(
- AmbariManagementController.class);
- Clusters clusters = ambariManagementController.getClusters();
- if (clusters == null) {
- return;
- }
- Map<String, Cluster> clusterMap = clusters.getClusters();
-
- if (clusterMap != null && !clusterMap.isEmpty()) {
- for (final Cluster cluster : clusterMap.values()) {
- Config config = cluster.getDesiredConfigByType(Configuration.GLOBAL_CONFIG_TAG);
- if (config == null) {
- LOG.info("Config " + Configuration.GLOBAL_CONFIG_TAG + " not found. Assuming upgrade already done.");
- return;
- }
-
- Map<String, Map<String, String>> newProperties = new HashMap<String, Map<String, String>>();
- Map<String, String> globalProperites = config.getProperties();
- Map<String, String> unmappedGlobalProperties = new HashMap<String, String>();
-
- for (Map.Entry<String, String> property : globalProperites.entrySet()) {
- String propertyName = property.getKey();
- String propertyValue = property.getValue();
- String newPropertyName = getNewPropertyName().get(propertyName);
-
- Set<String> newConfigTypes = configHelper.findConfigTypesByPropertyName(cluster.getCurrentStackVersion(),
- propertyName, cluster.getClusterName());
- // if it's custom user service global.xml can be still there.
- newConfigTypes.remove(Configuration.GLOBAL_CONFIG_TAG);
-
- String newConfigType = null;
- if(newConfigTypes.size() > 0) {
- newConfigType = newConfigTypes.iterator().next();
- } else {
- newConfigType = getAdditionalMappingGlobalToEnv().get(((newPropertyName == null)? propertyName : newPropertyName));
- }
-
- if(newConfigType==null) {
- LOG.warn("Cannot find where to map " + propertyName + " from " + Configuration.GLOBAL_CONFIG_TAG +
- " (value="+propertyValue+")");
- unmappedGlobalProperties.put(propertyName, propertyValue);
- continue;
- }
-
- LOG.info("Mapping config " + propertyName + " from " + Configuration.GLOBAL_CONFIG_TAG +
- " to " + ((newPropertyName == null)? propertyName : newPropertyName) + " property in " + newConfigType +
- " (value="+propertyValue+")");
-
- if(!newProperties.containsKey(newConfigType)) {
- newProperties.put(newConfigType, new HashMap<String, String>());
- }
- newProperties.get(newConfigType).put(((newPropertyName == null)? propertyName : newPropertyName), propertyValue);
- }
-
- for (Entry<String, Map<String, String>> newProperty : newProperties.entrySet()) {
- updateConfigurationProperties(newProperty.getKey(), newProperty.getValue(), false, true);
- }
-
- // if have some custom properties, for own services etc., leave that as it was
- if(unmappedGlobalProperties.size() != 0) {
- LOG.info("Not deleting globals because have custom properties");
- configHelper.createConfigType(cluster, ambariManagementController,
- Configuration.GLOBAL_CONFIG_TAG, unmappedGlobalProperties,
- "ambari-upgrade", null);
- } else {
- configHelper.removeConfigsByType(cluster, Configuration.GLOBAL_CONFIG_TAG);
- }
- }
- }
- }
-
- public static Map<String, String> getAdditionalMappingGlobalToEnv() {
- Map<String, String> result = new HashMap<String, String>();
-
- result.put("smokeuser_keytab","hadoop-env");
- result.put("hdfs_user_keytab","hadoop-env");
- result.put("hdfs_principal_name","hadoop-env");
- result.put("kerberos_domain","hadoop-env");
- result.put("hbase_user_keytab","hbase-env");
- result.put("hbase_principal_name", "hbase-env");
- result.put("nagios_principal_name","nagios-env");
- result.put("nagios_keytab_path","nagios-env");
- result.put("oozie_keytab","oozie-env");
- result.put("zookeeper_principal_name","zookeeper-env");
- result.put("zookeeper_keytab_path","zookeeper-env");
- result.put("storm_principal_name","storm-env");
- result.put("storm_keytab","storm-env");
- result.put("hive_hostname","hive-env");
- result.put("oozie_hostname","oozie-env");
- result.put("dataDir","zoo.cfg");
-
- return result;
- }
-
- public Map<String, String> getNewPropertyName() {
- Map<String, String> result = new HashMap<String, String>();
- result.put("zk_data_dir","dataDir");
- return result;
- }
-
- private void upgradePermissionModel() throws SQLException {
- final UserDAO userDAO = injector.getInstance(UserDAO.class);
- final PrincipalDAO principalDAO = injector.getInstance(PrincipalDAO.class);
- final PrincipalTypeDAO principalTypeDAO = injector.getInstance(PrincipalTypeDAO.class);
- final ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- final ResourceTypeDAO resourceTypeDAO = injector.getInstance(ResourceTypeDAO.class);
- final ResourceDAO resourceDAO = injector.getInstance(ResourceDAO.class);
- final ViewDAO viewDAO = injector.getInstance(ViewDAO.class);
- final ViewInstanceDAO viewInstanceDAO = injector.getInstance(ViewInstanceDAO.class);
- final PermissionDAO permissionDAO = injector.getInstance(PermissionDAO.class);
- final PrivilegeDAO privilegeDAO = injector.getInstance(PrivilegeDAO.class);
-
- final PrincipalTypeEntity userPrincipalType = principalTypeDAO.findById(PrincipalTypeEntity.USER_PRINCIPAL_TYPE);
- for (UserEntity user: userDAO.findAll()) {
- final PrincipalEntity principalEntity = new PrincipalEntity();
- principalEntity.setPrincipalType(userPrincipalType);
- principalDAO.create(principalEntity);
- user.setPrincipal(principalEntity);
- userDAO.merge(user);
- }
-
- final ResourceTypeEntity clusterResourceType = resourceTypeDAO.findById(ResourceType.CLUSTER.getId());
- for (ClusterEntity cluster: clusterDAO.findAll()) {
- final ResourceEntity resourceEntity = new ResourceEntity();
- resourceEntity.setResourceType(clusterResourceType);
- resourceDAO.create(resourceEntity);
- cluster.setResource(resourceEntity);
- clusterDAO.merge(cluster);
- }
-
- for (ViewEntity view: viewDAO.findAll()) {
- final ResourceTypeEntity resourceType = new ResourceTypeEntity();
- resourceType.setName(ViewEntity.getViewName(view.getCommonName(), view.getVersion()));
- resourceTypeDAO.create(resourceType);
- }
-
- for (ViewInstanceEntity viewInstance: viewInstanceDAO.findAll()) {
- final ResourceEntity resourceEntity = new ResourceEntity();
- viewInstance.getViewEntity();
- resourceEntity.setResourceType(resourceTypeDAO.findByName(
- ViewEntity.getViewName(
- viewInstance.getViewEntity().getCommonName(),
- viewInstance.getViewEntity().getVersion())));
- viewInstance.setResource(resourceEntity);
- resourceDAO.create(resourceEntity);
- viewInstanceDAO.merge(viewInstance);
- }
-
- final PermissionEntity adminPermission = permissionDAO.findAmbariAdminPermission();
- final PermissionEntity clusterOperatePermission = permissionDAO.findClusterOperatePermission();
- final PermissionEntity clusterReadPermission = permissionDAO.findClusterReadPermission();
- final ResourceEntity ambariResource = resourceDAO.findAmbariResource();
-
- final Map<UserEntity, List<String>> roles = new HashMap<UserEntity, List<String>>();
- Statement statement = null;
- ResultSet rs = null;
- try {
- statement = dbAccessor.getConnection().createStatement();
- if (statement != null) {
- rs = statement.executeQuery("SELECT role_name, user_id FROM user_roles");
- if (rs != null) {
- while (rs.next()) {
- final String roleName = rs.getString(1);
- final int userId = rs.getInt(2);
-
- final UserEntity user = userDAO.findByPK(userId);
- List<String> userRoles = roles.get(user);
- if (userRoles == null) {
- userRoles = new ArrayList<String>();
- roles.put(user, userRoles);
- }
- userRoles.add(roleName);
- }
- }
- }
- } finally {
- if (rs != null) {
- rs.close();
- }
- if (statement != null) {
- statement.close();
- }
- }
- for (UserEntity user: userDAO.findAll()) {
- List<String> userRoles = roles.get(user);
- if (userRoles.contains("admin")) {
- final PrivilegeEntity privilege = new PrivilegeEntity();
- privilege.setPermission(adminPermission);
- privilege.setPrincipal(user.getPrincipal());
- privilege.setResource(ambariResource);
- user.getPrincipal().getPrivileges().add(privilege);
- privilegeDAO.create(privilege);
- for (ClusterEntity cluster: clusterDAO.findAll()) {
- final PrivilegeEntity clusterPrivilege = new PrivilegeEntity();
- clusterPrivilege.setPermission(clusterOperatePermission);
- clusterPrivilege.setPrincipal(user.getPrincipal());
- clusterPrivilege.setResource(cluster.getResource());
- privilegeDAO.create(clusterPrivilege);
- user.getPrincipal().getPrivileges().add(clusterPrivilege);
- }
- userDAO.merge(user);
- } else if (userRoles.contains("user")) {
- for (ClusterEntity cluster: clusterDAO.findAll()) {
- final PrivilegeEntity privilege = new PrivilegeEntity();
- privilege.setPermission(clusterReadPermission);
- privilege.setPrincipal(user.getPrincipal());
- privilege.setResource(cluster.getResource());
- privilegeDAO.create(privilege);
- user.getPrincipal().getPrivileges().add(privilege);
- }
- userDAO.merge(user);
- }
- }
-
- dbAccessor.dropTable("user_roles");
- dbAccessor.dropTable("roles");
- }
-
- protected void addJobsViewPermissions() {
-
- final UserDAO userDAO = injector.getInstance(UserDAO.class);
- final ResourceTypeDAO resourceTypeDAO = injector.getInstance(ResourceTypeDAO.class);
- final ResourceDAO resourceDAO = injector.getInstance(ResourceDAO.class);
- final ViewDAO viewDAO = injector.getInstance(ViewDAO.class);
- final ViewInstanceDAO viewInstanceDAO = injector.getInstance(ViewInstanceDAO.class);
- final KeyValueDAO keyValueDAO = injector.getInstance(KeyValueDAO.class);
- final PermissionDAO permissionDAO = injector.getInstance(PermissionDAO.class);
- final PrivilegeDAO privilegeDAO = injector.getInstance(PrivilegeDAO.class);
- final ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- final ViewRegistry viewRegistry = injector.getInstance(ViewRegistry.class);
-
- List<ClusterEntity> clusters = clusterDAO.findAll();
- if (!clusters.isEmpty()) {
- ClusterEntity currentCluster = clusters.get(0);
- StackEntity currentStack = currentCluster.getClusterStateEntity().getCurrentStack();
-
- boolean isStackHdp21 = CLUSTER_STATE_STACK_HDP_2_1.getStackName().equals(
- currentStack.getStackName())
- && CLUSTER_STATE_STACK_HDP_2_1.getStackVersion().equals(
- currentStack.getStackVersion());
-
- if (isStackHdp21) {
- ViewRegistry.initInstance(viewRegistry);
- viewRegistry.readViewArchives(VIEW_NAME_REG_EXP);
- ViewEntity jobsView = viewDAO.findByCommonName(JOBS_VIEW_NAME);
-
- if (jobsView != null) {
- ViewInstanceEntity jobsInstance = jobsView.getInstanceDefinition(JOBS_VIEW_INSTANCE_NAME);
- if (jobsInstance == null) {
- jobsInstance = new ViewInstanceEntity(jobsView, JOBS_VIEW_INSTANCE_NAME, JOBS_VIEW_INSTANCE_LABEL);
- ResourceEntity resourceEntity = new ResourceEntity();
- resourceEntity.setResourceType(resourceTypeDAO.findByName(
- ViewEntity.getViewName(
- jobsView.getCommonName(),
- jobsView.getVersion())));
- String atsHost;
- String rmHost;
- try {
- ClusterConfigEntity currentYarnConfig = null;
- for (ClusterConfigMappingEntity configMappingEntity : currentCluster.getConfigMappingEntities()) {
- if (YARN_SITE.equals(configMappingEntity.getType()) && configMappingEntity.isSelected() > 0) {
- currentYarnConfig = clusterDAO.findConfig(currentCluster.getClusterId(),
- configMappingEntity.getType(), configMappingEntity.getTag());
- break;
- }
- }
- Type type = new TypeToken<Map<String, String>>() {}.getType();
- Map<String, String> yarnSiteProps = StageUtils.getGson().fromJson(currentYarnConfig.getData(), type);
- atsHost = yarnSiteProps.get(YARN_TIMELINE_SERVICE_WEBAPP_ADDRESS_PROPERTY);
- rmHost = yarnSiteProps.get(YARN_RESOURCEMANAGER_WEBAPP_ADDRESS_PROPERTY);
- } catch (Exception ex) {
- // Required properties failed to be set, therefore jobs instance should not be created
- return;
- }
- jobsInstance.setResource(resourceEntity);
- jobsInstance.putProperty(YARN_ATS_URL_PROPERTY, "http://" + atsHost);
- jobsInstance.putProperty(YARN_RESOURCEMANAGER_URL_PROPERTY, "http://" + rmHost);
- jobsView.addInstanceDefinition(jobsInstance);
- resourceDAO.create(resourceEntity);
- viewInstanceDAO.create(jobsInstance);
- viewDAO.merge(jobsView);
- }
- // get showJobsForNonAdmin value and remove it
- boolean showJobsForNonAdmin = false;
- KeyValueEntity showJobsKeyValueEntity = keyValueDAO.findByKey(SHOW_JOBS_FOR_NON_ADMIN_KEY);
- if (showJobsKeyValueEntity != null) {
- String value = showJobsKeyValueEntity.getValue();
- showJobsForNonAdmin = Boolean.parseBoolean(value);
- keyValueDAO.remove(showJobsKeyValueEntity);
- }
- if (showJobsForNonAdmin) {
- ResourceEntity jobsResource = jobsInstance.getResource();
- long jobsResourceId = jobsResource.getId();
- PermissionEntity viewUsePermission = permissionDAO.findViewUsePermission();
- PermissionEntity adminPermission = permissionDAO.findAmbariAdminPermission();
- int viewUsePermissionId = viewUsePermission.getId();
- int adminPermissionId = adminPermission.getId();
- for (UserEntity userEntity : userDAO.findAll()) {
- // check if user has VIEW.USE privilege for JOBS view
- List<PrivilegeEntity> privilegeEntities = privilegeDAO.findAllByPrincipal(
- Collections.singletonList(userEntity.getPrincipal()));
- boolean hasJobsUsePrivilege = false;
- for (PrivilegeEntity privilegeEntity : privilegeEntities) {
- int privilegePermissionId = privilegeEntity.getPermission().getId();
- Long privilegeResourceId = privilegeEntity.getResource().getId();
- if ((privilegePermissionId == viewUsePermissionId && privilegeResourceId == jobsResourceId)
- || privilegePermissionId == adminPermissionId) {
- hasJobsUsePrivilege = true;
- break;
- }
- }
- // if not - add VIEW.use privilege
- if (!hasJobsUsePrivilege) {
- PrivilegeEntity privilegeEntity = new PrivilegeEntity();
- privilegeEntity.setResource(jobsResource);
- privilegeEntity.setPermission(viewUsePermission);
- privilegeEntity.setPrincipal(userEntity.getPrincipal());
- privilegeDAO.create(privilegeEntity);
- }
- }
- }
- }
- }
- }
-
- }
-}
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog200.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog200.java b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog200.java
index 1458557..1f5b466 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog200.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog200.java
@@ -84,14 +84,6 @@ public class UpgradeCatalog200 extends AbstractUpgradeCatalog {
* {@inheritDoc}
*/
@Override
- public String getSourceVersion() {
- return "1.7.0";
- }
-
- /**
- * {@inheritDoc}
- */
- @Override
public String getTargetVersion() {
return "2.0.0";
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java b/ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java
index ee64ac9..82526e7 100644
--- a/ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java
+++ b/ambari-server/src/test/java/org/apache/ambari/server/topology/AmbariContextTest.java
@@ -330,7 +330,6 @@ public class AmbariContextTest {
expect(clusterController.ensureResourceProvider(Resource.Type.ConfigGroup)).andReturn(configGroupResourceProvider).once();
//todo: for now not using return value so just returning null
expect(configGroupResourceProvider.createResources(capture(configGroupRequestCapture))).andReturn(null).once();
- configHelper.moveDeprecatedGlobals(stackId, group1Configuration.getFullProperties(1), CLUSTER_NAME);
// replay all mocks
replayAll();
@@ -370,7 +369,6 @@ public class AmbariContextTest {
expect(clusterController.ensureResourceProvider(Resource.Type.ConfigGroup)).andReturn(configGroupResourceProvider).once();
//todo: for now not using return value so just returning null
expect(configGroupResourceProvider.createResources(capture(configGroupRequestCapture))).andReturn(null).once();
- configHelper.moveDeprecatedGlobals(stackId, group1Configuration.getFullProperties(1), CLUSTER_NAME);
reset(group1Info);
expect(group1Info.getConfiguration()).andReturn(group1Configuration).anyTimes();
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog150Test.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog150Test.java b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog150Test.java
deleted file mode 100644
index eebda98..0000000
--- a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog150Test.java
+++ /dev/null
@@ -1,258 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.ambari.server.upgrade;
-
-import javax.persistence.EntityManager;
-
-import junit.framework.Assert;
-
-import org.apache.ambari.server.AmbariException;
-import org.apache.ambari.server.api.services.AmbariMetaInfo;
-import org.apache.ambari.server.orm.GuiceJpaInitializer;
-import org.apache.ambari.server.orm.InMemoryDefaultTestModule;
-import org.apache.ambari.server.orm.dao.ClusterDAO;
-import org.apache.ambari.server.orm.dao.HostComponentDesiredStateDAO;
-import org.apache.ambari.server.orm.dao.KeyValueDAO;
-import org.apache.ambari.server.orm.dao.ServiceComponentDesiredStateDAO;
-import org.apache.ambari.server.orm.dao.StackDAO;
-import org.apache.ambari.server.orm.entities.ClusterConfigEntity;
-import org.apache.ambari.server.orm.entities.ClusterConfigMappingEntity;
-import org.apache.ambari.server.orm.entities.ClusterEntity;
-import org.apache.ambari.server.orm.entities.ClusterServiceEntity;
-import org.apache.ambari.server.orm.entities.HostComponentDesiredStateEntity;
-import org.apache.ambari.server.orm.entities.HostComponentStateEntity;
-import org.apache.ambari.server.orm.entities.HostEntity;
-import org.apache.ambari.server.orm.entities.KeyValueEntity;
-import org.apache.ambari.server.orm.entities.ServiceComponentDesiredStateEntity;
-import org.apache.ambari.server.orm.entities.StackEntity;
-import org.apache.ambari.server.state.HostComponentAdminState;
-import org.apache.ambari.server.state.StackId;
-import org.junit.After;
-import org.junit.Before;
-import org.junit.Ignore;
-import org.junit.Test;
-
-import com.google.inject.Guice;
-import com.google.inject.Injector;
-import com.google.inject.persist.PersistService;
-
-import java.util.Collections;
-
-/**
- * UpgradeCatalog150 tests.
- * @Deprecated
- */
-@Ignore
-public class UpgradeCatalog150Test {
- private Injector injector;
- private final String CLUSTER_NAME = "c1";
- private final String HOST_NAME = "h1";
-
- public static final StackId DESIRED_STACK = new StackId("HDP", "1.3.4");
-
- private UpgradeCatalogHelper upgradeCatalogHelper;
- private StackEntity desiredStackEntity;
-
- @Before
- public void setup() throws Exception {
- injector = Guice.createInjector(new InMemoryDefaultTestModule());
- injector.getInstance(GuiceJpaInitializer.class);
- upgradeCatalogHelper = injector.getInstance(UpgradeCatalogHelper.class);
-
- // inject AmbariMetaInfo to ensure that stacks get populated in the DB
- injector.getInstance(AmbariMetaInfo.class);
-
- // load the stack entity
- StackDAO stackDAO = injector.getInstance(StackDAO.class);
-
- desiredStackEntity = stackDAO.find(DESIRED_STACK.getStackName(),
- DESIRED_STACK.getStackVersion());
-
- Assert.assertNotNull(desiredStackEntity);
- }
-
- @After
- public void tearDown() {
- injector.getInstance(PersistService.class).stop();
- }
-
- protected void executeInTransaction(Runnable func) {
- EntityManager entityManager = injector.getProvider(EntityManager.class).get();
- if (entityManager.getTransaction().isActive()) { //already started, reuse
- func.run();
- } else {
- entityManager.getTransaction().begin();
- try {
- func.run();
- entityManager.getTransaction().commit();
- } catch (Exception e) {
- //LOG.error("Error in transaction ", e);
- if (entityManager.getTransaction().isActive()) {
- entityManager.getTransaction().rollback();
- }
- throw new RuntimeException(e);
- }
-
- }
- }
-
- @Test
- public void testAddHistoryServer() throws AmbariException {
- final ClusterEntity clusterEntity = upgradeCatalogHelper.createCluster(
- injector, CLUSTER_NAME, desiredStackEntity);
- final ClusterServiceEntity clusterServiceEntityMR = upgradeCatalogHelper.addService(
- injector, clusterEntity, "MAPREDUCE", desiredStackEntity);
- final HostEntity hostEntity = upgradeCatalogHelper.createHost(injector,
- clusterEntity, HOST_NAME);
-
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- upgradeCatalogHelper.addComponent(injector, clusterEntity,
- clusterServiceEntityMR, hostEntity, "JOBTRACKER",
- desiredStackEntity);
- }
- });
-
- UpgradeCatalog150 upgradeCatalog150 = injector.getInstance(UpgradeCatalog150.class);
- upgradeCatalog150.addHistoryServer();
- }
-
- @Test
- public void testProcessDecommissionedDatanodes() throws Exception {
- ClusterEntity clusterEntity = upgradeCatalogHelper.createCluster(injector,
- CLUSTER_NAME, desiredStackEntity);
- ClusterServiceEntity clusterServiceEntity = upgradeCatalogHelper.createService(
- injector, clusterEntity, "HDFS");
- HostEntity hostEntity = upgradeCatalogHelper.createHost(injector,
- clusterEntity, HOST_NAME);
-
- ServiceComponentDesiredStateEntity componentDesiredStateEntity =
- new ServiceComponentDesiredStateEntity();
- componentDesiredStateEntity.setClusterId(clusterEntity.getClusterId());
- componentDesiredStateEntity.setServiceName(clusterServiceEntity.getServiceName());
- componentDesiredStateEntity.setClusterServiceEntity(clusterServiceEntity);
- componentDesiredStateEntity.setComponentName("DATANODE");
- componentDesiredStateEntity.setDesiredStack(desiredStackEntity);
-
- ServiceComponentDesiredStateDAO componentDesiredStateDAO =
- injector.getInstance(ServiceComponentDesiredStateDAO.class);
-
- componentDesiredStateDAO.create(componentDesiredStateEntity);
-
- HostComponentDesiredStateDAO hostComponentDesiredStateDAO =
- injector.getInstance(HostComponentDesiredStateDAO.class);
-
- HostComponentDesiredStateEntity hostComponentDesiredStateEntity =
- new HostComponentDesiredStateEntity();
-
- hostComponentDesiredStateEntity.setClusterId(clusterEntity.getClusterId());
- hostComponentDesiredStateEntity.setComponentName("DATANODE");
- hostComponentDesiredStateEntity.setAdminState(HostComponentAdminState.INSERVICE);
- hostComponentDesiredStateEntity.setServiceName(clusterServiceEntity.getServiceName());
- hostComponentDesiredStateEntity.setServiceComponentDesiredStateEntity(componentDesiredStateEntity);
- hostComponentDesiredStateEntity.setHostEntity(hostEntity);
- hostComponentDesiredStateEntity.setDesiredStack(desiredStackEntity);
- componentDesiredStateEntity.setHostComponentDesiredStateEntities(
- Collections.singletonList(hostComponentDesiredStateEntity));
-
- hostComponentDesiredStateDAO.create(hostComponentDesiredStateEntity);
-
- HostComponentDesiredStateEntity entity = hostComponentDesiredStateDAO.findAll().get(0);
-
- Assert.assertEquals(HostComponentAdminState.INSERVICE.name(), entity.getAdminState().name());
-
- KeyValueDAO keyValueDAO = injector.getInstance(KeyValueDAO.class);
- KeyValueEntity keyValueEntity = new KeyValueEntity();
- keyValueEntity.setKey("decommissionDataNodesTag");
- keyValueEntity.setValue("1394147791230");
- keyValueDAO.create(keyValueEntity);
-
- ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
- ClusterConfigEntity configEntity = new ClusterConfigEntity();
- configEntity.setClusterEntity(clusterEntity);
- configEntity.setClusterId(clusterEntity.getClusterId());
- configEntity.setType("hdfs-exclude-file");
- configEntity.setTag("1394147791230");
- configEntity.setData("{\"datanodes\":\"" + HOST_NAME + "\"}");
- configEntity.setTimestamp(System.currentTimeMillis());
- configEntity.setStack(desiredStackEntity);
- configEntity.setStack(clusterEntity.getDesiredStack());
-
- clusterDAO.createConfig(configEntity);
-
- UpgradeCatalog150 upgradeCatalog150 = injector.getInstance(UpgradeCatalog150.class);
-
- upgradeCatalog150.processDecommissionedDatanodes();
-
- entity = hostComponentDesiredStateDAO.findAll().get(0);
-
- Assert.assertEquals(HostComponentAdminState.DECOMMISSIONED.name(), entity.getAdminState().name());
-
- keyValueEntity = keyValueDAO.findByKey("decommissionDataNodesTag");
- Assert.assertNull(keyValueEntity);
- keyValueEntity = keyValueDAO.findByKey("decommissionDataNodesTag-Moved");
- Assert.assertNotNull(keyValueEntity);
- Assert.assertEquals("1394147791230", keyValueEntity.getValue());
- }
-
- @Test
- public void testAddMissingLog4jConfigs() throws Exception {
- ClusterDAO clusterDAO = injector.getInstance(ClusterDAO.class);
-
- ClusterEntity clusterEntity = upgradeCatalogHelper.createCluster(injector,
- CLUSTER_NAME, desiredStackEntity);
-
- ClusterServiceEntity clusterServiceEntityMR = upgradeCatalogHelper.addService(
- injector, clusterEntity, "HDFS", desiredStackEntity);
-
- Long clusterId = clusterEntity.getClusterId();
-
- ClusterConfigEntity configEntity = clusterDAO.findConfig(clusterId, "hdfs-log4j", "version1");
- Assert.assertNull(configEntity);
-
- for (ClusterConfigMappingEntity ccme : clusterEntity.getConfigMappingEntities()) {
- if ("hdfs-log4j".equals(ccme.getType())) {
- Assert.fail();
- }
- }
-
- UpgradeCatalog150 upgradeCatalog150 = injector.getInstance(UpgradeCatalog150.class);
- upgradeCatalog150.addMissingLog4jConfigs();
-
- configEntity = clusterDAO.findConfig(clusterId, "hdfs-log4j", "version1");
- Assert.assertNotNull(configEntity);
-
- //Get updated cluster
- clusterEntity = clusterDAO.findById(1L);
-
- boolean failFlag = true;
- for (ClusterConfigMappingEntity ccme : clusterEntity.getConfigMappingEntities()) {
- if ("hdfs-log4j".equals(ccme.getType())) {
- failFlag = false;
- }
- }
- Assert.assertFalse(failFlag);
- }
-
- @Test
- public void testGetSourceVersion() {
- UpgradeCatalog150 upgradeCatalog150 = injector.getInstance(UpgradeCatalog150.class);
- Assert.assertNull(upgradeCatalog150.getSourceVersion());
- }
-}
[22/31] ambari git commit: AMBARI-18818 Config changes for Atlas to
Support KnoxSSO Authentication (mugdha)
Posted by nc...@apache.org.
AMBARI-18818 Config changes for Atlas to Support KnoxSSO Authentication (mugdha)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/5454bd61
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/5454bd61
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/5454bd61
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 5454bd612774d7f17704adaff24cbfa73c6a4f77
Parents: 8c3ad1d
Author: Mugdha Varadkar <mu...@apache.org>
Authored: Tue Nov 15 11:30:25 2016 +0530
Committer: Mugdha Varadkar <mu...@apache.org>
Committed: Wed Nov 16 10:23:32 2016 +0530
----------------------------------------------------------------------
.../configuration/application-properties.xml | 11 +
.../configuration/application-properties.xml | 95 +++
.../ATLAS/configuration/atlas-log4j.xml | 118 +++
.../stacks/HDP/2.6/services/ATLAS/metainfo.xml | 10 +-
.../services/ATLAS/themes/theme_version_2.json | 845 +++++++++++++++++++
.../stacks/HDP/2.6/services/stack_advisor.py | 381 +++++----
.../stacks/2.6/common/test_stack_advisor.py | 97 +++
7 files changed, 1374 insertions(+), 183 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/5454bd61/ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml b/ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
index 95715b3..fda16b2 100644
--- a/ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
+++ b/ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
@@ -77,18 +77,29 @@
<value>false</value>
<description>Indicates whether or not Kerberos is enabled.</description>
<on-ambari-upgrade add="false"/>
+ <value-attributes>
+ <type>boolean</type>
+ </value-attributes>
</property>
<property>
<name>atlas.authentication.method.file</name>
+ <display-name>Enable File Authentication</display-name>
<value>true</value>
<description>Indicates whether or not file based authentication is enabled.</description>
<on-ambari-upgrade add="false"/>
+ <value-attributes>
+ <type>boolean</type>
+ </value-attributes>
</property>
<property>
<name>atlas.authentication.method.ldap</name>
+ <display-name>Enable LDAP Authentication</display-name>
<value>false</value>
<description>Indicates whether or not LDAP authentication is enabled.</description>
<on-ambari-upgrade add="false"/>
+ <value-attributes>
+ <type>boolean</type>
+ </value-attributes>
</property>
<property>
<name>atlas.authentication.method.file.filename</name>
http://git-wip-us.apache.org/repos/asf/ambari/blob/5454bd61/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/application-properties.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/application-properties.xml b/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/application-properties.xml
new file mode 100644
index 0000000..0d6ee38
--- /dev/null
+++ b/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/application-properties.xml
@@ -0,0 +1,95 @@
+<?xml version="1.0"?>
+<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+<!--
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+-->
+<configuration supports_final="false">
+
+ <!-- Atlas knox sso properties -->
+ <property>
+ <name>atlas.sso.knox.enabled</name>
+ <display-name>Enable Atlas Knox SSO</display-name>
+ <value>false</value>
+ <description/>
+ <value-attributes>
+ <type>boolean</type>
+ </value-attributes>
+ <on-ambari-upgrade add="true"/>
+ </property>
+
+ <property>
+ <name>atlas.sso.knox.providerurl</name>
+ <value/>
+ <description/>
+ <value-attributes>
+ <empty-value-valid>true</empty-value-valid>
+ </value-attributes>
+ <depends-on>
+ <property>
+ <type>gateway-site</type>
+ <name>gateway.port</name>
+ </property>
+ </depends-on>
+ <on-ambari-upgrade add="true"/>
+ </property>
+
+ <property>
+ <name>atlas.sso.knox.publicKey</name>
+ <value/>
+ <description/>
+ <value-attributes>
+ <empty-value-valid>true</empty-value-valid>
+ </value-attributes>
+ <on-ambari-upgrade add="true"/>
+ </property>
+
+ <property>
+ <name>atlas.sso.knox.browser.useragent</name>
+ <value/>
+ <description/>
+ <value-attributes>
+ <empty-value-valid>true</empty-value-valid>
+ </value-attributes>
+ <on-ambari-upgrade add="true"/>
+ </property>
+
+ <property>
+ <name>atlas.authentication.method.ldap.type</name>
+ <display-name>LDAP Authentication Type</display-name>
+ <value>ldap</value>
+ <description>The LDAP type (ldap, ad, or none).</description>
+ <value-attributes>
+ <overridable>false</overridable>
+ <type>value-list</type>
+ <entries>
+ <entry>
+ <value>ldap</value>
+ <label>LDAP</label>
+ </entry>
+ <entry>
+ <value>ad</value>
+ <label>AD</label>
+ </entry>
+ </entries>
+ <selection-cardinality>1</selection-cardinality>
+ </value-attributes>
+ <on-ambari-upgrade add="true"/>
+ </property>
+
+</configuration>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/ambari/blob/5454bd61/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/atlas-log4j.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/atlas-log4j.xml b/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/atlas-log4j.xml
new file mode 100644
index 0000000..aebf241
--- /dev/null
+++ b/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/atlas-log4j.xml
@@ -0,0 +1,118 @@
+<?xml version="1.0"?>
+<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+<!--
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+-->
+<configuration supports_adding_forbidden="true">
+ <property>
+ <name>content</name>
+ <display-name>atlas-log4j template</display-name>
+ <description>Custom log4j.properties</description>
+ <value><![CDATA[<?xml version="1.0" encoding="UTF-8" ?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one
+ ~ or more contributor license agreements. See the NOTICE file
+ ~ distributed with this work for additional information
+ ~ regarding copyright ownership. The ASF licenses this file
+ ~ to you under the Apache License, Version 2.0 (the
+ ~ "License"); you may not use this file except in compliance
+ ~ with the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+
+<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
+
+<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
+ <appender name="console" class="org.apache.log4j.ConsoleAppender">
+ <param name="Target" value="System.out"/>
+ <layout class="org.apache.log4j.PatternLayout">
+ <param name="ConversionPattern" value="%d %-5p - [%t:%x] ~ %m (%c{1}:%L)%n"/>
+ </layout>
+ </appender>
+
+ <appender name="FILE" class="org.apache.log4j.DailyRollingFileAppender">
+ <param name="File" value="{{log_dir}}/application.log"/>
+ <param name="Append" value="true"/>
+ <param name="Threshold" value="info"/>
+ <layout class="org.apache.log4j.PatternLayout">
+ <param name="ConversionPattern" value="%d %-5p - [%t:%x] ~ %m (%c{1}:%L)%n"/>
+ </layout>
+ </appender>
+
+ <appender name="AUDIT" class="org.apache.log4j.DailyRollingFileAppender">
+ <param name="File" value="{{log_dir}}/audit.log"/>
+ <param name="Append" value="true"/>
+ <param name="Threshold" value="info"/>
+ <layout class="org.apache.log4j.PatternLayout">
+ <param name="ConversionPattern" value="%d %x %m%n"/>
+ </layout>
+ </appender>
+
+ <logger name="org.apache.atlas" additivity="false">
+ <level value="{{atlas_log_level}}"/>
+ <appender-ref ref="FILE"/>
+ </logger>
+
+
+ <logger name="com.thinkaurelius.titan" additivity="false">
+ <level value="info"/>
+ <appender-ref ref="FILE"/>
+ </logger>
+
+ <logger name="org.elasticsearch" additivity="false">
+ <level value="info"/>
+ <appender-ref ref="FILE"/>
+ </logger>
+
+ <logger name="org.apache.lucene" additivity="false">
+ <level value="info"/>
+ <appender-ref ref="FILE"/>
+ </logger>
+
+ <logger name="com.google" additivity="false">
+ <level value="info"/>
+ <appender-ref ref="FILE"/>
+ </logger>
+
+ <logger name="AUDIT" additivity="false">
+ <level value="{{audit_log_level}}"/>
+ <appender-ref ref="AUDIT"/>
+ </logger>
+
+ <root>
+ <priority value="info"/>
+ <appender-ref ref="FILE"/>
+ </root>
+
+</log4j:configuration>
+ ]]></value>
+ <value-attributes>
+ <type>content</type>
+ <show-property-name>false</show-property-name>
+ </value-attributes>
+ <on-ambari-upgrade add="true"/>
+ </property>
+</configuration>
http://git-wip-us.apache.org/repos/asf/ambari/blob/5454bd61/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/metainfo.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/metainfo.xml b/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/metainfo.xml
index 92db1f9..eaa5e21 100644
--- a/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/metainfo.xml
+++ b/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/metainfo.xml
@@ -20,7 +20,15 @@
<services>
<service>
<name>ATLAS</name>
- <version>0.7.0.2.5</version>
+ <version>0.8.0.2.6</version>
+
+ <themes>
+ <theme>
+ <fileName>theme_version_2.json</fileName>
+ <default>true</default>
+ </theme>
+ </themes>
+
</service>
</services>
</metainfo>
http://git-wip-us.apache.org/repos/asf/ambari/blob/5454bd61/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/themes/theme_version_2.json
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/themes/theme_version_2.json b/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/themes/theme_version_2.json
new file mode 100644
index 0000000..f72ea4b
--- /dev/null
+++ b/ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/themes/theme_version_2.json
@@ -0,0 +1,845 @@
+{
+ "name": "default",
+ "description": "Default theme for Atlas service",
+ "configuration": {
+ "layouts": [
+ {
+ "name": "default",
+ "tabs": [
+ {
+ "name": "authentication_settings",
+ "display-name": "Authentication",
+ "layout": {
+ "tab-columns": "1",
+ "tab-rows": "4",
+ "sections": [
+ {
+ "name": "section-authentication-type",
+ "display-name": "Authentication Methods",
+ "row-index": "0",
+ "column-index": "0",
+ "row-span": "1",
+ "column-span": "1",
+ "section-columns": "1",
+ "section-rows": "1",
+ "subsections": [
+ {
+ "name": "subsection-authentication-type",
+ "row-index": "0",
+ "column-index": "0",
+ "row-span": "1",
+ "column-span": "1"
+ }
+ ]
+ },
+ {
+ "name": "subsection-authentication-file",
+ "display-name": "File",
+ "row-index": "1",
+ "column-index": "0",
+ "row-span": "1",
+ "column-span": "1",
+ "section-columns": "1",
+ "section-rows": "1",
+ "subsections": [
+ {
+ "name": "subsection-authentication-file",
+ "row-index": "0",
+ "column-index": "0",
+ "row-span": "1",
+ "column-span": "1"
+ }
+ ]
+ },
+ {
+ "name": "section-authentication-ldap",
+ "display-name": "LDAP/AD",
+ "row-index": "2",
+ "column-index": "0",
+ "row-span": "1",
+ "column-span": "1",
+ "section-columns": "1",
+ "section-rows": "1",
+ "subsections": [
+ {
+ "name": "subsection-authentication-ldap",
+ "row-index": "0",
+ "column-index": "0",
+ "row-span": "1",
+ "column-span": "1",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ }
+ ]
+ },
+ {
+ "name": "section-atlas-sso",
+ "display-name": "Atlas Knox SSO",
+ "row-index": "3",
+ "column-index": "0",
+ "row-span": "1",
+ "column-span": "1",
+ "section-columns": "1",
+ "section-rows": "1",
+ "subsections": [
+ {
+ "name": "subsection-atlas-sso",
+ "row-index": "0",
+ "column-index": "0",
+ "row-span": "1",
+ "column-span": "1"
+ }
+ ]
+ }
+ ]
+ }
+ }
+ ]
+ }
+ ],
+ "placement": {
+ "configuration-layout": "default",
+ "configs": [
+ {
+ "config": "application-properties/atlas.authentication.method.file",
+ "subsection-name": "subsection-authentication-type"
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.file.filename",
+ "subsection-name": "subsection-authentication-file",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.file"
+ ],
+ "if": "${application-properties/atlas.authentication.method.file}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap",
+ "subsection-name": "subsection-authentication-type"
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.type",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.url",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.userDNpattern",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap} ",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.groupSearchBase",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.groupSearchFilter",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.groupRoleAttribute",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.base.dn",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.bind.dn",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.bind.password",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.referral",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.user.searchfilter",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.default.role",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ldap && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.url",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ad && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.domain",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ad && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.base.dn",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ad && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.bind.dn",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ad && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.bind.password",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ad && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.referral",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ad && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.user.searchfilter",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ad && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.default.role",
+ "subsection-name": "subsection-authentication-ldap",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.authentication.method.ldap.type",
+ "application-properties/atlas.authentication.method.ldap"
+ ],
+ "if": "${application-properties/atlas.authentication.method.ldap.type} === ad && ${application-properties/atlas.authentication.method.ldap}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.sso.enabled",
+ "subsection-name": "subsection-authentication-type"
+ },
+ {
+ "config": "application-properties/atlas.sso.providerurl",
+ "subsection-name": "subsection-atlas-sso",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.sso.enabled"
+ ],
+ "if": "${application-properties/atlas.sso.enabled}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.sso.publicKey",
+ "subsection-name": "subsection-atlas-sso",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.sso.enabled"
+ ],
+ "if": "${application-properties/atlas.sso.enabled}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ },
+ {
+ "config": "application-properties/atlas.sso.browser.useragent",
+ "subsection-name": "subsection-atlas-sso",
+ "depends-on": [
+ {
+ "configs":[
+ "application-properties/atlas.sso.enabled"
+ ],
+ "if": "${application-properties/atlas.sso.enabled}",
+ "then": {
+ "property_value_attributes": {
+ "visible": true
+ }
+ },
+ "else": {
+ "property_value_attributes": {
+ "visible": false
+ }
+ }
+ }
+ ]
+ }
+ ]
+ },
+ "widgets": [
+ {
+ "config": "application-properties/atlas.authentication.method.file",
+ "widget": {
+ "type": "checkbox"
+ }
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.file.filename",
+ "widget": {
+ "type": "text-field"
+ }
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap",
+ "widget": {
+ "type": "checkbox"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.url",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.userDNpattern",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.groupSearchBase",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.groupSearchFilter",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.groupRoleAttribute",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.base.dn",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.bind.dn",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.bind.password",
+ "widget":{
+ "type":"password"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.referral",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.user.searchfilter",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.default.role",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.type",
+ "widget": {
+ "type": "combo"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.ad.url",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.ad.domain",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config":"application-properties/atlas.authentication.method.ldap.ad.base.dn",
+ "widget":{
+ "type":"text-field"
+ }
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.bind.dn",
+ "widget": {
+ "type": "text-field"
+ }
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.bind.password",
+ "widget": {
+ "type": "password"
+ }
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.referral",
+ "widget": {
+ "type": "text-field"
+ }
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.user.searchfilter",
+ "widget": {
+ "type": "text-field"
+ }
+ },
+ {
+ "config": "application-properties/atlas.authentication.method.ldap.ad.default.role",
+ "widget": {
+ "type": "text-field"
+ }
+ },
+ {
+ "config": "application-properties/atlas.sso.enabled",
+ "widget": {
+ "type":"checkbox"
+ }
+ },
+ {
+ "config": "application-properties/atlas.sso.providerurl",
+ "widget": {
+ "type": "text-field"
+ }
+ },
+ {
+ "config": "application-properties/atlas.sso.publicKey",
+ "widget": {
+ "type": "text-field"
+ }
+ },
+ {
+ "config": "application-properties/atlas.sso.browser.useragent",
+ "widget": {
+ "type": "text-field"
+ }
+ }
+ ]
+ }
+}
http://git-wip-us.apache.org/repos/asf/ambari/blob/5454bd61/ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py b/ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
index 49dd086..718da2c 100644
--- a/ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
+++ b/ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
@@ -22,185 +22,202 @@ from resource_management.libraries.functions import format
class HDP26StackAdvisor(HDP25StackAdvisor):
- def __init__(self):
- super(HDP26StackAdvisor, self).__init__()
- Logger.initialize_logger()
-
- def getServiceConfigurationRecommenderDict(self):
- parentRecommendConfDict = super(HDP26StackAdvisor, self).getServiceConfigurationRecommenderDict()
- childRecommendConfDict = {
- "DRUID": self.recommendDruidConfigurations
- }
- parentRecommendConfDict.update(childRecommendConfDict)
- return parentRecommendConfDict
-
- def recommendDruidConfigurations(self, configurations, clusterData, services, hosts):
-
- componentsListList = [service["components"] for service in services["services"]]
- componentsList = [item["StackServiceComponents"] for sublist in componentsListList for item in sublist]
- servicesList = [service["StackServices"]["service_name"] for service in services["services"]]
- putCommonProperty = self.putProperty(configurations, "druid-common", services)
-
- putCommonProperty('druid.zk.service.host', self.getZKHostPortString(services))
- self.recommendDruidMaxMemoryLimitConfigurations(configurations, clusterData, services, hosts)
-
- # recommending the metadata storage uri
- database_name = services['configurations']["druid-common"]["properties"]["database_name"]
- metastore_hostname = services['configurations']["druid-common"]["properties"]["metastore_hostname"]
- database_type = services['configurations']["druid-common"]["properties"]["druid.metadata.storage.type"]
- metadata_storage_port = "1527"
- mysql_extension_name = "io.druid.extensions:mysql-metadata-storage"
- mysql_module_name = "mysql-metadata-storage"
- postgres_module_name = "postgresql-metadata-storage"
- extensions_load_list = services['configurations']['druid-common']['properties']['druid.extensions.loadList']
- extensions_pull_list = services['configurations']['druid-common']['properties']['druid.extensions.pullList']
- putDruidCommonProperty = self.putProperty(configurations, "druid-common", services)
-
- extensions_pull_list = self.removeFromList(extensions_pull_list, mysql_extension_name)
- extensions_load_list = self.removeFromList(extensions_load_list, mysql_module_name)
- extensions_load_list = self.removeFromList(extensions_load_list, postgres_module_name)
-
- if database_type == 'mysql':
- metadata_storage_port = "3306"
- extensions_pull_list = self.addToList(extensions_pull_list, mysql_extension_name)
- extensions_load_list = self.addToList(extensions_load_list, mysql_module_name)
-
- if database_type == 'postgres':
- extensions_load_list = self.addToList(extensions_load_list, postgres_module_name)
- metadata_storage_port = "5432"
-
- putDruidCommonProperty('druid.metadata.storage.connector.port', metadata_storage_port)
- putDruidCommonProperty('druid.metadata.storage.connector.connectURI',
- self.getMetadataConnectionString(database_type).format(metastore_hostname, database_name,
- metadata_storage_port))
- # HDFS is installed
- if "HDFS" in servicesList and "hdfs-site" in services["configurations"]:
- # recommend HDFS as default deep storage
- extensions_load_list = self.addToList(extensions_load_list, "druid-hdfs-storage")
- putCommonProperty("druid.storage.type", "hdfs")
- putCommonProperty("druid.storage.storageDirectory", "/user/druid/data")
- # configure indexer logs configs
- putCommonProperty("druid.indexer.logs.type", "hdfs")
- putCommonProperty("druid.indexer.logs.directory", "/user/druid/logs")
-
- if "KAFKA" in servicesList:
- extensions_load_list = self.addToList(extensions_load_list, "druid-kafka-indexing-service")
-
- putCommonProperty('druid.extensions.loadList', extensions_load_list)
- putCommonProperty('druid.extensions.pullList', extensions_pull_list)
-
- # JVM Configs go to env properties
- putEnvProperty = self.putProperty(configurations, "druid-env", services)
-
- # processing thread pool Config
- for component in ['DRUID_HISTORICAL', 'DRUID_BROKER']:
- component_hosts = self.getHostsWithComponent("DRUID", component, services, hosts)
- nodeType = self.DRUID_COMPONENT_NODE_TYPE_MAP[component]
- putComponentProperty = self.putProperty(configurations, format("druid-{nodeType}"), services)
- if (component_hosts is not None and len(component_hosts) > 0):
- totalAvailableCpu = self.getMinCpu(component_hosts)
- processingThreads = 1
- if totalAvailableCpu > 1:
- processingThreads = totalAvailableCpu - 1
- putComponentProperty('druid.processing.numThreads', processingThreads)
- putComponentProperty('druid.server.http.numThreads', max(10, (totalAvailableCpu * 17) / 16 + 2) + 30)
-
- def getMetadataConnectionString(self, database_type):
- driverDict = {
- 'mysql': 'jdbc:mysql://{0}:{2}/{1}?createDatabaseIfNotExist=true',
- 'derby': 'jdbc:derby://{0}:{2}/{1};create=true',
- 'postgres': 'jdbc:postgresql://{0}:{2}/{1}'
- }
- return driverDict.get(database_type.lower())
-
- def addToList(self, json_list, word):
- desr_list = json.loads(json_list)
- if word not in desr_list:
- desr_list.append(word)
- return json.dumps(desr_list)
-
- def removeFromList(self, json_list, word):
- desr_list = json.loads(json_list)
- if word in desr_list:
- desr_list.remove(word)
- return json.dumps(desr_list)
-
- def recommendDruidMaxMemoryLimitConfigurations(self, configurations, clusterData, services, hosts):
- putEnvPropertyAttribute = self.putPropertyAttribute(configurations, "druid-env")
- for component in ["DRUID_HISTORICAL", "DRUID_MIDDLEMANAGER", "DRUID_BROKER", "DRUID_OVERLORD",
- "DRUID_COORDINATOR"]:
- component_hosts = self.getHostsWithComponent("DRUID", component, services, hosts)
- if component_hosts is not None and len(component_hosts) > 0:
- totalAvailableMem = self.getMinMemory(component_hosts) / 1024 # In MB
- nodeType = self.DRUID_COMPONENT_NODE_TYPE_MAP[component]
- putEnvPropertyAttribute(format('druid.{nodeType}.jvm.heap.memory'), 'maximum',
- max(totalAvailableMem, 1024))
-
- DRUID_COMPONENT_NODE_TYPE_MAP = {
- 'DRUID_BROKER': 'broker',
- 'DRUID_COORDINATOR': 'coordinator',
- 'DRUID_HISTORICAL': 'historical',
- 'DRUID_MIDDLEMANAGER': 'middlemanager',
- 'DRUID_OVERLORD': 'overlord',
- 'DRUID_ROUTER': 'router'
- }
-
- def getMinMemory(self, component_hosts):
- min_ram_kb = 1073741824 # 1 TB
- for host in component_hosts:
- ram_kb = host['Hosts']['total_mem']
- min_ram_kb = min(min_ram_kb, ram_kb)
- return min_ram_kb
-
- def getMinCpu(self, component_hosts):
- min_cpu = 256
- for host in component_hosts:
- cpu_count = host['Hosts']['cpu_count']
- min_cpu = min(min_cpu, cpu_count)
- return min_cpu
-
- def getServiceConfigurationValidators(self):
- parentValidators = super(HDP26StackAdvisor, self).getServiceConfigurationValidators()
- childValidators = {
- "DRUID": {"druid-env": self.validateDruidEnvConfigurations,
- "druid-historical": self.validateDruidHistoricalConfigurations,
- "druid-broker": self.validateDruidBrokerConfigurations}
- }
- self.mergeValidators(parentValidators, childValidators)
- return parentValidators
-
- def validateDruidEnvConfigurations(self, properties, recommendedDefaults, configurations, services, hosts):
- validationItems = []
- # Minimum Direct memory Validation
- envProperties = services['configurations']['druid-env']['properties']
- for nodeType in ['broker', 'historical']:
- properties = services['configurations'][format('druid-{nodeType}')]['properties']
- intermediateBufferSize = int(properties['druid.processing.buffer.sizeBytes']) / (1024 * 1024) # In MBs
- processingThreads = int(properties['druid.processing.numThreads'])
- directMemory = int(envProperties[format('druid.{nodeType}.jvm.direct.memory')])
- if directMemory < (processingThreads + 1) * intermediateBufferSize:
- validationItems.extend(
- {"config-name": format("druid.{nodeType}.jvm.direct.memory"), "item": self.getErrorItem(
- format(
- "Not enough direct memory available for {nodeType} Node."
- "Please adjust druid.{nodeType}.jvm.direct.memory, druid.processing.buffer.sizeBytes, druid.processing.numThreads"
- )
- )
- })
- return self.toConfigurationValidationProblems(validationItems, "druid-env")
-
- def validateDruidHistoricalConfigurations(self, properties, recommendedDefaults, configurations, services, hosts):
- validationItems = [
- {"config-name": "druid.processing.numThreads",
- "item": self.validatorEqualsToRecommendedItem(properties, recommendedDefaults,
- "druid.processing.numThreads")}
- ]
- return self.toConfigurationValidationProblems(validationItems, "druid-historical")
-
- def validateDruidBrokerConfigurations(self, properties, recommendedDefaults, configurations, services, hosts):
- validationItems = [
- {"config-name": "druid.processing.numThreads",
- "item": self.validatorEqualsToRecommendedItem(properties, recommendedDefaults,
- "druid.processing.numThreads")}
- ]
- return self.toConfigurationValidationProblems(validationItems, "druid-broker")
+ def __init__(self):
+ super(HDP26StackAdvisor, self).__init__()
+ Logger.initialize_logger()
+
+ def getServiceConfigurationRecommenderDict(self):
+ parentRecommendConfDict = super(HDP26StackAdvisor, self).getServiceConfigurationRecommenderDict()
+ childRecommendConfDict = {
+ "DRUID": self.recommendDruidConfigurations,
+ "ATLAS": self.recommendAtlasConfigurations
+ }
+ parentRecommendConfDict.update(childRecommendConfDict)
+ return parentRecommendConfDict
+
+ def recommendAtlasConfigurations(self, configurations, clusterData, services, hosts):
+ super(HDP26StackAdvisor, self).recommendAtlasConfigurations(configurations, clusterData, services, hosts)
+ servicesList = [service["StackServices"]["service_name"] for service in services["services"]]
+ putAtlasApplicationProperty = self.putProperty(configurations, "application-properties", services)
+
+ knox_host = 'localhost'
+ knox_port = '8443'
+ if 'KNOX' in servicesList:
+ knox_hosts = self.getComponentHostNames(services, "KNOX", "KNOX_GATEWAY")
+ if len(knox_hosts) > 0:
+ knox_hosts.sort()
+ knox_host = knox_hosts[0]
+ if 'gateway-site' in services['configurations'] and 'gateway.port' in services['configurations']["gateway-site"]["properties"]:
+ knox_port = services['configurations']["gateway-site"]["properties"]['gateway.port']
+ putAtlasApplicationProperty('atlas.sso.providerurl', 'https://{0}:{1}/gateway/knoxsso/api/v1/websso'.format(knox_host, knox_port))
+
+ def recommendDruidConfigurations(self, configurations, clusterData, services, hosts):
+
+ componentsListList = [service["components"] for service in services["services"]]
+ componentsList = [item["StackServiceComponents"] for sublist in componentsListList for item in sublist]
+ servicesList = [service["StackServices"]["service_name"] for service in services["services"]]
+ putCommonProperty = self.putProperty(configurations, "druid-common", services)
+
+ putCommonProperty('druid.zk.service.host', self.getZKHostPortString(services))
+ self.recommendDruidMaxMemoryLimitConfigurations(configurations, clusterData, services, hosts)
+
+ # recommending the metadata storage uri
+ database_name = services['configurations']["druid-common"]["properties"]["database_name"]
+ metastore_hostname = services['configurations']["druid-common"]["properties"]["metastore_hostname"]
+ database_type = services['configurations']["druid-common"]["properties"]["druid.metadata.storage.type"]
+ metadata_storage_port = "1527"
+ mysql_extension_name = "io.druid.extensions:mysql-metadata-storage"
+ mysql_module_name = "mysql-metadata-storage"
+ postgres_module_name = "postgresql-metadata-storage"
+ extensions_load_list = services['configurations']['druid-common']['properties']['druid.extensions.loadList']
+ extensions_pull_list = services['configurations']['druid-common']['properties']['druid.extensions.pullList']
+ putDruidCommonProperty = self.putProperty(configurations, "druid-common", services)
+
+ extensions_pull_list = self.removeFromList(extensions_pull_list, mysql_extension_name)
+ extensions_load_list = self.removeFromList(extensions_load_list, mysql_module_name)
+ extensions_load_list = self.removeFromList(extensions_load_list, postgres_module_name)
+
+ if database_type == 'mysql':
+ metadata_storage_port = "3306"
+ extensions_pull_list = self.addToList(extensions_pull_list, mysql_extension_name)
+ extensions_load_list = self.addToList(extensions_load_list, mysql_module_name)
+
+ if database_type == 'postgres':
+ extensions_load_list = self.addToList(extensions_load_list, postgres_module_name)
+ metadata_storage_port = "5432"
+
+ putDruidCommonProperty('druid.metadata.storage.connector.port', metadata_storage_port)
+ putDruidCommonProperty('druid.metadata.storage.connector.connectURI',
+ self.getMetadataConnectionString(database_type).format(metastore_hostname, database_name,
+ metadata_storage_port))
+ # HDFS is installed
+ if "HDFS" in servicesList and "hdfs-site" in services["configurations"]:
+ # recommend HDFS as default deep storage
+ extensions_load_list = self.addToList(extensions_load_list, "druid-hdfs-storage")
+ putCommonProperty("druid.storage.type", "hdfs")
+ putCommonProperty("druid.storage.storageDirectory", "/user/druid/data")
+ # configure indexer logs configs
+ putCommonProperty("druid.indexer.logs.type", "hdfs")
+ putCommonProperty("druid.indexer.logs.directory", "/user/druid/logs")
+
+ if "KAFKA" in servicesList:
+ extensions_load_list = self.addToList(extensions_load_list, "druid-kafka-indexing-service")
+
+ putCommonProperty('druid.extensions.loadList', extensions_load_list)
+ putCommonProperty('druid.extensions.pullList', extensions_pull_list)
+
+ # JVM Configs go to env properties
+ putEnvProperty = self.putProperty(configurations, "druid-env", services)
+
+ # processing thread pool Config
+ for component in ['DRUID_HISTORICAL', 'DRUID_BROKER']:
+ component_hosts = self.getHostsWithComponent("DRUID", component, services, hosts)
+ nodeType = self.DRUID_COMPONENT_NODE_TYPE_MAP[component]
+ putComponentProperty = self.putProperty(configurations, format("druid-{nodeType}"), services)
+ if (component_hosts is not None and len(component_hosts) > 0):
+ totalAvailableCpu = self.getMinCpu(component_hosts)
+ processingThreads = 1
+ if totalAvailableCpu > 1:
+ processingThreads = totalAvailableCpu - 1
+ putComponentProperty('druid.processing.numThreads', processingThreads)
+ putComponentProperty('druid.server.http.numThreads', max(10, (totalAvailableCpu * 17) / 16 + 2) + 30)
+
+ def getMetadataConnectionString(self, database_type):
+ driverDict = {
+ 'mysql': 'jdbc:mysql://{0}:{2}/{1}?createDatabaseIfNotExist=true',
+ 'derby': 'jdbc:derby://{0}:{2}/{1};create=true',
+ 'postgres': 'jdbc:postgresql://{0}:{2}/{1}'
+ }
+ return driverDict.get(database_type.lower())
+
+ def addToList(self, json_list, word):
+ desr_list = json.loads(json_list)
+ if word not in desr_list:
+ desr_list.append(word)
+ return json.dumps(desr_list)
+
+ def removeFromList(self, json_list, word):
+ desr_list = json.loads(json_list)
+ if word in desr_list:
+ desr_list.remove(word)
+ return json.dumps(desr_list)
+
+ def recommendDruidMaxMemoryLimitConfigurations(self, configurations, clusterData, services, hosts):
+ putEnvPropertyAttribute = self.putPropertyAttribute(configurations, "druid-env")
+ for component in ["DRUID_HISTORICAL", "DRUID_MIDDLEMANAGER", "DRUID_BROKER", "DRUID_OVERLORD",
+ "DRUID_COORDINATOR"]:
+ component_hosts = self.getHostsWithComponent("DRUID", component, services, hosts)
+ if component_hosts is not None and len(component_hosts) > 0:
+ totalAvailableMem = self.getMinMemory(component_hosts) / 1024 # In MB
+ nodeType = self.DRUID_COMPONENT_NODE_TYPE_MAP[component]
+ putEnvPropertyAttribute(format('druid.{nodeType}.jvm.heap.memory'), 'maximum',
+ max(totalAvailableMem, 1024))
+
+ DRUID_COMPONENT_NODE_TYPE_MAP = {
+ 'DRUID_BROKER': 'broker',
+ 'DRUID_COORDINATOR': 'coordinator',
+ 'DRUID_HISTORICAL': 'historical',
+ 'DRUID_MIDDLEMANAGER': 'middlemanager',
+ 'DRUID_OVERLORD': 'overlord',
+ 'DRUID_ROUTER': 'router'
+ }
+
+ def getMinMemory(self, component_hosts):
+ min_ram_kb = 1073741824 # 1 TB
+ for host in component_hosts:
+ ram_kb = host['Hosts']['total_mem']
+ min_ram_kb = min(min_ram_kb, ram_kb)
+ return min_ram_kb
+
+ def getMinCpu(self, component_hosts):
+ min_cpu = 256
+ for host in component_hosts:
+ cpu_count = host['Hosts']['cpu_count']
+ min_cpu = min(min_cpu, cpu_count)
+ return min_cpu
+
+ def getServiceConfigurationValidators(self):
+ parentValidators = super(HDP26StackAdvisor, self).getServiceConfigurationValidators()
+ childValidators = {
+ "DRUID": {"druid-env": self.validateDruidEnvConfigurations,
+ "druid-historical": self.validateDruidHistoricalConfigurations,
+ "druid-broker": self.validateDruidBrokerConfigurations}
+ }
+ self.mergeValidators(parentValidators, childValidators)
+ return parentValidators
+
+ def validateDruidEnvConfigurations(self, properties, recommendedDefaults, configurations, services, hosts):
+ validationItems = []
+ # Minimum Direct memory Validation
+ envProperties = services['configurations']['druid-env']['properties']
+ for nodeType in ['broker', 'historical']:
+ properties = services['configurations'][format('druid-{nodeType}')]['properties']
+ intermediateBufferSize = int(properties['druid.processing.buffer.sizeBytes']) / (1024 * 1024) # In MBs
+ processingThreads = int(properties['druid.processing.numThreads'])
+ directMemory = int(envProperties[format('druid.{nodeType}.jvm.direct.memory')])
+ if directMemory < (processingThreads + 1) * intermediateBufferSize:
+ validationItems.extend(
+ {"config-name": format("druid.{nodeType}.jvm.direct.memory"), "item": self.getErrorItem(
+ format(
+ "Not enough direct memory available for {nodeType} Node."
+ "Please adjust druid.{nodeType}.jvm.direct.memory, druid.processing.buffer.sizeBytes, druid.processing.numThreads"
+ )
+ )
+ })
+ return self.toConfigurationValidationProblems(validationItems, "druid-env")
+
+ def validateDruidHistoricalConfigurations(self, properties, recommendedDefaults, configurations, services, hosts):
+ validationItems = [
+ {"config-name": "druid.processing.numThreads",
+ "item": self.validatorEqualsToRecommendedItem(properties, recommendedDefaults,
+ "druid.processing.numThreads")}
+ ]
+ return self.toConfigurationValidationProblems(validationItems, "druid-historical")
+
+ def validateDruidBrokerConfigurations(self, properties, recommendedDefaults, configurations, services, hosts):
+ validationItems = [
+ {"config-name": "druid.processing.numThreads",
+ "item": self.validatorEqualsToRecommendedItem(properties, recommendedDefaults,
+ "druid.processing.numThreads")}
+ ]
+ return self.toConfigurationValidationProblems(validationItems, "druid-broker")
http://git-wip-us.apache.org/repos/asf/ambari/blob/5454bd61/ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py b/ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py
index 8227d69..a7a45f8 100644
--- a/ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py
+++ b/ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py
@@ -568,6 +568,103 @@ class TestHDP26StackAdvisor(TestCase):
)
+ def test_recommendAtlasConfigurations(self):
+ configurations = {
+ "application-properties": {
+ "properties": {
+ "atlas.sso.providerurl": "",
+ "atlas.graph.index.search.solr.zookeeper-url": "",
+ "atlas.audit.hbase.zookeeper.quorum": "",
+ "atlas.graph.storage.hostname": "",
+ "atlas.kafka.bootstrap.servers": "",
+ "atlas.kafka.zookeeper.connect": "",
+ "atlas.authorizer.impl": "simple"
+ }
+ },
+ "infra-solr-env": {
+ "properties": {
+ "infra_solr_znode": "/infra-solr"
+ }
+ },
+ "ranger-atlas-plugin-properties": {
+ "properties": {
+ "ranger-atlas-plugin-enabled":"No"
+ }
+ },
+ "atlas-env": {
+ "properties": {
+ "atlas_server_max_new_size": "600",
+ "atlas_server_xmx": "2048"
+ }
+ }
+ }
+
+ clusterData = {}
+
+ expected = {
+ "application-properties": {
+ "properties": {
+ "atlas.sso.providerurl": "https://c6401.ambari.apache.org:8443/gateway/knoxsso/api/v1/websso",
+ "atlas.graph.index.search.solr.zookeeper-url": "",
+ "atlas.audit.hbase.zookeeper.quorum": "",
+ "atlas.graph.storage.hostname": "",
+ "atlas.kafka.bootstrap.servers": "",
+ "atlas.kafka.zookeeper.connect": "",
+ "atlas.authorizer.impl": "simple"
+ }
+ },
+ "infra-solr-env": {
+ "properties": {
+ "infra_solr_znode": "/infra-solr"
+ }
+ },
+ "ranger-atlas-plugin-properties": {
+ "properties": {
+ "ranger-atlas-plugin-enabled":"No"
+ }
+ },
+ "atlas-env": {
+ "properties": {
+ "atlas_server_max_new_size": "600",
+ "atlas_server_xmx": "2048"
+ }
+ }
+ }
+
+ services = {
+ "services": [
+ {
+ "href": "/api/v1/stacks/HDP/versions/2.6/services/KNOX",
+ "StackServices": {
+ "service_name": "KNOX",
+ "service_version": "0.9.0.2.5",
+ "stack_name": "HDP",
+ "stack_version": "2.6"
+ },
+ "components": [
+ {
+ "href": "/api/v1/stacks/HDP/versions/2.6/services/KNOX/components/KNOX_GATEWAY",
+ "StackServiceComponents": {
+ "advertise_version": "false",
+ "cardinality": "1+",
+ "component_category": "MASTER",
+ "component_name": "KNOX_GATEWAY",
+ "display_name": "Knox Gateway",
+ "is_client": "false",
+ "is_master": "true",
+ "hostnames": ["c6401.ambari.apache.org"]
+ },
+ "dependencies": []
+ }
+ ]
+ }
+ ],
+ "configurations": configurations
+ }
+
+ self.stackAdvisor.recommendAtlasConfigurations(configurations, clusterData, services, None)
+ self.assertEquals(configurations, expected)
+
def load_json(self, filename):
file = os.path.join(self.testDirectory, filename)
with open(file, 'rb') as f:
[06/31] ambari git commit: AMBARI-18851 : Collector host refresh in
Ambari Server is not working in single Metrics collector deployments.
(avijayan)
Posted by nc...@apache.org.
AMBARI-18851 : Collector host refresh in Ambari Server is not working in single Metrics collector deployments. (avijayan)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/ce460602
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/ce460602
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/ce460602
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: ce46060294b3ad33e3e32153277ca42c71aacc1a
Parents: 45adfdf
Author: Aravindan Vijayan <av...@hortonworks.com>
Authored: Mon Nov 14 14:49:29 2016 -0800
Committer: Aravindan Vijayan <av...@hortonworks.com>
Committed: Mon Nov 14 14:49:29 2016 -0800
----------------------------------------------------------------------
.../metrics/MetricsCollectorHAClusterState.java | 10 ++++++++++
1 file changed, 10 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/ce460602/ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsCollectorHAClusterState.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsCollectorHAClusterState.java b/ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsCollectorHAClusterState.java
index 05b4e05..2e6b091 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsCollectorHAClusterState.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsCollectorHAClusterState.java
@@ -168,6 +168,16 @@ public class MetricsCollectorHAClusterState {
return true;
}
}
+
+ //If no host is alive, check if some dead collectors have become live.
+ testAndAddDeadCollectorsToLiveList();
+
+ //try one more time
+ for (String host : liveCollectorHosts) {
+ if (HostStatusHelper.isHostLive(managementController, clusterName, host)) {
+ return true;
+ }
+ }
return false;
}
[19/31] ambari git commit: AMBARI-18852 HostCleanup.py to be able to
stop processes owned by users or java processes (dili)
Posted by nc...@apache.org.
AMBARI-18852 HostCleanup.py to be able to stop processes owned by users or java processes (dili)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/0e8474fe
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/0e8474fe
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/0e8474fe
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 0e8474fea1de0ed113b50bb89a18e8534f991db5
Parents: b413d92
Author: Di Li <di...@apache.org>
Authored: Tue Nov 15 16:47:00 2016 -0500
Committer: Di Li <di...@apache.org>
Committed: Tue Nov 15 16:47:00 2016 -0500
----------------------------------------------------------------------
.../src/main/python/ambari_agent/HostCleanup.py | 87 ++++++++++++++++++--
.../test/python/ambari_agent/TestHostCleanup.py | 18 ++--
2 files changed, 93 insertions(+), 12 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/0e8474fe/ambari-agent/src/main/python/ambari_agent/HostCleanup.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/HostCleanup.py b/ambari-agent/src/main/python/ambari_agent/HostCleanup.py
index cca79a8..87cd1b0 100644
--- a/ambari-agent/src/main/python/ambari_agent/HostCleanup.py
+++ b/ambari-agent/src/main/python/ambari_agent/HostCleanup.py
@@ -36,6 +36,8 @@ import shlex
import datetime
import tempfile
import glob
+import pwd
+import re
from AmbariConfig import AmbariConfig
from ambari_agent.Constants import AGENT_TMP_DIR
from ambari_commons import OSCheck, OSConst
@@ -76,6 +78,8 @@ CACHE_FILES_PATTERN = {
}
PROCESS_SECTION = "processes"
PROCESS_KEY = "proc_list"
+PROCESS_OWNER_KEY = "proc_owner_list"
+PROCESS_IDENTIFIER_KEY = "proc_identifier"
ALT_SECTION = "alternatives"
ALT_KEYS = ["symlink_list", "target_list"]
HADOOP_GROUP = "hadoop"
@@ -88,6 +92,7 @@ DIRNAME_PATTERNS = [
# resources that should not be cleaned
REPOSITORY_BLACK_LIST = ["ambari.repo"]
PACKAGES_BLACK_LIST = ["ambari-server", "ambari-agent"]
+USER_BLACK_LIST = ["root"]
def get_erase_cmd():
if OSCheck.is_redhat_family():
@@ -135,7 +140,10 @@ class HostCleanup:
homeDirList = argMap.get(USER_HOMEDIR_SECTION)
dirList = argMap.get(DIR_SECTION)
repoList = argMap.get(REPO_SECTION)
- procList = argMap.get(PROCESS_SECTION)
+ proc_map = argMap.get(PROCESS_SECTION)
+ procList = proc_map.get(PROCESS_KEY)
+ procUserList = proc_map.get(PROCESS_OWNER_KEY)
+ procIdentifierList = proc_map.get(PROCESS_IDENTIFIER_KEY)
alt_map = argMap.get(ALT_SECTION)
additionalDirList = self.get_additional_dirs()
@@ -144,6 +152,12 @@ class HostCleanup:
if procList and not PROCESS_SECTION in SKIP_LIST:
logger.info("\n" + "Killing pid's: " + str(procList) + "\n")
self.do_kill_processes(procList)
+ if procIdentifierList and not PROCESS_SECTION in SKIP_LIST:
+ self.do_kill_processes_by_identifier(procIdentifierList)
+ if procUserList and not PROCESS_SECTION in SKIP_LIST:
+ logger.info("\n" + "Killing pids owned by: " + str(procUserList) + "\n")
+ self.do_kill_processes_by_users(procUserList)
+
if packageList and not PACKAGE_SECTION in SKIP_LIST:
logger.info("Deleting packages: " + str(packageList) + "\n")
self.do_erase_packages(packageList)
@@ -163,7 +177,7 @@ class HostCleanup:
logger.info("\n" + "Deleting repo files: " + str(repoFiles))
self.do_erase_files_silent(repoFiles)
if alt_map and not ALT_SECTION in SKIP_LIST:
- logger.info("\n" + "Erasing alternatives:" + str(alt_map) + "\n")
+ logger.info("\n" + "Erasing alternatives: " + str(alt_map) + "\n")
self.do_erase_alternatives(alt_map)
return 0
@@ -190,11 +204,23 @@ class HostCleanup:
propertyMap[PACKAGE_SECTION] = config.get(PACKAGE_SECTION, PACKAGE_KEY).split(',')
except:
logger.warn("Cannot read package list: " + str(sys.exc_info()[0]))
+
try:
+ proc_map = {}
if config.has_option(PROCESS_SECTION, PROCESS_KEY):
- propertyMap[PROCESS_SECTION] = config.get(PROCESS_SECTION, PROCESS_KEY).split(',')
+ proc_map[PROCESS_KEY] = config.get(PROCESS_SECTION, PROCESS_KEY).split(',')
+
+ if config.has_option(PROCESS_SECTION, PROCESS_OWNER_KEY):
+ proc_map[PROCESS_OWNER_KEY] = config.get(PROCESS_SECTION, PROCESS_OWNER_KEY).split(',')
+
+ if config.has_option(PROCESS_SECTION, PROCESS_IDENTIFIER_KEY):
+ proc_map[PROCESS_IDENTIFIER_KEY] = config.get(PROCESS_SECTION, PROCESS_IDENTIFIER_KEY).split(',')
+
+ if proc_map:
+ propertyMap[PROCESS_SECTION] = proc_map
except:
- logger.warn("Cannot read process list: " + str(sys.exc_info()[0]))
+ logger.warn("Cannot read process list: " + str(sys.exc_info()))
+
try:
if config.has_option(USER_SECTION, USER_KEY):
propertyMap[USER_SECTION] = config.get(USER_SECTION, USER_KEY).split(',')
@@ -317,9 +343,60 @@ class HostCleanup:
command = PROC_KILL_CMD.format(pid)
(returncode, stdoutdata, stderrdata) = self.run_os_command(command)
if returncode != 0:
- logger.error("Unable to kill process with pid: " + pid + ", " + stderrdata)
+ logger.error("Unable to kill process with pid: " + str(pid) + ", " + str(stderrdata))
return 0
+ def getProcsByUsers(self, users, pidList):
+ logger.debug("User list: "+str(users))
+ pids = [pid for pid in os.listdir('/proc') if pid.isdigit()]
+ logger.debug("All pids under /proc: "+str(pids));
+ for pid in pids:
+ logger.debug("Checking " + str(pid))
+ try:
+ with open(os.path.join('/proc', pid, 'status'), 'r') as f:
+ for line in f:
+ if line.startswith('Uid:'):
+ uid = int(line.split()[1])
+ user = pwd.getpwuid(uid).pw_name
+ logger.debug("User: "+user);
+ if user in users and user not in USER_BLACK_LIST:
+ logger.info(user + " started process " + str(pid))
+ pidList.append(int(pid))
+ except:
+ logger.debug(str(sys.exc_info()))
+
+ def do_kill_processes_by_users(self, userList):
+ pidList = []
+ self.getProcsByUsers(userList, pidList)
+ logger.info("Killing pids: "+ str(pidList) + " owned by " + str(userList))
+ return self.do_kill_processes(pidList)
+
+ def do_kill_processes_by_identifier(self, identifierList):
+ pidList = []
+ cmd = "ps aux"
+ (returncode, stdoutdata, stderrdata) = self.run_os_command(cmd, True)
+
+ if 0 == returncode and stdoutdata:
+ lines = stdoutdata.split('\n')
+ for line in lines:
+ line = line.strip()
+ for identifier in identifierList:
+ identifier = identifier.strip()
+ if identifier in line:
+ logger.debug("Found " + line + " for " + identifier);
+ line = re.sub("\s\s+" , " ", line) #replace multi spaces with single space before calling the split
+ tokens = line.split(' ')
+ logger.debug(tokens)
+ logger.debug(len(tokens))
+ if len(tokens) > 1:
+ pid = str(tokens[1]);
+ pid = pid.strip()
+ if pid and pid not in pidList:
+ logger.info("Adding pid: "+str(pid) + " for " + identifier)
+ pidList.append(pid)
+
+ return self.do_kill_processes(pidList)
+
def get_files_in_dir(self, dirPath, filemask = None):
fileList = []
if dirPath:
http://git-wip-us.apache.org/repos/asf/ambari/blob/0e8474fe/ambari-agent/src/test/python/ambari_agent/TestHostCleanup.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/test/python/ambari_agent/TestHostCleanup.py b/ambari-agent/src/test/python/ambari_agent/TestHostCleanup.py
index 165c5f0..9e1c87a 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestHostCleanup.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestHostCleanup.py
@@ -40,13 +40,16 @@ DIR_SECTION = "directories"
DIR_KEY = "dir_list"
PROCESS_SECTION = "processes"
PROCESS_KEY = "proc_list"
+PROCESS_OWNER_KEY = "proc_owner_list"
ALT_SECTION = "alternatives"
ALT_KEYS = ["symlink_list", "target_list"]
ALT_ERASE_CMD = "alternatives --remove {0} {1}"
USER_HOMEDIR_SECTION = "usr_homedir"
-hostcheck_result_fileContent = """[processes]
+hostcheck_result_fileContent = """
+[processes]
proc_list = 323,434
+proc_owner_list = abc,efg
[users]
usr_list = rrdcached,ambari-qa,hive,oozie,hbase,hcat,mysql,mapred,hdfs,zookeeper,sqoop
@@ -102,7 +105,8 @@ class TestHostCleanup(TestCase):
patch_join_mock.return_value = f2.name
propMap = self.hostcleanup.read_host_check_file(tmpfile)
- self.assertTrue('434' in propMap["processes"])
+ self.assertTrue('434' in propMap["processes"]["proc_list"])
+ self.assertTrue('abc' in propMap["processes"]["proc_owner_list"])
self.assertTrue("mysql" in propMap["users"])
self.assertTrue("HDP-epel" in propMap["repositories"])
self.assertTrue("/etc/hadoop" in propMap["directories"])
@@ -220,7 +224,7 @@ class TestHostCleanup(TestCase):
get_additional_dirs_method.return_value = ['/tmp/hadoop-yarn','/tmp/hsperfdata_007']
propertyMap = {PACKAGE_SECTION:['abcd', 'pqrst'], USER_SECTION:['abcd', 'pqrst'],
REPO_SECTION:['abcd', 'pqrst'], DIR_SECTION:['abcd', 'pqrst'],
- PROCESS_SECTION:['abcd', 'pqrst'],
+ PROCESS_SECTION:{PROCESS_KEY:['1234']},
ALT_SECTION:{ALT_KEYS[0]:['alt1','alt2'], ALT_KEYS[1]:[
'dir1']}, USER_HOMEDIR_SECTION:['decf']}
get_os_type_method.return_value = 'redhat'
@@ -239,7 +243,7 @@ class TestHostCleanup(TestCase):
do_erase_packages_method.assert_called_once_with(['abcd', 'pqrst'])
do_erase_files_silent_method.assert_called_once_with(['abcd', 'pqrst'])
do_delete_users_method.assert_called_once_with(['abcd', 'pqrst'])
- do_kill_processes_method.assert_called_once_with(['abcd', 'pqrst'])
+ do_kill_processes_method.assert_called_once_with(['1234'])
do_erase_alternatives_method.assert_called_once_with({ALT_KEYS[0]:['alt1',
'alt2'], ALT_KEYS[1]:['dir1']})
@@ -270,7 +274,7 @@ class TestHostCleanup(TestCase):
sys.stdout = out
propertyMap = {PACKAGE_SECTION:['abcd', 'pqrst'], USER_SECTION:['abcd', 'pqrst'],
REPO_SECTION:['abcd', 'pqrst'], DIR_SECTION:['abcd', 'pqrst'],
- PROCESS_SECTION:['abcd', 'pqrst'],
+ PROCESS_SECTION:{PROCESS_KEY:['abcd', 'pqrst']},
ALT_SECTION:{ALT_KEYS[0]:['alt1','alt2'], ALT_KEYS[1]:[
'dir1']}}
get_os_type_method.return_value = 'redhat'
@@ -311,7 +315,7 @@ class TestHostCleanup(TestCase):
sys.stdout = out
propertyMap = {PACKAGE_SECTION:['abcd', 'pqrst'], USER_SECTION:['abcd', 'pqrst'],
REPO_SECTION:['abcd', 'pqrst'], DIR_SECTION:['abcd', 'pqrst'],
- PROCESS_SECTION:['abcd', 'pqrst']}
+ PROCESS_SECTION:{PROCESS_KEY:['abcd', 'pqrst']}}
get_os_type_method.return_value = 'redhat'
find_repo_files_for_repos_method.return_value = ['abcd', 'pqrst']
HostCleanup.SKIP_LIST = [PACKAGE_SECTION, REPO_SECTION]
@@ -369,7 +373,7 @@ class TestHostCleanup(TestCase):
self.assertTrue(propertyMap.has_key(USER_SECTION))
self.assertTrue(propertyMap.has_key(DIR_SECTION))
self.assertTrue(propertyMap.has_key(PROCESS_SECTION))
- self.assertEquals(propertyMap[PROCESS_SECTION][0], "323")
+ self.assertEquals(propertyMap[PROCESS_SECTION][PROCESS_KEY][0], "323")
sys.stdout = sys.__stdout__
[15/31] ambari git commit: AMBARI-12263. Support PAM as
authentication mechanism for accessing Ambari UI/REST (Vishal Ghugare via
rlevas)
Posted by nc...@apache.org.
AMBARI-12263. Support PAM as authentication mechanism for accessing Ambari UI/REST (Vishal Ghugare via rlevas)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/b5a2bb8d
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/b5a2bb8d
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/b5a2bb8d
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: b5a2bb8ddbc7badcdd459b443077d429c5e8235d
Parents: 7df30b1
Author: Vishal Ghugare <gh...@us.ibm.com>
Authored: Tue Nov 15 09:19:06 2016 -0500
Committer: Robert Levas <rl...@hortonworks.com>
Committed: Tue Nov 15 09:20:22 2016 -0500
----------------------------------------------------------------------
ambari-server/pom.xml | 10 +
ambari-server/sbin/ambari-server | 4 +
.../server/configuration/Configuration.java | 23 ++
.../AmbariManagementControllerImpl.java | 7 +-
.../ambari/server/controller/AmbariServer.java | 3 +
.../ambari/server/controller/GroupResponse.java | 14 ++
.../internal/GroupResourceProvider.java | 4 +
.../internal/UserPrivilegeResourceProvider.java | 3 +
.../apache/ambari/server/orm/dao/GroupDAO.java | 19 +-
.../ambari/server/orm/dao/ResourceDAO.java | 21 ++
.../ambari/server/orm/entities/GroupEntity.java | 18 ++
.../server/security/ClientSecurityType.java | 3 +-
.../AmbariPamAuthenticationProvider.java | 252 +++++++++++++++++++
.../server/security/authorization/Group.java | 6 +
.../security/authorization/GroupType.java | 25 ++
.../PamAuthenticationException.java | 36 +++
.../server/security/authorization/UserType.java | 3 +-
.../server/security/authorization/Users.java | 54 +++-
.../server/upgrade/UpgradeCatalog250.java | 11 +
ambari-server/src/main/python/ambari-server.py | 7 +-
.../main/python/ambari_server/setupActions.py | 1 +
.../main/python/ambari_server/setupSecurity.py | 53 +++-
.../main/resources/Ambari-DDL-MySQL-CREATE.sql | 1 +
.../main/resources/Ambari-DDL-Oracle-CREATE.sql | 1 +
.../resources/Ambari-DDL-Postgres-CREATE.sql | 1 +
.../resources/Ambari-DDL-SQLAnywhere-CREATE.sql | 1 +
.../resources/Ambari-DDL-SQLServer-CREATE.sql | 1 +
.../src/main/resources/properties.json | 1 +
.../webapp/WEB-INF/spring-security.xml | 1 +
.../AmbariPamAuthenticationProviderTest.java | 97 +++++++
.../security/authorization/TestUsers.java | 10 +-
.../server/upgrade/UpgradeCatalog250Test.java | 13 +
32 files changed, 686 insertions(+), 18 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/pom.xml
----------------------------------------------------------------------
diff --git a/ambari-server/pom.xml b/ambari-server/pom.xml
index e02b7a5..36c57de 100644
--- a/ambari-server/pom.xml
+++ b/ambari-server/pom.xml
@@ -1469,6 +1469,16 @@
<version>1.0.0.0-SNAPSHOT</version>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.kohsuke</groupId>
+ <artifactId>libpam4j</artifactId>
+ <version>1.8</version>
+ </dependency>
+ <dependency>
+ <groupId>net.java.dev.jna</groupId>
+ <artifactId>jna</artifactId>
+ <version>4.1.0</version>
+ </dependency>
</dependencies>
<pluginRepositories>
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/sbin/ambari-server
----------------------------------------------------------------------
diff --git a/ambari-server/sbin/ambari-server b/ambari-server/sbin/ambari-server
index bdbdd0f..f08db13 100755
--- a/ambari-server/sbin/ambari-server
+++ b/ambari-server/sbin/ambari-server
@@ -132,6 +132,10 @@ case "${1:-}" in
echo -e "Updating jce policy"
$PYTHON "$AMBARI_PYTHON_EXECUTABLE" $@
;;
+ setup-pam)
+ echo -e "Setting up PAM properties..."
+ $PYTHON "$AMBARI_PYTHON_EXECUTABLE" $@
+ ;;
setup-ldap)
echo -e "Setting up LDAP properties..."
$PYTHON "$AMBARI_PYTHON_EXECUTABLE" $@
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java b/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
index 0b8e195..b8b8f54 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
@@ -725,6 +725,21 @@ public class Configuration {
"jce.name", null);
/**
+ * The auto group creation by Ambari.
+ */
+ @Markdown(
+ description = "The auto group creation by Ambari")
+ public static final ConfigurationProperty<Boolean> AUTO_GROUP_CREATION = new ConfigurationProperty<>(
+ "auto.group.creation", Boolean.FALSE);
+
+ /**
+ * The PAM configuration file.
+ */
+ @Markdown(description = "The PAM configuration file.")
+ public static final ConfigurationProperty<String> PAM_CONFIGURATION_FILE = new ConfigurationProperty<>(
+ "pam.configuration", null);
+
+ /**
* The type of authentication mechanism used by Ambari.
*
* @see ClientSecurityType
@@ -5747,4 +5762,12 @@ public class Configuration {
String acceptors = getProperty(SRVR_API_ACCEPTOR_THREAD_COUNT);
return StringUtils.isEmpty(acceptors) ? null : Integer.parseInt(acceptors);
}
+
+ public String getPamConfigurationFile() {
+ return getProperty(PAM_CONFIGURATION_FILE);
+ }
+
+ public String getAutoGroupCreation() {
+ return getProperty(AUTO_GROUP_CREATION);
+ }
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java b/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
index b04fdd7..8e2fe74 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
@@ -129,6 +129,7 @@ import org.apache.ambari.server.scheduler.ExecutionScheduleManager;
import org.apache.ambari.server.security.authorization.AuthorizationException;
import org.apache.ambari.server.security.authorization.AuthorizationHelper;
import org.apache.ambari.server.security.authorization.Group;
+import org.apache.ambari.server.security.authorization.GroupType;
import org.apache.ambari.server.security.authorization.ResourceType;
import org.apache.ambari.server.security.authorization.RoleAuthorization;
import org.apache.ambari.server.security.authorization.User;
@@ -973,7 +974,7 @@ public class AmbariManagementControllerImpl implements AmbariManagementControlle
if (group != null) {
throw new AmbariException("Group already exists.");
}
- users.createGroup(request.getGroupName());
+ users.createGroup(request.getGroupName(), GroupType.LOCAL);
}
}
@@ -3685,7 +3686,7 @@ public class AmbariManagementControllerImpl implements AmbariManagementControlle
// get them all
if (null == request.getGroupName()) {
for (Group group: users.getAllGroups()) {
- final GroupResponse response = new GroupResponse(group.getGroupName(), group.isLdapGroup());
+ final GroupResponse response = new GroupResponse(group.getGroupName(), group.isLdapGroup(), group.getGroupType());
responses.add(response);
}
} else {
@@ -3698,7 +3699,7 @@ public class AmbariManagementControllerImpl implements AmbariManagementControlle
+ request.getGroupName() + "'");
}
} else {
- final GroupResponse response = new GroupResponse(group.getGroupName(), group.isLdapGroup());
+ final GroupResponse response = new GroupResponse(group.getGroupName(), group.isLdapGroup(), group.getGroupType());
responses.add(response);
}
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java b/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
index e54d54e..537ebc5 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
@@ -103,6 +103,7 @@ import org.apache.ambari.server.security.authorization.AmbariLocalUserProvider;
import org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter;
import org.apache.ambari.server.security.authorization.PermissionHelper;
import org.apache.ambari.server.security.authorization.Users;
+import org.apache.ambari.server.security.authorization.AmbariPamAuthenticationProvider;
import org.apache.ambari.server.security.authorization.internal.AmbariInternalAuthenticationProvider;
import org.apache.ambari.server.security.ldap.AmbariLdapDataPopulator;
import org.apache.ambari.server.security.unsecured.rest.CertificateDownload;
@@ -339,6 +340,8 @@ public class AmbariServer {
injector.getInstance(AmbariUserAuthorizationFilter.class));
factory.registerSingleton("ambariInternalAuthenticationProvider",
injector.getInstance(AmbariInternalAuthenticationProvider.class));
+ factory.registerSingleton("ambariPamAuthenticationProvider",
+ injector.getInstance(AmbariPamAuthenticationProvider.class));
// Spring Security xml config depends on this Bean
String[] contextLocations = {SPRING_CONTEXT_LOCATION};
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java b/ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
index ef28f61..0baccc7 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
@@ -17,16 +17,26 @@
*/
package org.apache.ambari.server.controller;
+import org.apache.ambari.server.security.authorization.GroupType;
+
/**
* Represents a user group maintenance response.
*/
public class GroupResponse {
private final String groupName;
private final boolean ldapGroup;
+ private final GroupType groupType;
+
+ public GroupResponse(String groupName, boolean ldapGroup, GroupType groupType) {
+ this.groupName = groupName;
+ this.ldapGroup = ldapGroup;
+ this.groupType = groupType;
+ }
public GroupResponse(String groupName, boolean ldapGroup) {
this.groupName = groupName;
this.ldapGroup = ldapGroup;
+ this.groupType = GroupType.LOCAL;
}
public String getGroupName() {
@@ -37,6 +47,10 @@ public class GroupResponse {
return ldapGroup;
}
+ public GroupType getGroupType() {
+ return groupType;
+ }
+
@Override
public boolean equals(Object o) {
if (this == o)
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java b/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
index e1aa5ac..e07dece 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
@@ -49,6 +49,7 @@ public class GroupResourceProvider extends AbstractControllerResourceProvider {
// Groups
public static final String GROUP_GROUPNAME_PROPERTY_ID = PropertyHelper.getPropertyId("Groups", "group_name");
public static final String GROUP_LDAP_GROUP_PROPERTY_ID = PropertyHelper.getPropertyId("Groups", "ldap_group");
+ public static final String GROUP_GROUPTYPE_PROPERTY_ID = PropertyHelper.getPropertyId("Groups", "group_type");
private static Set<String> pkPropertyIds =
new HashSet<String>(Arrays.asList(new String[]{
@@ -132,6 +133,9 @@ public class GroupResourceProvider extends AbstractControllerResourceProvider {
setResourceProperty(resource, GROUP_LDAP_GROUP_PROPERTY_ID,
groupResponse.isLdapGroup(), requestedIds);
+ setResourceProperty(resource, GROUP_GROUPTYPE_PROPERTY_ID,
+ groupResponse.getGroupType(), requestedIds);
+
resources.add(resource);
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java b/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
index ba32a5f..0575c1d 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
@@ -289,6 +289,9 @@ public class UserPrivilegeResourceProvider extends ReadOnlyResourceProvider {
}
if (userEntity == null) {
+ userEntity = userDAO.findUserByNameAndType(userName, UserType.PAM);
+ }
+ if (userEntity == null) {
throw new SystemException("User " + userName + " was not found");
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/orm/dao/GroupDAO.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/orm/dao/GroupDAO.java b/ambari-server/src/main/java/org/apache/ambari/server/orm/dao/GroupDAO.java
index 255c5e6..8b5902c 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/orm/dao/GroupDAO.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/orm/dao/GroupDAO.java
@@ -29,13 +29,14 @@ import javax.persistence.TypedQuery;
import org.apache.ambari.server.orm.RequiresSession;
import org.apache.ambari.server.orm.entities.GroupEntity;
+import org.apache.ambari.server.orm.entities.PrincipalEntity;
+import org.apache.ambari.server.security.authorization.GroupType;
+
import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.Singleton;
import com.google.inject.persist.Transactional;
-import org.apache.ambari.server.orm.entities.PrincipalEntity;
-
@Singleton
public class GroupDAO {
@Inject
@@ -65,6 +66,20 @@ public class GroupDAO {
}
}
+ @RequiresSession
+ public GroupEntity findGroupByNameAndType(String groupName, GroupType groupType) {
+ // do case insensitive compare
+ TypedQuery<GroupEntity> query = entityManagerProvider.get().createQuery(
+ "SELECT group_entity FROM GroupEntity group_entity WHERE group_entity.groupType=:type AND lower(group_entity.groupName)=lower(:name)", GroupEntity.class);
+ query.setParameter("type", groupType);
+ query.setParameter("name", groupName);
+ try {
+ return query.getSingleResult();
+ } catch (NoResultException e) {
+ return null;
+ }
+ }
+
/**
* Find the group entities for the given list of principals
*
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java b/ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java
index e4ed9c6..e57f265 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java
@@ -26,6 +26,7 @@ import org.apache.ambari.server.orm.RequiresSession;
import org.apache.ambari.server.orm.entities.ResourceEntity;
import javax.persistence.EntityManager;
+import javax.persistence.NoResultException;
import javax.persistence.TypedQuery;
import java.util.List;
@@ -55,6 +56,26 @@ public class ResourceDAO {
}
/**
+ * Find a resource with the given resource type id.
+ *
+ * @param id type id
+ *
+ * @return a matching resource or null
+ */
+ @RequiresSession
+ public ResourceEntity findByResourceTypeId(Integer id) {
+ TypedQuery<ResourceEntity> query = entityManagerProvider.get().createQuery(
+ "SELECT resource FROM ResourceEntity resource WHERE resource.resourceType.id =:resourceTypeId",
+ ResourceEntity.class);
+ query.setParameter("resourceTypeId", id);
+ try {
+ return query.getSingleResult();
+ } catch (NoResultException e) {
+ return null;
+ }
+ }
+
+ /**
* Find all resources.
*
* @return all resources or an empty List
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java b/ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
index 00e233e..58b2e5d 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
@@ -19,9 +19,12 @@ package org.apache.ambari.server.orm.entities;
import java.util.Set;
+import javax.persistence.Basic;
import javax.persistence.CascadeType;
import javax.persistence.Column;
import javax.persistence.Entity;
+import javax.persistence.EnumType;
+import javax.persistence.Enumerated;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
@@ -35,6 +38,8 @@ import javax.persistence.Table;
import javax.persistence.TableGenerator;
import javax.persistence.UniqueConstraint;
+import org.apache.ambari.server.security.authorization.GroupType;
+
@Entity
@Table(name = "groups", uniqueConstraints = {@UniqueConstraint(columnNames = {"group_name", "ldap_group"})})
@TableGenerator(name = "group_id_generator",
@@ -59,6 +64,11 @@ public class GroupEntity {
@Column(name = "ldap_group")
private Integer ldapGroup = 0;
+ @Column(name = "group_type")
+ @Enumerated(EnumType.STRING)
+ @Basic
+ private GroupType groupType = GroupType.LOCAL;
+
@OneToMany(mappedBy = "group", cascade = CascadeType.ALL)
private Set<MemberEntity> memberEntities;
@@ -99,6 +109,14 @@ public class GroupEntity {
}
}
+ public GroupType getGroupType() {
+ return groupType;
+ }
+
+ public void setgroupType(GroupType groupType) {
+ this.groupType = groupType;
+ }
+
public Set<MemberEntity> getMemberEntities() {
return memberEntities;
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java b/ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
index 26d4da7..fa853a6 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
@@ -19,7 +19,8 @@ package org.apache.ambari.server.security;
public enum ClientSecurityType {
LOCAL("local"),
- LDAP("ldap");
+ LDAP("ldap"),
+ PAM("pam");
private String value;
ClientSecurityType(String value) {
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
new file mode 100644
index 0000000..ab66271
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
@@ -0,0 +1,252 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.authorization;
+
+import java.security.Principal;
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.Set;
+
+import org.apache.ambari.server.AmbariException;
+import org.apache.ambari.server.configuration.Configuration;
+import org.apache.ambari.server.orm.dao.GroupDAO;
+import org.apache.ambari.server.orm.dao.UserDAO;
+import org.apache.ambari.server.orm.entities.GroupEntity;
+import org.apache.ambari.server.orm.entities.MemberEntity;
+import org.apache.ambari.server.orm.entities.UserEntity;
+import org.apache.ambari.server.security.ClientSecurityType;
+import org.jvnet.libpam.PAM;
+import org.jvnet.libpam.PAMException;
+import org.jvnet.libpam.UnixUser;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.security.authentication.AuthenticationProvider;
+import org.springframework.security.authentication.AuthenticationServiceException;
+import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
+import org.springframework.security.core.Authentication;
+import org.springframework.security.core.AuthenticationException;
+import org.springframework.security.core.context.SecurityContextHolder;
+
+import com.google.inject.Inject;
+
+/**
+ * Provides PAM user authentication & authorization logic for Ambari Server
+ */
+
+public class AmbariPamAuthenticationProvider implements AuthenticationProvider {
+
+ @Inject
+ private Users users;
+ @Inject
+ protected UserDAO userDAO;
+ @Inject
+ protected GroupDAO groupDAO;
+
+ private static Logger LOG = LoggerFactory.getLogger(AmbariPamAuthenticationProvider.class);
+
+ private final Configuration configuration;
+
+ @Inject
+ public AmbariPamAuthenticationProvider(Configuration configuration) {
+ this.configuration = configuration;
+ }
+
+ /**
+ * Performs PAM Initialization
+ *
+ * @param authentication
+ * @return authentication
+ */
+
+ @Override
+ public Authentication authenticate(Authentication authentication) throws AuthenticationException {
+ if(isPamEnabled()){
+ PAM pam;
+ try{
+ //Set PAM configuration file (found under /etc/pam.d)
+ String pamConfig = configuration.getPamConfigurationFile();
+ pam = new PAM(pamConfig);
+
+ } catch(PAMException ex) {
+ LOG.error("Unable to Initialize PAM." + ex.getMessage());
+ throw new AuthenticationServiceException("Unable to Initialize PAM - ", ex);
+ }
+
+ return authenticateViaPam(pam, authentication);
+ } else {
+ return null;
+ }
+ }
+
+ /**
+ * Performs PAM Authentication
+ *
+ * @param pam
+ * @param authentication
+ * @return authentication
+ */
+
+ protected Authentication authenticateViaPam(PAM pam, Authentication authentication) throws AuthenticationException{
+ if(isPamEnabled()){
+ try {
+ String userName = String.valueOf(authentication.getPrincipal());
+ String passwd = String.valueOf(authentication.getCredentials());
+
+ // authenticate using PAM
+ UnixUser unixUser = pam.authenticate(userName,passwd);
+
+ //Get all the groups that user belongs to
+ //Change all group names to lower case.
+ Set<String> groups = new HashSet<String>();
+
+ for(String group: unixUser.getGroups()){
+ groups.add(group.toLowerCase());
+ }
+
+ ambariPamAuthorization(userName,groups);
+
+ Collection<AmbariGrantedAuthority> userAuthorities =
+ users.getUserAuthorities(userName, UserType.PAM);
+
+ final User user = users.getUser(userName, UserType.PAM);
+
+ Principal principal = new Principal() {
+ @Override
+ public String getName() {
+ return user.getUserName();
+ }
+ };
+
+ UsernamePasswordAuthenticationToken token = new UsernamePasswordAuthenticationToken(principal, null, userAuthorities);
+ SecurityContextHolder.getContext().setAuthentication(token);
+ return token;
+
+ } catch (PAMException ex) {
+ LOG.error("Unable to sign in. Invalid username/password combination - " + ex.getMessage());
+ Throwable t = ex.getCause();
+ throw new PamAuthenticationException("Unable to sign in. Invalid username/password combination.",t);
+
+ } finally {
+ pam.dispose();
+ }
+
+ }
+ else {
+ return null;
+ }
+ }
+
+ @Override
+ public boolean supports(Class<?> authentication) {
+ return UsernamePasswordAuthenticationToken.class.isAssignableFrom(authentication);
+ }
+
+ /**
+ * Check if PAM authentication is enabled in server properties
+ * @return true if enabled
+ */
+ private boolean isPamEnabled() {
+ return configuration.getClientSecurityType() == ClientSecurityType.PAM;
+ }
+
+ /**
+ * Check if PAM authentication is enabled in server properties
+ * @return true if enabled
+ */
+ private boolean isAutoGroupCreationAllowed() {
+ return configuration.getAutoGroupCreation().equals("true");
+ }
+
+
+ /**
+ * Performs PAM authorization by creating user & group(s)
+ *
+ * @param userName user name
+ * @param userGroups Collection of groups
+ * @return
+ */
+ private void ambariPamAuthorization(String userName,Set<String> userGroups){
+ try {
+ User existingUser = users.getUser(userName,UserType.PAM);
+
+ if (existingUser == null ) {
+ users.createUser(userName, null, UserType.PAM, true, false);
+ }
+
+ UserEntity userEntity = userDAO.findUserByNameAndType(userName, UserType.PAM);
+
+ if(isAutoGroupCreationAllowed()){
+ for(String userGroup: userGroups){
+ if(users.getGroupByNameAndType(userGroup, GroupType.PAM) == null){
+ users.createGroup(userGroup, GroupType.PAM);
+ }
+
+ final GroupEntity groupEntity = groupDAO.findGroupByNameAndType(userGroup, GroupType.PAM);
+
+ if (!isUserInGroup(userEntity, groupEntity)){
+ users.addMemberToGroup(userGroup,userName);
+ }
+ }
+
+ Set<String> ambariUserGroups = getUserGroups(userName, UserType.PAM);
+
+ for(String group: ambariUserGroups){
+ if(userGroups == null || !userGroups.contains(group)){
+ users.removeMemberFromGroup(group, userName);
+ }
+ }
+ }
+
+ } catch (AmbariException e) {
+ e.printStackTrace();
+ }
+ }
+
+ /**
+ * Performs a check if given user belongs to given group.
+ *
+ * @param userEntity user entity
+ * @param groupEntity group entity
+ * @return true if user presents in group
+ */
+ private boolean isUserInGroup(UserEntity userEntity, GroupEntity groupEntity) {
+ for (MemberEntity memberEntity: userEntity.getMemberEntities()) {
+ if (memberEntity.getGroup().equals(groupEntity)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ /**
+ * Extracts all groups a user belongs to
+ *
+ * @param userName user name
+ * @return Collection of group names
+ */
+ private Set<String> getUserGroups(String userName, UserType userType) {
+ UserEntity userEntity = userDAO.findUserByNameAndType(userName, userType);
+ Set<String> groups = new HashSet<String>();
+ for (MemberEntity memberEntity: userEntity.getMemberEntities()) {
+ groups.add(memberEntity.getGroup().getGroupName());
+ }
+
+ return groups;
+ }
+
+}
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
index b20df8d..715c41c 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
@@ -26,11 +26,13 @@ public class Group {
private final int groupId;
private final String groupName;
private final boolean ldapGroup;
+ private final GroupType groupType;
Group(GroupEntity groupEntity) {
this.groupId = groupEntity.getGroupId();
this.groupName = groupEntity.getGroupName();
this.ldapGroup = groupEntity.getLdapGroup();
+ this.groupType = groupEntity.getGroupType();
}
public int getGroupId() {
@@ -45,6 +47,10 @@ public class Group {
return ldapGroup;
}
+ public GroupType getGroupType() {
+ return groupType;
+ }
+
@Override
public String toString() {
return "Group [groupId=" + groupId + ", groupName=" + groupName
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java
new file mode 100644
index 0000000..d427f3a
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java
@@ -0,0 +1,25 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.authorization;
+
+public enum GroupType {
+ LOCAL,
+ LDAP,
+ JWT,
+ PAM
+}
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java
new file mode 100644
index 0000000..6c09a67
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java
@@ -0,0 +1,36 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.authorization;
+
+import org.springframework.security.core.AuthenticationException;
+
+public class PamAuthenticationException extends AuthenticationException{
+
+ public PamAuthenticationException() {
+ this("The user authentication failed");
+ }
+
+ public PamAuthenticationException(String msg, Throwable t) {
+ super(msg, t);
+ }
+
+ public PamAuthenticationException(String msg) {
+ super(msg);
+ }
+
+}
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
index aa9f3e0..e60d58e 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
@@ -20,5 +20,6 @@ package org.apache.ambari.server.security.authorization;
public enum UserType {
LOCAL,
LDAP,
- JWT
+ JWT,
+ PAM
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
index 8ac7ebb..2cd538c 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
@@ -41,6 +41,7 @@ import org.apache.ambari.server.orm.dao.PrincipalDAO;
import org.apache.ambari.server.orm.dao.PrincipalTypeDAO;
import org.apache.ambari.server.orm.dao.PrivilegeDAO;
import org.apache.ambari.server.orm.dao.ResourceDAO;
+import org.apache.ambari.server.orm.dao.ResourceTypeDAO;
import org.apache.ambari.server.orm.dao.UserDAO;
import org.apache.ambari.server.orm.entities.GroupEntity;
import org.apache.ambari.server.orm.entities.MemberEntity;
@@ -48,7 +49,10 @@ import org.apache.ambari.server.orm.entities.PermissionEntity;
import org.apache.ambari.server.orm.entities.PrincipalEntity;
import org.apache.ambari.server.orm.entities.PrincipalTypeEntity;
import org.apache.ambari.server.orm.entities.PrivilegeEntity;
+import org.apache.ambari.server.orm.entities.ResourceEntity;
+import org.apache.ambari.server.orm.entities.ResourceTypeEntity;
import org.apache.ambari.server.orm.entities.UserEntity;
+import org.apache.ambari.server.security.ClientSecurityType;
import org.apache.ambari.server.security.ldap.LdapBatchDto;
import org.apache.ambari.server.security.ldap.LdapUserGroupMemberDto;
import org.apache.commons.lang.StringUtils;
@@ -88,6 +92,8 @@ public class Users {
@Inject
protected ResourceDAO resourceDAO;
@Inject
+ protected ResourceTypeDAO resourceTypeDAO;
+ @Inject
protected PrincipalTypeDAO principalTypeDAO;
@Inject
protected PasswordEncoder passwordEncoder;
@@ -127,6 +133,11 @@ public class Users {
if (userEntity == null) {
userEntity = userDAO.findUserByNameAndType(userName, UserType.JWT);
}
+
+ if (userEntity == null) {
+ userEntity = userDAO.findUserByNameAndType(userName, UserType.PAM);
+ }
+
return (null == userEntity) ? null : new User(userEntity);
}
@@ -369,6 +380,18 @@ public class Users {
}
/**
+ * Gets group by given name & type.
+ *
+ * @param groupName group name
+ * @param groupType group type
+ * @return group
+ */
+ public Group getGroupByNameAndType(String groupName, GroupType groupType) {
+ final GroupEntity groupEntity = groupDAO.findGroupByNameAndType(groupName, groupType);
+ return (null == groupEntity) ? null : new Group(groupEntity);
+ }
+
+ /**
* Gets group members.
*
* @param groupName group name
@@ -393,10 +416,10 @@ public class Users {
}
/**
- * Creates new local group with provided name
+ * Creates new group with provided name & type
*/
@Transactional
- public synchronized void createGroup(String groupName) {
+ public synchronized void createGroup(String groupName, GroupType groupType) {
// create an admin principal to represent this group
PrincipalTypeEntity principalTypeEntity = principalTypeDAO.findById(PrincipalTypeEntity.GROUP_PRINCIPAL_TYPE);
if (principalTypeEntity == null) {
@@ -412,6 +435,7 @@ public class Users {
final GroupEntity groupEntity = new GroupEntity();
groupEntity.setGroupName(groupName);
groupEntity.setPrincipal(principalEntity);
+ groupEntity.setgroupType(groupType);
groupDAO.create(groupEntity);
}
@@ -480,6 +504,32 @@ public class Users {
}
/**
+ * Grants privilege to provided group.
+ *
+ * @param groupId group id
+ * @param resourceId resource id
+ * @param resourceType resource type
+ * @param permissionName permission name
+ */
+ public synchronized void grantPrivilegeToGroup(Integer groupId, Long resourceId, ResourceType resourceType, String permissionName) {
+ final GroupEntity group = groupDAO.findByPK(groupId);
+ final PrivilegeEntity privilege = new PrivilegeEntity();
+ ResourceTypeEntity resourceTypeEntity = new ResourceTypeEntity();
+ resourceTypeEntity.setId(resourceType.getId());
+ resourceTypeEntity.setName(resourceType.name());
+ privilege.setPermission(permissionDAO.findPermissionByNameAndType(permissionName,resourceTypeEntity));
+ privilege.setPrincipal(group.getPrincipal());
+ privilege.setResource(resourceDAO.findById(resourceId));
+ if (!group.getPrincipal().getPrivileges().contains(privilege)) {
+ privilegeDAO.create(privilege);
+ group.getPrincipal().getPrivileges().add(privilege);
+ principalDAO.merge(group.getPrincipal()); //explicit merge for Derby support
+ groupDAO.merge(group);
+ privilegeDAO.merge(privilege);
+ }
+ }
+
+ /**
* Revokes AMBARI.ADMINISTRATOR privilege from provided user.
*
* @param userId user id
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
index 3425dd7..e81568c 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
@@ -46,6 +46,8 @@ import com.google.inject.Injector;
public class UpgradeCatalog250 extends AbstractUpgradeCatalog {
protected static final String HOST_VERSION_TABLE = "host_version";
+ protected static final String GROUPS_TABLE = "groups";
+ protected static final String GROUP_TYPE_COL = "group_type";
private static final String AMS_ENV = "ams-env";
private static final String KAFKA_BROKER = "kafka-broker";
private static final String KAFKA_TIMELINE_METRICS_HOST = "kafka.timeline.metrics.host";
@@ -109,6 +111,7 @@ public class UpgradeCatalog250 extends AbstractUpgradeCatalog {
protected void executeDDLUpdates() throws AmbariException, SQLException {
updateHostVersionTable();
createComponentVersionTable();
+ updateGroupsTable();
dbAccessor.addColumn("stage",
new DBAccessor.DBColumnInfo("command_execution_type", String.class, 32, CommandExecutionType.STAGE.toString(),
false));
@@ -140,6 +143,14 @@ public class UpgradeCatalog250 extends AbstractUpgradeCatalog {
dbAccessor.addUniqueConstraint(HOST_VERSION_TABLE, "UQ_host_repo", "repo_version_id", "host_id");
}
+ protected void updateGroupsTable() throws SQLException {
+ LOG.info("Updating the {} table", GROUPS_TABLE);
+
+ dbAccessor.addColumn(GROUPS_TABLE, new DBColumnInfo(GROUP_TYPE_COL, String.class, null, "LOCAL", false));
+ dbAccessor.executeQuery("UPDATE groups SET group_type='LDAP' WHERE ldap_group=1");
+ dbAccessor.addUniqueConstraint(GROUPS_TABLE, "UNQ_groups_0", "group_name", "group_type");
+ }
+
protected void updateAMSConfigs() throws AmbariException {
AmbariManagementController ambariManagementController = injector.getInstance(AmbariManagementController.class);
Clusters clusters = ambariManagementController.getClusters();
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/python/ambari-server.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari-server.py b/ambari-server/src/main/python/ambari-server.py
index d6c6c10..d43e0f2 100755
--- a/ambari-server/src/main/python/ambari-server.py
+++ b/ambari-server/src/main/python/ambari-server.py
@@ -52,8 +52,8 @@ from ambari_server.setupActions import BACKUP_ACTION, LDAP_SETUP_ACTION, LDAP_SY
SETUP_ACTION, SETUP_SECURITY_ACTION,START_ACTION, STATUS_ACTION, STOP_ACTION, RESTART_ACTION, UPGRADE_ACTION, \
UPGRADE_STACK_ACTION, SETUP_JCE_ACTION, SET_CURRENT_ACTION, START_ACTION, STATUS_ACTION, STOP_ACTION, UPGRADE_ACTION, \
UPGRADE_STACK_ACTION, SETUP_JCE_ACTION, SET_CURRENT_ACTION, ENABLE_STACK_ACTION, SETUP_SSO_ACTION, \
- DB_CLEANUP_ACTION, INSTALL_MPACK_ACTION, UPGRADE_MPACK_ACTION
-from ambari_server.setupSecurity import setup_ldap, sync_ldap, setup_master_key, setup_ambari_krb5_jaas
+ DB_CLEANUP_ACTION, INSTALL_MPACK_ACTION, UPGRADE_MPACK_ACTION, PAM_SETUP_ACTION
+from ambari_server.setupSecurity import setup_ldap, sync_ldap, setup_master_key, setup_ambari_krb5_jaas, setup_pam
from ambari_server.userInput import get_validated_string_input
from ambari_server_main import server_process_main
@@ -651,7 +651,8 @@ def create_user_action_map(args, options):
SETUP_SSO_ACTION: UserActionRestart(setup_sso, options),
DB_CLEANUP_ACTION: UserAction(db_cleanup, options),
INSTALL_MPACK_ACTION: UserAction(install_mpack, options),
- UPGRADE_MPACK_ACTION: UserAction(upgrade_mpack, options)
+ UPGRADE_MPACK_ACTION: UserAction(upgrade_mpack, options),
+ PAM_SETUP_ACTION: UserAction(setup_pam)
}
return action_map
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/python/ambari_server/setupActions.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/setupActions.py b/ambari-server/src/main/python/ambari_server/setupActions.py
index 697bc1d..c87e0b2 100644
--- a/ambari-server/src/main/python/ambari_server/setupActions.py
+++ b/ambari-server/src/main/python/ambari_server/setupActions.py
@@ -46,3 +46,4 @@ ENABLE_STACK_ACTION = "enable-stack"
DB_CLEANUP_ACTION = "db-cleanup"
INSTALL_MPACK_ACTION = "install-mpack"
UPGRADE_MPACK_ACTION = "upgrade-mpack"
+PAM_SETUP_ACTION = "setup-pam"
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/python/ambari_server/setupSecurity.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/setupSecurity.py b/ambari-server/src/main/python/ambari_server/setupSecurity.py
index ef27ced..1508d27 100644
--- a/ambari-server/src/main/python/ambari_server/setupSecurity.py
+++ b/ambari-server/src/main/python/ambari_server/setupSecurity.py
@@ -67,8 +67,12 @@ REGEX_ANYTHING = ".*"
CLIENT_SECURITY_KEY = "client.security"
+AUTO_GROUP_CREATION = "auto.group.creation"
+
SERVER_API_LDAP_URL = 'ldap_sync_events'
+PAM_CONFIG_FILE = 'pam.configuration'
+
def read_master_key(isReset=False, options = None):
passwordPattern = ".*"
@@ -271,12 +275,17 @@ def sync_ldap(options):
'root-level privileges'
raise FatalException(4, err)
+ properties = get_ambari_properties()
+
+ if get_value_from_properties(properties,CLIENT_SECURITY_KEY,"") == 'pam':
+ err = "PAM is configured. Can not sync LDAP."
+ raise FatalException(1, err)
+
server_status, pid = is_server_runing()
if not server_status:
err = 'Ambari Server is not running.'
raise FatalException(1, err)
- properties = get_ambari_properties()
if properties == -1:
raise FatalException(1, "Failed to read properties file.")
@@ -614,6 +623,11 @@ def setup_ldap(options):
raise FatalException(4, err)
properties = get_ambari_properties()
+
+ if get_value_from_properties(properties,CLIENT_SECURITY_KEY,"") == 'pam':
+ err = "PAM is configured. Can not setup LDAP."
+ raise FatalException(1, err)
+
isSecure = get_is_secure(properties)
ldap_property_list_reqd = init_ldap_properties_list_reqd(properties, options)
@@ -812,3 +826,40 @@ def ensure_can_start_under_current_user(ambari_user):
"command as root, as sudo or as user \"{1}\"".format(current_user, ambari_user)
raise FatalException(1, err)
return current_user
+
+class PamPropTemplate:
+ def __init__(self, properties, i_prop_name, i_prop_val_pattern, i_prompt_regex, i_allow_empty_prompt, i_prop_name_default=None):
+ self.prop_name = i_prop_name
+ self.pam_prop_name = get_value_from_properties(properties, i_prop_name, i_prop_name_default)
+ self.pam_prop_val_prompt = i_prop_val_pattern.format(get_prompt_default(self.pam_prop_name))
+ self.prompt_regex = i_prompt_regex
+ self.allow_empty_prompt = i_allow_empty_prompt
+
+def setup_pam():
+ if not is_root():
+ err = 'Ambari-server setup-pam should be run with ' \
+ 'root-level privileges'
+ raise FatalException(4, err)
+
+ properties = get_ambari_properties()
+
+ if get_value_from_properties(properties,CLIENT_SECURITY_KEY,"") == 'ldap':
+ err = "LDAP is configured. Can not setup PAM."
+ raise FatalException(1, err)
+
+ pam_property_value_map = {}
+ pam_property_value_map[CLIENT_SECURITY_KEY] = 'pam'
+
+ pamConfig = get_validated_string_input("Enter PAM configuration file: ", PAM_CONFIG_FILE, REGEX_ANYTHING,
+ "Invalid characters in the input!", False, False)
+
+ pam_property_value_map[PAM_CONFIG_FILE] = pamConfig
+
+ if get_YN_input("Do you want to allow automatic group creation [y/n] (y)? ", True):
+ pam_property_value_map[AUTO_GROUP_CREATION] = 'true'
+ else:
+ pam_property_value_map[AUTO_GROUP_CREATION] = 'false'
+
+ update_properties_2(properties, pam_property_value_map)
+ print 'Saving...done'
+ return 0
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql b/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql
index 37a9757..09042b5 100644
--- a/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql
+++ b/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql
@@ -295,6 +295,7 @@ CREATE TABLE groups (
principal_id BIGINT NOT NULL,
group_name VARCHAR(255) NOT NULL,
ldap_group INTEGER NOT NULL DEFAULT 0,
+ group_type VARCHAR(255) NOT NULL DEFAULT 'LOCAL',
CONSTRAINT PK_groups PRIMARY KEY (group_id),
CONSTRAINT FK_groups_principal_id FOREIGN KEY (principal_id) REFERENCES adminprincipal(principal_id),
CONSTRAINT UNQ_groups_0 UNIQUE (group_name, ldap_group));
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql b/ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql
index 15d6120..e2c2dd5 100644
--- a/ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql
+++ b/ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql
@@ -286,6 +286,7 @@ CREATE TABLE groups (
principal_id NUMBER(19) NOT NULL,
group_name VARCHAR2(255) NOT NULL,
ldap_group NUMBER(10) DEFAULT 0,
+ group_type VARCHAR(255) DEFAULT 'LOCAL' NOT NULL,
CONSTRAINT PK_groups PRIMARY KEY (group_id),
CONSTRAINT FK_groups_principal_id FOREIGN KEY (principal_id) REFERENCES adminprincipal(principal_id),
CONSTRAINT UNQ_groups_0 UNIQUE (group_name, ldap_group));
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql b/ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql
index 5a82a52..4e9a535 100644
--- a/ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql
+++ b/ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql
@@ -295,6 +295,7 @@ CREATE TABLE groups (
principal_id BIGINT NOT NULL,
group_name VARCHAR(255) NOT NULL,
ldap_group INTEGER NOT NULL DEFAULT 0,
+ group_type VARCHAR(255) NOT NULL DEFAULT 'LOCAL',
CONSTRAINT PK_groups PRIMARY KEY (group_id),
UNIQUE (ldap_group, group_name),
CONSTRAINT FK_groups_principal_id FOREIGN KEY (principal_id) REFERENCES adminprincipal(principal_id));
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql b/ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql
index 659e4dc..0ba7df6 100644
--- a/ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql
+++ b/ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql
@@ -284,6 +284,7 @@ CREATE TABLE groups (
principal_id NUMERIC(19) NOT NULL,
group_name VARCHAR(255) NOT NULL,
ldap_group INTEGER NOT NULL DEFAULT 0,
+ group_type VARCHAR(255) NOT NULL DEFAULT 'LOCAL',
CONSTRAINT PK_groups PRIMARY KEY (group_id),
CONSTRAINT FK_groups_principal_id FOREIGN KEY (principal_id) REFERENCES adminprincipal(principal_id),
CONSTRAINT UNQ_groups_0 UNIQUE (group_name, ldap_group));
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql b/ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql
index e9a258a..d8cad6f 100644
--- a/ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql
+++ b/ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql
@@ -299,6 +299,7 @@ CREATE TABLE groups (
principal_id BIGINT NOT NULL,
group_name VARCHAR(255) NOT NULL,
ldap_group INTEGER NOT NULL DEFAULT 0,
+ group_type VARCHAR(255) NOT NULL DEFAULT 'LOCAL',
CONSTRAINT PK_groups PRIMARY KEY CLUSTERED (group_id),
CONSTRAINT FK_groups_principal_id FOREIGN KEY (principal_id) REFERENCES adminprincipal(principal_id),
CONSTRAINT UNQ_groups_0 UNIQUE (group_name, ldap_group));
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/resources/properties.json
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/properties.json b/ambari-server/src/main/resources/properties.json
index 6bbb323..b7e0988 100644
--- a/ambari-server/src/main/resources/properties.json
+++ b/ambari-server/src/main/resources/properties.json
@@ -183,6 +183,7 @@
"Group":[
"Groups/group_name",
"Groups/ldap_group",
+ "Groups/group_type",
"_"
],
"Member":[
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml b/ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml
index 500c0bf..9eca920 100644
--- a/ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml
+++ b/ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml
@@ -32,6 +32,7 @@
<authentication-manager alias="authenticationManager">
<authentication-provider ref="ambariLocalAuthenticationProvider"/>
+ <authentication-provider ref="ambariPamAuthenticationProvider"/>
<authentication-provider ref="ambariLdapAuthenticationProvider"/>
<authentication-provider ref="ambariInternalAuthenticationProvider"/>
<authentication-provider ref="kerberosServiceAuthenticationProvider"/>
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java b/ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java
new file mode 100644
index 0000000..2a6c754
--- /dev/null
+++ b/ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java
@@ -0,0 +1,97 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ambari.server.security.authorization;
+
+import static junit.framework.Assert.assertEquals;
+import static org.easymock.EasyMock.createNiceMock;
+import static org.easymock.EasyMock.expect;
+
+import java.util.Arrays;
+import java.util.HashSet;
+
+import org.apache.ambari.server.audit.AuditLoggerModule;
+import org.apache.ambari.server.configuration.Configuration;
+import org.apache.ambari.server.orm.GuiceJpaInitializer;
+import org.apache.ambari.server.security.ClientSecurityType;
+import org.easymock.EasyMock;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.jvnet.libpam.PAM;
+import org.jvnet.libpam.UnixUser;
+import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
+import org.springframework.security.core.Authentication;
+import org.springframework.security.core.AuthenticationException;
+
+import com.google.inject.Guice;
+import com.google.inject.Inject;
+import com.google.inject.Injector;
+import com.google.inject.persist.PersistService;
+
+import junit.framework.Assert;
+
+public class AmbariPamAuthenticationProviderTest {
+
+ private static Injector injector;
+
+ @Inject
+ private AmbariPamAuthenticationProvider authenticationProvider;
+ @Inject
+ Configuration configuration;
+
+ @Before
+ public void setUp() {
+ injector = Guice.createInjector(new AuditLoggerModule(), new AuthorizationTestModule());
+ injector.injectMembers(this);
+ injector.getInstance(GuiceJpaInitializer.class);
+ configuration.setClientSecurityType(ClientSecurityType.PAM);
+ configuration.setProperty(Configuration.PAM_CONFIGURATION_FILE, "ambari-pam");
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ injector.getInstance(PersistService.class).stop();
+ }
+
+ @Test(expected = AuthenticationException.class)
+ public void testBadCredential() throws Exception {
+ Authentication authentication = new UsernamePasswordAuthenticationToken("notFound", "wrong");
+ authenticationProvider.authenticate(authentication);
+ }
+
+ @Test
+ public void testAuthenticate() throws Exception {
+ PAM pam = createNiceMock(PAM.class);
+ UnixUser unixUser = createNiceMock(UnixUser.class);
+ expect(pam.authenticate(EasyMock.anyObject(String.class), EasyMock.anyObject(String.class))).andReturn(unixUser).atLeastOnce();
+ expect(unixUser.getGroups()).andReturn(new HashSet<String>(Arrays.asList("group"))).atLeastOnce();
+ EasyMock.replay(unixUser);
+ EasyMock.replay(pam);
+ Authentication authentication = new UsernamePasswordAuthenticationToken("allowedUser", "password");
+ Authentication result = authenticationProvider.authenticateViaPam(pam,authentication);
+ assertEquals("allowedUser", result.getName());
+ }
+
+ @Test
+ public void testDisabled() throws Exception {
+ configuration.setClientSecurityType(ClientSecurityType.LOCAL);
+ Authentication authentication = new UsernamePasswordAuthenticationToken("allowedUser", "password");
+ Authentication auth = authenticationProvider.authenticate(authentication);
+ Assert.assertTrue(auth == null);
+ }
+}
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/test/java/org/apache/ambari/server/security/authorization/TestUsers.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/security/authorization/TestUsers.java b/ambari-server/src/test/java/org/apache/ambari/server/security/authorization/TestUsers.java
index f54ac5c..7d112fc 100644
--- a/ambari-server/src/test/java/org/apache/ambari/server/security/authorization/TestUsers.java
+++ b/ambari-server/src/test/java/org/apache/ambari/server/security/authorization/TestUsers.java
@@ -285,7 +285,7 @@ public class TestUsers {
@Test
public void testSetGroupLdap() throws Exception {
- users.createGroup("group");
+ users.createGroup("group", GroupType.LOCAL);
users.setGroupLdap("group");
Assert.assertNotNull(users.getGroup("group"));
@@ -302,8 +302,8 @@ public class TestUsers {
public void testCreateGetRemoveGroup() throws Exception {
final String groupName = "engineering1";
final String groupName2 = "engineering2";
- users.createGroup(groupName);
- users.createGroup(groupName2);
+ users.createGroup(groupName, GroupType.LOCAL);
+ users.createGroup(groupName2, GroupType.LOCAL);
final Group group = users.getGroup(groupName);
assertNotNull(group);
@@ -328,8 +328,8 @@ public class TestUsers {
public void testMembers() throws Exception {
final String groupName = "engineering";
final String groupName2 = "engineering2";
- users.createGroup(groupName);
- users.createGroup(groupName2);
+ users.createGroup(groupName, GroupType.LOCAL);
+ users.createGroup(groupName2, GroupType.LOCAL);
users.createUser("user1", "user1");
users.createUser("user2", "user2");
users.createUser("user3", "user3");
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5a2bb8d/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
index 8ed81df..14fc20b 100644
--- a/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
+++ b/ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
@@ -101,6 +101,11 @@ public class UpgradeCatalog250Test {
// !!! setup capture for host_version
dbAccessor.addUniqueConstraint("host_version", "UQ_host_repo", "repo_version_id", "host_id");
+
+ Capture<DBAccessor.DBColumnInfo> groupGroupType = newCapture();
+ dbAccessor.addColumn(eq(UpgradeCatalog250.GROUPS_TABLE), capture(groupGroupType));
+ dbAccessor.addUniqueConstraint("groups", "UNQ_groups_0", "group_name", "group_type");
+
expectLastCall().once();
// !!! setup capture for servicecomponent_version
@@ -143,6 +148,14 @@ public class UpgradeCatalog250Test {
UpgradeCatalog250 upgradeCatalog250 = injector.getInstance(UpgradeCatalog250.class);
upgradeCatalog250.executeDDLUpdates();
+ DBAccessor.DBColumnInfo capturedGroupTypeColumn = groupGroupType.getValue();
+ Assert.assertNotNull(capturedGroupTypeColumn);
+ Assert.assertEquals(UpgradeCatalog250.GROUP_TYPE_COL, capturedGroupTypeColumn.getName());
+ Assert.assertEquals(String.class, capturedGroupTypeColumn.getType());
+ Assert.assertEquals(null, capturedGroupTypeColumn.getLength());
+ Assert.assertEquals("LOCAL", capturedGroupTypeColumn.getDefaultValue());
+ Assert.assertEquals(false, capturedGroupTypeColumn.isNullable());
+
verify(dbAccessor);
// !!! check the captured for host_version
[28/31] ambari git commit: AMBARI-18899. Post Upgrade to Ambari 2.4.1
from 2.2.2 --> All stack component prompted for restart (dlysnichenko)
Posted by nc...@apache.org.
AMBARI-18899. Post Upgrade to Ambari 2.4.1 from 2.2.2 --> All stack component prompted for restart (dlysnichenko)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/61942e55
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/61942e55
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/61942e55
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 61942e55a8fd09b90f320fb2bf7dcc68eee0eeca
Parents: 7cf4d86
Author: Lisnichenko Dmitro <dl...@hortonworks.com>
Authored: Wed Nov 16 13:24:15 2016 +0200
Committer: Lisnichenko Dmitro <dl...@hortonworks.com>
Committed: Wed Nov 16 13:24:15 2016 +0200
----------------------------------------------------------------------
.../common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml | 2 +-
.../stacks/HDP/2.2/services/HDFS/configuration/core-site.xml | 2 +-
.../stacks/HDP/2.2/services/HDFS/configuration/hdfs-site.xml | 2 +-
.../stacks/PHD/3.0/services/HDFS/configuration/core-site.xml | 2 +-
.../stacks/PHD/3.0/services/HDFS/configuration/hdfs-site.xml | 2 +-
5 files changed, 5 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/61942e55/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml b/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml
index afaaee8..37da591 100644
--- a/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml
+++ b/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml
@@ -225,7 +225,7 @@
<name>namenode_backup_dir</name>
<description>Local directory for storing backup copy of NameNode images during upgrade</description>
<value>/tmp/upgrades</value>
- <on-ambari-upgrade add="true"/>
+ <on-ambari-upgrade add="false"/>
</property>
<property>
<name>hdfs_user_keytab</name>
http://git-wip-us.apache.org/repos/asf/ambari/blob/61942e55/ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/core-site.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/core-site.xml b/ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/core-site.xml
index 846b09e..cfda2b4 100644
--- a/ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/core-site.xml
+++ b/ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/core-site.xml
@@ -50,6 +50,6 @@
<name>ranger.service.https.attrib.ssl.enabled</name>
</property>
</depends-on>
- <on-ambari-upgrade add="true"/>
+ <on-ambari-upgrade add="false"/>
</property>
</configuration>
http://git-wip-us.apache.org/repos/asf/ambari/blob/61942e55/ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hdfs-site.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hdfs-site.xml b/ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hdfs-site.xml
index aad524d..c3eb763 100644
--- a/ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hdfs-site.xml
+++ b/ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hdfs-site.xml
@@ -77,6 +77,6 @@
<name>ranger.service.https.attrib.ssl.enabled</name>
</property>
</depends-on>
- <on-ambari-upgrade add="true"/>
+ <on-ambari-upgrade add="false"/>
</property>
</configuration>
http://git-wip-us.apache.org/repos/asf/ambari/blob/61942e55/ambari-server/src/main/resources/stacks/PHD/3.0/services/HDFS/configuration/core-site.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/PHD/3.0/services/HDFS/configuration/core-site.xml b/ambari-server/src/main/resources/stacks/PHD/3.0/services/HDFS/configuration/core-site.xml
index 4f18611..cd0e43a 100644
--- a/ambari-server/src/main/resources/stacks/PHD/3.0/services/HDFS/configuration/core-site.xml
+++ b/ambari-server/src/main/resources/stacks/PHD/3.0/services/HDFS/configuration/core-site.xml
@@ -50,7 +50,7 @@
<name>ranger.service.https.attrib.ssl.enabled</name>
</property>
</depends-on>
- <on-ambari-upgrade add="true"/>
+ <on-ambari-upgrade add="false"/>
</property>
<!-- HDFS properties required for HAWQ -->
<property>
http://git-wip-us.apache.org/repos/asf/ambari/blob/61942e55/ambari-server/src/main/resources/stacks/PHD/3.0/services/HDFS/configuration/hdfs-site.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/PHD/3.0/services/HDFS/configuration/hdfs-site.xml b/ambari-server/src/main/resources/stacks/PHD/3.0/services/HDFS/configuration/hdfs-site.xml
index 0fed58d..f1ecaa9 100644
--- a/ambari-server/src/main/resources/stacks/PHD/3.0/services/HDFS/configuration/hdfs-site.xml
+++ b/ambari-server/src/main/resources/stacks/PHD/3.0/services/HDFS/configuration/hdfs-site.xml
@@ -73,7 +73,7 @@
<name>ranger.service.https.attrib.ssl.enabled</name>
</property>
</depends-on>
- <on-ambari-upgrade add="true"/>
+ <on-ambari-upgrade add="false"/>
</property>
<!-- HDFS properties required for HAWQ -->
<property>
[21/31] ambari git commit: AMBARI-18866. Stack advisor should limit
call history it preserves (Madhuvanthi Radhakrishnan via smohanty)
Posted by nc...@apache.org.
AMBARI-18866. Stack advisor should limit call history it preserves (Madhuvanthi Radhakrishnan via smohanty)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/8c3ad1d6
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/8c3ad1d6
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/8c3ad1d6
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 8c3ad1d6469fbe9c095fcc563f1c1a536892a727
Parents: b7ee658
Author: Sumit Mohanty <sm...@hortonworks.com>
Authored: Tue Nov 15 16:49:14 2016 -0800
Committer: Sumit Mohanty <sm...@hortonworks.com>
Committed: Tue Nov 15 16:50:09 2016 -0800
----------------------------------------------------------------------
.../services/stackadvisor/StackAdvisorHelper.java | 18 +++++++++++++++---
.../server/configuration/Configuration.java | 11 +++++++++++
.../stackadvisor/StackAdvisorHelperTest.java | 10 +++++++++-
3 files changed, 35 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/8c3ad1d6/ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorHelper.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorHelper.java b/ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorHelper.java
index e175c69..05e7598 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorHelper.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorHelper.java
@@ -41,6 +41,7 @@ public class StackAdvisorHelper {
private File recommendationsDir;
private String recommendationsArtifactsLifetime;
+ private int recommendationsArtifactsRolloverMax;
private String stackAdvisorScript;
private final AmbariMetaInfo metaInfo;
@@ -53,6 +54,7 @@ public class StackAdvisorHelper {
AmbariMetaInfo metaInfo) throws IOException {
this.recommendationsDir = conf.getRecommendationsDir();
this.recommendationsArtifactsLifetime = conf.getRecommendationsArtifactsLifetime();
+ this.recommendationsArtifactsRolloverMax = conf.getRecommendationsArtifactsRolloverMax();
this.stackAdvisorScript = conf.getStackAdvisorScript();
this.saRunner = saRunner;
this.metaInfo = metaInfo;
@@ -68,7 +70,7 @@ public class StackAdvisorHelper {
*/
public synchronized ValidationResponse validate(StackAdvisorRequest request)
throws StackAdvisorException {
- requestId += 1;
+ requestId = generateRequestId();
StackAdvisorCommand<ValidationResponse> command = createValidationCommand(request
.getRequestType());
@@ -103,9 +105,9 @@ public class StackAdvisorHelper {
*/
public synchronized RecommendationResponse recommend(StackAdvisorRequest request)
throws StackAdvisorException {
- requestId += 1;
+ requestId = generateRequestId();
- StackAdvisorCommand<RecommendationResponse> command = createRecommendationCommand(request
+ StackAdvisorCommand<RecommendationResponse> command = createRecommendationCommand(request
.getRequestType());
return command.invoke(request);
@@ -131,4 +133,14 @@ public class StackAdvisorHelper {
return command;
}
+ /**
+ * Returns an incremented requestId. Rollsover back to 0 in case the requestId >= recommendationsArtifactsrollovermax
+ * @return {int requestId}
+ */
+ private int generateRequestId(){
+ requestId += 1;
+ return requestId % recommendationsArtifactsRolloverMax;
+
+ }
+
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/8c3ad1d6/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java b/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
index b8b8f54..ff91b36 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
@@ -403,6 +403,12 @@ public class Configuration {
public static final ConfigurationProperty<String> RECOMMENDATIONS_ARTIFACTS_LIFETIME = new ConfigurationProperty<>(
"recommendations.artifacts.lifetime", "1w");
+ @Markdown(
+ description = "Maximum number of recommendations artifacts at a given time",
+ examples = {"50","10","100"} )
+ public static final ConfigurationProperty<Integer> RECOMMENDATIONS_ARTIFACTS_ROLLOVER_MAX = new ConfigurationProperty<>(
+ "recommendations.artifacts.rollover.max",100);
+
/**
* The directory on the Ambari Server file system used for storing
* Recommendation API artifacts.
@@ -3179,6 +3185,11 @@ public class Configuration {
return getProperty(RECOMMENDATIONS_ARTIFACTS_LIFETIME);
}
+ public int getRecommendationsArtifactsRolloverMax() {
+ int rollovermax = Integer.parseInt(getProperty(RECOMMENDATIONS_ARTIFACTS_ROLLOVER_MAX));
+ return (rollovermax == 0) ? 100 : rollovermax;
+ }
+
public String areHostsSysPrepped(){
return getProperty(SYS_PREPPED_HOSTS);
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/8c3ad1d6/ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorHelperTest.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorHelperTest.java b/ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorHelperTest.java
index 5190dea..5b0fdda 100644
--- a/ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorHelperTest.java
+++ b/ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorHelperTest.java
@@ -44,11 +44,11 @@ import org.junit.Test;
* StackAdvisorHelper unit tests.
*/
public class StackAdvisorHelperTest {
-
@Test
@SuppressWarnings("unchecked")
public void testValidate_returnsCommandResult() throws StackAdvisorException, IOException {
Configuration configuration = mock(Configuration.class);
+ when(configuration.getRecommendationsArtifactsRolloverMax()).thenReturn(100);
StackAdvisorRunner saRunner = mock(StackAdvisorRunner.class);
AmbariMetaInfo metaInfo = mock(AmbariMetaInfo.class);
StackAdvisorHelper helper = spy(new StackAdvisorHelper(configuration, saRunner, metaInfo));
@@ -61,6 +61,7 @@ public class StackAdvisorHelperTest {
when(command.invoke(request)).thenReturn(expected);
doReturn(command).when(helper).createValidationCommand(requestType);
+
ValidationResponse response = helper.validate(request);
assertEquals(expected, response);
@@ -71,6 +72,7 @@ public class StackAdvisorHelperTest {
public void testValidate_commandThrowsException_throwsException() throws StackAdvisorException,
IOException {
Configuration configuration = mock(Configuration.class);
+ when(configuration.getRecommendationsArtifactsRolloverMax()).thenReturn(100);
StackAdvisorRunner saRunner = mock(StackAdvisorRunner.class);
AmbariMetaInfo metaInfo = mock(AmbariMetaInfo.class);
StackAdvisorHelper helper = spy(new StackAdvisorHelper(configuration, saRunner, metaInfo));
@@ -91,6 +93,7 @@ public class StackAdvisorHelperTest {
@SuppressWarnings("unchecked")
public void testRecommend_returnsCommandResult() throws StackAdvisorException, IOException {
Configuration configuration = mock(Configuration.class);
+ when(configuration.getRecommendationsArtifactsRolloverMax()).thenReturn(100);
StackAdvisorRunner saRunner = mock(StackAdvisorRunner.class);
AmbariMetaInfo metaInfo = mock(AmbariMetaInfo.class);
StackAdvisorHelper helper = spy(new StackAdvisorHelper(configuration, saRunner, metaInfo));
@@ -113,6 +116,7 @@ public class StackAdvisorHelperTest {
public void testRecommend_commandThrowsException_throwsException() throws StackAdvisorException,
IOException {
Configuration configuration = mock(Configuration.class);
+ when(configuration.getRecommendationsArtifactsRolloverMax()).thenReturn(100);
StackAdvisorRunner saRunner = mock(StackAdvisorRunner.class);
AmbariMetaInfo metaInfo = mock(AmbariMetaInfo.class);
StackAdvisorHelper helper = spy(new StackAdvisorHelper(configuration, saRunner, metaInfo));
@@ -133,6 +137,7 @@ public class StackAdvisorHelperTest {
public void testCreateRecommendationCommand_returnsComponentLayoutRecommendationCommand()
throws IOException, StackAdvisorException {
Configuration configuration = mock(Configuration.class);
+ when(configuration.getRecommendationsArtifactsRolloverMax()).thenReturn(100);
StackAdvisorRunner saRunner = mock(StackAdvisorRunner.class);
AmbariMetaInfo metaInfo = mock(AmbariMetaInfo.class);
StackAdvisorHelper helper = new StackAdvisorHelper(configuration, saRunner, metaInfo);
@@ -148,6 +153,7 @@ public class StackAdvisorHelperTest {
public void testCreateValidationCommand_returnsComponentLayoutValidationCommand()
throws IOException, StackAdvisorException {
Configuration configuration = mock(Configuration.class);
+ when(configuration.getRecommendationsArtifactsRolloverMax()).thenReturn(100);
StackAdvisorRunner saRunner = mock(StackAdvisorRunner.class);
AmbariMetaInfo metaInfo = mock(AmbariMetaInfo.class);
StackAdvisorHelper helper = new StackAdvisorHelper(configuration, saRunner, metaInfo);
@@ -162,6 +168,7 @@ public class StackAdvisorHelperTest {
public void testCreateValidationCommand_returnsConfigurationValidationCommand()
throws IOException, StackAdvisorException {
Configuration configuration = mock(Configuration.class);
+ when(configuration.getRecommendationsArtifactsRolloverMax()).thenReturn(100);
StackAdvisorRunner saRunner = mock(StackAdvisorRunner.class);
AmbariMetaInfo metaInfo = mock(AmbariMetaInfo.class);
StackAdvisorHelper helper = new StackAdvisorHelper(configuration, saRunner, metaInfo);
@@ -176,6 +183,7 @@ public class StackAdvisorHelperTest {
public void testCreateRecommendationDependencyCommand_returnsConfigurationDependencyRecommendationCommand()
throws IOException, StackAdvisorException {
Configuration configuration = mock(Configuration.class);
+ when(configuration.getRecommendationsArtifactsRolloverMax()).thenReturn(100);
StackAdvisorRunner saRunner = mock(StackAdvisorRunner.class);
AmbariMetaInfo metaInfo = mock(AmbariMetaInfo.class);
StackAdvisorHelper helper = new StackAdvisorHelper(configuration, saRunner, metaInfo);
[09/31] ambari git commit: AMBARI-18893. Remove redundant css (part
2) (onechiporenko)
Posted by nc...@apache.org.
AMBARI-18893. Remove redundant css (part 2) (onechiporenko)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/07cde1d7
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/07cde1d7
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/07cde1d7
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 07cde1d74d94bbcf380d195be271491ee5035746
Parents: 5a32eda
Author: Oleg Nechiporenko <on...@apache.org>
Authored: Tue Nov 15 11:55:02 2016 +0200
Committer: Oleg Nechiporenko <on...@apache.org>
Committed: Tue Nov 15 11:55:02 2016 +0200
----------------------------------------------------------------------
ambari-web/app/styles/alerts.less | 55 -
ambari-web/app/styles/application.less | 997 ++++++++-----------
ambari-web/app/styles/bootstrap_overrides.less | 14 +
ambari-web/app/styles/common.less | 298 +-----
ambari-web/app/styles/config_history_flow.less | 6 -
ambari-web/app/styles/modal_popups.less | 3 -
ambari-web/app/styles/stack_versions.less | 24 +-
ambari-web/app/styles/widgets.less | 4 +-
.../common/configs/selectCreateConfigGroup.hbs | 4 +-
.../templates/common/host_progress_popup.hbs | 4 +-
.../common/modal_popups/log_tail_popup.hbs | 2 +-
ambari-web/app/templates/login.hbs | 2 +-
.../main/alerts/definition_details.hbs | 8 +-
.../main/dashboard/plus_button_filter.hbs | 8 +-
.../main/dashboard/widgets/yarn_links.hbs | 2 +-
ambari-web/app/templates/main/host/summary.hbs | 2 +-
.../manage_configuration_groups_popup.hbs | 8 +-
.../templates/main/service/services/flume.hbs | 4 +-
ambari-web/app/templates/main/views.hbs | 5 +-
ambari-web/app/templates/wizard/step3.hbs | 4 +-
ambari-web/app/views/main/dashboard/widgets.js | 2 +
.../host_component_recommendation_mixin_test.js | 24 +-
.../host_component_validation_mixin_test.js | 24 +-
23 files changed, 476 insertions(+), 1028 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/styles/alerts.less
----------------------------------------------------------------------
diff --git a/ambari-web/app/styles/alerts.less b/ambari-web/app/styles/alerts.less
index 1ea3492..4569e25 100644
--- a/ambari-web/app/styles/alerts.less
+++ b/ambari-web/app/styles/alerts.less
@@ -184,25 +184,8 @@
.alert-definition-details-header {
margin-bottom: 30px;
}
- .definition-details-block {
- margin-top: 30px;
- .multiline-text {
- padding-top: 4px;
- height: 60px;
- }
- }
- hr {
- margin: 10px 0 0 0;
- }
- .edit-description-button {
- font-size: 14px;
- }
- .edit-description-button:hover {
- text-decoration: none;
- }
.status {
- text-align: right;
.label {
font-size: 14px;
padding: 5px 8px;
@@ -215,8 +198,6 @@
width: 100%;
}
.properties-list {
- padding: 10px;
- text-align: left;
.property-name {
font-weight: bold;
}
@@ -231,45 +212,9 @@
font-weight: bold;
}
- .definition-name {
- .name-text-field{
- margin-left: 2px;
- margin-bottom: 0;
- input {
- width: 99%;
- margin-bottom: 0;
- margin-top: -4px;
- margin-left: -5px;
- }
- }
- .edit-buttons {
- margin-bottom: 0;
- margin-top: -4px;
- float: left;
- margin-left: 7px;
- }
- }
-
.error-msg {
color: #b94a48;
}
-
- .edit-buttons {
- text-align: right;
- margin-bottom: 10px;
- }
-
- .text-area-edit {
- textarea {
- width: 99%;
- height: 100px;
- margin-left: -5px;
- }
- }
-
- .edit-link {
- cursor: pointer;
- }
}
.alert-configs {
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/styles/application.less
----------------------------------------------------------------------
diff --git a/ambari-web/app/styles/application.less b/ambari-web/app/styles/application.less
index 3d2375e..5312ee2 100644
--- a/ambari-web/app/styles/application.less
+++ b/ambari-web/app/styles/application.less
@@ -91,6 +91,286 @@ footer {
height: @footer-height - 60px;
}
+
+.editable-list-container {
+ &:hover {
+ .pencil-box {
+ opacity: 1;
+ }
+ }
+ &.disabled {
+ .pencil-box {
+ display: none;
+ }
+ }
+ &.well{
+ padding: 10px;
+ position: relative;
+ margin-bottom: 25px;
+ cursor: pointer;
+ &.edit-mode{
+ cursor: default;
+ }
+ &.disabled{
+ background: white;
+ }
+ }
+ &.edit-mode {
+ &:hover {
+ .pencil-box {
+ opacity: 0;
+ }
+ }
+ .items-box {
+ ul.items-list {
+ li.item {
+ &.add-item-input .ember-text-field{
+ display: inline-block!important;
+ text-overflow: ellipsis;
+ white-space: nowrap;
+ position: relative;
+ padding: 0;
+ margin: 0;
+ border: none;
+ width: 50px;
+ height: 18px;
+ box-shadow: none;
+ -webkit-transition: none;
+ -moz-transition: none;
+ -o-transition: none;
+ transition: none;
+ }
+ .close {
+ display: inline-block;
+ }
+ }
+ }
+ }
+ }
+ .actions-panel{
+ position: absolute;
+ right: 5px;
+ bottom: -42px;
+ padding: 2px 5px 5px 5px;
+ background: #f5f5f5;
+ border: 1px solid #e3e3e3;
+ border-top: none;
+ border-radius: 0 0 4px 4px;
+
+ -webkit-transition: all 0.3s;
+ -o-transition: all 0.3s;
+ transition: all 0.3s;
+
+ -ms-transform-origin: 0% 0%; /* IE 9 */
+ -webkit-transform-origin: 0% 0%; /* Chrome, Safari, Opera */
+ transform-origin: 0% 0%;
+
+ -webkit-transform: rotateX(0deg);
+ -ms-transform: rotateX(0deg);
+ -o-transform: rotateX(0deg);
+ transform: rotateX(0deg);
+
+ .btn-xs {
+ padding: 1px 5px;
+ font-size: 12px;
+ line-height: 1.5;
+ border-radius: 3px;
+ }
+ .glyphicon-remove {
+ color: #000;
+ }
+ .glyphicon-ok {
+ color: white;
+ }
+ }
+ .items-box {
+ ul.items-list{
+ list-style-type: none;
+ margin: 0;
+ padding: 0;
+ li.item{
+ display: inline-block;
+ padding: 4px 8px;
+ margin: 0 5px 5px 2px;
+ background: white;
+ border: 1px solid #ebebeb;
+ max-width: 100%;
+ white-space: nowrap;
+ position: relative;
+ &.add-item{
+ color: #ddd;
+ }
+ &.deletable{
+ padding-right: 25px;
+ }
+ &.global{
+ background-color: #f5f5f5;
+ }
+ li {
+ a {
+ text-decoration: none;
+ }
+ }
+ a {
+ color: #555;
+ }
+ .close{
+ margin: -2px 0 0 5px;
+ width: 13px;
+ outline: none;
+ position: absolute;
+ display: none;
+ }
+ }
+ }
+ }
+ .pencil-box{
+ position: absolute;
+ right: 5px;
+ top: 5px;
+ opacity: 0;
+ -webkit-transition: all 0.3s;
+ -o-transition: all 0.3s;
+ transition: all 0.3s;
+ }
+}
+
+.typeahead-box {
+ position: absolute;
+ left: 0;
+ margin-top: 5px;
+ background: white;
+ border: 1px solid #ebebeb;
+ z-index: 1000;
+ min-width: 65px;
+ max-height: 150px;
+ overflow: auto;
+ ul {
+ list-style-type: none;
+ margin: 0;
+ padding: 0;
+ li {
+ padding: 3px 5px;
+ display: block;
+ cursor: pointer;
+ &.selected, .typeahead-box ul li:hover{
+ background: #eee;
+ }
+ }
+ }
+}
+
+
+.table-overlay {
+ position: absolute;
+ opacity: 0.9;
+ background-color: #fefefe;
+ .spinner {
+ display: none;
+ }
+}
+
+.host-elements {
+ border: 1px solid #ddd;
+ height: 120px;
+ overflow: auto;
+ padding: 5px;
+ ul {
+ margin-bottom: 0;
+ }
+ table {
+ width: 100%;
+ td {
+ width: 50%;
+ padding: 0;
+ }
+ }
+}
+
+.export-graph-list {
+ top: 25px;
+ min-width: 60px;
+ font-size: 14px;
+ cursor: auto;
+ a {
+ cursor: pointer;
+ }
+}
+
+.pre-styled {
+ display: block;
+ padding: 9.5px;
+ margin: 0 0 10px;
+ font-size: 11px;
+ line-height: 14px;
+ font-family: monospace;
+ word-break: break-all;
+ word-wrap: break-word;
+ white-space: pre-wrap;
+ background-color: #f5f5f5;
+ border: 1px solid rgba(0, 0, 0, 0.15);
+ border-radius: 4px;
+}
+
+.clear {
+ clear: both;
+}
+
+.noDisplay {
+ display: none !important;
+}
+
+.display {
+ display: block !important;
+}
+
+.display-inline-block {
+ display: inline-block;
+}
+
+.popover {
+ small {
+ font-size: 1.3rem;
+ }
+}
+
+.form-group {
+ .help-block {
+ &.help-inline {
+ margin-bottom: 0;
+ }
+ }
+}
+
+.alert {
+ & > ul, ol {
+ padding-left: inherit;
+ }
+}
+
+.dropdown-toggle.btn-icon {
+ padding-left: 8px;
+ padding-right: 8px;
+}
+
+::-webkit-scrollbar {
+ -webkit-appearance: none;
+}
+
+::-webkit-scrollbar:vertical {
+ width: 11px;
+}
+
+::-webkit-scrollbar:horizontal {
+ height: 11px;
+}
+
+::-webkit-scrollbar-thumb {
+ border-radius: 8px;
+ border: 2px solid white;
+ background-color: rgba(0, 0, 0, .5);
+}
+
#content {
padding: 15px 0;
}
@@ -122,11 +402,6 @@ table.diff {
line-height: @spinner-default-height;
}
-.heatmap-select-metric-btn, .heatmap-select-metric-btn:focus {
- font-size: 13px;
- padding: 10px;
-}
-
.row.row-no-pad {
.col-md-1, .col-md-2, .col-md-3, .col-md-4, .col-md-5, .col-md-6, .col-md-7, .col-md-8, .col-md-9, .col-md-10, .col-md-11, .col-md-12,
.col-lg-1, .col-lg-2, .col-lg-3, .col-lg-4, .col-lg-5, .col-lg-6, .col-lg-7, .col-lg-8, .col-lg-9, .col-lg-10, .col-lg-11, .col-lg-12 {
@@ -147,15 +422,15 @@ table.diff {
text-decoration: none;
}
.upgrade-in-progress {
- .infoPulse;
+ .infoPulse();
}
.upgrade-holding {
- .warnPulse;
+ .warnPulse();
}
.upgrade-aborted {
- .dangerPulse;
+ .dangerPulse();
.glyphicon-remove {
color: #fff;
}
@@ -173,6 +448,22 @@ table.diff {
cursor: pointer;
}
+.health-status-HEALTHY, .health-status-LIVE, .health-status-STARTING, .health-status-started, .health-status-starting {
+ color: @health-status-green;
+}
+.health-status-color-red, .health-status-installed, .health-status-stopping, .health-status-DEAD-RED, .health-status-STOPPING {
+ color: @health-status-red !important;
+}
+.health-status-unknown, .health-status-DEAD-YELLOW {
+ color: @health-status-yellow !important;
+}
+.health-status-DEAD-ORANGE {
+ color: @health-status-orange;
+}
+.health-status-color-blue {
+ color:@health-status-blue;
+}
+
#top-nav {
.navbar.navbar-static-top {
@@ -182,8 +473,6 @@ table.diff {
background-image: -o-linear-gradient(top, @top-nav-bg-color-from, @top-nav-bg-color-to);
background-image: linear-gradient(to bottom, @top-nav-bg-color-from, @top-nav-bg-color-to);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr= @top-nav-bg-color-from, endColorstr=@top-nav-bg-color-to); //for IE9-
- -webkit-box-shadow: inset 0 0 0 rgba(0, 0, 0, 0.1), 0 1px 10px rgba(0, 0, 0, 0.1);
- -moz-box-shadow: inset 0 0 0 rgba(0, 0, 0, 0.1), 0 1px 10px rgba(0, 0, 0, 0.1);
box-shadow: inset 0 0 0 rgba(0, 0, 0, 0.1), 0 1px 10px rgba(0, 0, 0, 0.1);
.navbar-text.brand-wrapper {
@@ -204,7 +493,7 @@ table.diff {
text-shadow: 0 -1px 0 rgba(0, 0, 0, 0.25);
background-color: @top-nav-ops-count-bg-color;
&.operations-count {
- .infoPulse;
+ .infoPulse();
}
&.alert-crit-count {
background-color: @health-status-red;
@@ -224,7 +513,7 @@ table.diff {
.ops-count {
margin-right: -1px;
color: #006DCC;
- .infoPulseInner;
+ .infoPulseInner();
}
}
a:hover {
@@ -266,8 +555,6 @@ table.diff {
.active > a, .active > a:hover, .active > a:focus {
color: @top-nav-menu-active-text-color;
background-color: @top-nav-menu-active-bg-color;
- -webkit-box-shadow: inset 0 3px 8px rgba(0, 0, 0, 0.125);
- -moz-box-shadow: inset 0 3px 8px rgba(0, 0, 0, 0.125);
box-shadow: inset 0 3px 8px rgba(0, 0, 0, 0.125);
}
li > a:focus, li > a:hover {
@@ -299,30 +586,12 @@ table.diff {
list-style: none;
background-color: @top-nav-menu-dropdown-bg-color;
border: 3px solid @top-nav-menu-dropdown-border-color;
- -webkit-box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
- -moz-box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
-webkit-background-clip: padding-box;
-moz-background-clip: padding;
background-clip: padding-box;
- .health-status-LIVE, .health-status-STARTING {
- color: @health-status-green;
- }
- .health-status-DEAD-RED, .health-status-STOPPING {
- color: @health-status-red;
- }
- .health-status-DEAD-YELLOW {
- color: @health-status-yellow;
- }
- .label {
- padding: 0 0 0 3px;
- }
.alerts-count {
- line-height: 14px;
- padding: 1px 4px;
- float: right;
margin-right: 5px;
- margin-top: 3px;
}
}
.top-nav-dropdown-menu > li{
@@ -333,9 +602,6 @@ table.diff {
text-align: left;
display: block;
padding: 3px 0 3px 5px;
- clear: both;
- font-weight: normal;
- line-height: 20px;
color: @top-nav-menu-dropdown-text-color;
white-space: nowrap;
.glyphicon-health-block > span:before {
@@ -373,7 +639,6 @@ table.diff {
border-radius: 8px;
font-size: 13px;
font-weight: bold;
- line-height: 19px;
margin: 1px 10px 2px 0;
padding: 7px 14px;
text-decoration: none;
@@ -397,9 +662,6 @@ table.diff {
.service-caption {
display: inline-block;
width: 100%;
- &.with-alert {
- width: 80%;
- }
}
.glyphicon-caption {
display: inline-block;
@@ -486,21 +748,12 @@ h1 {
width: 8px;
}
-.glyphicon-caret-right {
+.glyphicon-caret-right, .glyphicon-caret-left {
min-width: 8px;
padding-top: 2px;
}
-.glyphicon-caret-left {
- min-width: 8px;
- padding-top: 2px;
-}
-
-.glyphicon-remove {
- color: #FF4B4B;
-}
-
-.glyphicon-remove-sign {
+.glyphicon-remove, .glyphicon-remove-sign {
color: #FF4B4B;
}
@@ -663,8 +916,6 @@ h1 {
li.filter-status.active,
li.filter-status.active:hover {
background-color: #666;
- -webkit-box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
- -moz-box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
border-radius: 5px;
}
@@ -768,13 +1019,11 @@ h1 {
font-size: 12px;
li.filter-status.active a {
text-decoration: none;
- color: #FFFFFF;
+ color: #fff;
}
li.filter-status.active,
li.filter-status.active:hover {
background-color: #666;
- -webkit-box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
- -moz-box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
border-radius: 5px;
}
@@ -954,11 +1203,6 @@ h1 {
#summary-restart-bar {
margin-top: 20px;
- .alert {
- .glyphicon-refresh {
- color: #fdb82f;
- }
- }
}
#serviceConfig {
@@ -1067,20 +1311,21 @@ h1 {
.action {
cursor: pointer;
}
- .glyphicon-plus-sign {
- color: @health-status-green;
+ .a-icon(@color) {
+ color: @color;
margin-right: 2px;
}
+ .glyphicon-plus-sign {
+ .a-icon(@health-status-green);
+ }
.glyphicon-minus-sign {
- color: #FF4B4B;
- margin-right: 2px;
+ .a-icon(#FF4B4B);
}
.icon-undo {
- color: rgb(243, 178, 11);
- margin-right: 2px;
+ .a-icon(rgb(243, 178, 11));
}
.btn-final{
- background: transparent repeat scroll 0 0 rgba(255, 255, 255, 0);
+ background: transparent repeat scroll 0 0 #fff;
}
.btn-final .glyphicon-lock{
color: #a6a6a6;
@@ -1094,8 +1339,6 @@ h1 {
background-color: #d9d9d9 \9;
background-image: none;
outline: 0;
- -webkit-box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
- -moz-box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
box-shadow: inset 0 2px 4px rgba(0, 0, 0, 0.15), 0 1px 2px rgba(0, 0, 0, 0.05);
}
.btn-final.active[disabled] {
@@ -1182,9 +1425,6 @@ h1 {
font-size: 27px;
line-height: 30px;
}
- .glyphicon-ok-sign {
- color: @health-status-green;
- }
.glyphicon-warning-sign {
color: @health-status-red;
}
@@ -1683,7 +1923,9 @@ a:focus {
padding: 5px;
}
- .tab-marker-position {
+ .health-status-LIVE, .health-status-STARTING,
+ .health-status-DEAD-RED, .health-status-STOPPING,
+ .health-status-DEAD-ORANGE, .health-status-DEAD-YELLOW {
list-style: none;
min-height: 20px;
min-width: 20px;
@@ -1691,29 +1933,6 @@ a:focus {
float: none;
display: inline;
}
- .health-status-LIVE, .health-status-STARTING {
- .tab-marker-position;
- color: @health-status-green;
- }
- .health-status-DEAD-RED, .health-status-STOPPING {
- .tab-marker-position;
- color: @health-status-red;
- }
- .health-status-DEAD-ORANGE {
- .tab-marker-position;
- color: @health-status-orange;
- }
- .health-status-DEAD-YELLOW {
- .tab-marker-position;
- color: @health-status-yellow;
- }
- dt {
- text-align: left;
- width: 120px;
- }
- dd {
- margin-left: 145px;
- }
.like_pointer {
cursor: pointer;
}
@@ -1758,7 +1977,8 @@ a:focus {
}
.services, .summary-info {
- .tab-marker-position {
+ .tab-marker-position(@color) {
+ color: @color;
background-position: 0 3px;
background-repeat: no-repeat;
list-style: none;
@@ -1769,16 +1989,13 @@ a:focus {
float: none !important;
}
.STARTING, .STARTED {
- .tab-marker-position;
- color: @health-status-green;
+ .tab-marker-position (@health-status-green);
}
.STOPPING, .INSTALLED {
- .tab-marker-position;
- color: @health-status-red;
+ .tab-marker-position (@health-status-red);
}
.UNKNOWN {
- .tab-marker-position;
- color: @health-status-yellow;
+ .tab-marker-position (@health-status-yellow);
}
.summary-view-host{
@@ -1798,7 +2015,7 @@ a:focus {
}
}
.service-summary-component-red-dead {
- color: #ff0000;
+ color: @health-status-red;
display: inline;
}
.service-summary-component-green-live {
@@ -1875,7 +2092,7 @@ a:focus {
width: 100%;
padding: 7px 0;
text-align: center;
- color: #333333;
+ color: #333;
&:hover {
background-color: #eee;
color: #333;
@@ -1901,7 +2118,7 @@ a:focus {
li {
border-bottom: 1px solid #eee;
list-style: none;
- padding: 5px 5px 5px 5px;
+ padding: 5px;
background-position: 14px 9px;
background-repeat: no-repeat;
.date-time {
@@ -1918,9 +2135,6 @@ a:focus {
font-weight: normal;
font-size: 13px;
}
- .row [class*="span"] {
- min-height: 0;
- }
.status-icon {
padding-top: 7px;
}
@@ -2056,7 +2270,7 @@ a:focus {
.x_tick {
margin-top: 5px;
.title {
- padding: 0 2px 0 2px;
+ padding: 0 2px;
opacity: 1 !important;
top: 148px;
}
@@ -2146,7 +2360,9 @@ a:focus {
/*Services*/
.services-menu {
.nav-list {
- .tab-marker-position {
+ .health-status-LIVE, .health-status-STARTING,
+ .health-status-DEAD-RED, .health-status-STOPPING,
+ .health-status-DEAD-YELLOW {
list-style: none;
height: 20px;
width: 20px;
@@ -2154,35 +2370,12 @@ a:focus {
display: inline;
float: none;
}
- .health-status-LIVE, .health-status-STARTING {
- .tab-marker-position;
- color: @health-status-green;
- }
- .health-status-DEAD-RED, .health-status-STOPPING {
- .tab-marker-position;
- color: @health-status-red;
- }
- .health-status-DEAD-YELLOW {
- .tab-marker-position;
- color: @health-status-yellow;
- }
li {
line-height: 24px;
a {
padding: 3px 4px;
white-space: nowrap;
}
- .label {
- padding: 0 0 0 3px;
- }
- .label.alerts-count {
- line-height: 14px;
- padding: 1px 4px;
- background: @health-status-red;
- float: right;
- margin-right: 1px;
- margin-top: 3px;
- }
}
li.clients {
@@ -2202,10 +2395,6 @@ a:focus {
}
- .start-stop-all-service-button {
- margin: 5px 5px 10px 10px;
- text-align: center;
- }
.icon-medkit {
padding-left:6px;
}
@@ -2248,13 +2437,6 @@ a.services-menu-blocks{
margin: 0 2px;
}
}
-.host-rack-id-popup {
- .control-label {
- width: 100px;
- margin-right: 10px;
- }
-}
-
.service-block {
margin-top: 20px;
@@ -2286,10 +2468,6 @@ a.services-menu-blocks{
}
.summary-metric-graphs {
- [class*="span"] {
- float: left;
- margin-left: 10px;
- }
.chart-container {
.chart-x-axis {
left: 0;
@@ -2336,47 +2514,16 @@ table.graphs {
/*Dashboard Widgets/heatmaps Start*/
#dashboard-widgets-container{
- > ul.nav.nav-tabs {
- margin-bottom: 10px;
- }
#widgets-options-menu {
.add-widgets-text .dropdown-menu {
- top: 0;
- left: 99%;
- margin-top: -6px;
- margin-left: -1px;
- max-height: 375px;
overflow: auto;
width: 210px;
- li {
- display: block;
- padding: 3px 0 3px 5px;
- .checkbox {
- margin: 0;
- }
- ul {
- margin-left: 0;
- }
- }
- &>li {
- &>ul {
- width: 180px;
- margin-left: 0;
- }
- }
- }
- .add-widgets-text:hover .dropdown-menu {
- display: block;
}
}
.dashboard-widgets-box {
margin-top: 4px;
}
- h4{
- line-height: 30px;
- margin-bottom: 0;
- margin-top: 0;
- }
+
#dashboard-widgets {
.caption {
height: 25px;
@@ -2394,25 +2541,14 @@ table.graphs {
width: 19.60%;
*width: 19.60%;
}
- .span4p8 {
- float: left;
- width: 39.55%;
- *width: 39.55%;
- }
.thumbnails > div {
- margin-left: 0;
- margin-right: 3px;
- margin-top: 0;
- margin-bottom: 3px;
+ margin: 0 3px 3px 0;
height: 163px;
}
.thumbnails li {
height: 160px;
width: 100%;
- margin-left: 0;
- margin-right: 3px;
- margin-top: 3px;
- margin-bottom: 0;
+ margin: 3px 3px 0 0;
}
.img-thumbnail .corner-icon {
@@ -2420,14 +2556,14 @@ table.graphs {
position: relative;
padding: 7px 0;
.glyphicon-remove-sign{
- color: #000000;
+ color: #000;
text-shadow: #fff 0 0 15px;
position: absolute;
left: -10px;
top: -10px;
}
.glyphicon-edit, .glyphicon-save {
- color: #555555;
+ color: #555;
}
}
.export-graph-list {
@@ -2438,7 +2574,7 @@ table.graphs {
}
}
.img-thumbnail .hidden-info-general{
- color: #555555;
+ color: #555;
font-size: 12px;
font-weight: bold;
text-align: center;
@@ -2466,11 +2602,8 @@ table.graphs {
}
}
.img-thumbnail .caption {
- padding-left: 0;
- padding-top: 7px;
- padding-bottom: 7px;
- padding-right: 0;
- color: #555555;
+ padding: 7px 0;
+ color: #555;
font-weight:bold;
font-size: 12px;
text-align: left;
@@ -2490,7 +2623,7 @@ table.graphs {
width: 90%;
height: 105px;
border: 1px solid silver;
- color: #ffffff;
+ color: #fff;
i {
font-size: inherit;
}
@@ -2505,7 +2638,7 @@ table.graphs {
position: relative;
}
.img-thumbnail{
- background-color: #ffffff;
+ background-color: #fff;
z-index: 3;
}
.has-hidden-info {
@@ -2515,7 +2648,6 @@ table.graphs {
}
.has-hidden-info .img-thumbnail:hover {
cursor: move;
- //background-color: #d3d3d3;
.corner-icon{
display:block;
text-decoration: none;
@@ -2532,15 +2664,13 @@ table.graphs {
margin-left: -13px;
z-index: 7;
}
- .widget-content{
- top: -104px;
+ .content-mx(@top) {
+ top: @top;
opacity: 0.3;
z-index: 5;
}
- .widget-content-isNA{
- top: -104px;
- opacity: 0.3;
- z-index: 5;
+ .widget-content, .widget-content-isNA {
+ .content-mx(-104px);
}
.uptime-content{
top: -116px;
@@ -2552,36 +2682,26 @@ table.graphs {
top: -82px;
}
.content-hidden-two-line{
- top: -105px;
- opacity: 0.3;
- z-index: 5;
+ .content-mx(-105px);
}
.content-hidden-three-line{
- top: -116px;
- opacity: 0.3;
- z-index: 5;
+ .content-mx(-116px);
}
.content-hidden-four-line{
- top: -126px;
- opacity: 0.3;
- z-index: 5;
+ .content-mx(-126px);
}
.content-hidden-five-line{
- top: -140px;
- opacity: 0.3;
- z-index: 5;
+ .content-mx(-140px);
}
.content-hidden-six-line{
- top: -143px;
- opacity: 0.3;
- z-index: 5;
+ .content-mx(-143px);
}
}
.cluster-metrics {
position: relative;
.chart-container{
- margin: 0 10px 0 10px;
+ margin: 0 10px;
.chart-y-axis{
margin-top: 10px;
}
@@ -2621,30 +2741,23 @@ table.graphs {
}
}
- .links {
- ul {
- margin: 0;
- }
- }
.links .img-thumbnail{
li{
height:20px;
- margin: 3px;
}
.link-button{
- padding-top: 7px;
- padding-left: 70px;
+ float: right;
.disabled-hdfs-quick-link {
- pointer-events: none;
- color: #808080;
- cursor: default;
- }
+ pointer-events: none;
+ color: #808080;
+ cursor: default;
+ }
}
.widget-content{
text-align: center;
font-size: 11px;
- color: #555555;
+ color: #555;
padding-top: 40px;
table{
vertical-position: center;
@@ -2672,58 +2785,31 @@ table.graphs {
position: relative;
}
- #map-reduce-slots-text{
- padding-left: 5px;
- padding-top: 40px;
- color: #555555;
- font-weight:bold;
- font-size: 10px;
- >ul{
- margin-top:5px;
- margin-bottom: 0;
- }
- #map-reduce-slots-bar1{
- margin-left: 10px;
- }
- #map-reduce-slots-bar2{
- margin-left: 3px;
- }
- #map-reduce-slots-num1{
- margin-left: 5px;
- }
- #map-reduce-slots-num2{
- margin-left: 5px;
- }
+ .widget-cn(@color) {
+ color: @color;
+ padding-top: 70px;
+ font-weight: bold;
}
-
.is-red{
.widget-content {
- color: @health-status-red;
- padding-top: 70px;
- font-weight: bold;
+ .widget-cn(@health-status-red);
}
}
.is-orange{
.widget-content {
- color: @health-status-orange;
- padding-top: 70px;
- font-weight: bold;
+ .widget-cn(@health-status-orange);
}
}
.is-green {
.widget-content {
- color: @health-status-green;
- padding-top: 70px;
- font-weight: bold;
+ .widget-cn(@health-status-green);
}
}
.is-na {
position: relative;
.widget-content {
- color: #D6DDDF;
+ .widget-cn(#D6DDDF);
text-shadow: none;
- padding-top: 70px;
- font-weight: bold;
}
}
}
@@ -2731,7 +2817,7 @@ table.graphs {
/*Dashboard Widgets END*/
.dialog-delete-component {
- border: 1px solid #FDB82F;
+ border: 1px solid @restart-indicator-color;
border-radius: 4px;
padding: 4px;
background: #FFFDF6;
@@ -2775,34 +2861,17 @@ table.graphs {
}
}
- .status-dot-position {
+ .health-status-HEALTHY,
+ .health-status-LIVE,
+ .health-status-DEAD-RED,
+ .health-status-DEAD-ORANGE,
+ .health-status-DEAD-YELLOW {
background-position: center;
background-repeat: no-repeat;
height: 20px;
width: 13px;
display: inline !important;
float: none !important;
- }
- .health-status-HEALTHY,
- .health-status-LIVE {
- .status-dot-position;
- color: @health-status-green;
- margin-right: 1px;
- margin-left: 1px;
- }
- .health-status-DEAD-RED {
- .status-dot-position;
- color: @health-status-red;
- }
- .health-status-DEAD-ORANGE {
- .status-dot-position;
- color: @health-status-orange;
- margin-right: 1px;
- margin-left: 1px;
- }
- .health-status-DEAD-YELLOW {
- .status-dot-position;
- color: @health-status-yellow;
margin-right: 1px;
margin-left: 1px;
}
@@ -2968,40 +3037,6 @@ table.graphs {
margin: 4px 4px 2px 2px;
}
}
- li#title-bar {
- text-align: left;
- border-bottom: 1px solid #e4e4e4;
- a.close {
- background: #777;
- display: inline;
- color: #fff;
- padding-left: 35px;
- padding-right: 12px;
- text-shadow: 0 1px 0 #fff;
- float: none;
- font-size: 10px;
- opacity: 0.6;
- }
- a.close:hover {
- background: #777;
- opacity: 1.0;
- }
- }
- li#selector-bar {
- text-align: left;
- border-bottom: 1px solid #e4e4e4;
- font-size: 6px;
- }
- li#list-area {
- font-weight: normal;
- text-align: left;
- }
- li#button-bar {
- text-align: center;
- button {
- font-size: 12px;
- }
- }
select {
width: 150px;
}
@@ -3041,39 +3076,9 @@ table.graphs {
.dropdown {
margin-top: 3px;
}
- .dropdown {
- margin-top: 3px;
- }
.dropdown select {
width: 60px;
}
- .page-listing a {
- line-height: 0;
- border: none;
- margin: 0;
- margin-right: 10px;
- cursor: pointer;
- color: #0088CC;
- padding: 8px 0;
- float: left;
- text-decoration: underline;
- }
- .page-listing a:hover {
- text-decoration: none;
- }
- .page-listing {
- width: 100px;
- .table {
- th.name {
- width: 300px;
- a.filter-label {
- width: 57px;
- display: block;
- float: left;
- }
- }
- }
- }
}
.collapsed-list {
padding-left: 10px;
@@ -3127,9 +3132,7 @@ table.graphs {
#host-details {
- .set-rack-id {
- cursor: pointer;
- }
+
.status-info {
.host-title {
font-size: 18px;
@@ -3151,7 +3154,14 @@ table.graphs {
border-bottom-color: #000;
}
- .marker {
+ .components-health.glyphicon-arrow-up {
+ color:@health-status-red;
+ }
+
+ .health-status-started, .health-status-starting, .health-status-LIVE,
+ .health-status-installed, .health-status-stopping, .health-status-DEAD-RED,
+ .health-status-unknown, .health-status-DEAD-YELLOW,
+ .health-status-DEAD-ORANGE {
background-repeat: no-repeat;
height: 13px;
width: 13px;
@@ -3159,53 +3169,6 @@ table.graphs {
float: none !important;
}
- .components-health.glyphicon-arrow-up{
- color:@health-status-red;
- }
-
- .health-status-started, .health-status-starting {
- .marker;
- color: @health-status-green;
- }
-
- .health-status-installed, .health-status-stopping {
- .marker;
- color: @health-status-red;
- }
-
- .health-status-unknown {
- .marker;
- color: @health-status-yellow;
- }
-
- .health-status-LIVE {
- .marker;
- color: @health-status-green;
- }
-
- .health-status-DEAD-RED {
- .marker;
- color: @health-status-red;
- }
-
- .health-status-color-blue {
- color:#0000ff;
- }
-
- .health-status-color-red {
- color: @health-status-red;
- }
-
- .health-status-DEAD-ORANGE {
- .marker;
- color: @health-status-orange;
- }
-
- .health-status-DEAD-YELLOW {
- .marker;
- color: @health-status-yellow;
- }
-
.host-details-summary-list {
.summary-label {
text-align: right;
@@ -3280,23 +3243,11 @@ table.graphs {
.services-menu {
.glyphicon-refresh {
- color: @restart-indicator-color;
margin-left: 4px;
}
}
-#host-details, #serviceConfig {
- .glyphicon-refresh {
- color: @restart-indicator-color;
- }
-}
#hosts{
- .glyphicon-refresh {
- color: @restart-indicator-color;
- }
- .alerts-crit-count {
- float: right;
- }
- .alerts-warn-count {
+ .alerts-crit-count, .alerts-warn-count {
float: right;
}
}
@@ -3335,15 +3286,6 @@ table.graphs {
}
/*End Admin*/
-/* Start Admin Access*/
-.ha-btn {
- margin-left: 0;
-}
-
-.rm-ha {
- margin-top: 20px;
-}
-/* End Admin Access*/
/*Start About*/
.about {
@@ -3415,7 +3357,6 @@ table.graphs {
max-width: 250px;
}
-
.hostName {
word-wrap: break-word!important;
white-space: normal;
@@ -3472,14 +3413,6 @@ ul.filter {
#main-admin-menu {
padding: 8px 0;
- .nav-list {
- li {
- line-height: 24px;
- a {
- padding: 3px 20px;
- }
- }
- }
}
.button-caret-margin {
@@ -3670,9 +3603,6 @@ ul.filter {
.container-fluid {
padding: 0;
}
- .row [class*="span"] {
- margin-left: 0;
- }
.legend {
margin-top: 20px;
margin-bottom: 20px;
@@ -3705,10 +3635,6 @@ ul.filter {
/*End Heatmap*/
-.noDisplay {
- display: none !important;
-}
-
/* CHARTS */
.chart {
overflow: hidden;
@@ -3727,36 +3653,6 @@ ul.filter {
font-weight: bold;
}
- .horizon {
- border-bottom: 1px solid #000;
- overflow: hidden;
- position: relative;
- }
- .horizon {
- border-bottom: 1px solid #000;
- border-top: 1px solid #000;
- }
- .horizon + .horizon {
- border-top: medium none;
- }
- .horizon canvas {
- display: block;
- }
- .horizon .title, .horizon .value {
- bottom: 0;
- line-height: 30px;
- margin: 0 6px;
- position: absolute;
- text-shadow: 0 1px 0 rgba(255, 255, 255, 0.5);
- white-space: nowrap;
- }
- .horizon .title {
- left: 0;
- }
- .horizon .value {
- right: 0;
- }
-
.rule {
display: none;
width: 30px;
@@ -3808,7 +3704,6 @@ ul.filter {
}
text {
- font-family: Arial;
font-size: 9pt;
}
}
@@ -3832,10 +3727,6 @@ ul.filter {
border: none !important;
}
-ul.noStyle {
- list-style: none;
-}
-
ul.inline li {
display: inline;
}
@@ -3936,8 +3827,6 @@ ul.inline li {
color: #555;
text-decoration: none;
background-color: #e5e5e5;
- -webkit-box-shadow: inset 0 -5px 8px rgba(0, 0, 0, 0.05);
- -moz-box-shadow: inset 0 -5px 8px rgba(0, 0, 0, 0.05);
box-shadow: inset 0 -5px 8px rgba(0, 0, 0, 0.05);
}
.filter-btn {
@@ -4122,12 +4011,6 @@ ul.inline li {
}
}
-#step10-alert-message {
- .glyphicon-refresh {
- color: @restart-indicator-color;
- }
-}
-
#step10-content {
ul {
li {
@@ -4397,11 +4280,6 @@ i.glyphicon-asterisks {
width: 19.47%;
*width: 19.47%;
}
- .span4p8 {
- float: left;
- width: 39.45%;
- *width: 39.45%;
- }
.thumbnails > div {
margin-right: 5px;
}
@@ -4419,20 +4297,10 @@ i.glyphicon-asterisks {
font-size: 12px;
}
}
- #map-reduce-slots-text{
- font-size: 12px;
- #map-reduce-slots-bar1{
- margin-left: 11px;
- }
- }
}
}
.summary-metric-graphs {
- [class*="span"] {
- float: left;
- margin-left: 10px;
- }
.chart-container {
.chart-x-axis {
left: 0;
@@ -4463,25 +4331,9 @@ i.glyphicon-asterisks {
}
}
-.row .host-component-block {
-
- .refresh-icon-block {
- width: 6%;
- float: right;
- }
-
- .medkit-icon-block {
- width: 6%;
- float: left;
- padding-left: 3px;
- }
-}
-
.filter-combobox{
> input{
width: 170px;
- -webkit-border-radius: 4px 0 0 4px;
- -moz-border-radius: 4px 0 0 4px;
border-radius: 4px 0 0 4px;
padding-right: 20px;
}
@@ -4508,12 +4360,10 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
#config-group-select-create-dialog {
.select-create-config-group-div {
- margin-left: 20px;
- margin-bottom: 20px;
- .btn-group >.dropdown-menu {
+ .dropdown-menu {
max-height: 160px;
max-width: 300px;
- overflow: scroll;
+ overflow-y: scroll;
}
}
.message{
@@ -4524,7 +4374,6 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
#config-group-confirm-save {
.message{
color: #777;
- padding-left: 10px;
}
}
@@ -4558,11 +4407,6 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
}
}
-.prrl {
- position: relative;
- right: @space-m;
-}
-
.pts {
padding-top: @space-s;
}
@@ -4667,29 +4511,14 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
cursor: pointer;
}
td.icon {
- width: 6%;
- padding-left: 20px;
- padding-right: 5px;
+ vertical-align: middle;
+ width: 40px;
.view-icon-image {
img {
height: 32px;
}
}
}
- td.label-version-description {
- padding-top: 10px;
- .view-label-text {
- display: inline;
- font-size: 16px;
- font-weight: bold;
- }
- .view-version-text,
- .view-description-text {
- font-size: 14px;
- display: inline;
- color: #808080;
- }
- }
}
.no-views-text {
padding-left: 20px;
@@ -4744,32 +4573,16 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
overflow: hidden;
max-width: 90px;
}
- .glyphicon-ok-sign {
- color: @health-status-green;
- }
- .glyphicon-question-sign {
- color: @health-status-yellow;
- }
.glyphicon-warning-sign {
- color: #FDB82F;
- }
- .sorting_asc {
- background:
- url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABMAAAATCAYAAAByUDbMAAAKQWlDQ1BJQ0MgUHJvZmlsZQAASA2dlndUU9kWh8+9N73QEiIgJfQaegkg0jtIFQRRiUmAUAKGhCZ2RAVGFBEpVmRUwAFHhyJjRRQLg4Ji1wnyEFDGwVFEReXdjGsJ7601896a/cdZ39nnt9fZZ+9917oAUPyCBMJ0WAGANKFYFO7rwVwSE8vE9wIYEAEOWAHA4WZmBEf4RALU/L09mZmoSMaz9u4ugGS72yy/UCZz1v9/kSI3QyQGAApF1TY8fiYX5QKUU7PFGTL/BMr0lSkyhjEyFqEJoqwi48SvbPan5iu7yZiXJuShGlnOGbw0noy7UN6aJeGjjAShXJgl4GejfAdlvVRJmgDl9yjT0/icTAAwFJlfzOcmoWyJMkUUGe6J8gIACJTEObxyDov5OWieAHimZ+SKBIlJYqYR15hp5ejIZvrxs1P5YjErlMNN4Yh4TM/0tAyOMBeAr2+WRQElWW2ZaJHtrRzt7VnW5mj5v9nfHn5T/T3IevtV8Sbsz55BjJ5Z32zsrC+9FgD2JFqbHbO+lVUAtG0GQOXhrE/vIADyBQC03pzzHoZsXpLE4gwnC4vs7GxzAZ9rLivoN/ufgm/Kv4Y595nL7vtWO6YXP4EjSRUzZUXlpqemS0TMzAwOl89k/fcQ/+PAOWnNycMsnJ/AF/GF6FVR6JQJhIlou4U8gViQLmQKhH/V4X8YNicHGX6daxRodV8AfYU5ULhJB8hvPQBDIwMkbj96An3rWxAxCsi+vGitka9zjzJ6/uf6Hwtcim7hTEEiU+b2DI9kciWiLBmj34RswQISkAd0oAo0gS4wAixgDRyAM3AD3iAAhIBIEAOWAy5IAmlABLJBPtgACkEx2AF2g2pwANSBetAEToI2cAZcBFfADXALDIBHQAqGwUswAd6BaQiC8BA
VokGqkBakD5lC1hAbWgh5Q0FQOBQDxUOJkBCSQPnQJqgYKoOqoUNQPfQjdBq6CF2D+qAH0CA0Bv0BfYQRmALTYQ3YALaA2bA7HAhHwsvgRHgVnAcXwNvhSrgWPg63whfhG/AALIVfwpMIQMgIA9FGWAgb8URCkFgkAREha5EipAKpRZqQDqQbuY1IkXHkAwaHoWGYGBbGGeOHWYzhYlZh1mJKMNWYY5hWTBfmNmYQM4H5gqVi1bGmWCesP3YJNhGbjS3EVmCPYFuwl7ED2GHsOxwOx8AZ4hxwfrgYXDJuNa4Etw/XjLuA68MN4SbxeLwq3hTvgg/Bc/BifCG+Cn8cfx7fjx/GvyeQCVoEa4IPIZYgJGwkVBAaCOcI/YQRwjRRgahPdCKGEHnEXGIpsY7YQbxJHCZOkxRJhiQXUiQpmbSBVElqIl0mPSa9IZPJOmRHchhZQF5PriSfIF8lD5I/UJQoJhRPShxFQtlOOUq5QHlAeUOlUg2obtRYqpi6nVpPvUR9Sn0vR5Mzl/OX48mtk6uRa5Xrl3slT5TXl3eXXy6fJ18hf0r+pvy4AlHBQMFTgaOwVqFG4bTCPYVJRZqilWKIYppiiWKD4jXFUSW8koGStxJPqUDpsNIlpSEaQtOledK4tE20Otpl2jAdRzek+9OT6cX0H+i99AllJWVb5SjlHOUa5bPKUgbCMGD4M1IZpYyTjLuMj/M05rnP48/bNq9pXv+8KZX5Km4qfJUilWaVAZWPqkxVb9UU1Z2qbapP1DBqJmphatlq+9Uuq43Pp893ns+dXzT/5PyH6rC6iXq4+mr1w+o96pMamhq+GhkaVRqXNMY1GZpumsma5ZrnNMe0aFoLtQRa5VrntV4wlZnuzFRmJbOLOaGtru2nLdE+pN2rPa1jqLNYZ6NOs84TXZIuWzdBt1y3U3dCT0svWC9fr1HvoT5Rn62fpL9Hv1t/ysDQINpgi0GbwaihiqG/YZ5ho+FjI6qRq9Eq
o1qjO8Y4Y7ZxivE+41smsImdSZJJjclNU9jU3lRgus+0zwxr5mgmNKs1u8eisNxZWaxG1qA5wzzIfKN5m/krCz2LWIudFt0WXyztLFMt6ywfWSlZBVhttOqw+sPaxJprXWN9x4Zq42Ozzqbd5rWtqS3fdr/tfTuaXbDdFrtOu8/2DvYi+yb7MQc9h3iHvQ732HR2KLuEfdUR6+jhuM7xjOMHJ3snsdNJp9+dWc4pzg3OowsMF/AX1C0YctFx4bgccpEuZC6MX3hwodRV25XjWuv6zE3Xjed2xG3E3dg92f24+ysPSw+RR4vHlKeT5xrPC16Il69XkVevt5L3Yu9q76c+Oj6JPo0+E752vqt9L/hh/QL9dvrd89fw5/rX+08EOASsCegKpARGBFYHPgsyCRIFdQTDwQHBu4IfL9JfJFzUFgJC/EN2hTwJNQxdFfpzGC4sNKwm7Hm4VXh+eHcELWJFREPEu0iPyNLIR4uNFksWd0bJR8VF1UdNRXtFl0VLl1gsWbPkRoxajCCmPRYfGxV7JHZyqffS3UuH4+ziCuPuLjNclrPs2nK15anLz66QX8FZcSoeGx8d3xD/iRPCqeVMrvRfuXflBNeTu4f7kufGK+eN8V34ZfyRBJeEsoTRRJfEXYljSa5JFUnjAk9BteB1sl/ygeSplJCUoykzqdGpzWmEtPi000IlYYqwK10zPSe9L8M0ozBDuspp1e5VE6JA0ZFMKHNZZruYjv5M9UiMJJslg1kLs2qy3mdHZZ/KUcwR5vTkmuRuyx3J88n7fjVmNXd1Z752/ob8wTXuaw6thdauXNu5Tnddwbrh9b7rj20gbUjZ8MtGy41lG99uit7UUaBRsL5gaLPv5sZCuUJR4b0tzlsObMVsFWzt3WazrWrblyJe0fViy+KK4k8l3JLr31l9V/ndzPaE7b2l9qX7d+B2CHfc3em681iZYlle2dCu4F2t5czyovK3u1fsvlZhW3FgD2mPZI+0M
qiyvUqvakfVp+qk6oEaj5rmvep7t+2d2sfb17/fbX/TAY0DxQc+HhQcvH/I91BrrUFtxWHc4azDz+ui6rq/Z39ff0TtSPGRz0eFR6XHwo911TvU1zeoN5Q2wo2SxrHjccdv/eD1Q3sTq+lQM6O5+AQ4ITnx4sf4H++eDDzZeYp9qukn/Z/2ttBailqh1tzWibakNml7THvf6YDTnR3OHS0/m/989Iz2mZqzymdLz5HOFZybOZ93fvJCxoXxi4kXhzpXdD66tOTSna6wrt7LgZevXvG5cqnbvfv8VZerZ645XTt9nX297Yb9jdYeu56WX+x+aem172296XCz/ZbjrY6+BX3n+l37L972un3ljv+dGwOLBvruLr57/17cPel93v3RB6kPXj/Mejj9aP1j7OOiJwpPKp6qP6391fjXZqm99Oyg12DPs4hnj4a4Qy//lfmvT8MFz6nPK0a0RupHrUfPjPmM3Xqx9MXwy4yX0+OFvyn+tveV0auffnf7vWdiycTwa9HrmT9K3qi+OfrW9m3nZOjk03dp76anit6rvj/2gf2h+2P0x5Hp7E/4T5WfjT93fAn88ngmbWbm3/eE8/syOll+AAAACXBIWXMAAAsTAAALEwEAmpwYAAAA4klEQVQ4Ee2RPw8BQRDF3x4dCokL0SqUKqVSr/ZRruWTaEnUWgkShwji3yWCwoXQOCKCHXPq24hSmGJ3srvz5vdmga8NIhK1GhW2B8q+M+F/96DRRHE0hUEagegUEyK4VdVoqgv3fL2h3HAMQ3I+sQDLCpRdUlWNUux8prjZltXTRUIQ4X4T6HSRcRwkPxLj7r7ZHPXFSgO7A3xgwQfsncRghJKKzpPMPiBv9pBwDQmhgaTgnRU5zD7S86U3necH2CtQJIyKHkWKyXTGCrFZh4XtxxWt4x6eda9u/+U/gZ+dwBODrVwv7HA8iwAAAABJRU5ErkJggg==) no-repeat right 50%;
- }
- .sorting_desc {
- background: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABMAAAATCAYAAAByUDbMAAAKQWlDQ1BJQ0MgUHJvZmlsZQAASA2dlndUU9kWh8+9N73QEiIgJfQaegkg0jtIFQRRiUmAUAKGhCZ2RAVGFBEpVmRUwAFHhyJjRRQLg4Ji1wnyEFDGwVFEReXdjGsJ7601896a/cdZ39nnt9fZZ+9917oAUPyCBMJ0WAGANKFYFO7rwVwSE8vE9wIYEAEOWAHA4WZmBEf4RALU/L09mZmoSMaz9u4ugGS72yy/UCZz1v9/kSI3QyQGAApF1TY8fiYX5QKUU7PFGTL/BMr0lSkyhjEyFqEJoqwi48SvbPan5iu7yZiXJuShGlnOGbw0noy7UN6aJeGjjAShXJgl4GejfAdlvVRJmgDl9yjT0/icTAAwFJlfzOcmoWyJMkUUGe6J8gIACJTEObxyDov5OWieAHimZ+SKBIlJYqYR15hp5ejIZvrxs1P5YjErlMNN4Yh4TM/0tAyOMBeAr2+WRQElWW2ZaJHtrRzt7VnW5mj5v9nfHn5T/T3IevtV8Sbsz55BjJ5Z32zsrC+9FgD2JFqbHbO+lVUAtG0GQOXhrE/vIADyBQC03pzzHoZsXpLE4gwnC4vs7GxzAZ9rLivoN/ufgm/Kv4Y595nL7vtWO6YXP4EjSRUzZUXlpqemS0TMzAwOl89k/fcQ/+PAOWnNycMsnJ/AF/GF6FVR6JQJhIlou4U8gViQLmQKhH/V4X8YNicHGX6daxRodV8AfYU5ULhJB8hvPQBDIwMkbj96An3rWxAxCsi+vGitka9zjzJ6/uf6Hwtcim7hTEEiU+b2DI9kciWiLBmj34RswQISkAd0oAo0gS4wAixgDRyAM3AD3iAAhIBIEAOWAy5IAmlABLJBPtgACkEx2AF2g2pwANSBetAEToI2cAZcBFfADXALDIBHQAqGwUswA
d6BaQiC8BAVokGqkBakD5lC1hAbWgh5Q0FQOBQDxUOJkBCSQPnQJqgYKoOqoUNQPfQjdBq6CF2D+qAH0CA0Bv0BfYQRmALTYQ3YALaA2bA7HAhHwsvgRHgVnAcXwNvhSrgWPg63whfhG/AALIVfwpMIQMgIA9FGWAgb8URCkFgkAREha5EipAKpRZqQDqQbuY1IkXHkAwaHoWGYGBbGGeOHWYzhYlZh1mJKMNWYY5hWTBfmNmYQM4H5gqVi1bGmWCesP3YJNhGbjS3EVmCPYFuwl7ED2GHsOxwOx8AZ4hxwfrgYXDJuNa4Etw/XjLuA68MN4SbxeLwq3hTvgg/Bc/BifCG+Cn8cfx7fjx/GvyeQCVoEa4IPIZYgJGwkVBAaCOcI/YQRwjRRgahPdCKGEHnEXGIpsY7YQbxJHCZOkxRJhiQXUiQpmbSBVElqIl0mPSa9IZPJOmRHchhZQF5PriSfIF8lD5I/UJQoJhRPShxFQtlOOUq5QHlAeUOlUg2obtRYqpi6nVpPvUR9Sn0vR5Mzl/OX48mtk6uRa5Xrl3slT5TXl3eXXy6fJ18hf0r+pvy4AlHBQMFTgaOwVqFG4bTCPYVJRZqilWKIYppiiWKD4jXFUSW8koGStxJPqUDpsNIlpSEaQtOledK4tE20Otpl2jAdRzek+9OT6cX0H+i99AllJWVb5SjlHOUa5bPKUgbCMGD4M1IZpYyTjLuMj/M05rnP48/bNq9pXv+8KZX5Km4qfJUilWaVAZWPqkxVb9UU1Z2qbapP1DBqJmphatlq+9Uuq43Pp893ns+dXzT/5PyH6rC6iXq4+mr1w+o96pMamhq+GhkaVRqXNMY1GZpumsma5ZrnNMe0aFoLtQRa5VrntV4wlZnuzFRmJbOLOaGtru2nLdE+pN2rPa1jqLNYZ6NOs84TXZIuWzdBt1y3U3dCT0svWC9fr1HvoT5Rn62fpL9Hv1t/ysDQINpgi0GbwaihiqG/YZ5ho+
FjI6qRq9Eqo1qjO8Y4Y7ZxivE+41smsImdSZJJjclNU9jU3lRgus+0zwxr5mgmNKs1u8eisNxZWaxG1qA5wzzIfKN5m/krCz2LWIudFt0WXyztLFMt6ywfWSlZBVhttOqw+sPaxJprXWN9x4Zq42Ozzqbd5rWtqS3fdr/tfTuaXbDdFrtOu8/2DvYi+yb7MQc9h3iHvQ732HR2KLuEfdUR6+jhuM7xjOMHJ3snsdNJp9+dWc4pzg3OowsMF/AX1C0YctFx4bgccpEuZC6MX3hwodRV25XjWuv6zE3Xjed2xG3E3dg92f24+ysPSw+RR4vHlKeT5xrPC16Il69XkVevt5L3Yu9q76c+Oj6JPo0+E752vqt9L/hh/QL9dvrd89fw5/rX+08EOASsCegKpARGBFYHPgsyCRIFdQTDwQHBu4IfL9JfJFzUFgJC/EN2hTwJNQxdFfpzGC4sNKwm7Hm4VXh+eHcELWJFREPEu0iPyNLIR4uNFksWd0bJR8VF1UdNRXtFl0VLl1gsWbPkRoxajCCmPRYfGxV7JHZyqffS3UuH4+ziCuPuLjNclrPs2nK15anLz66QX8FZcSoeGx8d3xD/iRPCqeVMrvRfuXflBNeTu4f7kufGK+eN8V34ZfyRBJeEsoTRRJfEXYljSa5JFUnjAk9BteB1sl/ygeSplJCUoykzqdGpzWmEtPi000IlYYqwK10zPSe9L8M0ozBDuspp1e5VE6JA0ZFMKHNZZruYjv5M9UiMJJslg1kLs2qy3mdHZZ/KUcwR5vTkmuRuyx3J88n7fjVmNXd1Z752/ob8wTXuaw6thdauXNu5Tnddwbrh9b7rj20gbUjZ8MtGy41lG99uit7UUaBRsL5gaLPv5sZCuUJR4b0tzlsObMVsFWzt3WazrWrblyJe0fViy+KK4k8l3JLr31l9V/ndzPaE7b2l9qX7d+B2CHfc3em681iZYlle2dCu4F2t5czyovK3u1fsvlZhW3F
gD2mPZI+0MqiyvUqvakfVp+qk6oEaj5rmvep7t+2d2sfb17/fbX/TAY0DxQc+HhQcvH/I91BrrUFtxWHc4azDz+ui6rq/Z39ff0TtSPGRz0eFR6XHwo911TvU1zeoN5Q2wo2SxrHjccdv/eD1Q3sTq+lQM6O5+AQ4ITnx4sf4H++eDDzZeYp9qukn/Z/2ttBailqh1tzWibakNml7THvf6YDTnR3OHS0/m/989Iz2mZqzymdLz5HOFZybOZ93fvJCxoXxi4kXhzpXdD66tOTSna6wrt7LgZevXvG5cqnbvfv8VZerZ645XTt9nX297Yb9jdYeu56WX+x+aem172296XCz/ZbjrY6+BX3n+l37L972un3ljv+dGwOLBvruLr57/17cPel93v3RB6kPXj/Mejj9aP1j7OOiJwpPKp6qP6391fjXZqm99Oyg12DPs4hnj4a4Qy//lfmvT8MFz6nPK0a0RupHrUfPjPmM3Xqx9MXwy4yX0+OFvyn+tveV0auffnf7vWdiycTwa9HrmT9K3qi+OfrW9m3nZOjk03dp76anit6rvj/2gf2h+2P0x5Hp7E/4T5WfjT93fAn88ngmbWbm3/eE8/syOll+AAAACXBIWXMAAAsTAAALEwEAmpwYAAABEUlEQVQ4Ee2SMUsDQRSE52U3Z3FpjIgQo+a0CCQehisimDa2Fmlt/EX+ATs7LWy0VFCwsLKJtWgRiYWFWAjmdsc9IU1c5Ehrtln2zbzv7Q4LzNYsgf+cgPgef3PL/ccn9IIgjWn1UlEQpsJ3Kxh8ffJurVI47XblcrJXTxay80qEj/6D6b2NFEgDQkFDyoYoF5XE1Q7une0XrOCDRRVctBPVl9SpVMhM1hqHBJpNPNfXceTr88JExDYa2F1exQ9I0cFcIPMLQKuNHaeb3LDMWCrJ63YiB3oOGJEIlELSwt5iKC8+UFbz3mxsrtVwHNdxpZ1rI8Lh1qacj7Wp9uGQ4ckZr0n+OTg3
3IG8Xyg3YBrjN2mnRpK2GkKGAAAAAElFTkSuQmCC) no-repeat right 50%;
- }
- .sorting {
- background: url( data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABMAAAATCAYAAAByUDbMAAAKQWlDQ1BJQ0MgUHJvZmlsZQAASA2dlndUU9kWh8+9N73QEiIgJfQaegkg0jtIFQRRiUmAUAKGhCZ2RAVGFBEpVmRUwAFHhyJjRRQLg4Ji1wnyEFDGwVFEReXdjGsJ7601896a/cdZ39nnt9fZZ+9917oAUPyCBMJ0WAGANKFYFO7rwVwSE8vE9wIYEAEOWAHA4WZmBEf4RALU/L09mZmoSMaz9u4ugGS72yy/UCZz1v9/kSI3QyQGAApF1TY8fiYX5QKUU7PFGTL/BMr0lSkyhjEyFqEJoqwi48SvbPan5iu7yZiXJuShGlnOGbw0noy7UN6aJeGjjAShXJgl4GejfAdlvVRJmgDl9yjT0/icTAAwFJlfzOcmoWyJMkUUGe6J8gIACJTEObxyDov5OWieAHimZ+SKBIlJYqYR15hp5ejIZvrxs1P5YjErlMNN4Yh4TM/0tAyOMBeAr2+WRQElWW2ZaJHtrRzt7VnW5mj5v9nfHn5T/T3IevtV8Sbsz55BjJ5Z32zsrC+9FgD2JFqbHbO+lVUAtG0GQOXhrE/vIADyBQC03pzzHoZsXpLE4gwnC4vs7GxzAZ9rLivoN/ufgm/Kv4Y595nL7vtWO6YXP4EjSRUzZUXlpqemS0TMzAwOl89k/fcQ/+PAOWnNycMsnJ/AF/GF6FVR6JQJhIlou4U8gViQLmQKhH/V4X8YNicHGX6daxRodV8AfYU5ULhJB8hvPQBDIwMkbj96An3rWxAxCsi+vGitka9zjzJ6/uf6Hwtcim7hTEEiU+b2DI9kciWiLBmj34RswQISkAd0oAo0gS4wAixgDRyAM3AD3iAAhIBIEAOWAy5IAmlABLJBPtgACkEx2AF2g2pwANSBetAEToI2cAZcBFfADXALDIBHQAqGwUsw
Ad6BaQiC8BAVokGqkBakD5lC1hAbWgh5Q0FQOBQDxUOJkBCSQPnQJqgYKoOqoUNQPfQjdBq6CF2D+qAH0CA0Bv0BfYQRmALTYQ3YALaA2bA7HAhHwsvgRHgVnAcXwNvhSrgWPg63whfhG/AALIVfwpMIQMgIA9FGWAgb8URCkFgkAREha5EipAKpRZqQDqQbuY1IkXHkAwaHoWGYGBbGGeOHWYzhYlZh1mJKMNWYY5hWTBfmNmYQM4H5gqVi1bGmWCesP3YJNhGbjS3EVmCPYFuwl7ED2GHsOxwOx8AZ4hxwfrgYXDJuNa4Etw/XjLuA68MN4SbxeLwq3hTvgg/Bc/BifCG+Cn8cfx7fjx/GvyeQCVoEa4IPIZYgJGwkVBAaCOcI/YQRwjRRgahPdCKGEHnEXGIpsY7YQbxJHCZOkxRJhiQXUiQpmbSBVElqIl0mPSa9IZPJOmRHchhZQF5PriSfIF8lD5I/UJQoJhRPShxFQtlOOUq5QHlAeUOlUg2obtRYqpi6nVpPvUR9Sn0vR5Mzl/OX48mtk6uRa5Xrl3slT5TXl3eXXy6fJ18hf0r+pvy4AlHBQMFTgaOwVqFG4bTCPYVJRZqilWKIYppiiWKD4jXFUSW8koGStxJPqUDpsNIlpSEaQtOledK4tE20Otpl2jAdRzek+9OT6cX0H+i99AllJWVb5SjlHOUa5bPKUgbCMGD4M1IZpYyTjLuMj/M05rnP48/bNq9pXv+8KZX5Km4qfJUilWaVAZWPqkxVb9UU1Z2qbapP1DBqJmphatlq+9Uuq43Pp893ns+dXzT/5PyH6rC6iXq4+mr1w+o96pMamhq+GhkaVRqXNMY1GZpumsma5ZrnNMe0aFoLtQRa5VrntV4wlZnuzFRmJbOLOaGtru2nLdE+pN2rPa1jqLNYZ6NOs84TXZIuWzdBt1y3U3dCT0svWC9fr1HvoT5Rn62fpL9Hv1t/ysDQINpgi0GbwaihiqG/YZ5ho
+FjI6qRq9Eqo1qjO8Y4Y7ZxivE+41smsImdSZJJjclNU9jU3lRgus+0zwxr5mgmNKs1u8eisNxZWaxG1qA5wzzIfKN5m/krCz2LWIudFt0WXyztLFMt6ywfWSlZBVhttOqw+sPaxJprXWN9x4Zq42Ozzqbd5rWtqS3fdr/tfTuaXbDdFrtOu8/2DvYi+yb7MQc9h3iHvQ732HR2KLuEfdUR6+jhuM7xjOMHJ3snsdNJp9+dWc4pzg3OowsMF/AX1C0YctFx4bgccpEuZC6MX3hwodRV25XjWuv6zE3Xjed2xG3E3dg92f24+ysPSw+RR4vHlKeT5xrPC16Il69XkVevt5L3Yu9q76c+Oj6JPo0+E752vqt9L/hh/QL9dvrd89fw5/rX+08EOASsCegKpARGBFYHPgsyCRIFdQTDwQHBu4IfL9JfJFzUFgJC/EN2hTwJNQxdFfpzGC4sNKwm7Hm4VXh+eHcELWJFREPEu0iPyNLIR4uNFksWd0bJR8VF1UdNRXtFl0VLl1gsWbPkRoxajCCmPRYfGxV7JHZyqffS3UuH4+ziCuPuLjNclrPs2nK15anLz66QX8FZcSoeGx8d3xD/iRPCqeVMrvRfuXflBNeTu4f7kufGK+eN8V34ZfyRBJeEsoTRRJfEXYljSa5JFUnjAk9BteB1sl/ygeSplJCUoykzqdGpzWmEtPi000IlYYqwK10zPSe9L8M0ozBDuspp1e5VE6JA0ZFMKHNZZruYjv5M9UiMJJslg1kLs2qy3mdHZZ/KUcwR5vTkmuRuyx3J88n7fjVmNXd1Z752/ob8wTXuaw6thdauXNu5Tnddwbrh9b7rj20gbUjZ8MtGy41lG99uit7UUaBRsL5gaLPv5sZCuUJR4b0tzlsObMVsFWzt3WazrWrblyJe0fViy+KK4k8l3JLr31l9V/ndzPaE7b2l9qX7d+B2CHfc3em681iZYlle2dCu4F2t5czyovK3u1fsvlZhW3
FgD2mPZI+0MqiyvUqvakfVp+qk6oEaj5rmvep7t+2d2sfb17/fbX/TAY0DxQc+HhQcvH/I91BrrUFtxWHc4azDz+ui6rq/Z39ff0TtSPGRz0eFR6XHwo911TvU1zeoN5Q2wo2SxrHjccdv/eD1Q3sTq+lQM6O5+AQ4ITnx4sf4H++eDDzZeYp9qukn/Z/2ttBailqh1tzWibakNml7THvf6YDTnR3OHS0/m/989Iz2mZqzymdLz5HOFZybOZ93fvJCxoXxi4kXhzpXdD66tOTSna6wrt7LgZevXvG5cqnbvfv8VZerZ645XTt9nX297Yb9jdYeu56WX+x+aem172296XCz/ZbjrY6+BX3n+l37L972un3ljv+dGwOLBvruLr57/17cPel93v3RB6kPXj/Mejj9aP1j7OOiJwpPKp6qP6391fjXZqm99Oyg12DPs4hnj4a4Qy//lfmvT8MFz6nPK0a0RupHrUfPjPmM3Xqx9MXwy4yX0+OFvyn+tveV0auffnf7vWdiycTwa9HrmT9K3qi+OfrW9m3nZOjk03dp76anit6rvj/2gf2h+2P0x5Hp7E/4T5WfjT93fAn88ngmbWbm3/eE8/syOll+AAAACXBIWXMAAAsTAAALEwEAmpwYAAABmElEQVQ4EdWSv0vDQBTH7y4ZUkKhTdtYHArOUvwPdHAVpeBY3PwH/BfEycF/wclR6NzBxUFxKrgokRLaSkmhTZr+ADWJ32s5DeXaSkHBW97du/c+73vvHiF/vaIooj+pyZYFAaTbtn0DuzR2YQBX1G63K57n7TQajfNlhRfCfN8/6na7u4AS13VPOp3O/iLgXBgAa0i+/Hh7J5RSEoYh6fV6FfjX5wGlMCQwgKpQNs0Lo4kdjUYEz77FvSIDSmGA7DmOU+SKxGJkukeRDfTwWPjjVo0fxH48Hic1TbtmjBX5c2F1WA/3rSAI7obDoSVif81+vyNWAmNQHgwGB6qqbqHxOUVRklD
kQ2ELCu+h+qJQKDzGUiZb6TPT6TTt9/uHABLeK947QFKE0RSyNg3DkM6c9AN0Xb9CwguUCNDXeKDQQyaTeZpVxc9SZVASQMk2frWFzyCTwUBDElqCmKZZxv10VmaIUmU8Bgmv+Xy+JNRxXzabraJfz3y/0mo2m2e1Wi2q1+sQG+VWgogkAKhlWaeY/pLw/T/7CTBQv9a27vsbAAAAAElFTkSuQmCC) no-repeat right 50%;
+ color: @restart-indicator-color;
}
#flume-host-agent-row {
cursor: pointer;
text-align: left;
- background-color: #ffffff;
+ background-color: #fff;
}
.empty-label {
- border-top: 1px solid #dddddd;
+ border-top: 1px solid #ddd;
text-align: center;
}
}
@@ -5014,10 +4827,18 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
font-style: italic;
}
}
- td:first-child, td:first-child + td, th:first-child, th:first-child + th {
+ td:first-child,
+ td:first-child + td,
+ th:first-child,
+ th:first-child + th {
width: 12.5%;
}
- td:first-child + td + td, td:first-child + td + td + td, td:first-child + td + td + td + td, th:first-child + th + th, th:first-child + th + th + th, th:first-child + th + th + th + th {
+ td:first-child + td + td,
+ td:first-child + td + td + td,
+ td:first-child + td + td + td + td,
+ th:first-child + th + th,
+ th:first-child + th + th + th,
+ th:first-child + th + th + th + th {
width: 25%;
}
}
@@ -5031,21 +4852,6 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
}
}
-.table {
- background-color: #fff;
-}
-
-.panel {
- background-color: #fff;
-}
-
-.panel-heading {
- background-color: #f0f0f0;
- .panel-title {
- line-height: 34px;
- }
-}
-
.config-manage-nav {
.config-groups-dropdown {
display: inline-block;
@@ -5077,7 +4883,7 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
}
.icon-undo {
- color: rgb(243, 178, 11);
+ color: #F3B20B;
}
.view-permission-header th {
@@ -5129,11 +4935,6 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
font-weight: normal;
cursor: pointer;
}
- #upload-definition-file-panel.disabled {
- .local-option-label {
- color: #999;
- }
- }
}
.register-version-options {
input.disabled {
@@ -5177,9 +4978,6 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
color: grey;
}
}
- .border-bottom {
- border-bottom: 1px solid #ebebeb;
- }
.repos-inner-table {
margin-bottom: 0;
background-color: transparent;
@@ -5188,14 +4986,11 @@ input[type="radio"].align-checkbox, input[type="checkbox"].align-checkbox {
width: 90%;
height: 24px;
}
- #skip-validation {
+ #skip-validation, #use-redhat {
span.disabled {
opacity: 0.7;
}
}
- #use-redhat span.disabled {
- opacity: 0.7;
- }
}
}
#combo_search_box {
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/styles/bootstrap_overrides.less
----------------------------------------------------------------------
diff --git a/ambari-web/app/styles/bootstrap_overrides.less b/ambari-web/app/styles/bootstrap_overrides.less
index f6efa0a..dd427cb 100644
--- a/ambari-web/app/styles/bootstrap_overrides.less
+++ b/ambari-web/app/styles/bootstrap_overrides.less
@@ -51,3 +51,17 @@
.modal-open {
overflow-y: scroll;
}
+
+.table {
+ background-color: #fff;
+}
+
+.panel {
+ background-color: #fff;
+ .panel-heading {
+ background-color: #f0f0f0;
+ .panel-title {
+ line-height: 34px;
+ }
+ }
+}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/styles/common.less
----------------------------------------------------------------------
diff --git a/ambari-web/app/styles/common.less b/ambari-web/app/styles/common.less
index 9eaf932..8099e8a 100644
--- a/ambari-web/app/styles/common.less
+++ b/ambari-web/app/styles/common.less
@@ -16,6 +16,10 @@
* limitations under the License.
*/
+/**
+ * This file only for variables and mixins!
+ */
+
/************************************************************************
* Health status(service/host/host component health)icons class names
***********************************************************************/
@@ -32,10 +36,11 @@
* Health status(service/host/host component health)icon colors
***********************************************************************/
@health-status-red: #ff0000;
+@health-status-blue: #0000ff;
@health-status-green: #5AB400;
@health-status-yellow: #FFD13D;
@health-status-orange: #FF8E00;
-@maintenance-black: #000000;
+@maintenance-black: #000;
/************************************************************************
* Health status(service/host/host component health)icon colors ends
***********************************************************************/
@@ -176,21 +181,21 @@
animation-iteration-count: infinite;
}
-.infoPulse {
+.infoPulse() {
.pulse(bluePulse);
background-color: #006DCC;
}
-.infoPulseInner {
+.infoPulseInner() {
.pulse(bluePulseInner);
}
-.warnPulse {
+.warnPulse() {
.pulse(orangePulse);
background-color: @health-status-orange;
}
-.dangerPulse {
+.dangerPulse() {
.pulse(redPulse);
background-color: @health-status-red;
}
@@ -209,286 +214,3 @@
@modal-header-height: 50px;
// modal footer height
@modal-footer-height: 60px;
-
-.editable-list-container.well{
- padding: 10px;
- position: relative;
- margin-bottom: 25px;
- cursor: pointer;
-}
-.editable-list-container.well.edit-mode{
- cursor: default;
-}
-.editable-list-container.well.disabled{
- background: white;
-}
-
-.editable-list-container .items-box{
-
-}
-.editable-list-container .items-box ul.items-list{
- list-style-type: none;
- margin: 0;
- padding: 0;
-}
-
-.editable-list-container .items-box ul.items-list li.item{
- display: inline-block;
- padding: 4px 8px;
- margin: 0 5px 5px 2px;
- background: white;
- border: 1px solid #ebebeb;
- max-width: 100%;
- white-space: nowrap;
- position: relative;
- a {
- color: #555;
- }
-}
-
-.editable-list-container.edit-mode .items-box ul.items-list li.item.deletable{
- padding-right: 25px;
-}
-.editable-list-container .items-box ul.items-list li.item.global{
- background-color: #f5f5f5;
-}
-
-.editable-list-container .items-box ul.items-list li.item .close{
- margin: -2px 0 0 5px;
- width: 13px;
- outline: none;
- position: absolute;
- display: none;
-}
-.editable-list-container.edit-mode .items-box ul.items-list li.item .close{
- display: inline-block;
-}
-
-.editable-list-container .items-box ul.items-list li a{
- text-decoration: none;
-}
-
-.editable-list-container.edit-mode .items-box ul.items-list li.item.add-item-input .ember-text-field{
- display: inline-block!important;
- text-overflow: ellipsis;
- white-space: nowrap;
- position: relative;
- padding: 0 0;
- margin: 0 0;
- border: none;
- width: 50px;
- height: 18px;
- -webkit-box-shadow: none;
- -moz-box-shadow: none;
- box-shadow: none;
- -webkit-transition: none;
- -moz-transition: none;
- -o-transition: none;
- transition: none;
-}
-
-.editable-list-container .items-box ul.items-list li.item.add-item{
- color: #ddd;
-}
-
-.typeahead-box{
- position: absolute;
- left: 0;
- margin-top: 5px;
- background: white;
- border: 1px solid #ebebeb;
- z-index: 1000;
- min-width: 65px;
- max-height: 150px;
- overflow: auto;
-}
-.typeahead-box ul{
- list-style-type: none;
- margin: 0;
- padding: 0;
-}
-.typeahead-box ul li{
- padding: 3px 5px;
- display: block;
- cursor: pointer;
-}
-
-.typeahead-box ul li.selected, .typeahead-box ul li:hover{
- background: #eee;
-}
-
-.editable-list-container .actions-panel{
- position: absolute;
- right: 5px;
- bottom: -30px;
- padding: 2px 5px 5px 5px;
- background: #f5f5f5;
- border: 1px solid #e3e3e3;
- border-top: none;
- border-radius: 0 0 4px 4px;
-
- -webkit-transition: all 0.3s;
- -o-transition: all 0.3s;
- transition: all 0.3s;
-
- -ms-transform-origin: 0% 0%; /* IE 9 */
- -webkit-transform-origin: 0% 0%; /* Chrome, Safari, Opera */
- transform-origin: 0% 0%;
-
- -webkit-transform: rotateX(0deg);
- -ms-transform: rotateX(0deg);
- -o-transform: rotateX(0deg);
- transform: rotateX(0deg);
-
- .btn-xs {
- padding: 1px 5px;
- font-size: 12px;
- line-height: 1.5;
- border-radius: 3px;
- }
- .glyphicon-remove {
- color: #000;
- }
- .glyphicon-ok {
- color: white;
- }
-}
-
-.editable-list-container.disabled .pencil-box{
- display: none;
-}
-.editable-list-container .pencil-box{
- position: absolute;
- right: 5px;
- top: 5px;
- opacity: 0;
- -webkit-transition: all 0.3s;
- -o-transition: all 0.3s;
- transition: all 0.3s;
-}
-.editable-list-container:hover .pencil-box{
- opacity: 1;
-}
-.editable-list-container.edit-mode:hover .pencil-box{
- opacity: 0;
-}
-
-.table-overlay {
- position: absolute;
- opacity: 0.9;
- background-color: #fefefe;
- .spinner {
- display: none;
- }
-}
-
-.host-elements {
- border: 1px solid #ddd;
- height: 120px;
- overflow: auto;
- padding: 5px;
- ul {
- margin-bottom: 0;
- }
- table {
- width: 100%;
- td {
- width: 50%;
- padding: 0;
- }
- }
-}
-
-.export-graph-list {
- top: 25px;
- min-width: 60px;
- font-size: 14px;
- cursor: auto;
- a {
- cursor: pointer;
- }
-}
-
-.lh-btn {
- line-height: 30px;
-}
-
-.text-bold {
- font-weight: bold;
-}
-
-.pre-styled {
- display: block;
- padding: 9.5px;
- margin: 0 0 10px;
- font-size: 11px;
- line-height: 14px;
- font-family: monospace;
- word-break: break-all;
- word-wrap: break-word;
- white-space: pre-wrap;
- background-color: #f5f5f5;
- border: 1px solid rgba(0, 0, 0, 0.15);
- -webkit-border-radius: 4px;
- -moz-border-radius: 4px;
- border-radius: 4px;
-}
-
-.clear {
- clear: both;
-}
-
-.noDisplay {
- display: none !important;
-}
-
-.display {
- display: block !important;
-}
-
-.display-inline-block {
- display: inline-block;
-}
-
-.popover {
- small {
- font-size: 1.3rem;
- }
-}
-
-.form-group {
- .help-block {
- &.help-inline {
- margin-bottom: 0;
- }
- }
-}
-
-.alert {
- & > ul, ol {
- padding-left: inherit;
- }
-}
-
-.dropdown-toggle.btn-icon {
- padding-left: 8px;
- padding-right: 8px;
-}
-
-::-webkit-scrollbar {
- -webkit-appearance: none;
-}
-
-::-webkit-scrollbar:vertical {
- width: 11px;
-}
-
-::-webkit-scrollbar:horizontal {
- height: 11px;
-}
-
-::-webkit-scrollbar-thumb {
- border-radius: 8px;
- border: 2px solid white;
- background-color: rgba(0, 0, 0, .5);
-}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/styles/config_history_flow.less
----------------------------------------------------------------------
diff --git a/ambari-web/app/styles/config_history_flow.less b/ambari-web/app/styles/config_history_flow.less
index dae2709..e3fe0e7 100644
--- a/ambari-web/app/styles/config_history_flow.less
+++ b/ambari-web/app/styles/config_history_flow.less
@@ -401,9 +401,6 @@
}
}
-.version-in-dropdown [class*="span"] {
- min-height: 24px;
-}
// Firefox specific styles
@-moz-document url-prefix() {
@@ -416,7 +413,4 @@
}
}
}
- .version-in-dropdown [class*="span"] {
- min-height: 30px;
- }
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/styles/modal_popups.less
----------------------------------------------------------------------
diff --git a/ambari-web/app/styles/modal_popups.less b/ambari-web/app/styles/modal_popups.less
index 4c2ba8c..48c02b5 100644
--- a/ambari-web/app/styles/modal_popups.less
+++ b/ambari-web/app/styles/modal_popups.less
@@ -192,9 +192,6 @@
clear:both;
}
.override-controls{
- .glyphicon-ok-sign {
- color: #5AB400;
- }
#filter-dropdown-div {
.dropdown-menu {
right: 0;
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/styles/stack_versions.less
----------------------------------------------------------------------
diff --git a/ambari-web/app/styles/stack_versions.less b/ambari-web/app/styles/stack_versions.less
index e014958..aca9c65 100644
--- a/ambari-web/app/styles/stack_versions.less
+++ b/ambari-web/app/styles/stack_versions.less
@@ -107,8 +107,6 @@
}
.textfield-error input {
border-color: #b94a48;
- -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
- -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
}
.disabled-label {
@@ -278,22 +276,22 @@
.version-column {
min-height: 500px;
width: 115px;
- margin: 15px 7px 0px 7px;
+ margin: 15px 7px 0 7px;
.version-column-display-name {
color: #000;
text-align: center;
font-weight: 500;
font-size: 16px;
line-height: 18px;
- padding: 5px 5px;
- margin-bottom: 0px;
+ padding: 5px;
+ margin-bottom: 0;
}
.version-column-show-details {
text-align: center;
font-size: 13px;
- padding: 5px 5px;
+ padding: 5px;
border-bottom: 1px solid #e4e4e4;
- margin-bottom: 0px;
+ margin-bottom: 0;
a {
cursor: pointer;
}
@@ -335,7 +333,7 @@
height: 2px;
background: #e4e4e4;
position: relative;
- margin: 0px 0px 0px -15px;
+ margin: 0 0 0 -15px;
}
.line-separator {
top: -7px;
@@ -435,7 +433,7 @@
.repository-list {
.os-block {
- border-top: 1px solid #dddddd;
+ border-top: 1px solid #ddd;
padding-top: 10px;
}
input[type="text"] {
@@ -473,7 +471,7 @@
.ROLLING, .NON_ROLLING {
background-color: #e4e4e4;
.method-name,.method-icon,.method-description {
- color: #333333;
+ color: #333;
}
}
.ROLLING {
@@ -578,7 +576,7 @@
.admin-cluster {
.repositories-table {
margin-bottom: 10px;
- border: 1px solid #dddddd;
+ border: 1px solid #ddd;
overflow: auto;
div {
float: left;
@@ -604,7 +602,7 @@
width: 100%;
.trow {
width: 100%;
- border-top: 1px solid #dddddd;
+ border-top: 1px solid #ddd;
padding-top: 8px;
.os-td {
padding-top: 4px;
@@ -660,8 +658,6 @@
}
.textfield-error input {
border-color: #b94a48;
- -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
- -moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
}
.disabled-textfield input {
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/styles/widgets.less
----------------------------------------------------------------------
diff --git a/ambari-web/app/styles/widgets.less b/ambari-web/app/styles/widgets.less
index 98d4fb0..af76e4f 100644
--- a/ambari-web/app/styles/widgets.less
+++ b/ambari-web/app/styles/widgets.less
@@ -244,10 +244,10 @@
}
&.slider-disabled {
.tooltip-arrow {
- border-top-color: lighten(@green, 20%);;
+ border-top-color: lighten(@green, 20%);
}
.tooltip-inner {
- background-color: lighten(@green, 20%);;
+ background-color: lighten(@green, 20%);
}
}
}
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/common/configs/selectCreateConfigGroup.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/common/configs/selectCreateConfigGroup.hbs b/ambari-web/app/templates/common/configs/selectCreateConfigGroup.hbs
index 47dcb2a..ef731b3 100644
--- a/ambari-web/app/templates/common/configs/selectCreateConfigGroup.hbs
+++ b/ambari-web/app/templates/common/configs/selectCreateConfigGroup.hbs
@@ -32,7 +32,7 @@
</div>
{{#if view.parentView.availableConfigGroups.length}}
<div class="select-create-config-group-div">
- <span class="btn-group">
+ <div class="btn-group">
<button class="btn btn-default" {{bindAttr disabled="view.parentView.optionCreateConfigGroup"}}>{{view.parentView.selectedConfigGroup.name}}</button>
<button class="btn btn-default dropdown-toggle btn-icon" data-toggle="dropdown" {{bindAttr disabled="view.parentView.optionCreateConfigGroup"}}>
<span class="caret"></span>
@@ -47,7 +47,7 @@
</li>
{{/each}}
</ul>
- </span>
+ </div>
<div class="message">{{t config.group.selection.dialog.option.select.msg}}</div>
</div>
{{else}}
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/common/host_progress_popup.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/common/host_progress_popup.hbs b/ambari-web/app/templates/common/host_progress_popup.hbs
index 4536ce3..0a7592b 100644
--- a/ambari-web/app/templates/common/host_progress_popup.hbs
+++ b/ambari-web/app/templates/common/host_progress_popup.hbs
@@ -265,8 +265,8 @@
{{#each hostLog in view.hostComponentLogs}}
<div {{bindAttr class=":tab-pane :log-component-tab hostLog.tabClassName"}}>
<p {{bindAttr class="view.isClipBoardActive:hidden"}}>
- <span class="text-bold">{{t common.file}}: </span>
- <span class="text-bold muted">{{hostLog.fileName}}</span>
+ <strong>{{t common.file}}: </strong>
+ <strong class="muted">{{hostLog.fileName}}</strong>
{{#view App.LogSearchUILinkView linkQueryParamsBinding="hostLog.linkTail" tagName="span"}}
<a {{bindAttr href="view.formatedLink" class=":pull-right view.isLodaded::disabled"}} target="_blank">
<i class="icon-external-link"></i>
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/common/modal_popups/log_tail_popup.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/common/modal_popups/log_tail_popup.hbs b/ambari-web/app/templates/common/modal_popups/log_tail_popup.hbs
index 48e5b9d..1b0a6d0 100644
--- a/ambari-web/app/templates/common/modal_popups/log_tail_popup.hbs
+++ b/ambari-web/app/templates/common/modal_popups/log_tail_popup.hbs
@@ -18,7 +18,7 @@
<div class="top-wrap top-wrap-header">
<div class="log-tail-popup-info">
- <span class="text-bold">{{t common.file}}: </span>
+ <strong>{{t common.file}}: </strong>
<span class="muted">{{view.content.filePath}}</span>
<div class="pull-right action-bar">
<a href="#" {{action toggleCopy target="view"}}>
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/login.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/login.hbs b/ambari-web/app/templates/login.hbs
index c17c0fe..ed16734 100644
--- a/ambari-web/app/templates/login.hbs
+++ b/ambari-web/app/templates/login.hbs
@@ -17,7 +17,7 @@
}}
{{#unless isServerClientVersionMismatch}}
-<div class="well login col-md-4 col-md-offset-4">
+<div class="well login col-md-4 col-md-offset-4 col-sm-offset-4">
<h2>{{t login.header}}</h2>
{{#if errorMessage}}
<div class="alert alert-danger">
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/main/alerts/definition_details.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/main/alerts/definition_details.hbs b/ambari-web/app/templates/main/alerts/definition_details.hbs
index 7e2e2ed..5789de6 100644
--- a/ambari-web/app/templates/main/alerts/definition_details.hbs
+++ b/ambari-web/app/templates/main/alerts/definition_details.hbs
@@ -25,10 +25,10 @@
{{! Alert Definition Name }}
<div class="definition-name row">
{{#if controller.editing.label.isEditing}}
- <div {{bindAttr class="controller.editing.label.isError:error :col-md-5 :name-text-field :form-group"}}>
+ <div {{bindAttr class="controller.editing.label.isError:error :col-md-8 :name-text-field"}}>
{{view Em.TextField valueBinding="controller.editing.label.value" maxlength="255" class="form-control"}}
</div>
- <div class="edit-buttons">
+ <div class="edit-buttons align-right col-md-4">
<button {{action cancelEdit controller.editing.label target="controller"}}
class="btn btn-default cancel-button">{{t common.cancel}}</button>
<button {{bindAttr disabled="controller.editing.label.isError"}} {{action saveEdit controller.editing.label target="controller"}}
@@ -55,7 +55,7 @@
</div>
</div>
- <div class="status col-md-4">
+ <div class="status align-right col-md-4">
{{view App.AlertDefinitionSummary contentBinding="view.controller.content"}}
</div>
</div>
@@ -85,7 +85,7 @@
<div class="panel-body">
{{view App.AlertDefinitionConfigsView contentBinding="view.controller.content" alertDefinitionTypeBinding="view.controller.content.type" canEdit=false}}
{{#if App.router.mainAlertDefinitionConfigsController.canEdit}}
- <div class="edit-buttons">
+ <div class="edit-buttons align-right">
<button {{action cancelEditConfigs target="App.router.mainAlertDefinitionConfigsController"}}
class="btn btn-default">{{t common.cancel}}</button>
<button {{bindAttr disabled="App.router.mainAlertDefinitionConfigsController.hasErrors"}} {{action saveConfigs target="App.router.mainAlertDefinitionConfigsController"}}
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/main/dashboard/plus_button_filter.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/main/dashboard/plus_button_filter.hbs b/ambari-web/app/templates/main/dashboard/plus_button_filter.hbs
index a8792fe..df7714c 100644
--- a/ambari-web/app/templates/main/dashboard/plus_button_filter.hbs
+++ b/ambari-web/app/templates/main/dashboard/plus_button_filter.hbs
@@ -16,15 +16,12 @@
* limitations under the License.
}}
-<ul class="dropdown-menu" >
{{#if view.hiddenWidgets.length}}
<li class="add-widgets-dropdown-list">
<ul>
{{#each widget in view.hiddenWidgets}}
<li>
- <div class="checkbox">
- {{view view.widgetCheckbox checkedBinding="widget.checked" labelBinding="widget.displayName"}}
- </div>
+ {{view view.widgetCheckbox class="checkbox" checkedBinding="widget.checked" labelBinding="widget.displayName"}}
</li>
{{/each}}
</ul>
@@ -36,11 +33,10 @@
{{else}}
<li>
<ul>
- <li>{{t dashboard.widgets.nothing}}</li>
+ <li>{{t dashboard.widgets.nothing}}</li>
</ul>
</li>
<li>
<button class="btn btn-default" {{action "closeFilter" target="view"}}>{{t common.cancel}}</button>
</li>
{{/if}}
-</ul>
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/main/dashboard/widgets/yarn_links.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/main/dashboard/widgets/yarn_links.hbs b/ambari-web/app/templates/main/dashboard/widgets/yarn_links.hbs
index e346abd..a8612f7 100644
--- a/ambari-web/app/templates/main/dashboard/widgets/yarn_links.hbs
+++ b/ambari-web/app/templates/main/dashboard/widgets/yarn_links.hbs
@@ -48,7 +48,7 @@
{{#if view.model.quickLinks.length}}
{{#view App.QuickLinksView contentBinding="view.model"}}
<div class="btn-group">
- <button class="btn btn-xs dropdown-toggle" data-toggle="dropdown" href="#">
+ <button class="btn btn-default btn-xs dropdown-toggle" data-toggle="dropdown" href="#">
{{t common.more}}
<span class="caret"></span>
</button>
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/main/host/summary.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/main/host/summary.hbs b/ambari-web/app/templates/main/host/summary.hbs
index 98619be..5d0b09c 100644
--- a/ambari-web/app/templates/main/host/summary.hbs
+++ b/ambari-web/app/templates/main/host/summary.hbs
@@ -164,7 +164,7 @@
<div class="col-md-4 summary-label summary-rack-label">{{t common.rack}}:</div>
<div class="col-md-8 summary-rack-value"> {{formatWordBreak view.content.rack}}
{{#isAuthorized "HOST.TOGGLE_MAINTENANCE"}}
- <a class="set-rack-id" {{action setRackId view.content target="controller"}}><i
+ <a href="#" class="set-rack-id" {{action setRackId view.content target="controller"}}><i
class="glyphicon glyphicon-pencil"></i></a>
{{/isAuthorized}}
</div>
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/main/service/manage_configuration_groups_popup.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/main/service/manage_configuration_groups_popup.hbs b/ambari-web/app/templates/main/service/manage_configuration_groups_popup.hbs
index fe6bf8a..12b08d7 100644
--- a/ambari-web/app/templates/main/service/manage_configuration_groups_popup.hbs
+++ b/ambari-web/app/templates/main/service/manage_configuration_groups_popup.hbs
@@ -67,15 +67,15 @@
</div>
</div>
<div class="row">
- <div class="col-md-2 col-lg-2">{{t common.overrides}}</div>
- <div class="col-md-10 col-lg-10">
+ <div class="col-md-3 col-lg-3 align-right">{{t common.overrides}}</div>
+ <div class="col-md-9 col-lg-9">
<a href="#" class="properties-link" {{action showProperties target="controller"}}
{{bindAttr data-original-title="tooltipText" }}>{{selectedConfigGroup.properties.length}} {{t common.properties}}</a>
</div>
</div>
<div class="row">
- <div class="col-md-2 col-lg-2"><span class="prrl">{{t common.description}}</span></div>
- <div class="col-md-10 col-lg-10 group-description">{{selectedConfigGroup.description}}</div>
+ <div class="col-md-3 col-lg-3 align-right">{{t common.description}}</div>
+ <div class="col-md-9 col-lg-9 group-description">{{selectedConfigGroup.description}}</div>
</div>
</div>
</div>
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/main/service/services/flume.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/main/service/services/flume.hbs b/ambari-web/app/templates/main/service/services/flume.hbs
index d9f053a..1e57cd5 100644
--- a/ambari-web/app/templates/main/service/services/flume.hbs
+++ b/ambari-web/app/templates/main/service/services/flume.hbs
@@ -22,7 +22,7 @@
<a href="#" {{action filterHosts view.flumeHandlerComponent}}>{{view.summaryHeader}}</a>
<a href="#" class="pull-right" {{action gotoConfigs target="controller"}}>{{t dashboard.services.flume.summary.configure}}</a>
<div id="flume-agent-table-wrap" class="scrollable-container">
- <table class="table table-hover table-bordered table-striped" id="flume-agents-table">
+ <table class="table table-hover advanced-header-table table-bordered table-striped" id="flume-agents-table">
<thead>
{{#view view.sortView contentBinding="view.filteredContent" class="label-row"}}
{{view view.parentView.hostSort}}
@@ -130,7 +130,7 @@
{{/each}}
{{else}}
<tr>
- <td colspan="6" id="flume-agents-empty-label" class="empty-label">
+ <td colspan="5" id="flume-agents-empty-label" class="empty-label">
{{t services.service.summary.flume.noAgents}}
</td>
</tr>
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/main/views.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/main/views.hbs b/ambari-web/app/templates/main/views.hbs
index 7fb3bf8..299c98c 100644
--- a/ambari-web/app/templates/main/views.hbs
+++ b/ambari-web/app/templates/main/views.hbs
@@ -25,11 +25,10 @@
{{#if instance.visible}}
<tr {{action "setView" instance target="controller"}}>
<td class="icon">
- <a class="view-icon-image" target="_blank"><img {{bindAttr src="instance.iconPath"}}></a>
+ <a class="view-icon-image"><img {{bindAttr src="instance.iconPath"}}></a>
</td>
<td class="label-version-description">
- <p class="view-label-text">{{instance.label}}</p>
- <p class="view-version-text"> ({{instance.version}})</p> <br />
+ <h4 class="view-label-text">{{instance.label}} <small class="view-version-text"> ({{instance.version}})</small></h4>
<p class="view-description-text">{{instance.description}}</p>
</td>
</tr>
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/templates/wizard/step3.hbs
----------------------------------------------------------------------
diff --git a/ambari-web/app/templates/wizard/step3.hbs b/ambari-web/app/templates/wizard/step3.hbs
index 07cf7f2..64c76ef 100644
--- a/ambari-web/app/templates/wizard/step3.hbs
+++ b/ambari-web/app/templates/wizard/step3.hbs
@@ -30,13 +30,13 @@
<button class="btn btn-primary step3-remove-selected-btn" {{bindAttr disabled="view.noHostsSelected"}}
{{action removeSelectedHosts target="controller" }}>
<i class="glyphicon glyphicon-trash glyphicon-white"></i>
- {{t installer.step3.removeSelected}}
+ <span class="hidden-sm">{{t installer.step3.removeSelected}}</span>
</button>
{{/unless}}
{{#unless isRetryDisabled}}
<a class="btn btn-primary decommission" href="#" {{action retrySelectedHosts target="view"}}>
<i class="glyphicon glyphicon-repeat glyphicon-white"></i>
- {{t installer.step3.retryFailed}}
+ <span class="hidden-sm">{{t installer.step3.retryFailed}}</span>
</a>
{{/unless}}
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/app/views/main/dashboard/widgets.js
----------------------------------------------------------------------
diff --git a/ambari-web/app/views/main/dashboard/widgets.js b/ambari-web/app/views/main/dashboard/widgets.js
index 57f5f50..f4603a2 100644
--- a/ambari-web/app/views/main/dashboard/widgets.js
+++ b/ambari-web/app/views/main/dashboard/widgets.js
@@ -228,6 +228,8 @@ App.MainDashboardWidgetsView = Em.View.extend(App.UserPref, App.LocalStorage, Ap
* @class
*/
plusButtonFilterView: Ember.View.extend({
+ tagName: 'ul',
+ classNames: ['dropdown-menu'],
templateName: require('templates/main/dashboard/plus_button_filter'),
hiddenWidgetsBinding: 'parentView.hiddenWidgets',
visibleWidgetsBinding: 'parentView.visibleWidgets',
http://git-wip-us.apache.org/repos/asf/ambari/blob/07cde1d7/ambari-web/test/mixins/common/hosts/host_component_recommendation_mixin_test.js
----------------------------------------------------------------------
diff --git a/ambari-web/test/mixins/common/hosts/host_component_recommendation_mixin_test.js b/ambari-web/test/mixins/common/hosts/host_component_recommendation_mixin_test.js
index 2e95a06..15bddd0 100644
--- a/ambari-web/test/mixins/common/hosts/host_component_recommendation_mixin_test.js
+++ b/ambari-web/test/mixins/common/hosts/host_component_recommendation_mixin_test.js
@@ -17,6 +17,7 @@
*/
var App = require('app');
+var helpers = require('test/helpers');
describe('App.HostComponentRecommendationMixin', function() {
var mixedObject;
@@ -25,7 +26,7 @@ describe('App.HostComponentRecommendationMixin', function() {
});
describe('#formatRecommendComponents', function() {
- var cases = [
+ [
{
components: undefined,
hosts: [],
@@ -51,9 +52,7 @@ describe('App.HostComponentRecommendationMixin', function() {
Em.Object.create({ componentName: 'C2', hostName: 'h3'}),
]
}
- ];
-
- cases.forEach(function(test) {
+ ].forEach(function(test) {
it(test.m, function() {
expect(mixedObject.formatRecommendComponents(test.components, test.hosts)).to.be.eql(test.e);
});
@@ -61,7 +60,6 @@ describe('App.HostComponentRecommendationMixin', function() {
});
describe('#getRecommendedHosts', function() {
- var cases;
beforeEach(function() {
sinon.stub(mixedObject, 'loadComponentsRecommendationsFromServer');
sinon.stub(mixedObject, 'getRecommendationRequestData');
@@ -74,7 +72,7 @@ describe('App.HostComponentRecommendationMixin', function() {
mixedObject.formatRecommendComponents.restore();
});
- cases = [
+ [
{
opts: null,
e: {
@@ -100,9 +98,7 @@ describe('App.HostComponentRecommendationMixin', function() {
},
m: 'should normaly merge defaults with passed options'
}
- ];
-
- cases.forEach(function(test) {
+ ].forEach(function(test) {
it(test.m, function() {
mixedObject.getRecommendedHosts(test.opts);
expect(mixedObject.getRecommendationRequestData.args[0][0]).to.be.eql(test.e);
@@ -113,7 +109,8 @@ describe('App.HostComponentRecommendationMixin', function() {
describe('#loadComponentsRecommedationsFromServer', function() {
it('default request options checking', function() {
mixedObject.loadComponentsRecommendationsFromServer('someData');
- expect(App.ajax.send.args[0][0]).to.be.eql({
+ var args = helpers.findAjaxRequest('name', 'wizard.loadrecommendations');
+ expect(args[0]).to.be.eql({
name: 'wizard.loadrecommendations',
sender: mixedObject,
data: 'someData',
@@ -124,14 +121,13 @@ describe('App.HostComponentRecommendationMixin', function() {
});
describe('#getRecommendationRequestData', function() {
- var cases;
beforeEach(function() {
sinon.stub(App, 'get').withArgs('stackVersionURL').returns('/stack/url');
});
afterEach(function() {
App.get.restore();
});
- cases = [
+ [
{
options: {
hosts: ['h1'],
@@ -177,9 +173,7 @@ describe('App.HostComponentRecommendationMixin', function() {
},
m: 'when blueprint passed it should be used instead of generated blueprint'
}
- ];
-
- cases.forEach(function(test) {
+ ].forEach(function(test) {
it(test.m, function() {
expect(mixedObject.getRecommendationRequestData(test.options)).to.be.eql(test.e);
});
[11/31] ambari git commit: AMBARI-18878. After entering wrong admin
kdc credentials UI does not handle it and shows HTTP error 500 (intermittent)
(dlysnichenko)
Posted by nc...@apache.org.
AMBARI-18878. After entering wrong admin kdc credentials UI does not handle it and shows HTTP error 500 (intermittent) (dlysnichenko)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/3c7d8bda
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/3c7d8bda
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/3c7d8bda
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 3c7d8bda6f0f481fbb7743aacddaf64d635ebd1f
Parents: 4665c81
Author: Lisnichenko Dmitro <dl...@hortonworks.com>
Authored: Tue Nov 15 13:13:34 2016 +0200
Committer: Lisnichenko Dmitro <dl...@hortonworks.com>
Committed: Tue Nov 15 13:13:34 2016 +0200
----------------------------------------------------------------------
.../server/serveraction/kerberos/MITKerberosOperationHandler.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/3c7d8bda/ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java b/ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
index a21d330..32aa19d 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
@@ -502,7 +502,7 @@ public class MITKerberosOperationHandler extends KerberosOperationHandler {
} else if (stdErr.contains("Cannot find KDC for requested realm while initializing kadmin interface")) {
throw new KerberosRealmException(stdErr);
} else {
- throw new KerberosOperationException("Unexpected error condition executing the kadmin command");
+ throw new KerberosOperationException(String.format("Unexpected error condition executing the kadmin command. STDERR: %s", stdErr));
}
}
[26/31] ambari git commit: AMBARI-18876. Remove old UpgradeCatalog
and tests (aonishuk)
Posted by nc...@apache.org.
http://git-wip-us.apache.org/repos/asf/ambari/blob/7cf4d866/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog161.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog161.java b/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog161.java
deleted file mode 100644
index 528c6b1..0000000
--- a/ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog161.java
+++ /dev/null
@@ -1,343 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ambari.server.upgrade;
-
-import java.sql.Connection;
-import java.sql.ResultSet;
-import java.sql.SQLException;
-import java.sql.Statement;
-import java.sql.Types;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-import javax.persistence.EntityManager;
-import javax.persistence.TypedQuery;
-
-import org.apache.ambari.server.AmbariException;
-import org.apache.ambari.server.configuration.Configuration.DatabaseType;
-import org.apache.ambari.server.orm.DBAccessor;
-import org.apache.ambari.server.orm.DBAccessor.DBColumnInfo;
-import org.apache.ambari.server.orm.entities.ClusterEntity;
-import org.apache.ambari.server.state.State;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import com.google.inject.Inject;
-import com.google.inject.Injector;
-
-/**
- * Upgrade catalog for version 1.6.1.
- */
-public class UpgradeCatalog161 extends AbstractUpgradeCatalog {
-
- //SourceVersion is only for book-keeping purpos
- @Override
- public String getSourceVersion() {
- return "1.6.0";
- }
-
-
- @Override
- public String getTargetVersion() {
- return "1.6.1";
- }
-
- /**
- * Logger.
- */
- private static final Logger LOG = LoggerFactory.getLogger
- (UpgradeCatalog161.class);
-
- // ----- Constructors ------------------------------------------------------
-
- @Inject
- public UpgradeCatalog161(Injector injector) {
- super(injector);
- }
-
-
- // ----- AbstractUpgradeCatalog --------------------------------------------
-
- @Override
- protected void executeDDLUpdates() throws AmbariException, SQLException {
- DatabaseType databaseType = configuration.getDatabaseType();
-
- List<DBColumnInfo> columns;
-
- // Operation level
- columns = new ArrayList<DBColumnInfo>();
- columns.add(new DBColumnInfo("operation_level_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("request_id", Long.class, null, null, false));
- columns.add(new DBColumnInfo("level_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("cluster_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("service_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("host_component_name", String.class, 255, null, true));
- columns.add(new DBColumnInfo("host_name", String.class, 255, null, true));
-
- dbAccessor.createTable("requestoperationlevel", columns, "operation_level_id");
-
- // 1.6.0 initially shipped with restart_required as a BOOELAN so some
- // upgrades might be BOOLEAN but most are probably SMALLINT
- if (databaseType == DatabaseType.POSTGRES) {
- int columnType = dbAccessor.getColumnType("hostcomponentdesiredstate",
- "restart_required");
-
- if (columnType == Types.BOOLEAN || columnType == Types.BIT) {
- dbAccessor.executeQuery(
- "ALTER TABLE hostcomponentdesiredstate ALTER column restart_required TYPE SMALLINT USING CASE WHEN restart_required=true THEN 1 ELSE 0 END",
- true);
- }
- }
-
- if (databaseType == DatabaseType.ORACLE) {
- dbAccessor.executeQuery(
- "ALTER TABLE hostcomponentdesiredstate MODIFY (restart_required DEFAULT 0)",
- true);
-
- } else {
- dbAccessor.executeQuery(
- "ALTER TABLE hostcomponentdesiredstate ALTER column restart_required SET DEFAULT 0",
- true);
- }
-
- //=========================================================================
- // Add columns
- dbAccessor.addColumn("viewmain",
- new DBAccessor.DBColumnInfo("icon", String.class, 255, null, true));
-
- dbAccessor.addColumn("viewmain",
- new DBAccessor.DBColumnInfo("icon64", String.class, 255, null, true));
-
- dbAccessor.addColumn("viewinstancedata",
- new DBAccessor.DBColumnInfo("user_name", String.class, 255, " ", false));
-
- dbAccessor.dropFKConstraint("viewinstancedata", "FK_viewinstdata_view_name");
- dbAccessor.dropFKConstraint("viewinstanceproperty", "FK_viewinstprop_view_name");
- dbAccessor.dropFKConstraint("viewentity", "FK_viewentity_view_name");
- dbAccessor.dropFKConstraint("viewinstance", "FK_viewinst_view_name");
-
- //modify primary key of viewinstancedata
- if (databaseType == DatabaseType.ORACLE
- || databaseType == DatabaseType.MYSQL
- || databaseType == DatabaseType.DERBY) {
- dbAccessor.executeQuery("ALTER TABLE viewinstance DROP PRIMARY KEY", true);
- dbAccessor.executeQuery("ALTER TABLE viewinstancedata DROP PRIMARY KEY", true);
- } else if (databaseType == DatabaseType.POSTGRES) {
- dbAccessor.executeQuery("ALTER TABLE viewinstance DROP CONSTRAINT viewinstance_pkey CASCADE", true);
- dbAccessor.executeQuery("ALTER TABLE viewinstancedata DROP CONSTRAINT viewinstancedata_pkey CASCADE", true);
- }
-
-
- dbAccessor.addColumn("viewinstance", new DBAccessor.DBColumnInfo("view_instance_id", Long.class, null, null, true));
- dbAccessor.addColumn("viewinstancedata",
- new DBAccessor.DBColumnInfo("view_instance_id", Long.class, null, null, true));
-
- if (databaseType == DatabaseType.ORACLE) {
- //sequence looks to be simpler than rownum
- if (dbAccessor.tableHasData("viewinstancedata")) {
- dbAccessor.executeQuery("CREATE SEQUENCE TEMP_SEQ " +
- " START WITH 1 " +
- " MAXVALUE 999999999999999999999999999 " +
- " MINVALUE 1 " +
- " NOCYCLE " +
- " NOCACHE " +
- " NOORDER");
- dbAccessor.executeQuery("UPDATE viewinstance SET view_instance_id = TEMP_SEQ.NEXTVAL");
- dbAccessor.dropSequence("TEMP_SEQ");
- }
- } else if (databaseType == DatabaseType.MYSQL) {
- if (dbAccessor.tableHasData("viewinstance")) {
- dbAccessor.executeQuery("UPDATE viewinstance " +
- "SET view_instance_id = (SELECT @a := @a + 1 FROM (SELECT @a := 1) s)");
- }
- } else if (databaseType == DatabaseType.POSTGRES) {
- if (dbAccessor.tableHasData("viewinstance")) {
- //window functions like row_number were added in 8.4, workaround for earlier versions (redhat/centos 5)
- dbAccessor.executeQuery("CREATE SEQUENCE temp_seq START WITH 1");
- dbAccessor.executeQuery("UPDATE viewinstance SET view_instance_id = nextval('temp_seq')");
- dbAccessor.dropSequence("temp_seq");
- }
-
-
- }
-
- if (databaseType == DatabaseType.DERBY) {
- dbAccessor.executeQuery("ALTER TABLE viewinstance ALTER COLUMN view_instance_id DEFAULT 0");
- dbAccessor.executeQuery("ALTER TABLE viewinstance ALTER COLUMN view_instance_id NOT NULL");
- dbAccessor.executeQuery("ALTER TABLE viewinstancedata ALTER COLUMN view_instance_id DEFAULT 0");
- dbAccessor.executeQuery("ALTER TABLE viewinstancedata ALTER COLUMN view_instance_id NOT NULL");
- dbAccessor.executeQuery("ALTER TABLE viewinstancedata ALTER COLUMN user_name DEFAULT ' '");
- dbAccessor.executeQuery("ALTER TABLE viewinstancedata ALTER COLUMN user_name NOT NULL");
- }
-
- dbAccessor.executeQuery("alter table viewinstance add primary key (view_instance_id)");
- dbAccessor.executeQuery("ALTER TABLE viewinstance ADD CONSTRAINT UQ_viewinstance_name UNIQUE (view_name, name)", true);
- dbAccessor.executeQuery("ALTER TABLE viewinstance ADD CONSTRAINT UQ_viewinstance_name_id UNIQUE (view_instance_id, view_name, name)", true);
-
- dbAccessor.addFKConstraint("viewinstanceproperty", "FK_viewinstprop_view_name",
- new String[]{"view_name", "view_instance_name"}, "viewinstance", new String[]{"view_name", "name"}, true);
- dbAccessor.addFKConstraint("viewentity", "FK_viewentity_view_name",
- new String[]{"view_name", "view_instance_name"}, "viewinstance", new String[]{"view_name", "name"}, true);
- dbAccessor.addFKConstraint("viewinstance", "FK_viewinst_view_name", "view_name", "viewmain", "view_name", true);
-
- if (databaseType == DatabaseType.POSTGRES) {
- dbAccessor.executeQuery("UPDATE viewinstancedata " +
- "SET view_instance_id = vi.view_instance_id FROM viewinstance AS vi " +
- "WHERE vi.name = viewinstancedata.view_instance_name AND vi.view_name = viewinstancedata.view_name");
- } else if (databaseType == DatabaseType.ORACLE) {
- dbAccessor.executeQuery("UPDATE viewinstancedata vid SET view_instance_id = (" +
- "SELECT view_instance_id FROM viewinstance vi WHERE vi.name = vid.view_instance_name AND vi.view_name = vid.view_name)");
- } else if (databaseType == DatabaseType.MYSQL) {
- dbAccessor.executeQuery("UPDATE viewinstancedata AS vid JOIN viewinstance AS vi " +
- "ON vi.name = vid.view_instance_name AND vi.view_name = vid.view_name " +
- "SET vid.view_instance_id = vi.view_instance_id");
- }
-
- dbAccessor.executeQuery("alter table viewinstancedata add primary key (view_instance_id, name, user_name)");
-
- dbAccessor.addFKConstraint("viewinstancedata", "FK_viewinstdata_view_name", new String[]{"view_instance_id", "view_name", "view_instance_name"},
- "viewinstance", new String[]{"view_instance_id", "view_name", "name"}, true);
-
-
- long count = 1;
- Statement statement = null;
- ResultSet rs = null;
- try {
- statement = dbAccessor.getConnection().createStatement();
- if (statement != null) {
- rs = statement.executeQuery("SELECT count(*) FROM viewinstance");
- if (rs != null) {
- if (rs.next()) {
- count = rs.getLong(1) + 2;
- }
- }
- }
- } finally {
- if (rs != null) {
- rs.close();
- }
- if (statement != null) {
- statement.close();
- }
- }
-
- String valueColumnName = "\"value\"";
- if (databaseType == DatabaseType.ORACLE
- || databaseType == DatabaseType.MYSQL) {
- valueColumnName = "value";
- }
-
- dbAccessor.executeQuery("INSERT INTO ambari_sequences(sequence_name, " + valueColumnName + ") " +
- "VALUES('view_instance_id_seq', " + count + ")", true);
-
- dbAccessor.addColumn("viewinstance",
- new DBAccessor.DBColumnInfo("label", String.class, 255, null, true));
-
- dbAccessor.addColumn("viewinstance",
- new DBAccessor.DBColumnInfo("description", String.class, 255, null, true));
-
- dbAccessor.addColumn("viewinstance",
- new DBAccessor.DBColumnInfo("visible", Character.class, 1, null, true));
-
- dbAccessor.addColumn("viewinstance",
- new DBAccessor.DBColumnInfo("icon", String.class, 255, null, true));
-
- dbAccessor.addColumn("viewinstance",
- new DBAccessor.DBColumnInfo("icon64", String.class, 255, null, true));
-
- // ========================================================================
- // Add constraints
- dbAccessor.addFKConstraint("requestoperationlevel", "FK_req_op_level_req_id",
- "request_id", "request", "request_id", true);
-
- // Clusters
- dbAccessor.addColumn("clusters", new DBColumnInfo("provisioning_state", String.class, 255, State.INIT.name(), false));
-
- dbAccessor.dropFKConstraint("stage", "FK_stage_cluster_id", true);
- dbAccessor.dropFKConstraint("request", "FK_request_cluster_id", true);
- }
-
-
- // ----- UpgradeCatalog ----------------------------------------------------
- /**
- * {@inheritDoc}
- */
- @Override
- public void executePreDMLUpdates() {
- ;
- }
-
- @Override
- protected void executeDMLUpdates() throws AmbariException, SQLException {
- //add new sequences for operation level
- dbAccessor.executeQuery("INSERT INTO ambari_sequences(sequence_name, sequence_value) " +
- "VALUES('operation_level_id_seq', 1)", true);
-
- // upgrade cluster provision state
- executeInTransaction(new Runnable() {
- @Override
- public void run() {
- // it should be safe to bulk update the current cluster state since
- // this field is not currently used and since all clusters stored in
- // the database must (at this point) be installed
- final EntityManager em = getEntityManagerProvider().get();
- final TypedQuery<ClusterEntity> query = em.createQuery(
- "UPDATE ClusterEntity SET provisioningState = :provisioningState",
- ClusterEntity.class);
-
- query.setParameter("provisioningState", State.INSTALLED);
- final int updatedClusterProvisionedStateCount = query.executeUpdate();
-
- LOG.info("Updated {} cluster provisioning states to {}",
- updatedClusterProvisionedStateCount, State.INSTALLED);
- }
- });
-
- addMissingConfigs();
- }
-
- protected void addMissingConfigs() throws AmbariException {
- updateConfigurationProperties("hbase-site", Collections.singletonMap("hbase.regionserver.info.port", "60030"), false, false);
- updateConfigurationProperties("hbase-site", Collections.singletonMap("hbase.master.info.port", "60010"), false, false);
- updateConfigurationProperties("hive-site", Collections.singletonMap("hive.heapsize", "1024"), false, false);
- updateConfigurationProperties("pig-properties", Collections.singletonMap("pig-content", "\n# Licensed to the Apache " +
- "Software Foundation (ASF) under one\n# or more contributor license agreements. See the NOTICE file\n# " +
- "distributed with this work for additional information\n# regarding copyright ownership. The ASF " +
- "licenses this file\n# to you under the Apache License, Version 2.0 (the\n# \"License\"); you may " +
- "not use this file except in compliance\n# with the License. You may obtain a copy of the License " +
- "at\n#\n#http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to " +
- "in writing,\n# software distributed under the License is distributed on an\n# \"AS IS\" BASIS, WITHOUT " +
- "WARRANTIES OR CONDITIONS OF ANY\n# KIND, either express or implied. See the License for the\n# " +
- "specific language governing permissions and limitations\n# under the License.\n\n# Pig default " +
- "configuration file. All values can be overwritten by pig.properties and command line arguments.\n# " +
- "see bin/pig -help\n\n# brief logging (no timestamps)\nbrief=false\n\n# debug level, INFO is default" +
- "\ndebug=INFO\n\n# verbose print all log messages to screen (default to print only INFO and above to " +
- "screen)\nverbose=false\n\n# exectype local|mapreduce, mapreduce is default\nexectype=mapreduce\n\n# " +
- "Enable insertion of information about script into hadoop job conf \npig.script.info.enabled=true\n\n# " +
- "Do not spill temp files smaller than this size (bytes)\npig.spill.size.threshold=5000000\n\n# " +
- "EXPERIMENT: Activate garbage collection when spilling a file bigger than this size (bytes)\n# " +
- "This should help reduce the number of files being spilled.\npig.spill.gc.activation.size=40000000\n\n# " +
- "the following two parameters are to help estimate the reducer number\npig.exec.reducers.bytes.per." +
- "reducer=1000000000\npig.exec.reducers.max=999\n\n# Temporary location to store the intermediate " +
- "data.\npig.temp.dir=/tmp/\n\n# Threshold for merging FRJoin fragment files\npig.files.concatenation." +
- "threshold=100\npig.optimistic.files.concatenation=false;\n\npig.disable.counter=false\n\n" +
- "hcat.bin=/usr/bin/hcat"), true, false);
- }
-}
[07/31] ambari git commit: AMBARI-18829. Allow Ambari to manage
Kafka's Custom JAAS Config (smohanty)
Posted by nc...@apache.org.
AMBARI-18829. Allow Ambari to manage Kafka's Custom JAAS Config (smohanty)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/5a32eda8
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/5a32eda8
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/5a32eda8
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 5a32eda86a01c558dc191d5516d4d7b4e8051d57
Parents: ce46060
Author: Sumit Mohanty <sm...@hortonworks.com>
Authored: Mon Nov 14 21:04:41 2016 -0800
Committer: Sumit Mohanty <sm...@hortonworks.com>
Committed: Mon Nov 14 21:05:32 2016 -0800
----------------------------------------------------------------------
.../configuration/kafka_client_jaas_conf.xml | 41 ++++++++++++++
.../0.8.1/configuration/kafka_jaas_conf.xml | 59 ++++++++++++++++++++
.../common-services/KAFKA/0.8.1/metainfo.xml | 2 +
.../KAFKA/0.8.1/package/scripts/kafka.py | 12 ++++
.../KAFKA/0.8.1/package/scripts/params.py | 2 +
5 files changed, 116 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/5a32eda8/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka_client_jaas_conf.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka_client_jaas_conf.xml b/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka_client_jaas_conf.xml
new file mode 100644
index 0000000..164d91e
--- /dev/null
+++ b/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka_client_jaas_conf.xml
@@ -0,0 +1,41 @@
+<?xml version="1.0"?>
+<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+<!--
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+-->
+<configuration supports_final="false" supports_adding_forbidden="true">
+ <property>
+ <name>content</name>
+ <display-name>kafka_client_jaas template</display-name>
+ <description>Kafka client jaas config</description>
+ <value>
+KafkaClient {
+com.sun.security.auth.module.Krb5LoginModule required
+useTicketCache=true
+renewTicket=true
+serviceName="{{kafka_bare_jaas_principal}}";
+};
+ </value>
+ <value-attributes>
+ <type>content</type>
+ <show-property-name>false</show-property-name>
+ </value-attributes>
+ <on-ambari-upgrade add="true"/>
+ </property>
+</configuration>
http://git-wip-us.apache.org/repos/asf/ambari/blob/5a32eda8/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka_jaas_conf.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka_jaas_conf.xml b/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka_jaas_conf.xml
new file mode 100644
index 0000000..fdde8f2
--- /dev/null
+++ b/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka_jaas_conf.xml
@@ -0,0 +1,59 @@
+<?xml version="1.0"?>
+<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+<!--
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+-->
+<configuration supports_final="false" supports_adding_forbidden="true">
+ <property>
+ <name>content</name>
+ <display-name>kafka_jaas template</display-name>
+ <description>Kafka jaas config</description>
+ <value>
+KafkaServer {
+com.sun.security.auth.module.Krb5LoginModule required
+useKeyTab=true
+keyTab="{{kafka_keytab_path}}"
+storeKey=true
+useTicketCache=false
+serviceName="{{kafka_bare_jaas_principal}}"
+principal="{{kafka_jaas_principal}}";
+};
+KafkaClient {
+com.sun.security.auth.module.Krb5LoginModule required
+useTicketCache=true
+renewTicket=true
+serviceName="{{kafka_bare_jaas_principal}}";
+};
+Client {
+com.sun.security.auth.module.Krb5LoginModule required
+useKeyTab=true
+keyTab="{{kafka_keytab_path}}"
+storeKey=true
+useTicketCache=false
+serviceName="zookeeper"
+principal="{{kafka_jaas_principal}}";
+};
+ </value>
+ <value-attributes>
+ <type>content</type>
+ <show-property-name>false</show-property-name>
+ </value-attributes>
+ <on-ambari-upgrade add="true"/>
+ </property>
+</configuration>
http://git-wip-us.apache.org/repos/asf/ambari/blob/5a32eda8/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/metainfo.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/metainfo.xml b/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/metainfo.xml
index 8630c87..049705d 100644
--- a/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/metainfo.xml
+++ b/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/metainfo.xml
@@ -83,6 +83,8 @@
<config-type>zookeeper-env</config-type>
<config-type>zoo.cfg</config-type>
<config-type>ams-ssl-client</config-type>
+ <config-type>kafka_jaas_conf</config-type>
+ <config-type>kafka_client_jaas_conf</config-type>
</configuration-dependencies>
<osSpecifics>
<osSpecific>
http://git-wip-us.apache.org/repos/asf/ambari/blob/5a32eda8/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py b/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
index a53a547..3f7104a 100644
--- a/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
+++ b/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
@@ -133,9 +133,21 @@ def kafka(upgrade_type=None):
)
if params.security_enabled and params.kafka_kerberos_enabled:
+ if params.kafka_jaas_conf_template:
+ File(format("{conf_dir}/kafka_jaas.conf"),
+ owner=params.kafka_user,
+ content=InlineTemplate(params.kafka_jaas_conf_template)
+ )
+ else:
TemplateConfig(format("{conf_dir}/kafka_jaas.conf"),
owner=params.kafka_user)
+ if params.kafka_client_jaas_conf_template:
+ File(format("{conf_dir}/kafka_client_jaas.conf"),
+ owner=params.kafka_user,
+ content=InlineTemplate(params.kafka_client_jaas_conf_template)
+ )
+ else:
TemplateConfig(format("{conf_dir}/kafka_client_jaas.conf"),
owner=params.kafka_user)
http://git-wip-us.apache.org/repos/asf/ambari/blob/5a32eda8/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py b/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
index 107bf6e..0cb88fe 100644
--- a/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
+++ b/ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
@@ -93,6 +93,8 @@ kafka_managed_log_dir = "/var/log/kafka"
user_group = config['configurations']['cluster-env']['user_group']
java64_home = config['hostLevelParams']['java_home']
kafka_env_sh_template = config['configurations']['kafka-env']['content']
+kafka_jaas_conf_template = default("/configurations/kafka_jaas_conf/content", None)
+kafka_client_jaas_conf_template = default("/configurations/kafka_client_jaas_conf/content", None)
kafka_hosts = config['clusterHostInfo']['kafka_broker_hosts']
kafka_hosts.sort()
[13/31] ambari git commit: AMBARI-18825. Make agent to server connect
delay configurable (magyari_sandor)
Posted by nc...@apache.org.
AMBARI-18825. Make agent to server connect delay configurable (magyari_sandor)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/fcd0d2ad
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/fcd0d2ad
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/fcd0d2ad
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: fcd0d2adbfdce42fe63c4a1ad648595fd2f90bdb
Parents: 3809ebc
Author: Sandor Magyari <sm...@hortonworks.com>
Authored: Thu Nov 10 12:28:12 2016 +0100
Committer: Sandor Magyari <sm...@hortonworks.com>
Committed: Tue Nov 15 13:48:56 2016 +0100
----------------------------------------------------------------------
ambari-agent/conf/unix/ambari-agent.ini | 3 +++
ambari-agent/conf/windows/ambari-agent.ini | 2 ++
ambari-agent/src/main/python/ambari_agent/Controller.py | 8 ++++----
ambari-agent/src/main/python/ambari_agent/NetUtil.py | 10 ++++++----
4 files changed, 15 insertions(+), 8 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/fcd0d2ad/ambari-agent/conf/unix/ambari-agent.ini
----------------------------------------------------------------------
diff --git a/ambari-agent/conf/unix/ambari-agent.ini b/ambari-agent/conf/unix/ambari-agent.ini
index b0b64c9..61948d4 100644
--- a/ambari-agent/conf/unix/ambari-agent.ini
+++ b/ambari-agent/conf/unix/ambari-agent.ini
@@ -16,6 +16,9 @@
hostname=localhost
url_port=8440
secured_url_port=8441
+connect_retry_delay=10
+max_reconnect_retry_delay=30
+
[agent]
logdir=/var/log/ambari-agent
http://git-wip-us.apache.org/repos/asf/ambari/blob/fcd0d2ad/ambari-agent/conf/windows/ambari-agent.ini
----------------------------------------------------------------------
diff --git a/ambari-agent/conf/windows/ambari-agent.ini b/ambari-agent/conf/windows/ambari-agent.ini
index 3612414..42d65c1 100644
--- a/ambari-agent/conf/windows/ambari-agent.ini
+++ b/ambari-agent/conf/windows/ambari-agent.ini
@@ -16,6 +16,8 @@
hostname=localhost
url_port=8440
secured_url_port=8441
+connect_retry_delay=10
+max_reconnect_retry_delay=30
[agent]
prefix=data
http://git-wip-us.apache.org/repos/asf/ambari/blob/fcd0d2ad/ambari-agent/src/main/python/ambari_agent/Controller.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/Controller.py b/ambari-agent/src/main/python/ambari_agent/Controller.py
index cece30c..234795b 100644
--- a/ambari-agent/src/main/python/ambari_agent/Controller.py
+++ b/ambari-agent/src/main/python/ambari_agent/Controller.py
@@ -59,7 +59,7 @@ AGENT_RAM_OVERUSE_MESSAGE = "Ambari-agent RAM usage {used_ram} MB went above {co
class Controller(threading.Thread):
- def __init__(self, config, server_hostname, heartbeat_stop_callback = None, range=30):
+ def __init__(self, config, server_hostname, heartbeat_stop_callback = None):
threading.Thread.__init__(self)
logger.debug('Initializing Controller RPC thread.')
if heartbeat_stop_callback is None:
@@ -82,7 +82,7 @@ class Controller(threading.Thread):
self.repeatRegistration = False
self.isRegistered = False
self.cachedconnect = None
- self.range = range
+ self.max_reconnect_retry_delay = int(config.get('server','max_reconnect_retry_delay', default=30))
self.hasMappedComponents = True
self.statusCommandsExecutor = None
# Event is used for synchronizing heartbeat iterations (to make possible
@@ -210,7 +210,7 @@ class Controller(threading.Thread):
return
except Exception, ex:
# try a reconnect only after a certain amount of random time
- delay = randint(0, self.range)
+ delay = randint(0, self.max_reconnect_retry_delay)
logger.error("Unable to connect to: " + self.registerUrl, exc_info=True)
logger.error("Error:" + str(ex))
logger.warn(""" Sleeping for {0} seconds and then trying again """.format(delay,))
@@ -432,7 +432,7 @@ class Controller(threading.Thread):
retry = True
#randomize the heartbeat
- delay = randint(0, self.range)
+ delay = randint(0, self.max_reconnect_retry_delay)
time.sleep(delay)
# Sleep for some time
http://git-wip-us.apache.org/repos/asf/ambari/blob/fcd0d2ad/ambari-agent/src/main/python/ambari_agent/NetUtil.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/NetUtil.py b/ambari-agent/src/main/python/ambari_agent/NetUtil.py
index 2e9381b..c3cd621 100644
--- a/ambari-agent/src/main/python/ambari_agent/NetUtil.py
+++ b/ambari-agent/src/main/python/ambari_agent/NetUtil.py
@@ -30,7 +30,7 @@ logger = logging.getLogger(__name__)
class NetUtil:
- CONNECT_SERVER_RETRY_INTERVAL_SEC = 10
+ DEFAULT_CONNECT_RETRY_DELAY_SEC = 10
HEARTBEAT_IDLE_INTERVAL_DEFAULT_MIN_SEC = 1
HEARTBEAT_IDLE_INTERVAL_DEFAULT_MAX_SEC = 10
MINIMUM_INTERVAL_BETWEEN_HEARTBEATS = 0.1
@@ -52,6 +52,8 @@ class NetUtil:
stop_callback = HeartbeatStopHandlers()
self.stopCallback = stop_callback
self.config = config
+ self.connect_retry_delay = int(config.get('server','connect_retry_delay',
+ default=self.DEFAULT_CONNECT_RETRY_DELAY_SEC))
def checkURL(self, url):
"""Try to connect to a given url. Result is True if url returns HTTP code 200, in any other case
@@ -94,7 +96,7 @@ class NetUtil:
return False, responseBody
def try_to_connect(self, server_url, max_retries, logger=None):
- """Try to connect to a given url, sleeping for CONNECT_SERVER_RETRY_INTERVAL_SEC seconds
+ """Try to connect to a given url, sleeping for connect_retry_delay seconds
between retries. No more than max_retries is performed. If max_retries is -1, connection
attempts will be repeated forever until server is not reachable
@@ -113,10 +115,10 @@ class NetUtil:
else:
if logger is not None:
logger.warn('Server at {0} is not reachable, sleeping for {1} seconds...'.format(server_url,
- self.CONNECT_SERVER_RETRY_INTERVAL_SEC))
+ self.connect_retry_delay))
retries += 1
- if 0 == self.stopCallback.wait(self.CONNECT_SERVER_RETRY_INTERVAL_SEC):
+ if 0 == self.stopCallback.wait(self.connect_retry_delay):
#stop waiting
if logger is not None:
logger.info("Stop event received")
[23/31] ambari git commit: AMBARI-18637: Management pack purge option
should warn user and ask for confirmation before purging - addendum (jluniya)
Posted by nc...@apache.org.
AMBARI-18637: Management pack purge option should warn user and ask for confirmation before purging - addendum (jluniya)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/56b4d57d
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/56b4d57d
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/56b4d57d
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 56b4d57dcf747ee84b07a24a6f663254ab1eeaff
Parents: 5454bd6
Author: Jayush Luniya <jl...@hortonworks.com>
Authored: Tue Nov 15 23:25:44 2016 -0800
Committer: Jayush Luniya <jl...@hortonworks.com>
Committed: Tue Nov 15 23:30:46 2016 -0800
----------------------------------------------------------------------
ambari-server/src/main/python/ambari_server/setupMpacks.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/56b4d57d/ambari-server/src/main/python/ambari_server/setupMpacks.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/setupMpacks.py b/ambari-server/src/main/python/ambari_server/setupMpacks.py
old mode 100644
new mode 100755
index b6d9051..3b8a67a
--- a/ambari-server/src/main/python/ambari_server/setupMpacks.py
+++ b/ambari-server/src/main/python/ambari_server/setupMpacks.py
@@ -640,9 +640,7 @@ def _install_mpack(options, replay_mode=False, is_upgrade=False):
_execute_hook(mpack_metadata, BEFORE_INSTALL_HOOK_NAME, tmp_root_dir)
# Purge previously installed stacks and management packs
- if options.purge:
- if not 'purge_list' in options:
- options.purge_list = ",".join([STACK_DEFINITIONS_RESOURCE_NAME, MPACKS_RESOURCE_NAME])
+ if options.purge and options.purge_list:
purge_resources = options.purge_list.split(",")
validate_purge(options, purge_resources, tmp_root_dir, mpack_metadata, replay_mode)
purge_stacks_and_mpacks(purge_resources, replay_mode)
@@ -827,6 +825,8 @@ def replay_mpack_logs():
print_info_msg(replay_log)
print_info_msg("===========================================================================================")
replay_options = _named_dict(ast.literal_eval(replay_log))
+ if 'purge_list' not in replay_options:
+ replay_options.purge_list = ",".join([STACK_DEFINITIONS_RESOURCE_NAME, MPACKS_RESOURCE_NAME])
if replay_options.mpack_command == INSTALL_MPACK_ACTION:
install_mpack(replay_options, replay_mode=True)
elif replay_options.mpack_command == UPGRADE_MPACK_ACTION:
[10/31] ambari git commit: AMBARI-18877. Ambari change
hcat.proxy.hosts value during add service (dlysnichenko)
Posted by nc...@apache.org.
AMBARI-18877. Ambari change hcat.proxy.hosts value during add service (dlysnichenko)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/4665c812
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/4665c812
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/4665c812
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 4665c812f4d7b4ebd009db07beb76be1ec587fe7
Parents: 07cde1d
Author: Lisnichenko Dmitro <dl...@hortonworks.com>
Authored: Tue Nov 15 13:10:59 2016 +0200
Committer: Lisnichenko Dmitro <dl...@hortonworks.com>
Committed: Tue Nov 15 13:10:59 2016 +0200
----------------------------------------------------------------------
.../stacks/HDP/2.0.6/services/stack_advisor.py | 29 +++++++++++++++++---
.../stacks/2.0.6/common/test_stack_advisor.py | 4 +--
2 files changed, 27 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/4665c812/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py b/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
index 5e7698d..9b5ff68 100644
--- a/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
+++ b/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
@@ -205,7 +205,10 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
if "HDFS" in servicesList:
ambari_user = self.getAmbariUser(services)
ambariHostName = socket.getfqdn()
- putCoreSiteProperty("hadoop.proxyuser.{0}.hosts".format(ambari_user), ambariHostName)
+ is_wildcard_value, hosts = self.get_hosts_for_proxyuser(ambari_user, services)
+ if not is_wildcard_value:
+ hosts.add(ambariHostName)
+ putCoreSiteProperty("hadoop.proxyuser.{0}.hosts".format(ambari_user), ",".join(hosts))
putCoreSiteProperty("hadoop.proxyuser.{0}.groups".format(ambari_user), "*")
old_ambari_user = self.getOldAmbariUser(services)
if old_ambari_user is not None:
@@ -251,7 +254,7 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
for proxyPropertyName, hostSelector in hostSelectorMap.iteritems():
componentHostNamesString = hostSelector if isinstance(hostSelector, basestring) else '*'
if isinstance(hostSelector, (list, tuple)):
- componentHostNames = set()
+ _, componentHostNames = self.get_hosts_for_proxyuser(user, services) # preserve old values
for component in hostSelector:
componentHosts = self.getHostsWithComponent(serviceName, component, services, hosts)
if componentHosts is not None:
@@ -285,6 +288,21 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
return users
+ def get_hosts_for_proxyuser(self, user_name, services):
+ if "core-site" in services["configurations"]:
+ coreSite = services["configurations"]["core-site"]['properties']
+ else:
+ coreSite = {}
+
+ property_name = "hadoop.proxyuser.{0}.hosts".format(user_name)
+ if property_name in coreSite:
+ property_value = coreSite[property_name]
+ if property_value == "*":
+ return True, set()
+ else:
+ return False, set(property_value.split(","))
+ return False, set()
+
def getServiceHadoopProxyUsersConfigurationDict(self):
"""
Returns a map that is used by 'getHadoopProxyUsers' to determine service
@@ -347,9 +365,12 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
services["forced-configurations"].append({"type" : "core-site", "name" : "hadoop.proxyuser.{0}.hosts".format(hive_user)})
for user_name, user_properties in users.iteritems():
+ is_wildcard_value, _ = self.get_hosts_for_proxyuser(user_name, services)
+
# Add properties "hadoop.proxyuser.*.hosts", "hadoop.proxyuser.*.groups" to core-site for all users
- putCoreSiteProperty("hadoop.proxyuser.{0}.hosts".format(user_name) , user_properties["propertyHosts"])
- Logger.info("Updated hadoop.proxyuser.{0}.hosts as : {1}".format(user_name, user_properties["propertyHosts"]))
+ if not is_wildcard_value:
+ putCoreSiteProperty("hadoop.proxyuser.{0}.hosts".format(user_name) , user_properties["propertyHosts"])
+ Logger.info("Updated hadoop.proxyuser.{0}.hosts as : {1}".format(user_name, user_properties["propertyHosts"]))
if "propertyGroups" in user_properties:
putCoreSiteProperty("hadoop.proxyuser.{0}.groups".format(user_name) , user_properties["propertyGroups"])
http://git-wip-us.apache.org/repos/asf/ambari/blob/4665c812/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py b/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py
index 125c9ca..00ee563 100644
--- a/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py
+++ b/ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py
@@ -1326,7 +1326,7 @@ class TestHDP206StackAdvisor(TestCase):
'hadoop.proxyuser.webhcat.groups': '*',
'hadoop.proxyuser.hdfs.groups': '*',
'hadoop.proxyuser.hdfs.hosts': '*',
- 'hadoop.proxyuser.hive.hosts': 'c6401.ambari.apache.org,c6402.ambari.apache.org',
+ 'hadoop.proxyuser.hive.hosts': 'c6401.ambari.apache.org,c6402.ambari.apache.org,c6403.ambari.apache.org',
'hadoop.proxyuser.oozie.hosts': 'c6401.ambari.apache.org,c6402.ambari.apache.org',
'hadoop.proxyuser.falcon.groups': '*'}},
'falcon-env':
@@ -1459,7 +1459,7 @@ class TestHDP206StackAdvisor(TestCase):
'hadoop.proxyuser.webhcat.groups': '*',
'hadoop.proxyuser.hdfs.groups': '*',
'hadoop.proxyuser.hdfs.hosts': '*',
- 'hadoop.proxyuser.hive.hosts': 'c6401.ambari.apache.org,c6402.ambari.apache.org',
+ 'hadoop.proxyuser.hive.hosts': 'c6401.ambari.apache.org,c6402.ambari.apache.org,c6403.ambari.apache.org',
'hadoop.proxyuser.oozie.hosts': 'c6401.ambari.apache.org,c6402.ambari.apache.org',
'hadoop.proxyuser.falcon.groups': '*'},
'property_attributes':
[05/31] ambari git commit: AMBARI-18887 - Hive Service Check Fails
During Upgrade Due to Missing Imports (jonathanhurley)
Posted by nc...@apache.org.
AMBARI-18887 - Hive Service Check Fails During Upgrade Due to Missing Imports (jonathanhurley)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/45adfdf7
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/45adfdf7
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/45adfdf7
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 45adfdf778b44e28c607264f8026404e02633463
Parents: bc7a29f
Author: Jonathan Hurley <jh...@hortonworks.com>
Authored: Mon Nov 14 15:52:31 2016 -0500
Committer: Jonathan Hurley <jh...@hortonworks.com>
Committed: Mon Nov 14 16:44:30 2016 -0500
----------------------------------------------------------------------
.../0.12.0.2.0/package/scripts/params_linux.py | 2 -
.../0.12.0.2.0/package/scripts/service_check.py | 10 ++---
.../2.0.6/HIVE/test_hive_service_check.py | 43 +++++++++++++++++++-
3 files changed, 46 insertions(+), 9 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/45adfdf7/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py b/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
index 3caaefc..e9372f9 100644
--- a/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
+++ b/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
@@ -108,8 +108,6 @@ hive_user_home_dir = "/home/hive"
hive_server2_hive2_dir = None
hive_server2_hive2_lib = None
-version = default("/commandParams/version", None)
-
if check_stack_feature(StackFeature.HIVE_SERVER_INTERACTIVE, version_for_stack_feature_checks):
# the name of the hiveserver2-hive2 component
hive_server2_hive2_component = status_params.SERVER_ROLE_DIRECTORY_MAP["HIVE_SERVER_INTERACTIVE"]
http://git-wip-us.apache.org/repos/asf/ambari/blob/45adfdf7/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py b/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py
index 90a97fe..d144c34 100644
--- a/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py
+++ b/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py
@@ -18,8 +18,7 @@ limitations under the License.
"""
-import socket
-import sys
+import os
import time
import subprocess
@@ -27,15 +26,14 @@ from hcat_service_check import hcat_service_check
from webhcat_service_check import webhcat_service_check
from ambari_commons import OSConst
from ambari_commons.os_family_impl import OsFamilyImpl
-from resource_management.core import shell
from resource_management.core.logger import Logger
from resource_management.libraries.functions import get_unique_id_and_date
from resource_management.libraries.script.script import Script
from resource_management.libraries.functions.format import format
from resource_management.libraries.functions.hive_check import check_thrift_port_sasl
-from resource_management.core.resources.system import Execute, File
+from resource_management.core.resources.system import Execute
from resource_management.core.exceptions import Fail
-from resource_management.core.source import StaticFile
+
class HiveServiceCheck(Script):
pass
@@ -173,7 +171,7 @@ class HiveServiceCheckDefault(HiveServiceCheck):
beeline_url.append('principal={key}')
exec_path = params.execute_path
- if params.version and params.stack_root:
+ if params.version:
upgrade_hive_bin = format("{stack_root}/{version}/hive2/bin")
exec_path = os.environ['PATH'] + os.pathsep + params.hadoop_bin_dir + os.pathsep + upgrade_hive_bin
http://git-wip-us.apache.org/repos/asf/ambari/blob/45adfdf7/ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py b/ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py
index 30b4eb7..9c73d30 100644
--- a/ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py
+++ b/ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py
@@ -259,7 +259,6 @@ class TestServiceCheck(RMFTestCase):
self.assertNoMoreResources()
-
def test_service_check_during_upgrade(self, socket_mock):
config_file = self.get_src_folder() + "/test/python/stacks/2.2/configs/hive-upgrade.json"
with open(config_file, 'r') as f:
@@ -289,3 +288,45 @@ class TestServiceCheck(RMFTestCase):
tries = 3,
user = 'ambari-qa',
try_sleep = 5)
+
+
+ def test_service_check_during_upgrade_for_llap(self, socket_mock):
+ config_file = self.get_src_folder() + "/test/python/stacks/2.2/configs/hive-upgrade.json"
+ with open(config_file, 'r') as f:
+ json_content = json.load(f)
+
+ # populate version and an LLAP instance to trigger the LLAP service check
+ json_content['commandParams']['version'] = "2.3.0.0-1234"
+ json_content['clusterHostInfo']['hive_server_interactive_hosts'] = ["c6402.ambari.apache.org"]
+ json_content['configurations']['hive-interactive-env'] = {}
+ json_content['configurations']['hive-interactive-env']['enable_hive_interactive'] = True
+
+
+ self.executeScript(self.COMMON_SERVICES_PACKAGE_DIR + "/scripts/service_check.py",
+ classname = "HiveServiceCheck",
+ command = "service_check",
+ config_dict = json_content,
+ stack_version = self.STACK_VERSION,
+ target = RMFTestCase.TARGET_COMMON_SERVICES)
+
+ self.assertResourceCalled('Execute',
+ "! beeline -u 'jdbc:hive2://c6402.ambari.apache.org:10010/;transportMode=binary' -e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL'",
+ path = ['/bin/', '/usr/bin/', '/usr/lib/hive/bin/', '/usr/sbin/'],
+ timeout = 30,
+ user = 'ambari-qa')
+
+ self.assertResourceCalled('Execute',
+ "! beeline -u 'jdbc:hive2://c6402.ambari.apache.org:10500/;transportMode=binary' -e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL'",
+ path = ['/bin/', '/usr/bin/', '/usr/lib/hive/bin/', '/usr/sbin/'],
+ timeout = 30,
+ user = 'ambari-qa')
+
+ # LLAP call
+ self.assertResourceCalled('Execute',
+ "! beeline -u 'jdbc:hive2://c6402.ambari.apache.org:10500/;transportMode=binary' --hiveconf \"hiveLlapServiceCheck=\" -f /usr/hdp/current/hive-server2-hive2/scripts/llap/sql/serviceCheckScript.sql -e '' 2>&1| awk '{print}'|grep -i -e 'Invalid status\|Invalid URL\|command not found\|Connection refused'",
+ path = ['/usr/sbin', '/usr/local/bin', '/bin', '/usr/bin', '/bin:/usr/hdp/current/hadoop-client/bin:/usr/hdp/2.3.0.0-1234/hive2/bin'],
+ tries = 1,
+ stderr = -1,
+ wait_for_finish = True,
+ logoutput = True,
+ user = 'hive')
[02/31] ambari git commit: Revert "AMBARI-18846 - Custom services
should be able to easily specify their own dashboards"
Posted by nc...@apache.org.
Revert "AMBARI-18846 - Custom services should be able to easily specify their own dashboards"
This reverts commit fa23822bd185a94e241145b0c3cedd394aed99d8.
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/3868cbe4
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/3868cbe4
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/3868cbe4
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 3868cbe4d249a02d923f5502c9a9a24787550991
Parents: 4d7d3cd
Author: Tim Thorpe <tt...@apache.org>
Authored: Mon Nov 14 11:50:52 2016 -0800
Committer: Tim Thorpe <tt...@apache.org>
Committed: Mon Nov 14 11:50:52 2016 -0800
----------------------------------------------------------------------
.../ambari_agent/CustomServiceOrchestrator.py | 3 +-
.../src/main/python/ambari_agent/FileCache.py | 10 -------
.../TestCustomServiceOrchestrator.py | 11 ++-----
.../test/python/ambari_agent/TestFileCache.py | 12 --------
.../python/ambari_server/resourceFilesKeeper.py | 8 ++----
.../python/ambari_server/serverConfiguration.py | 11 -------
.../package/scripts/metrics_grafana_util.py | 8 +++---
.../0.1.0/package/scripts/params.py | 3 --
.../0.1.0/package/scripts/split_points.py | 30 +++-----------------
.../stacks/HDP/2.0.6/services/stack_advisor.py | 3 +-
.../src/test/python/TestResourceFilesKeeper.py | 16 +++++------
11 files changed, 22 insertions(+), 93 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py b/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
index fbc1509..57416a4 100644
--- a/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
+++ b/ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py
@@ -153,8 +153,7 @@ class CustomServiceOrchestrator():
self.file_cache.get_host_scripts_base_dir(server_url_prefix)
hook_dir = self.file_cache.get_hook_base_dir(command, server_url_prefix)
base_dir = self.file_cache.get_service_base_dir(command, server_url_prefix)
- self.file_cache.get_dashboard_base_dir(server_url_prefix)
-
+
script_path = self.resolve_script_path(base_dir, script)
script_tuple = (script_path, base_dir)
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-agent/src/main/python/ambari_agent/FileCache.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/main/python/ambari_agent/FileCache.py b/ambari-agent/src/main/python/ambari_agent/FileCache.py
index 83ac373..b7c5dee 100644
--- a/ambari-agent/src/main/python/ambari_agent/FileCache.py
+++ b/ambari-agent/src/main/python/ambari_agent/FileCache.py
@@ -45,7 +45,6 @@ class FileCache():
STACKS_CACHE_DIRECTORY="stacks"
COMMON_SERVICES_DIRECTORY="common-services"
CUSTOM_ACTIONS_CACHE_DIRECTORY="custom_actions"
- DASHBOARD_DIRECTORY="dashboards"
HOST_SCRIPTS_CACHE_DIRECTORY="host_scripts"
HASH_SUM_FILE=".hash"
ARCHIVE_NAME="archive.zip"
@@ -100,15 +99,6 @@ class FileCache():
server_url_prefix)
- def get_dashboard_base_dir(self, server_url_prefix):
- """
- Returns a base directory for dashboards
- """
- return self.provide_directory(self.cache_dir,
- self.DASHBOARD_DIRECTORY,
- server_url_prefix)
-
-
def get_host_scripts_base_dir(self, server_url_prefix):
"""
Returns a base directory for host scripts (host alerts, etc) which
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py b/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
index 9f1241e..c9724b7 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestCustomServiceOrchestrator.py
@@ -220,7 +220,7 @@ class TestCustomServiceOrchestrator(TestCase):
except AgentException:
pass # Expected
- @patch.object(FileCache, "get_dashboard_base_dir")
+
@patch.object(CustomServiceOrchestrator, "resolve_script_path")
@patch.object(CustomServiceOrchestrator, "resolve_hook_script_path")
@patch.object(FileCache, "get_host_scripts_base_dir")
@@ -234,8 +234,7 @@ class TestCustomServiceOrchestrator(TestCase):
get_hook_base_dir_mock, get_service_base_dir_mock,
get_host_scripts_base_dir_mock,
resolve_hook_script_path_mock,
- resolve_script_path_mock,
- get_dashboard_base_dir_mock):
+ resolve_script_path_mock):
FileCache_mock.return_value = None
command = {
@@ -266,7 +265,6 @@ class TestCustomServiceOrchestrator(TestCase):
unix_process_id = 111
orchestrator.commands_in_progress = {command['taskId']: unix_process_id}
get_hook_base_dir_mock.return_value = "/hooks/"
- get_dashboard_base_dir_mock.return_value = "/dashboards/"
# normal run case
run_file_mock.return_value = {
'stdout' : 'sss',
@@ -311,7 +309,6 @@ class TestCustomServiceOrchestrator(TestCase):
pass
- @patch.object(FileCache, "get_dashboard_base_dir")
@patch("ambari_commons.shell.kill_process_with_children")
@patch.object(CustomServiceOrchestrator, "resolve_script_path")
@patch.object(CustomServiceOrchestrator, "resolve_hook_script_path")
@@ -326,8 +323,7 @@ class TestCustomServiceOrchestrator(TestCase):
get_hook_base_dir_mock, get_service_base_dir_mock,
get_host_scripts_base_dir_mock,
resolve_hook_script_path_mock, resolve_script_path_mock,
- kill_process_with_children_mock,
- get_dashboard_base_dir_mock):
+ kill_process_with_children_mock):
FileCache_mock.return_value = None
command = {
'role' : 'REGION_SERVER',
@@ -357,7 +353,6 @@ class TestCustomServiceOrchestrator(TestCase):
unix_process_id = 111
orchestrator.commands_in_progress = {command['taskId']: unix_process_id}
get_hook_base_dir_mock.return_value = "/hooks/"
- get_dashboard_base_dir_mock.return_value = "/dashboards/"
run_file_mock_return_value = {
'stdout' : 'killed',
'stderr' : 'killed',
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
----------------------------------------------------------------------
diff --git a/ambari-agent/src/test/python/ambari_agent/TestFileCache.py b/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
index fbefc2b..5933daa 100644
--- a/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
+++ b/ambari-agent/src/test/python/ambari_agent/TestFileCache.py
@@ -117,18 +117,6 @@ class TestFileCache(TestCase):
"('/var/lib/ambari-agent/cache', 'custom_actions', 'server_url_pref')")
self.assertEquals(res, "dummy value")
-
- @patch.object(FileCache, "provide_directory")
- def test_get_dashboard_base_dir(self, provide_directory_mock):
- provide_directory_mock.return_value = "dummy value"
- fileCache = FileCache(self.config)
- res = fileCache.get_dashboard_base_dir("server_url_pref")
- self.assertEquals(
- pprint.pformat(provide_directory_mock.call_args_list[0][0]),
- "('/var/lib/ambari-agent/cache', 'dashboards', 'server_url_pref')")
- self.assertEquals(res, "dummy value")
-
-
@patch.object(FileCache, "build_download_url")
def test_provide_directory_no_update(self, build_download_url_mock):
try:
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py b/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
index ed77700..f2a3ebd 100644
--- a/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
+++ b/ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
@@ -39,7 +39,6 @@ class ResourceFilesKeeper():
COMMON_SERVICES_DIR="common-services"
CUSTOM_ACTIONS_DIR="custom_actions"
HOST_SCRIPTS_DIR="host_scripts"
- DASHBOARDS_DIR="dashboards"
# For these directories archives are created
ARCHIVABLE_DIRS = [HOOKS_DIR, PACKAGE_DIR]
@@ -69,7 +68,7 @@ class ResourceFilesKeeper():
"""
Performs housekeeping operations on resource files
"""
- self.update_directory_archives()
+ self.update_directory_archieves()
# probably, later we will need some additional operations
@@ -88,7 +87,7 @@ class ResourceFilesKeeper():
# update the directories so that the .hash is generated
self.update_directory_archive(archive_root)
- def update_directory_archives(self):
+ def update_directory_archieves(self):
"""
Please see AMBARI-4481 for more details
"""
@@ -113,9 +112,6 @@ class ResourceFilesKeeper():
# agent host scripts
self._update_resources_subdir_archive(self.HOST_SCRIPTS_DIR)
- # custom service dashboards
- self._update_resources_subdir_archive(self.DASHBOARDS_DIR)
-
def _list_metainfo_dirs(self, root_dir):
valid_items = [] # Format: <stack_dir, ignore(True|False)>
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-server/src/main/python/ambari_server/serverConfiguration.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/serverConfiguration.py b/ambari-server/src/main/python/ambari_server/serverConfiguration.py
index 3086003..04509cf 100644
--- a/ambari-server/src/main/python/ambari_server/serverConfiguration.py
+++ b/ambari-server/src/main/python/ambari_server/serverConfiguration.py
@@ -397,7 +397,6 @@ class ServerConfigDefaults(object):
self.EXTENSION_LOCATION_DEFAULT = ""
self.COMMON_SERVICES_LOCATION_DEFAULT = ""
self.MPACKS_STAGING_LOCATION_DEFAULT = ""
- self.DASHBOARD_LOCATION_DEFAULT = ""
self.SERVER_TMP_DIR_DEFAULT = ""
self.DEFAULT_VIEWS_DIR = ""
@@ -469,7 +468,6 @@ class ServerConfigDefaultsWindows(ServerConfigDefaults):
self.EXTENSION_LOCATION_DEFAULT = "resources\\extensions"
self.COMMON_SERVICES_LOCATION_DEFAULT = "resources\\common-services"
self.MPACKS_STAGING_LOCATION_DEFAULT = "resources\\mpacks"
- self.DASHBOARD_LOCATION_DEFAULT = "resources\\dashboards"
self.SERVER_TMP_DIR_DEFAULT = "data\\tmp"
self.DEFAULT_VIEWS_DIR = "resources\\views"
@@ -556,7 +554,6 @@ class ServerConfigDefaultsLinux(ServerConfigDefaults):
self.EXTENSION_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/extensions")
self.COMMON_SERVICES_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/common-services")
self.MPACKS_STAGING_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/mpacks")
- self.DASHBOARD_LOCATION_DEFAULT = AmbariPath.get("/var/lib/ambari-server/resources/dashboards")
self.SERVER_TMP_DIR_DEFAULT = AmbariPath.get("/var/lib/ambari-server/data/tmp")
self.DEFAULT_VIEWS_DIR = AmbariPath.get("/var/lib/ambari-server/resources/views")
@@ -1442,14 +1439,6 @@ def get_mpacks_staging_location(properties):
mpacks_staging_location = configDefaults.MPACKS_STAGING_LOCATION_DEFAULT
return mpacks_staging_location
-
-#
-# Dashboard location
-#
-def get_dashboard_location(properties):
- dashboard_location = configDefaults.DASHBOARD_LOCATION_DEFAULT
- return dashboard_location
-
#
# Server temp location
#
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
index 6030fc4..b98dc1d 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana_util.py
@@ -36,7 +36,7 @@ import os
GRAFANA_CONNECT_TRIES = 5
GRAFANA_CONNECT_TIMEOUT = 10
-GRAFANA_SEARCH_BUILTIN_DASHBOARDS = "/api/search?tag=builtin"
+GRAFANA_SEARCH_BULTIN_DASHBOARDS = "/api/search?tag=builtin"
GRAFANA_DATASOURCE_URL = "/api/datasources"
GRAFANA_DASHBOARDS_URL = "/api/dashboards/db"
METRICS_GRAFANA_DATASOURCE_NAME = "AMBARI_METRICS"
@@ -313,14 +313,14 @@ def create_ams_dashboards():
Dashboard = namedtuple('Dashboard', ['uri', 'id', 'title', 'tags'])
existing_dashboards = []
- response = perform_grafana_get_call(GRAFANA_SEARCH_BUILTIN_DASHBOARDS, server)
+ response = perform_grafana_get_call(GRAFANA_SEARCH_BULTIN_DASHBOARDS, server)
if response and response.status == 200:
data = response.read()
try:
dashboards = json.loads(data)
except:
Logger.error("Unable to parse JSON response from grafana request: %s" %
- GRAFANA_SEARCH_BUILTIN_DASHBOARDS)
+ GRAFANA_SEARCH_BULTIN_DASHBOARDS)
Logger.info(data)
return
@@ -336,7 +336,7 @@ def create_ams_dashboards():
else:
Logger.error("Failed to execute search query on Grafana dashboards. "
"query = %s\n statuscode = %s\n reason = %s\n data = %s\n" %
- (GRAFANA_SEARCH_BUILTIN_DASHBOARDS, response.status, response.reason, response.read()))
+ (GRAFANA_SEARCH_BULTIN_DASHBOARDS, response.status, response.reason, response.read()))
return
Logger.debug('Dashboard definitions found = %s' % str(dashboard_files))
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
index 1a754f6..b052999 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
@@ -83,9 +83,6 @@ dashboards_dirs.append(os.path.join(agent_cache_dir, service_package_folder,
dashboards_dirs.append(os.path.join(agent_cache_dir, service_package_folder,
'files', 'grafana-dashboards', 'default'))
-# Custom services
-dashboards_dirs.append(os.path.join(agent_cache_dir, 'dashboards', 'grafana-dashboards'))
-
def get_grafana_dashboard_defs():
dashboard_defs = []
for dashboards_dir in dashboards_dirs:
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
index fda0199..aa03d197 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
@@ -67,12 +67,11 @@ def format_Xmx_size_to_bytes(value, default='b'):
# pre-splits based on selected services also passed as a parameter to the class.
class FindSplitPointsForAMSRegions():
- def __init__(self, ams_hbase_site, ams_hbase_env, serviceMetricsDir, customServiceMetricsDir,
+ def __init__(self, ams_hbase_site, ams_hbase_env, serviceMetricsDir,
operation_mode = 'embedded', services = None):
self.ams_hbase_site = ams_hbase_site
self.ams_hbase_env = ams_hbase_env
self.serviceMetricsDir = serviceMetricsDir
- self.customServiceMetricsDir = customServiceMetricsDir
self.services = services
self.mode = operation_mode
# Add host metrics if not present as input
@@ -118,19 +117,13 @@ class FindSplitPointsForAMSRegions():
pass
def initialize_ordered_set_of_metrics(self):
- stackServiceFiles = [ f for f in os.listdir(self.serviceMetricsDir) if
+ onlyServicefiles = [ f for f in os.listdir(self.serviceMetricsDir) if
os.path.isfile(os.path.join(self.serviceMetricsDir, f)) ]
- if os.path.exists(self.customServiceMetricsDir):
- customServiceFiles = [ f for f in os.listdir(self.customServiceMetricsDir) if
- os.path.isfile(os.path.join(self.customServiceMetricsDir, f)) ]
- else:
- customServiceFiles = []
-
metrics = set()
- for file in stackServiceFiles:
- # Process for stack services selected at deploy time or all stack services if
+ for file in onlyServicefiles:
+ # Process for services selected at deploy time or all services if
# services arg is not passed
if self.services is None or file.rstrip(metric_filename_ext) in self.services:
print 'Processing file: %s' % os.path.join(self.serviceMetricsDir, file)
@@ -144,21 +137,6 @@ class FindSplitPointsForAMSRegions():
pass
pass
- for file in customServiceFiles:
- # Process for custom services selected at deploy time or all custom services if
- # services arg is not passed
- if self.services is None or file.rstrip(metric_filename_ext) in self.services:
- print 'Processing file: %s' % os.path.join(self.customServiceMetricsDir, file)
- service_metrics = set()
- with open(os.path.join(self.customServiceMetricsDir, file), 'r') as f:
- for metric in f:
- service_metrics.add(metric.strip())
- pass
- pass
- metrics.update(self.find_equidistant_metrics(list(sorted(service_metrics))))
- pass
- pass
-
self.metrics = sorted(metrics)
print 'metrics length: %s' % len(self.metrics)
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py b/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
index 5e7698d..ee63f1e 100644
--- a/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
+++ b/ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py
@@ -811,7 +811,6 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
scriptDir = os.path.dirname(os.path.abspath(__file__))
metricsDir = os.path.join(scriptDir, '../../../../common-services/AMBARI_METRICS/0.1.0/package')
serviceMetricsDir = os.path.join(metricsDir, 'files', 'service-metrics')
- customServiceMetricsDir = os.path.join(scriptDir, '../../../../dashboards/service-metrics')
sys.path.append(os.path.join(metricsDir, 'scripts'))
servicesList = [service["StackServices"]["service_name"] for service in services["services"]]
@@ -833,7 +832,7 @@ class HDP206StackAdvisor(DefaultStackAdvisor):
ams_hbase_env = configurations["ams-hbase-env"]["properties"]
split_point_finder = FindSplitPointsForAMSRegions(
- ams_hbase_site, ams_hbase_env, serviceMetricsDir, customServiceMetricsDir, operatingMode, servicesList)
+ ams_hbase_site, ams_hbase_env, serviceMetricsDir, operatingMode, servicesList)
result = split_point_finder.get_split_points()
precision_splits = ' '
http://git-wip-us.apache.org/repos/asf/ambari/blob/3868cbe4/ambari-server/src/test/python/TestResourceFilesKeeper.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/python/TestResourceFilesKeeper.py b/ambari-server/src/test/python/TestResourceFilesKeeper.py
index ba29354..966d3b1 100644
--- a/ambari-server/src/test/python/TestResourceFilesKeeper.py
+++ b/ambari-server/src/test/python/TestResourceFilesKeeper.py
@@ -82,8 +82,7 @@ class TestResourceFilesKeeper(TestCase):
"call('../resources/TestAmbaryServer.samples/" \
"dummy_common_services/HIVE/0.11.0.2.0.5.0/package'),\n " \
"call('../resources/custom_actions'),\n " \
- "call('../resources/host_scripts'),\n " \
- "call('../resources/dashboards')]"
+ "call('../resources/host_scripts')]"
else:
UPDATE_DIRECTORY_ARCHIVE_CALL_LIST = \
"[call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_stack\\\\HIVE\\\\package'),\n " \
@@ -92,18 +91,17 @@ class TestResourceFilesKeeper(TestCase):
"call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_common_services\\\\HIVE\\\\0.11.0.2.0.5.0\\\\package'),\n " \
"call('..\\\\resources\\\\TestAmbaryServer.samples\\\\dummy_common_services\\\\HIVE\\\\0.11.0.2.0.5.0\\\\package'),\n " \
"call('..\\\\resources\\\\custom_actions'),\n " \
- "call('..\\\\resources\\\\host_scripts'),\n " \
- "call('..\\\\resources\\\\dashboards')]"
+ "call('..\\\\resources\\\\host_scripts')]"
def setUp(self):
logging.basicConfig(level=logging.ERROR)
- @patch.object(ResourceFilesKeeper, "update_directory_archives")
- def test_perform_housekeeping(self, update_directory_archives_mock):
+ @patch.object(ResourceFilesKeeper, "update_directory_archieves")
+ def test_perform_housekeeping(self, update_directory_archieves_mock):
resource_files_keeper = ResourceFilesKeeper(os.sep + "dummy-resources", os.sep + "dummy-path")
resource_files_keeper.perform_housekeeping()
- update_directory_archives_mock.assertCalled()
+ update_directory_archieves_mock.assertCalled()
pass
@@ -111,7 +109,7 @@ class TestResourceFilesKeeper(TestCase):
@patch.object(ResourceFilesKeeper, "list_common_services")
@patch.object(ResourceFilesKeeper, "list_stacks")
@patch("os.path.abspath")
- def test_update_directory_archives(self, abspath_mock,
+ def test_update_directory_archieves(self, abspath_mock,
list_active_stacks_mock,
list_common_services_mock,
update_directory_archive_mock):
@@ -122,7 +120,7 @@ class TestResourceFilesKeeper(TestCase):
self.DUMMY_UNCHANGEABLE_COMMON_SERVICES]
abspath_mock.side_effect = lambda s : s
resource_files_keeper = ResourceFilesKeeper(self.TEST_RESOURCES_DIR, self.TEST_STACKS_DIR)
- resource_files_keeper.update_directory_archives()
+ resource_files_keeper.update_directory_archieves()
self.assertEquals(pprint.pformat(
update_directory_archive_mock.call_args_list),
self.UPDATE_DIRECTORY_ARCHIVE_CALL_LIST)
[30/31] ambari git commit: AMBARI-18902 Minor typo fix in
TimeliveMetricsStore class in AMS (dili)
Posted by nc...@apache.org.
AMBARI-18902 Minor typo fix in TimeliveMetricsStore class in AMS (dili)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/58ef3946
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/58ef3946
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/58ef3946
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: 58ef394602fcae9a5cdea99db5146ed3206a74d6
Parents: b5e6a88
Author: Di Li <di...@apache.org>
Authored: Wed Nov 16 09:02:11 2016 -0500
Committer: Di Li <di...@apache.org>
Committed: Wed Nov 16 09:02:11 2016 -0500
----------------------------------------------------------------------
.../metrics/timeline/TimelineMetricStore.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/58ef3946/ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java
----------------------------------------------------------------------
diff --git a/ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java b/ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java
index 15644ed..57186f5 100644
--- a/ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java
+++ b/ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java
@@ -32,7 +32,7 @@ import java.util.Set;
public interface TimelineMetricStore {
/**
- * This method retrieves metrics stored byu the Timeline store.
+ * This method retrieves metrics stored by the Timeline store.
*
* @param metricNames Names of the metric, e.g.: cpu_user
* @param hostnames Names of the host where the metric originated from
[17/31] ambari git commit: AMBARI-18637: Management pack purge option
should warn user and ask for confirmation before purging - addendum (jluniya)
Posted by nc...@apache.org.
AMBARI-18637: Management pack purge option should warn user and ask for confirmation before purging - addendum (jluniya)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/d2118f58
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/d2118f58
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/d2118f58
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: d2118f58c9cf36720710222deac2d06e8024af00
Parents: a201d09
Author: Jayush Luniya <jl...@hortonworks.com>
Authored: Tue Nov 15 11:39:16 2016 -0800
Committer: Jayush Luniya <jl...@hortonworks.com>
Committed: Tue Nov 15 11:41:13 2016 -0800
----------------------------------------------------------------------
ambari-server/src/main/python/ambari_server/setupMpacks.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/d2118f58/ambari-server/src/main/python/ambari_server/setupMpacks.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/python/ambari_server/setupMpacks.py b/ambari-server/src/main/python/ambari_server/setupMpacks.py
index f0c4f86..b6d9051 100644
--- a/ambari-server/src/main/python/ambari_server/setupMpacks.py
+++ b/ambari-server/src/main/python/ambari_server/setupMpacks.py
@@ -640,7 +640,9 @@ def _install_mpack(options, replay_mode=False, is_upgrade=False):
_execute_hook(mpack_metadata, BEFORE_INSTALL_HOOK_NAME, tmp_root_dir)
# Purge previously installed stacks and management packs
- if options.purge and options.purge_list:
+ if options.purge:
+ if not 'purge_list' in options:
+ options.purge_list = ",".join([STACK_DEFINITIONS_RESOURCE_NAME, MPACKS_RESOURCE_NAME])
purge_resources = options.purge_list.split(",")
validate_purge(options, purge_resources, tmp_root_dir, mpack_metadata, replay_mode)
purge_stacks_and_mpacks(purge_resources, replay_mode)
[29/31] ambari git commit: AMBARI-18895. Cluster install fails if
user accidentally creates a config group (onechiporenko)
Posted by nc...@apache.org.
AMBARI-18895. Cluster install fails if user accidentally creates a config group (onechiporenko)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/b5e6a88c
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/b5e6a88c
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/b5e6a88c
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: b5e6a88c6e1f060bfe8e064cdce808f28c121701
Parents: 61942e5
Author: Oleg Nechiporenko <on...@apache.org>
Authored: Tue Nov 15 14:20:10 2016 +0200
Committer: Oleg Nechiporenko <on...@apache.org>
Committed: Wed Nov 16 15:55:46 2016 +0200
----------------------------------------------------------------------
ambari-web/app/mixins/main/service/configs/config_overridable.js | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/b5e6a88c/ambari-web/app/mixins/main/service/configs/config_overridable.js
----------------------------------------------------------------------
diff --git a/ambari-web/app/mixins/main/service/configs/config_overridable.js b/ambari-web/app/mixins/main/service/configs/config_overridable.js
index d84e31c..93dea5f 100644
--- a/ambari-web/app/mixins/main/service/configs/config_overridable.js
+++ b/ambari-web/app/mixins/main/service/configs/config_overridable.js
@@ -152,6 +152,7 @@ App.ConfigOverridable = Em.Mixin.create({
popup.hide();
});
} else {
+ newConfigGroup.is_temporary = true;
App.store.load(App.ServiceConfigGroup, newConfigGroup);
App.store.commit();
newConfigGroup = App.ServiceConfigGroup.find(newConfigGroup.id);
[04/31] ambari git commit: AMBARI-18355: Conditional dependencies in
stack defition to handle blueprint validation gracefully (Amruta Borkar via
dili)
Posted by nc...@apache.org.
AMBARI-18355: Conditional dependencies in stack defition to handle blueprint validation gracefully (Amruta Borkar via dili)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/bc7a29f5
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/bc7a29f5
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/bc7a29f5
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: bc7a29f54046978b79f659d94a17f6335f693191
Parents: a190600
Author: Di Li <di...@apache.org>
Authored: Mon Nov 14 15:33:36 2016 -0500
Committer: Di Li <di...@apache.org>
Committed: Mon Nov 14 15:33:36 2016 -0500
----------------------------------------------------------------------
.../server/state/DependencyConditionInfo.java | 104 +++++++++++++++++++
.../ambari/server/state/DependencyInfo.java | 37 ++++++-
.../server/topology/BlueprintValidatorImpl.java | 18 +++-
.../common-services/HDFS/2.1.0.2.0/metainfo.xml | 43 +++++++-
.../server/topology/BlueprintImplTest.java | 41 --------
.../topology/BlueprintValidatorImplTest.java | 75 ++++++++++++-
6 files changed, 270 insertions(+), 48 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/bc7a29f5/ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java b/ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
new file mode 100644
index 0000000..66c3e2b
--- /dev/null
+++ b/ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
@@ -0,0 +1,104 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ambari.server.state;
+
+import java.util.Map;
+import javax.xml.bind.annotation.XmlElement;
+import javax.xml.bind.annotation.XmlSeeAlso;
+import javax.xml.bind.annotation.XmlTransient;
+
+/**
+ * Represents stack component dependency condition information provided in
+ * metainfo.xml file.
+ */
+@XmlTransient
+@XmlSeeAlso({ PropertyExists.class, PropertyValueEquals.class })
+public interface DependencyConditionInfo {
+ /**
+ * Returns true if the dependency condition is satisfied
+ *
+ * @param properties
+ * @return boolean
+ */
+ boolean isResolved(Map<String, Map<String, String>> properties);
+}
+
+/**
+ * This class represents a dependency condition that is based on existence of a
+ * certain property.
+ */
+class PropertyExists implements DependencyConditionInfo {
+ /**
+ * configType of the conditional dependency
+ */
+ protected String configType;
+
+ /**
+ * property of the conditional dependency
+ */
+ protected String property;
+
+ @XmlElement
+ public String getProperty() {
+ return property;
+ }
+
+ public void setProperty(String property) {
+ this.property = property;
+ }
+
+ @XmlElement
+ public String getConfigType() {
+ return configType;
+ }
+
+ public void setConfigType(String configType) {
+ this.configType = configType;
+ }
+
+ @Override
+ public boolean isResolved(Map<String, Map<String, String>> properties) {
+ return (properties.get(configType).containsKey(property));
+ }
+}
+
+/**
+ * This class represents a dependency condition that is based on value of a
+ * certain property.
+ */
+class PropertyValueEquals extends PropertyExists {
+ /**
+ * propertyValue of the property
+ */
+ protected String propertyValue;
+
+ @XmlElement
+ public String getPropertyValue() {
+ return propertyValue;
+ }
+
+ public void setPropertyValue(String propertyValue) {
+ this.propertyValue = propertyValue;
+ }
+
+ @Override
+ public boolean isResolved(Map<String, Map<String, String>> properties) {
+ return (super.isResolved(properties) && propertyValue.equals(properties.get(configType).get(property)));
+ }
+}
http://git-wip-us.apache.org/repos/asf/ambari/blob/bc7a29f5/ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java b/ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java
index e3db662..540767c 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java
@@ -19,7 +19,12 @@
package org.apache.ambari.server.state;
+import java.util.ArrayList;
+import java.util.List;
import javax.xml.bind.annotation.XmlElement;
+import javax.xml.bind.annotation.XmlElementWrapper;
+import javax.xml.bind.annotation.XmlElements;
+import org.apache.commons.collections.CollectionUtils;
/**
* Represents stack component dependency information.
@@ -54,7 +59,10 @@ public class DependencyInfo {
@XmlElement(name="auto-deploy")
private AutoDeployInfo m_autoDeploy;
-
+ /**
+ * Conditions for Component dependency to other components.
+ */
+ private List<DependencyConditionInfo> dependencyConditions = new ArrayList<DependencyConditionInfo>();
/**
* Setter for name property.
*
@@ -135,6 +143,33 @@ public class DependencyInfo {
public String getServiceName() {
return serviceName;
}
+ /**
+ * Get the dependencyConditions list
+ *
+ * @return dependencyConditions
+ */
+ @XmlElementWrapper(name="conditions")
+ @XmlElements(@XmlElement(name="condition"))
+ public List<DependencyConditionInfo> getDependencyConditions() {
+ return dependencyConditions;
+ }
+
+ /**
+ * Set dependencyConditions
+ *
+ * @param dependencyConditions
+ */
+ public void setDependencyConditions(List<DependencyConditionInfo> dependencyConditions) {
+ this.dependencyConditions = dependencyConditions;
+ }
+
+ /**
+ * Confirms if dependency have any condition or not
+ * @return true if dependencies are based on a condition
+ */
+ public boolean hasDependencyConditions(){
+ return !CollectionUtils.isEmpty(dependencyConditions);
+ }
@Override
public String toString() {
http://git-wip-us.apache.org/repos/asf/ambari/blob/bc7a29f5/ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java b/ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
index a5f33ff..c5647c3 100644
--- a/ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
+++ b/ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
@@ -26,10 +26,10 @@ import java.util.HashSet;
import java.util.Iterator;
import java.util.Map;
import java.util.Set;
-import java.util.regex.Pattern;
import org.apache.ambari.server.controller.internal.Stack;
import org.apache.ambari.server.state.AutoDeployInfo;
+import org.apache.ambari.server.state.DependencyConditionInfo;
import org.apache.ambari.server.state.DependencyInfo;
import org.apache.ambari.server.utils.SecretReference;
import org.apache.ambari.server.utils.VersionUtils;
@@ -135,9 +135,6 @@ public class BlueprintValidatorImpl implements BlueprintValidator {
}
}
if (ClusterTopologyImpl.isNameNodeHAEnabled(clusterConfigurations) && component.equals("NAMENODE")) {
- if(!hostGroup.getComponentNames().contains("ZKFC")){
- throw new InvalidTopologyException("Compoenent ZKFC is mandatory for hostgroup " + hostGroup+" when NAMENODE HA is enabled");
- }
Map<String, String> hadoopEnvConfig = clusterConfigurations.get("hadoop-env");
if(hadoopEnvConfig != null && !hadoopEnvConfig.isEmpty() && hadoopEnvConfig.containsKey("dfs_ha_initial_namenode_active") && hadoopEnvConfig.containsKey("dfs_ha_initial_namenode_standby")) {
ArrayList<HostGroup> hostGroupsForComponent = new ArrayList<HostGroup>( blueprint.getHostGroupsForComponent(component));
@@ -289,6 +286,19 @@ public class BlueprintValidatorImpl implements BlueprintValidator {
AutoDeployInfo autoDeployInfo = dependency.getAutoDeploy();
boolean resolved = false;
+ //check if conditions are met, if any
+ if(dependency.hasDependencyConditions()) {
+ boolean conditionsSatisfied = true;
+ for (DependencyConditionInfo dependencyCondition : dependency.getDependencyConditions()) {
+ if (!dependencyCondition.isResolved(blueprint.getConfiguration().getFullProperties())) {
+ conditionsSatisfied = false;
+ break;
+ }
+ }
+ if(!conditionsSatisfied){
+ continue;
+ }
+ }
if (dependencyScope.equals("cluster")) {
Collection<String> missingDependencyInfo = verifyComponentCardinalityCount(
componentName, new Cardinality("1+"), autoDeployInfo);
http://git-wip-us.apache.org/repos/asf/ambari/blob/bc7a29f5/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml b/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml
index 9c1387d..2b7a684 100644
--- a/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml
+++ b/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml
@@ -15,7 +15,7 @@
See the License for the specific language governing permissions and
limitations under the License.
-->
-<metainfo>
+<metainfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<schemaVersion>2.0</schemaVersion>
<services>
<service>
@@ -32,6 +32,47 @@
<cardinality>1-2</cardinality>
<versionAdvertised>true</versionAdvertised>
<reassignAllowed>true</reassignAllowed>
+ <dependencies>
+ <dependency>
+ <name>HDFS/ZKFC</name>
+ <scope>host</scope>
+ <auto-deploy>
+ <enabled>false</enabled>
+ </auto-deploy>
+ <conditions>
+ <condition xsi:type="propertyExists">
+ <configType>hdfs-site</configType>
+ <property>dfs.nameservices</property>
+ </condition>
+ </conditions>
+ </dependency>
+ <dependency>
+ <name>ZOOKEEPER/ZOOKEEPER_SERVER</name>
+ <scope>host</scope>
+ <auto-deploy>
+ <enabled>false</enabled>
+ </auto-deploy>
+ <conditions>
+ <condition xsi:type="propertyExists">
+ <configType>hdfs-site</configType>
+ <property>dfs.nameservices</property>
+ </condition>
+ </conditions>
+ </dependency>
+ <dependency>
+ <name>HDFS/JOURNALNODE</name>
+ <scope>host</scope>
+ <auto-deploy>
+ <enabled>false</enabled>
+ </auto-deploy>
+ <conditions>
+ <condition xsi:type="propertyExists">
+ <configType>hdfs-site</configType>
+ <property>dfs.nameservices</property>
+ </condition>
+ </conditions>
+ </dependency>
+ </dependencies>
<commandScript>
<script>scripts/namenode.py</script>
<scriptType>PYTHON</scriptType>
http://git-wip-us.apache.org/repos/asf/ambari/blob/bc7a29f5/ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java b/ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java
index ff9af17..6763b7c 100644
--- a/ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java
+++ b/ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java
@@ -212,47 +212,6 @@ public class BlueprintImplTest {
assertTrue(entity.getSecurityType() == SecurityType.NONE);
assertTrue(entity.getSecurityDescriptorReference() == null);
}
- @Test(expected = InvalidTopologyException.class)
- public void testValidateConfigurations__hostGroupConfigForNameNodeHANOZKFC() throws Exception {
- Map<String, Map<String, String>> group2Props = new HashMap<>();
- Map<String, String> group2Category2Props = new HashMap<>();
- group2Props.put("category2", group2Category2Props);
- group2Category2Props.put("prop2", "val");
- // set config for group2 which contains a required property
- Configuration group2Configuration = new Configuration(group2Props, EMPTY_ATTRIBUTES, configuration);
- expect(group2.getConfiguration()).andReturn(group2Configuration).atLeastOnce();
-
- expect(group1.getCardinality()).andReturn("1").atLeastOnce();
- expect(group1.getComponents()).andReturn(Arrays.asList(new Component("NAMENODE"),new Component("ZKFC"))).atLeastOnce();
- expect(group2.getCardinality()).andReturn("1").atLeastOnce();
- expect(group2.getComponents()).andReturn(Arrays.asList(new Component("NAMENODE"),new Component("ZKFC"))).atLeastOnce();
- Map<String, String> category2Props = new HashMap<>();
- properties.put("category2", category2Props);
- category2Props.put("prop2", "val");
- group1Components.add("NAMENODE");
- group1Components.add("ZKFC");
- group2Components.add("NAMENODE");
- expect(stack.getServiceForComponent("NAMENODE")).andReturn("SERVICE2").atLeastOnce();
- expect(stack.getServiceForComponent("ZKFC")).andReturn("SERVICE2").atLeastOnce();
- Map<String, String> hdfsProps = new HashMap<String, String>();
- properties.put("hdfs-site", hdfsProps);
- hdfsProps.put("foo", "val");
- hdfsProps.put("bar", "val");
- hdfsProps.put("dfs.nameservices", "val");
- Map<String, String> hadoopProps = new HashMap<String, String>();
- properties.put("hadoop-env", hadoopProps);
- hadoopProps.put("dfs_ha_initial_namenode_active", "%HOSTGROUP::group1%");
- hadoopProps.put("dfs_ha_initial_namenode_standby", "%HOSTGROUP::group2%");
- replay(stack, group1, group2);
-
- Blueprint blueprint = new BlueprintImpl("test", hostGroups, stack, configuration, null);
- blueprint.validateRequiredProperties();
- BlueprintEntity entity = blueprint.toEntity();
-
- verify(stack, group1, group2);
- assertTrue(entity.getSecurityType() == SecurityType.NONE);
- assertTrue(entity.getSecurityDescriptorReference() == null);
- }
@Test(expected= IllegalArgumentException.class)
public void testValidateConfigurations__hostGroupConfigForNameNodeHAInCorrectHostGroups() throws Exception {
http://git-wip-us.apache.org/repos/asf/ambari/blob/bc7a29f5/ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
----------------------------------------------------------------------
diff --git a/ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java b/ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
index b1de8ef..1501c53 100644
--- a/ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
+++ b/ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
@@ -34,7 +34,9 @@ import java.util.Map;
import org.apache.ambari.server.controller.internal.Stack;
import org.apache.ambari.server.state.AutoDeployInfo;
import org.apache.ambari.server.state.ComponentInfo;
+import org.apache.ambari.server.state.DependencyConditionInfo;
import org.apache.ambari.server.state.DependencyInfo;
+import org.easymock.EasyMock;
import org.easymock.EasyMockRule;
import org.easymock.Mock;
import org.easymock.MockType;
@@ -66,15 +68,22 @@ public class BlueprintValidatorImplTest {
@Mock(type = MockType.NICE)
private DependencyInfo dependency1;
+ @Mock(type = MockType.NICE)
+ private DependencyInfo dependency2;
@Mock(type = MockType.NICE)
private ComponentInfo dependencyComponentInfo;
+ @Mock(type = MockType.NICE)
+ private DependencyConditionInfo dependencyConditionInfo1;
+ @Mock(type = MockType.NICE)
+ private DependencyConditionInfo dependencyConditionInfo2;
private final Collection<String> group1Components = new ArrayList<String>();
private final Collection<String> group2Components = new ArrayList<String>();
private final Collection<String> services = new ArrayList<String>();
private Collection<DependencyInfo> dependencies1 = new ArrayList<DependencyInfo>();
+ private List<DependencyConditionInfo> dependenciesConditionInfos1 = new ArrayList<DependencyConditionInfo>();
private AutoDeployInfo autoDeploy = new AutoDeployInfo();
private Map<String, Map<String, String>> configProperties = new HashMap<String, Map<String, String>>();
private Configuration configuration = new Configuration(configProperties, Collections.<String, Map<String, Map<String, String>>>emptyMap());
@@ -105,13 +114,15 @@ public class BlueprintValidatorImplTest {
expect(stack.getCardinality("component1")).andReturn(new Cardinality("1"));
expect(stack.getCardinality("component2")).andReturn(new Cardinality("1+"));
expect(stack.getCardinality("component3")).andReturn(new Cardinality("1+"));
+ dependenciesConditionInfos1.add(dependencyConditionInfo1);
+ dependenciesConditionInfos1.add(dependencyConditionInfo2);
expect(blueprint.getConfiguration()).andReturn(configuration).anyTimes();
}
@After
public void tearDown() {
- reset(blueprint, stack, group1, group2, dependency1);
+ reset(blueprint, stack, group1, group2, dependency1, dependency2, dependencyConditionInfo1, dependencyConditionInfo2);
}
@Test
@@ -341,4 +352,66 @@ public class BlueprintValidatorImplTest {
verify(group1);
}
+ @Test(expected=InvalidTopologyException.class)
+ public void testWhenComponentIsConditionallyDependentAndOnlyOneOfTheConditionsIsSatisfied() throws Exception {
+ // GIVEN
+ hostGroups.clear();
+ hostGroups.put("group1", group1);
+
+ group1Components.add("component-1");
+ dependencies1.add(dependency1);
+ dependencies1.add(dependency2);
+ services.addAll(Collections.singleton("service-1"));
+
+
+ expect(blueprint.getHostGroupsForComponent("component-1")).andReturn(Arrays.asList(group1)).anyTimes();
+ expect(blueprint.getName()).andReturn("blueprint-1").anyTimes();
+ Map<String, Map<String, String>> properties = new HashMap<String, Map<String, String>>();
+ Map<String, String> typeProps = new HashMap<String, String>();
+ typeProps.put("yarn.resourcemanager.hostname", "testhost");
+ properties.put("yarn-site", typeProps);
+
+ Configuration clusterConfig = new Configuration(properties,
+ Collections.<String, Map<String, Map<String, String>>>emptyMap());
+
+ Cardinality cardinality = new Cardinality("1");
+
+ expect(stack.getComponents("service-1")).andReturn(Arrays.asList("component-1")).anyTimes();
+ expect(stack.getAutoDeployInfo("component-1")).andReturn(autoDeploy).anyTimes();
+ expect(stack.getDependenciesForComponent("component-1")).andReturn(dependencies1).anyTimes();
+ expect(stack.getCardinality("component-1")).andReturn(cardinality).anyTimes();
+
+ AutoDeployInfo dependencyAutoDeploy = null;
+
+ expect(dependency1.getScope()).andReturn("host").anyTimes();
+ expect(dependency1.getAutoDeploy()).andReturn(dependencyAutoDeploy).anyTimes();
+ expect(dependency1.getComponentName()).andReturn("component-d").anyTimes();
+ expect(dependency1.getServiceName()).andReturn("service-d").anyTimes();
+ expect(dependency1.getName()).andReturn("dependency-1").anyTimes();
+ expect(dependency1.hasDependencyConditions()).andReturn(true).anyTimes();
+ expect(dependency1.getDependencyConditions()).andReturn(dependenciesConditionInfos1).anyTimes();
+ expect(dependency2.getScope()).andReturn("host").anyTimes();
+ expect(dependency2.getAutoDeploy()).andReturn(dependencyAutoDeploy).anyTimes();
+ expect(dependency2.getComponentName()).andReturn("component-d").anyTimes();
+ expect(dependency2.getServiceName()).andReturn("service-d").anyTimes();
+ expect(dependency2.getName()).andReturn("dependency-2").anyTimes();
+ expect(dependency2.hasDependencyConditions()).andReturn(false).anyTimes();
+
+ expect(dependencyConditionInfo1.isResolved(EasyMock.anyObject(Map.class))).andReturn(true).anyTimes();
+ expect(dependencyConditionInfo2.isResolved(EasyMock.anyObject(Map.class))).andReturn(false).anyTimes();
+
+
+ expect(dependencyComponentInfo.isClient()).andReturn(false).anyTimes();
+ expect(stack.getComponentInfo("component-d")).andReturn(dependencyComponentInfo).anyTimes();
+
+ replay(blueprint, stack, group1, group2, dependency1, dependency2, dependencyComponentInfo,dependencyConditionInfo1,dependencyConditionInfo2);
+
+ // WHEN
+ BlueprintValidator validator = new BlueprintValidatorImpl(blueprint);
+ validator.validateTopology();
+
+ // THEN
+ verify(group1);
+
+ }
}
[16/31] ambari git commit: Revert "AMBARI-17898 Add Kerberos HTTP
SPNEGO authentication support to Ambari Metrics Monitor (qinliu via dsen)"
Posted by nc...@apache.org.
Revert "AMBARI-17898 Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor (qinliu via dsen)"
This reverts commit 18a88ebb43260f01ff1b5370bc15fb7b6a64055c.
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/a201d092
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/a201d092
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/a201d092
Branch: refs/heads/branch-feature-AMBARI-18634
Commit: a201d092d1dc4ef32dcbf3ba37ee957f33418637
Parents: b5a2bb8
Author: Dmytro Sen <ds...@apache.org>
Authored: Tue Nov 15 18:15:42 2016 +0200
Committer: Dmytro Sen <ds...@apache.org>
Committed: Tue Nov 15 18:15:42 2016 +0200
----------------------------------------------------------------------
ambari-metrics/ambari-metrics-assembly/pom.xml | 1 -
.../conf/unix/metric_monitor.ini | 2 -
.../src/main/python/core/config_reader.py | 10 +-
.../src/main/python/core/emitter.py | 57 +------
.../src/main/python/core/krberr.py | 42 -----
.../main/python/core/spnego_kerberos_auth.py | 163 -------------------
.../src/test/python/core/TestEmitter.py | 26 ---
.../AMBARI_METRICS/0.1.0/kerberos.json | 26 ---
.../AMBARI_METRICS/0.1.0/package/scripts/ams.py | 13 +-
.../0.1.0/package/scripts/params.py | 7 +-
.../package/templates/metric_monitor.ini.j2 | 2 -
11 files changed, 20 insertions(+), 329 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-metrics/ambari-metrics-assembly/pom.xml
----------------------------------------------------------------------
diff --git a/ambari-metrics/ambari-metrics-assembly/pom.xml b/ambari-metrics/ambari-metrics-assembly/pom.xml
index cd45d35..d73a0af 100644
--- a/ambari-metrics/ambari-metrics-assembly/pom.xml
+++ b/ambari-metrics/ambari-metrics-assembly/pom.xml
@@ -535,7 +535,6 @@
<require>${python.ver}</require>
<require>gcc</require>
<require>${python.devel}</require>
- <require>python-kerberos</require>
</requires>
<preremoveScriptlet>
<scriptFile>src/main/package/rpm/preremove.sh</scriptFile>
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini
----------------------------------------------------------------------
diff --git a/ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini b/ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini
index 8f7bcc8..e98c65c 100644
--- a/ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini
+++ b/ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini
@@ -25,8 +25,6 @@ skip_disk_patterns =
[emitter]
send_interval = 60
-kinit_cmd = /usr/bin/kinit -kt /etc/security/keytabs/ams.monitor.keytab amsmon/localhost
-klist_cmd = /usr/bin/klist
[collector]
collector_sleep_interval = 5
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
----------------------------------------------------------------------
diff --git a/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py b/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
index 3ad3aac..890d3ce 100644
--- a/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
+++ b/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
@@ -108,8 +108,6 @@ enable_value_threshold = false
[emitter]
send_interval = 60
-kinit_cmd = /usr/bin/kinit -kt /etc/security/keytabs/ams.monitor.keytab amsmon/localhost
-klist_cmd = /usr/bin/klist
[collector]
collector_sleep_interval = 5
@@ -211,12 +209,6 @@ class Configuration:
def get_send_interval(self):
return int(self.get("emitter", "send_interval", 60))
- def get_kinit_cmd(self):
- return self.get("emitter", "kinit_cmd")
-
- def get_klist_cmd(self):
- return self.get("emitter", "klist_cmd")
-
def get_collector_sleep_interval(self):
return int(self.get("collector", "collector_sleep_interval", 10))
@@ -264,4 +256,4 @@ class Configuration:
return self._ca_cert_file_path
def get_disk_metrics_skip_pattern(self):
- return self.get("default", "skip_disk_patterns")
+ return self.get("default", "skip_disk_patterns")
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py
----------------------------------------------------------------------
diff --git a/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py b/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py
index 27cfa0f..ba3f18e 100644
--- a/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py
+++ b/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py
@@ -20,7 +20,6 @@ limitations under the License.
import logging
import threading
-from spnego_kerberos_auth import SPNEGOKerberosAuth
from security import CachedHTTPSConnection, CachedHTTPConnection
from blacklisted_set import BlacklistedSet
@@ -32,10 +31,6 @@ class Emitter(threading.Thread):
AMS_METRICS_POST_URL = "/ws/v1/timeline/metrics/"
RETRY_SLEEP_INTERVAL = 5
MAX_RETRY_COUNT = 3
- cookie_cached = {}
- kinit_cmd = None
- klist_cmd = None
- spnego_krb_auth = None
"""
Wake up every send interval seconds and empty the application metric map.
"""
@@ -44,10 +39,6 @@ class Emitter(threading.Thread):
logger.debug('Initializing Emitter thread.')
self.lock = threading.Lock()
self.send_interval = config.get_send_interval()
- self.kinit_cmd = config.get_kinit_cmd()
- if self.kinit_cmd:
- logger.debug(self.kinit_cmd)
- self.klist_cmd = config.get_klist_cmd()
self.hostname = config.get_hostname_config()
self.hostname_hash = self.compute_hash(self.hostname)
self._stop_handler = stop_handler
@@ -72,7 +63,6 @@ class Emitter(threading.Thread):
self.submit_metrics()
except Exception, e:
logger.warn('Unable to emit events. %s' % str(e))
- self.cookie_cached = {}
pass
#Wait for the service stop event instead of sleeping blindly
if 0 == self._stop_handler.wait(self.send_interval):
@@ -110,50 +100,17 @@ class Emitter(threading.Thread):
headers = {"Content-Type" : "application/json", "Accept" : "*/*"}
connection = self.get_connection(collector_host)
logger.debug("message to send: %s" % data)
-
- try:
- if self.cookie_cached[connection.host]:
- headers["Cookie"] = self.cookie_cached[connection.host]
- logger.debug("Cookie: %s" % self.cookie_cached[connection.host])
- except Exception, e:
- self.cookie_cached = {}
- pass
-
retry_count = 0
while retry_count < self.MAX_RETRY_COUNT:
response = self.get_response_from_submission(connection, data, headers)
- if response:
- if response.status == 200:
- return True
- if response.status == 401 or response.status == 403:
- self.cookie_cached = {}
- auth_header = response.getheader('www-authenticate', None)
- if auth_header == None:
- logger.warn('www-authenticate header not found')
- else:
- self.spnego_krb_auth = SPNEGOKerberosAuth()
- if self.spnego_krb_auth.get_negotiate_value(auth_header) == '':
- response = self.spnego_krb_auth.authenticate_handshake(connection, "POST", self.AMS_METRICS_POST_URL, data, headers, self.kinit_cmd, self.klist_cmd)
- if response:
- logger.debug("response from authenticate_client: retcode = {0}, reason = {1}"
- .format(response.status, response.reason))
- logger.debug(str(response.read()))
- if response.status == 200:
- logger.debug("response headers: {0}".format(response.getheaders()))
- logger.debug("cookie_cached: %s" % self.cookie_cached)
- set_cookie_header = response.getheader('set-cookie', None)
- if set_cookie_header and self.spnego_krb_auth:
- set_cookie_val = self.spnego_krb_auth.get_hadoop_auth_cookie(set_cookie_header)
- logger.debug("set_cookie: %s" % set_cookie_val)
- if set_cookie_val:
- self.cookie_cached[connection.host] = set_cookie_val
- return True
-
- logger.warn("Retrying after {0} ...".format(self.RETRY_SLEEP_INTERVAL))
- retry_count += 1
- #Wait for the service stop event instead of sleeping blindly
- if 0 == self._stop_handler.wait(self.RETRY_SLEEP_INTERVAL):
+ if response and response.status == 200:
return True
+ else:
+ logger.warn("Retrying after {0} ...".format(self.RETRY_SLEEP_INTERVAL))
+ retry_count += 1
+ #Wait for the service stop event instead of sleeping blindly
+ if 0 == self._stop_handler.wait(self.RETRY_SLEEP_INTERVAL):
+ return True
pass
if retry_count >= self.MAX_RETRY_COUNT:
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py
----------------------------------------------------------------------
diff --git a/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py b/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py
deleted file mode 100644
index 25d7890..0000000
--- a/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/krberr.py
+++ /dev/null
@@ -1,42 +0,0 @@
-#!/usr/bin/env python
-
-'''
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-'''
-
-"""
-Python Kerberos GSS APIs used by spnego_kerberos_auth.py.
-It is used as a place holder for kerberos.py which is not available.
-"""
-
-class KrbError(Exception):
- pass
-
-class GSSError(KrbError):
- pass
-
-def authGSSClientInit(service):
- pass
-
-def authGSSClientClean(context):
- pass
-
-def authGSSClientStep(context, challenge):
- pass
-
-def authGSSClientResponse(context):
- pass
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
----------------------------------------------------------------------
diff --git a/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py b/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
deleted file mode 100644
index a16ca86..0000000
--- a/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py
+++ /dev/null
@@ -1,163 +0,0 @@
-#!/usr/bin/env python
-
-'''
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-'''
-
-import logging
-import httplib
-import os
-
-logger = logging.getLogger()
-try:
- import kerberos
-except ImportError:
- import krberr as kerberos
- logger.warn('import kerberos exception: %s' % str(ImportError))
-pass
-
-class SPNEGOKerberosAuth:
- def __init__(self):
- self.krb_context = None
-
- def authenticate_handshake (self, connection, method, service_url, body, headers, kinit_cmd, klist_cmd):
- # kinit to ensure ticket valid
- self.execute_kinit(kinit_cmd, klist_cmd)
-
- try:
- # Authenticate the client request
- response = self.authenticate_client(connection, method, service_url, body, headers)
-
- # Authenticate the response from the server
- if response:
- self.authenticate_server(response)
- return response
- finally:
- # Clean the client context after the handshake
- self.clean_client_context()
- pass
-
- def execute_kinit(self, kinit_cmd, klist_cmd):
- exit_status = os.system(kinit_cmd)
- logger.debug("kinit exit_status: {0}".format(exit_status))
- logger.debug(os.system(klist_cmd))
- return exit_status
-
- def authenticate_client(self, connection, method, service_url, body, headers):
- service = "HTTP@%s" % connection.host.lower()
- logger.debug("connection: %s", connection)
- logger.debug("service: %s", service)
-
- auth_header = self.get_authorization_header(service)
- logger.debug("Authorization: %s" % auth_header)
-
- # Send 2nd HTTP request with authorization header
- headers['Authorization'] = auth_header
- try:
- connection.request(method, service_url, body, headers)
- response = connection.getresponse()
- except Exception, e:
- logger.warn('2nd HTTP request exception from server: %s' % str(e))
- return None
- pass
- if response:
- logger.debug("2nd HTTP response from server: retcode = {0}, reason = {1}"
- .format(response.status, response.reason))
- logger.debug(str(response.read()))
- logger.debug("response headers: {0}".format(response.getheaders()))
- return response
-
- def get_authorization_header(self, service):
- # Initialize the context object for client-side authentication with a service principal
- try:
- result, self.krb_context = kerberos.authGSSClientInit(service)
- if result == -1:
- logger.warn('authGSSClientInit result: {0}'.format(result))
- return None
- except kerberos.GSSError, e:
- logger.warn('authGSSClientInit exception: %s' % str(e))
- return None
- pass
-
- # Process the first client-side step with the context
- try:
- result = kerberos.authGSSClientStep(self.krb_context, "")
- if result == -1:
- logger.warn('authGSSClientStep result for authenticate client: {0}'.format(result))
- return None
- except kerberos.GSSError, e:
- logger.warn('authGSSClientStep exception for authenticate client: %s' % str(e))
- return None
- pass
-
- # Get the client response from the first client-side step
- try:
- negotiate_value = kerberos.authGSSClientResponse(self.krb_context)
- logger.debug("authGSSClientResponse response:{0}".format(negotiate_value))
- except kerberos.GSSError, e:
- logger.warn('authGSSClientResponse exception: %s' % str(e))
- return None
- pass
-
- # Build the authorization header
- return "Negotiate %s" % negotiate_value
-
- def authenticate_server(self, response):
- auth_header = response.getheader('www-authenticate', None)
- negotiate_value = self.get_negotiate_value(auth_header)
- if negotiate_value == None:
- logger.warn('www-authenticate header not found')
-
- # Process the client-side step with the context and the negotiate value from 2nd HTTP response
- try:
- result = kerberos.authGSSClientStep(self.krb_context, negotiate_value)
- if result == -1:
- logger.warn('authGSSClientStep result for authenticate server: {0}'.format(result))
- except kerberos.GSSError, e:
- logger.warn('authGSSClientStep exception for authenticate server: %s' % str(e))
- result = -1
- pass
- return result
-
- def clean_client_context(self):
- # Destroy the context for client-side authentication
- try:
- result = kerberos.authGSSClientClean(self.krb_context)
- logger.debug("authGSSClientClean result:{0}".format(result))
- except kerberos.GSSError, e:
- logger.warn('authGSSClientClean exception: %s' % str(e))
- result = -1
- pass
- return result
-
- def get_hadoop_auth_cookie(self, set_cookie_header):
- if set_cookie_header:
- for field in set_cookie_header.split(";"):
- if field.startswith('hadoop.auth='):
- return field
- else:
- return None
- return None
-
- def get_negotiate_value(self, auth_header):
- if auth_header:
- for field in auth_header.split(","):
- key, __, value = field.strip().partition(" ")
- if key.lower() == "negotiate":
- return value.strip()
- else:
- return None
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py
----------------------------------------------------------------------
diff --git a/ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py b/ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py
index 722a176..de97292 100644
--- a/ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py
+++ b/ambari-metrics/ambari-metrics-host-monitoring/src/test/python/core/TestEmitter.py
@@ -27,7 +27,6 @@ from only_for_platform import get_platform, PLATFORM_WINDOWS
from mock.mock import patch, MagicMock
from security import CachedHTTPConnection
from blacklisted_set import BlacklistedSet
-from spnego_kerberos_auth import SPNEGOKerberosAuth
if get_platform() != PLATFORM_WINDOWS:
os_distro_value = ('Suse','11','Final')
@@ -87,31 +86,6 @@ class TestEmitter(TestCase):
self.assertEqual(request_mock.call_count, 9)
self.assertUrlData(request_mock)
- @patch.object(OSCheck, "os_distribution", new = MagicMock(return_value = os_distro_value))
- @patch.object(CachedHTTPConnection, "create_connection", new = MagicMock())
- @patch.object(SPNEGOKerberosAuth, "authenticate_handshake")
- @patch.object(CachedHTTPConnection, "getresponse")
- @patch.object(CachedHTTPConnection, "request")
- def test_spnego_negotiation(self, request_mock, getresponse_mock, auth_mock):
- request_mock.return_value = MagicMock()
- getresponse_mock.return_value.status = 401
- getresponse_mock.return_value.getheader.return_value = "Negotiate "
-
- auth_mock.return_value.status = 200
-
- stop_handler = bind_signal_handlers()
-
- config = Configuration()
- application_metric_map = ApplicationMetricMap("host","10.10.10.10")
- application_metric_map.clear()
- application_metric_map.put_metric("APP1", {"metric1":1}, 1)
- emitter = Emitter(config, application_metric_map, stop_handler)
- emitter.submit_metrics()
-
-
- self.assertEqual(request_mock.call_count, 1)
- self.assertUrlData(request_mock)
-
def assertUrlData(self, request_mock):
self.assertEqual(len(request_mock.call_args), 2)
data = request_mock.call_args[0][2]
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json
index a6afda5..51f541f 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json
@@ -9,32 +9,6 @@
],
"components": [
{
- "name": "METRICS_MONITOR",
- "identities": [
- {
- "name": "ams_monitor",
- "principal": {
- "value": "amsmon/_HOST@${realm}",
- "type": "service",
- "configuration": "ams-hbase-security-site/ams.monitor.principal",
- "local_username": "${ams-env/ambari_metrics_user}"
- },
- "keytab": {
- "file": "${keytab_dir}/ams-monitor.keytab",
- "owner": {
- "name": "${ams-env/ambari_metrics_user}",
- "access": "r"
- },
- "group": {
- "name": "${cluster-env/user_group}",
- "access": ""
- },
- "configuration": "ams-hbase-security-site/ams.monitor.keytab"
- }
- }
- ]
- },
- {
"name": "METRICS_COLLECTOR",
"identities": [
{
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py
index 28aba35..6cf743e 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py
@@ -320,9 +320,18 @@ def ams(name=None, action=None):
mode=0644
)
+ # Remove spnego configs from core-site, since AMS does not support spnego (AMBARI-14384)
+ truncated_core_site = {}
+ truncated_core_site.update(params.config['configurations']['core-site'])
+ if 'core-site' in params.config['configurations']:
+ if 'hadoop.http.authentication.type' in params.config['configurations']['core-site']:
+ truncated_core_site.pop('hadoop.http.authentication.type')
+ if 'hadoop.http.filter.initializers' in params.config['configurations']['core-site']:
+ truncated_core_site.pop('hadoop.http.filter.initializers')
+
XmlConfig("core-site.xml",
conf_dir=params.ams_collector_conf_dir,
- configurations=params.config['configurations']['core-site'],
+ configurations=truncated_core_site,
configuration_attributes=params.config['configuration_attributes']['core-site'],
owner=params.ams_user,
group=params.user_group,
@@ -331,7 +340,7 @@ def ams(name=None, action=None):
XmlConfig("core-site.xml",
conf_dir=params.hbase_conf_dir,
- configurations=params.config['configurations']['core-site'],
+ configurations=truncated_core_site,
configuration_attributes=params.config['configuration_attributes']['core-site'],
owner=params.ams_user,
group=params.user_group,
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
index b052999..f04f5c2 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
@@ -270,10 +270,7 @@ service_check_data = functions.get_unique_id_and_date()
user_group = config['configurations']['cluster-env']["user_group"]
hadoop_user = "hadoop"
-kinit_path_local = functions.get_kinit_path(default('/configurations/kerberos-env/executable_search_paths', None))
kinit_cmd = ""
-klist_path_local = functions.get_klist_path(default('/configurations/kerberos-env/executable_search_paths', None))
-klist_cmd = ""
if security_enabled:
_hostname_lowercase = config['hostname'].lower()
@@ -297,9 +294,6 @@ if security_enabled:
regionserver_keytab_path = config['configurations']['ams-hbase-security-site']['hbase.regionserver.keytab.file']
regionserver_jaas_princ = config['configurations']['ams-hbase-security-site']['hbase.regionserver.kerberos.principal'].replace('_HOST',_hostname_lowercase)
- kinit_cmd = '%s -kt %s %s' % (kinit_path_local, config['configurations']['ams-hbase-security-site']['ams.monitor.keytab'], config['configurations']['ams-hbase-security-site']['ams.monitor.principal'].replace('_HOST',_hostname_lowercase))
- klist_cmd = '%s' % klist_path_local
-
#log4j.properties
if (('ams-hbase-log4j' in config['configurations']) and ('content' in config['configurations']['ams-hbase-log4j'])):
hbase_log4j_props = config['configurations']['ams-hbase-log4j']['content']
@@ -322,6 +316,7 @@ hbase_staging_dir = default("/configurations/ams-hbase-site/hbase.bulkload.stagi
hdfs_user_keytab = config['configurations']['hadoop-env']['hdfs_user_keytab']
hdfs_user = config['configurations']['hadoop-env']['hdfs_user']
hdfs_principal_name = config['configurations']['hadoop-env']['hdfs_principal_name']
+kinit_path_local = functions.get_kinit_path(default('/configurations/kerberos-env/executable_search_paths', None))
http://git-wip-us.apache.org/repos/asf/ambari/blob/a201d092/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
----------------------------------------------------------------------
diff --git a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2 b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
index 1f1f117..3823912 100644
--- a/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
+++ b/ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/metric_monitor.ini.j2
@@ -26,8 +26,6 @@ skip_disk_patterns = {{skip_disk_metrics_patterns}}
[emitter]
send_interval = {{metrics_report_interval}}
-kinit_cmd = {{kinit_cmd}}
-klist_cmd = {{klist_cmd}}
[collector]
collector_sleep_interval = 10