You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2015/08/10 15:02:45 UTC
[jira] [Commented] (AMBARI-12688) HDFS check fails after move
NameNode on NN HA cluster
[ https://issues.apache.org/jira/browse/AMBARI-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680063#comment-14680063 ]
Hadoop QA commented on AMBARI-12688:
------------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12749571/AMBARI-12688.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in ambari-server:
org.apache.ambari.server.controller.AmbariManagementControllerTest
Test results: https://builds.apache.org/job/Ambari-trunk-test-patch/3554//testReport/
Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/3554//console
This message is automatically generated.
> HDFS check fails after move NameNode on NN HA cluster
> -----------------------------------------------------
>
> Key: AMBARI-12688
> URL: https://issues.apache.org/jira/browse/AMBARI-12688
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server, blueprints
> Affects Versions: 2.1.1
> Reporter: Dmytro Sen
> Assignee: Dmytro Sen
> Priority: Critical
> Fix For: 2.1.1
>
> Attachments: AMBARI-12688.patch
>
>
> This ticket is related to
> https://issues.apache.org/jira/browse/AMBARI-10750 Initial merge of advanced api provisioning work
> Detailed scenario
> 1. Deploy 3-node cluster from a blueprint (NameNode is on host1. Here NN is mapped to host1 in the cluster topology)
> 2. Enable NN HA (now NameNodes are on host1 and host2)
> 3. Move NN from host1 to host3(now NameNodes are on host3 and host2, but according to cluster topology NN also mapped to host1)
> 4. HDFS service check fails
> Because of the code below, we always have phantom host components, added from topology manager.
> org/apache/ambari/server/utils/StageUtils.java:354
> {code}
> // add components from topology manager
> for (Map.Entry<String, Collection<String>> entry : pendingHostComponents.entrySet()) {
> String hostname = entry.getKey();
> Collection<String> hostComponents = entry.getValue();
> for (String hostComponent : hostComponents) {
> String roleName = componentToClusterInfoKeyMap.get(hostComponent);
> if (null == roleName) {
> roleName = additionalComponentToClusterInfoKeyMap.get(hostComponent);
> }
> if (null == roleName) {
> // even though all mappings are being added, componentToClusterInfoKeyMap is
> // a higher priority lookup
> for (Service service : cluster.getServices().values()) {
> for (ServiceComponent sc : service.getServiceComponents().values()) {
> if (!sc.isClientComponent() && sc.getName().equals(hostComponent)) {
> roleName = hostComponent.toLowerCase() + "_hosts";
> additionalComponentToClusterInfoKeyMap.put(hostComponent, roleName);
> }
> }
> }
> }
> if (roleName != null) {
> SortedSet<Integer> hostsForComponentsHost = hostRolesInfo.get(roleName);
> if (hostsForComponentsHost == null) {
> hostsForComponentsHost = new TreeSet<Integer>();
> hostRolesInfo.put(roleName, hostsForComponentsHost);
> }
> int hostIndex = hostsList.indexOf(hostname);
> if (hostIndex != -1) {
> if (!hostsForComponentsHost.contains(hostIndex)) {
> hostsForComponentsHost.add(hostIndex);
> }
> } else {
> //todo: I don't think that this can happen
> //todo: determine if it can and if so, handle properly
> //todo: if it 'cant' should probably enforce invariant
> throw new RuntimeException("Unable to get host index for host: " + hostname);
> }
> }
> }
> }
> {code}
> clusterHost info is a merged result of actual host components mapping and components mapping from topolgy manager(which is stale if component have been moved/removed)
> {noformat}
> "clusterHostInfo": {
> ...
> "namenode_host": [
> "host1",
> "host2",
> "host3"
> ],
> {noformat}
> I think clusterHostInfo is incorrect if any component has been moved or removed from the node where it had been initially deployed by a blueprint.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)