You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ambari.apache.org by JOAQUIN GUANTER GONZALBEZ <xi...@tid.es> on 2013/04/03 16:49:28 UTC
INSTALL_FAILED status but no logs?
Hello,
I am trying to create a cluster with HDFS programmatically by targeting the REST APIs in the Ambari Server directly instead of using the Web frontend. I set the HDFS service state to "INSTALLED" and I can see my host components entering the state "INSTALLING", but after a while, they transition to "INSTALL_FAILED". The problem is that neither the ambari-server.log/out nor the ambari-agent.log/out have any error messages.
I'm not sure if there are any other sources of information I should be looking at for errors, or if I'm doing anything wrong when calling the REST APIs (all the API calls return successfully, though). My test is a NodeJS program which I have attached in case anyone wants to take a look at it. The ambari.js code is a supporting object model that enables easy calls to the REST API.
Does anyone know how can investigate why the state of my host components is ending up in INSTALL_FAILED?
Thanks,
Ximo.
________________________________
Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra pol?tica de env?o y recepci?n de correo electr?nico en el enlace situado m?s abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
http://www.tid.es/ES/PAGINAS/disclaimer.aspx
Re: INSTALL_FAILED status but no logs?
Posted by Yusaku Sako <yu...@hortonworks.com>.
Also, you can look under cd /var/lib/ambari-agent/data.
You will see all the Puppet scripts ran by ambari-agent, stdout, and stderr.
Yusaku
On Wed, Apr 3, 2013 at 8:22 AM, JOAQUIN GUANTER GONZALBEZ <xi...@tid.es>wrote:
> Sweet, I think I’ve got a lead on what’s wrong (DNS/FQDN issues) J
>
>
>
> Thanks!
>
> Ximo.
>
>
>
> *De:* Yusaku Sako [mailto:yusaku@hortonworks.com]
> *Enviado el:* miércoles, 03 de abril de 2013 17:07
>
> *Para:* ambari-user@incubator.apache.org
> *Asunto:* Re: INSTALL_FAILED status but no logs?
>
>
>
> Hi Ximo,
>
>
>
> You can poll on the status of the service install.
>
> When you issue the API call to change the service state to INSTALLED, the
> server responds with status code 202 with an href for the request progress.
>
> It looks something like
> http://localhost:8080/api/v1/clusters/cluster/requests/1.
>
> If you do a GET on that URL, you can see the tasks (but not the details
> like stdout and stderr).
>
> You can see all the details of the install tasks by appending additional
> query parameters like so:
>
> http://localhost:8080/api/v1/clusters/cluster/requests/1?fields=tasks/*
>
>
>
> Note that this pattern is the same across all API operations that are
> synchronous.
>
>
>
> Let me know if this works for you.
>
>
>
> Yusaku
>
>
>
> On Wed, Apr 3, 2013 at 7:59 AM, JOAQUIN GUANTER GONZALBEZ <xi...@tid.es>
> wrote:
>
> Hi Sylvain,
>
> Thanks for the tips. Unfortunately, /var/log/ambari-* has no useful
> information and /var/log/hadoop doesn't even exist :(
>
> Ximo.
>
> -----Mensaje original-----
> De: Sylvain Trias [mailto:bozo@ovh.net]
> Enviado el: miércoles, 03 de abril de 2013 16:53
> Para: ambari-user@incubator.apache.org
> Asunto: Re: INSTALL_FAILED status but no logs?
>
>
> On Wed, Apr 03, 2013 at 02:49:28PM +0000, JOAQUIN GUANTER GONZALBEZ wrote:
> > Hello,
> >
> > I am trying to create a cluster with HDFS programmatically by targeting
> the REST APIs in the Ambari Server directly instead of using the Web
> frontend. I set the HDFS service state to "INSTALLED" and I can see my host
> components entering the state "INSTALLING", but after a while, they
> transition to "INSTALL_FAILED". The problem is that neither the
> ambari-server.log/out nor the ambari-agent.log/out have any error messages.
> >
> > I'm not sure if there are any other sources of information I should be
> looking at for errors, or if I'm doing anything wrong when calling the REST
> APIs (all the API calls return successfully, though). My test is a NodeJS
> program which I have attached in case anyone wants to take a look at it.
> The ambari.js code is a supporting object model that enables easy calls to
> the REST API.
> >
> > Does anyone know how can investigate why the state of my host components
> is ending up in INSTALL_FAILED?
>
> Hi, I'm doing the same thing (and manage to install my first cluster
> today) When I have that kind of failure I try to log in to the host and
> manually check logs in /var/log/hadoop... or /var/log/ambari-* it helped me
> quite a lot
>
>
> >
> > Thanks,
> > Ximo.
> >
> > ________________________________
> >
> > Este mensaje se dirige exclusivamente a su destinatario. Puede consultar
> nuestra pol?tica de env?o y recepci?n de correo electr?nico en el enlace
> situado m?s abajo.
> > This message is intended exclusively for its addressee. We only send and
> receive email on the basis of the terms set out at:
> > http://www.tid.es/ES/PAGINAS/disclaimer.aspx
>
> > ;'use strict';
> >
> > var Q = require('q');
> > var http = require('http');
> > var PATH_PREFIX = '/api/v1/';
> >
> > var performRequest = function (request_options) {
> > var result = Q.defer();
> > var body = request_options.body;
> > delete request_options.body;
> > var req = http.request(request_options, function(res) {
> > var body = '';
> > res.setEncoding('utf8');
> > res.on('data', function(chunk) {
> > body += chunk;
> > });
> > res.on('end', function() {
> > if (body) {
> > res.body = body;
> > try {
> > var json = JSON.parse(body);
> > res.body = json;
> > } catch(ignored) { }
> > }
> > if (Math.floor(res.statusCode / 100) === 2)
> result.resolve(res);
> > else result.reject(res);
> > });
> > });
> > if (body) req.write(JSON.stringify(body));
> > req.end();
> > return result.promise;
> > };
> >
> > var isString = function(obj) {
> > return Object.prototype.toString.call(obj) == '[object String]';
> > };
> >
> > var Service = function(cluster, serviceInfo) {
> > var service = this;
> > this.cluster = cluster;
> > this.name = serviceInfo.ServiceInfo.service_name;
> > this.state = serviceInfo.ServiceInfo.state;
> > this.componentNames =
> serviceInfo.components.map(function(componentInfo) {
> > return componentInfo.ServiceComponentInfo.component_name;
> > });
> > this.desiredConfigs = serviceInfo.ServiceInfo.desired_configs;
> > this.addComponent = function(componentName) {
> > var request_options = service.cluster.server.getBaseRequest();
> > request_options.method = 'POST';
> > request_options.path = PATH_PREFIX + 'clusters/' +
> service.cluster.name + '/services/' + service.name + '/components/' +
> componentName;
> > return performRequest(request_options).then(function(res) {
> > return componentName;
> > });
> > };
> > this.updateCluster = function() { throw new Error('Not
> implemented'); };
> > this.removeComponent = function() { throw new Error('Not
> implemented'); };
> > this.applyConfiguration = function(type, tag) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'PUT';
> > request_options.path = PATH_PREFIX + 'clusters/' +
> service.cluster.name + '/services/' + service.name;
> > request_options.body = {
> > config: {}
> > };
> > request_options.body.config[type] = tag;
> > return performRequest(request_options);
> > };
> > this.install = function() {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'PUT';
> > request_options.path = PATH_PREFIX + 'clusters/' +
> service.cluster.name + '/services/' + service.name;
> > request_options.body = {
> > ServiceInfo: {
> > state: 'INSTALLED'
> > }
> > };
> > return performRequest(request_options);
> > };
> > this.start = function() {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'PUT';
> > request_options.path = PATH_PREFIX + 'clusters/' +
> service.cluster.name + '/services/' + service.name;
> > request_options.body = {
> > ServiceInfo: {
> > state: 'STARTED'
> > }
> > };
> > return performRequest(request_options);
> > };
> > return this;
> > };
> >
> > var Host = function(cluster, hostInfo) {
> > var host = this;
> > this.cluster = cluster;
> > this.name = hostInfo.Hosts.host_name;
> > this.ip = hostInfo.Hosts.ip;
> > this.status = hostInfo.Hosts.host_status;
> > this.addComponents = function(components) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'POST';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/hosts?Hosts/host_name=' +
> host.name;
> > request_options.body = {
> > "host_components": components.map(function(component) {
> > return { HostRoles: {
> > "component_name": component
> > }};
> > })
> > };
> > return performRequest(request_options);
> > };
> > this.addComponent = function(component) {
> > return host.addComponents([component]);
> > };
> > };
> >
> > var Cluster = function(server, clusterInfo) {
> > var cluster = this;
> > this.server = server;
> > this.name = clusterInfo.Clusters.cluster_name;
> > this.serviceNames = clusterInfo.services.map(function(service) {
> > return service.ServiceInfo.service_name;
> > });
> > this.hostNames = clusterInfo.hosts.map(function(host) {
> > return host.Hosts.host_name;
> > });
> > this.configurations =
> clusterInfo.configurations.map(function(config) {
> > return {
> > type: config.type,
> > tag: config.tag
> > };
> > });
> > this.getService = function(serviceName) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'GET';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/services/' + serviceName;
> > return performRequest(request_options).then(function(res) {
> > return new Service(cluster, res.body);
> > });
> > };
> > this.addService = function(serviceName) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'POST';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/services';
> > request_options.body = {
> > ServiceInfo: { service_name: serviceName }
> > };
> > return performRequest(request_options).then(function(res) {
> > return cluster.getService(serviceName);
> > });
> > };
> > this.removeService = function(service) {
> > if (!isString(service)) {
> > service = service.name;
> > }
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'DELETE';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/services/' + service;
> > return performRequest(request_options);
> > };
> > this.getConfiguration = function() { throw new Error('Not
> implemented'); };
> > this.addConfiguration = function(type, tag, config) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'POST';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/configurations';
> > request_options.body = {
> > type: type,
> > tag: tag,
> > properties: config,
> > };
> > return performRequest(request_options);
> > };
> > this.removeConfiguration = function() { throw new Error('Not
> implemented'); };
> > this.updateConfiguration = function() { throw new Error('Not
> implemented'); };
> > this.getHost = function(hostname) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'GET';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/hosts/' + hostname;
> > return performRequest(request_options).then(function(res) {
> > return new Host(cluster, res.body);
> > });
> > };
> > this.addHost = function(hostname) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'POST';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/hosts';
> > request_options.body = { Hosts: { host_name: hostname} };
> > return performRequest(request_options).then(function(res) {
> > return cluster.getHost(hostname);
> > });
> > };
> > this.removeHost = function(hostname) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'DELETE';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/hosts';
> > return performRequest(request_options);
> > };
> > this.updateHost = function() { throw new Error('Not implemented'); };
> > return this;
> > };
> >
> > exports.Server = function(host, port, user, pass) {
> > var server = this;
> > this.host = host;
> > this.port = port;
> > this.user = user;
> > this.pass = pass;
> > this.getBaseRequest = function() {
> > return {
> > host: host,
> > port: port.toString(),
> > auth: user + ':' + pass
> > };
> > };
> > this.listClusterNames = function() {
> > var request = server.getBaseRequest();
> > request.path = PATH_PREFIX + 'clusters';
> > request.method = 'GET';
> > return performRequest(request).then(function(res) {
> > return res.body.items.map(function(clusterInfo) {
> > return clusterInfo.Clusters.cluster_name;
> > });
> > });
> > };
> > this.getCluster = function(clusterName) {
> > var request = server.getBaseRequest();
> > request.path = PATH_PREFIX + 'clusters/' + clusterName;
> > request.method = 'GET';
> > return performRequest(request).then(function(res) {
> > return new Cluster(server, res.body);
> > });
> > };
> > this.createCluster = function(clusterName, version) {
> > var request = server.getBaseRequest();
> > request.path = PATH_PREFIX + 'clusters/' + clusterName;
> > request.method = 'POST';
> > request.body = { "Clusters": { "version": version }};
> > return performRequest(request).then(function(res) {
> > return server.getCluster(clusterName);
> > });
> > };
> > this.updateCluster = function() { throw new Error('Not
> implemented'); };
> > this.removeCluster = function(clusterName) {
> > var request = server.getBaseRequest();
> > request.path = PATH_PREFIX + 'clusters/' + clusterName;
> > request.method = 'DELETE';
> > return performRequest(request);
> > };
> > return this;
> > };
>
> > ;'use strict';
> >
> > process.on('uncaughtException', function(err) {
> > console.log(err);
> > console.log(err.stack);
> > });
> >
> > var Q = require('q');
> > var ambari = require('./ambari');
> > var server = new ambari.Server('localhost', 8080, 'admin', 'admin');
> >
> > server.createCluster('test', 'HDP-1.2.0').then(function(cluster) {
> > var serviceD = Q.defer();
> > var servicePromise = serviceD.promise;
> > Q.all([
> > cluster.addService('HDFS'),
> > cluster.addConfiguration('core-site', '1', {
> > 'fs.default.name' : 'local',
> > 'fs.checkpoint.dir': '/hdfs/checkpoint',
> > 'hadoop.security.authentication': 'simple',
> > 'hadoop.security.authorization': 'false'
> > }),
> > cluster.addConfiguration('hdfs-site', '1', {
> > 'dfs.name.dir': '/hdfs/name',
> > 'dfs.support.append': 'true',
> > 'dfs.webhdfs.enabled': 'false',
> > 'dfs.datanode.failed.volume.tolerated': '0',
> > 'dfs.block.local-path-access.user': 'root',
> > 'dfs.data.dir': '/hdfs/data',
> > 'dfs.replication': '1',
> > 'dfs.datanode.address': '0.0.0.0:50010',
> > 'dfs.datanode.http.address': '0.0.0.0:50075',
> > 'dfs.http.address': '0.0.0.0:50070',
> >
> > })
> > ]).spread(function(service) {
> > serviceD.resolve(service);
> > });
> > return servicePromise.then(function(service) {
> > return Q.all([
> > service.addComponent('NAMENODE'),
> > service.addComponent('DATANODE'),
> > service.addComponent('HDFS_CLIENT')
> > ]).then(function() { return service });
> > }).then(function(service) {
> > return Q.all([
> >
> service.cluster.addHost('1ddf42d1-0539-45f7-807a-a846a0cab9d8').then(function(host)
> {
> > return host.addComponents(['NAMENODE', 'DATANODE',
> 'HDFS_CLIENT']);
> > }),
> > service.applyConfiguration('core-site', '1'),
> > service.applyConfiguration('hdfs-site', '1')
> > ]).then(function() { return service; });
> > }).then(function(service) {
> > return service.install().then(function() { return service; });
> > });
> > }).done();
>
>
> ________________________________
>
> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar
> nuestra política de envío y recepción de correo electrónico en el enlace
> situado más abajo.
>
> This message is intended exclusively for its addressee. We only send and
> receive email on the basis of the terms set out at:
> http://www.tid.es/ES/PAGINAS/disclaimer.aspx
>
>
>
> ------------------------------
>
> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar
> nuestra política de envío y recepción de correo electrónico en el enlace
> situado más abajo.
> This message is intended exclusively for its addressee. We only send and
> receive email on the basis of the terms set out at:
> http://www.tid.es/ES/PAGINAS/disclaimer.aspx
>
RE: INSTALL_FAILED status but no logs?
Posted by JOAQUIN GUANTER GONZALBEZ <xi...@tid.es>.
Sweet, I think I've got a lead on what's wrong (DNS/FQDN issues) :)
Thanks!
Ximo.
De: Yusaku Sako [mailto:yusaku@hortonworks.com]
Enviado el: miércoles, 03 de abril de 2013 17:07
Para: ambari-user@incubator.apache.org
Asunto: Re: INSTALL_FAILED status but no logs?
Hi Ximo,
You can poll on the status of the service install.
When you issue the API call to change the service state to INSTALLED, the server responds with status code 202 with an href for the request progress.
It looks something like http://localhost:8080/api/v1/clusters/cluster/requests/1.
If you do a GET on that URL, you can see the tasks (but not the details like stdout and stderr).
You can see all the details of the install tasks by appending additional query parameters like so:
http://localhost:8080/api/v1/clusters/cluster/requests/1?fields=tasks/*
Note that this pattern is the same across all API operations that are synchronous.
Let me know if this works for you.
Yusaku
On Wed, Apr 3, 2013 at 7:59 AM, JOAQUIN GUANTER GONZALBEZ <xi...@tid.es>> wrote:
Hi Sylvain,
Thanks for the tips. Unfortunately, /var/log/ambari-* has no useful information and /var/log/hadoop doesn't even exist :(
Ximo.
-----Mensaje original-----
De: Sylvain Trias [mailto:bozo@ovh.net<ma...@ovh.net>]
Enviado el: miércoles, 03 de abril de 2013 16:53
Para: ambari-user@incubator.apache.org<ma...@incubator.apache.org>
Asunto: Re: INSTALL_FAILED status but no logs?
On Wed, Apr 03, 2013 at 02:49:28PM +0000, JOAQUIN GUANTER GONZALBEZ wrote:
> Hello,
>
> I am trying to create a cluster with HDFS programmatically by targeting the REST APIs in the Ambari Server directly instead of using the Web frontend. I set the HDFS service state to "INSTALLED" and I can see my host components entering the state "INSTALLING", but after a while, they transition to "INSTALL_FAILED". The problem is that neither the ambari-server.log/out nor the ambari-agent.log/out have any error messages.
>
> I'm not sure if there are any other sources of information I should be looking at for errors, or if I'm doing anything wrong when calling the REST APIs (all the API calls return successfully, though). My test is a NodeJS program which I have attached in case anyone wants to take a look at it. The ambari.js code is a supporting object model that enables easy calls to the REST API.
>
> Does anyone know how can investigate why the state of my host components is ending up in INSTALL_FAILED?
Hi, I'm doing the same thing (and manage to install my first cluster today) When I have that kind of failure I try to log in to the host and manually check logs in /var/log/hadoop... or /var/log/ambari-* it helped me quite a lot
>
> Thanks,
> Ximo.
>
> ________________________________
>
> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra pol?tica de env?o y recepci?n de correo electr?nico en el enlace situado m?s abajo.
> This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
> http://www.tid.es/ES/PAGINAS/disclaimer.aspx
> ;'use strict';
>
> var Q = require('q');
> var http = require('http');
> var PATH_PREFIX = '/api/v1/';
>
> var performRequest = function (request_options) {
> var result = Q.defer();
> var body = request_options.body;
> delete request_options.body;
> var req = http.request(request_options, function(res) {
> var body = '';
> res.setEncoding('utf8');
> res.on('data', function(chunk) {
> body += chunk;
> });
> res.on('end', function() {
> if (body) {
> res.body = body;
> try {
> var json = JSON.parse(body);
> res.body = json;
> } catch(ignored) { }
> }
> if (Math.floor(res.statusCode / 100) === 2) result.resolve(res);
> else result.reject(res);
> });
> });
> if (body) req.write(JSON.stringify(body));
> req.end();
> return result.promise;
> };
>
> var isString = function(obj) {
> return Object.prototype.toString.call(obj) == '[object String]';
> };
>
> var Service = function(cluster, serviceInfo) {
> var service = this;
> this.cluster = cluster;
> this.name<http://this.name> = serviceInfo.ServiceInfo.service_name;
> this.state = serviceInfo.ServiceInfo.state;
> this.componentNames = serviceInfo.components.map(function(componentInfo) {
> return componentInfo.ServiceComponentInfo.component_name;
> });
> this.desiredConfigs = serviceInfo.ServiceInfo.desired_configs;
> this.addComponent = function(componentName) {
> var request_options = service.cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name<http://service.cluster.name> + '/services/' + service.name<http://service.name> + '/components/' + componentName;
> return performRequest(request_options).then(function(res) {
> return componentName;
> });
> };
> this.updateCluster = function() { throw new Error('Not implemented'); };
> this.removeComponent = function() { throw new Error('Not implemented'); };
> this.applyConfiguration = function(type, tag) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'PUT';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name<http://service.cluster.name> + '/services/' + service.name<http://service.name>;
> request_options.body = {
> config: {}
> };
> request_options.body.config[type] = tag;
> return performRequest(request_options);
> };
> this.install = function() {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'PUT';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name<http://service.cluster.name> + '/services/' + service.name<http://service.name>;
> request_options.body = {
> ServiceInfo: {
> state: 'INSTALLED'
> }
> };
> return performRequest(request_options);
> };
> this.start = function() {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'PUT';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name<http://service.cluster.name> + '/services/' + service.name<http://service.name>;
> request_options.body = {
> ServiceInfo: {
> state: 'STARTED'
> }
> };
> return performRequest(request_options);
> };
> return this;
> };
>
> var Host = function(cluster, hostInfo) {
> var host = this;
> this.cluster = cluster;
> this.name<http://this.name> = hostInfo.Hosts.host_name;
> this.ip = hostInfo.Hosts.ip;
> this.status = hostInfo.Hosts.host_status;
> this.addComponents = function(components) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name<http://cluster.name> + '/hosts?Hosts/host_name=' + host.name<http://host.name>;
> request_options.body = {
> "host_components": components.map(function(component) {
> return { HostRoles: {
> "component_name": component
> }};
> })
> };
> return performRequest(request_options);
> };
> this.addComponent = function(component) {
> return host.addComponents([component]);
> };
> };
>
> var Cluster = function(server, clusterInfo) {
> var cluster = this;
> this.server = server;
> this.name<http://this.name> = clusterInfo.Clusters.cluster_name;
> this.serviceNames = clusterInfo.services.map(function(service) {
> return service.ServiceInfo.service_name;
> });
> this.hostNames = clusterInfo.hosts.map(function(host) {
> return host.Hosts.host_name;
> });
> this.configurations = clusterInfo.configurations.map(function(config) {
> return {
> type: config.type,
> tag: config.tag
> };
> });
> this.getService = function(serviceName) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'GET';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name<http://cluster.name> + '/services/' + serviceName;
> return performRequest(request_options).then(function(res) {
> return new Service(cluster, res.body);
> });
> };
> this.addService = function(serviceName) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name<http://cluster.name> + '/services';
> request_options.body = {
> ServiceInfo: { service_name: serviceName }
> };
> return performRequest(request_options).then(function(res) {
> return cluster.getService(serviceName);
> });
> };
> this.removeService = function(service) {
> if (!isString(service)) {
> service = service.name<http://service.name>;
> }
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'DELETE';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name<http://cluster.name> + '/services/' + service;
> return performRequest(request_options);
> };
> this.getConfiguration = function() { throw new Error('Not implemented'); };
> this.addConfiguration = function(type, tag, config) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name<http://cluster.name> + '/configurations';
> request_options.body = {
> type: type,
> tag: tag,
> properties: config,
> };
> return performRequest(request_options);
> };
> this.removeConfiguration = function() { throw new Error('Not implemented'); };
> this.updateConfiguration = function() { throw new Error('Not implemented'); };
> this.getHost = function(hostname) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'GET';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name<http://cluster.name> + '/hosts/' + hostname;
> return performRequest(request_options).then(function(res) {
> return new Host(cluster, res.body);
> });
> };
> this.addHost = function(hostname) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name<http://cluster.name> + '/hosts';
> request_options.body = { Hosts: { host_name: hostname} };
> return performRequest(request_options).then(function(res) {
> return cluster.getHost(hostname);
> });
> };
> this.removeHost = function(hostname) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'DELETE';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name<http://cluster.name> + '/hosts';
> return performRequest(request_options);
> };
> this.updateHost = function() { throw new Error('Not implemented'); };
> return this;
> };
>
> exports.Server = function(host, port, user, pass) {
> var server = this;
> this.host = host;
> this.port = port;
> this.user = user;
> this.pass = pass;
> this.getBaseRequest = function() {
> return {
> host: host,
> port: port.toString(),
> auth: user + ':' + pass
> };
> };
> this.listClusterNames = function() {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters';
> request.method = 'GET';
> return performRequest(request).then(function(res) {
> return res.body.items.map(function(clusterInfo) {
> return clusterInfo.Clusters.cluster_name;
> });
> });
> };
> this.getCluster = function(clusterName) {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters/' + clusterName;
> request.method = 'GET';
> return performRequest(request).then(function(res) {
> return new Cluster(server, res.body);
> });
> };
> this.createCluster = function(clusterName, version) {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters/' + clusterName;
> request.method = 'POST';
> request.body = { "Clusters": { "version": version }};
> return performRequest(request).then(function(res) {
> return server.getCluster(clusterName);
> });
> };
> this.updateCluster = function() { throw new Error('Not implemented'); };
> this.removeCluster = function(clusterName) {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters/' + clusterName;
> request.method = 'DELETE';
> return performRequest(request);
> };
> return this;
> };
> ;'use strict';
>
> process.on('uncaughtException', function(err) {
> console.log(err);
> console.log(err.stack);
> });
>
> var Q = require('q');
> var ambari = require('./ambari');
> var server = new ambari.Server('localhost', 8080, 'admin', 'admin');
>
> server.createCluster('test', 'HDP-1.2.0').then(function(cluster) {
> var serviceD = Q.defer();
> var servicePromise = serviceD.promise;
> Q.all([
> cluster.addService('HDFS'),
> cluster.addConfiguration('core-site', '1', {
> 'fs.default.name<http://fs.default.name>' : 'local',
> 'fs.checkpoint.dir': '/hdfs/checkpoint',
> 'hadoop.security.authentication': 'simple',
> 'hadoop.security.authorization': 'false'
> }),
> cluster.addConfiguration('hdfs-site', '1', {
> 'dfs.name.dir': '/hdfs/name',
> 'dfs.support.append': 'true',
> 'dfs.webhdfs.enabled': 'false',
> 'dfs.datanode.failed.volume.tolerated': '0',
> 'dfs.block.local-path-access.user': 'root',
> 'dfs.data.dir': '/hdfs/data',
> 'dfs.replication': '1',
> 'dfs.datanode.address': '0.0.0.0:50010<http://0.0.0.0:50010>',
> 'dfs.datanode.http.address': '0.0.0.0:50075<http://0.0.0.0:50075>',
> 'dfs.http.address': '0.0.0.0:50070<http://0.0.0.0:50070>',
>
> })
> ]).spread(function(service) {
> serviceD.resolve(service);
> });
> return servicePromise.then(function(service) {
> return Q.all([
> service.addComponent('NAMENODE'),
> service.addComponent('DATANODE'),
> service.addComponent('HDFS_CLIENT')
> ]).then(function() { return service });
> }).then(function(service) {
> return Q.all([
> service.cluster.addHost('1ddf42d1-0539-45f7-807a-a846a0cab9d8').then(function(host) {
> return host.addComponents(['NAMENODE', 'DATANODE', 'HDFS_CLIENT']);
> }),
> service.applyConfiguration('core-site', '1'),
> service.applyConfiguration('hdfs-site', '1')
> ]).then(function() { return service; });
> }).then(function(service) {
> return service.install().then(function() { return service; });
> });
> }).done();
________________________________
Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
http://www.tid.es/ES/PAGINAS/disclaimer.aspx
________________________________
Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
http://www.tid.es/ES/PAGINAS/disclaimer.aspx
Re: INSTALL_FAILED status but no logs?
Posted by Yusaku Sako <yu...@hortonworks.com>.
Hi Ximo,
You can poll on the status of the service install.
When you issue the API call to change the service state to INSTALLED, the
server responds with status code 202 with an href for the request progress.
It looks something like
http://localhost:8080/api/v1/clusters/cluster/requests/1.
If you do a GET on that URL, you can see the tasks (but not the details
like stdout and stderr).
You can see all the details of the install tasks by appending additional
query parameters like so:
http://localhost:8080/api/v1/clusters/cluster/requests/1?fields=tasks/*
Note that this pattern is the same across all API operations that are
synchronous.
Let me know if this works for you.
Yusaku
On Wed, Apr 3, 2013 at 7:59 AM, JOAQUIN GUANTER GONZALBEZ <xi...@tid.es>wrote:
> Hi Sylvain,
>
> Thanks for the tips. Unfortunately, /var/log/ambari-* has no useful
> information and /var/log/hadoop doesn't even exist :(
>
> Ximo.
>
> -----Mensaje original-----
> De: Sylvain Trias [mailto:bozo@ovh.net]
> Enviado el: miércoles, 03 de abril de 2013 16:53
> Para: ambari-user@incubator.apache.org
> Asunto: Re: INSTALL_FAILED status but no logs?
>
> On Wed, Apr 03, 2013 at 02:49:28PM +0000, JOAQUIN GUANTER GONZALBEZ wrote:
> > Hello,
> >
> > I am trying to create a cluster with HDFS programmatically by targeting
> the REST APIs in the Ambari Server directly instead of using the Web
> frontend. I set the HDFS service state to "INSTALLED" and I can see my host
> components entering the state "INSTALLING", but after a while, they
> transition to "INSTALL_FAILED". The problem is that neither the
> ambari-server.log/out nor the ambari-agent.log/out have any error messages.
> >
> > I'm not sure if there are any other sources of information I should be
> looking at for errors, or if I'm doing anything wrong when calling the REST
> APIs (all the API calls return successfully, though). My test is a NodeJS
> program which I have attached in case anyone wants to take a look at it.
> The ambari.js code is a supporting object model that enables easy calls to
> the REST API.
> >
> > Does anyone know how can investigate why the state of my host components
> is ending up in INSTALL_FAILED?
>
> Hi, I'm doing the same thing (and manage to install my first cluster
> today) When I have that kind of failure I try to log in to the host and
> manually check logs in /var/log/hadoop... or /var/log/ambari-* it helped me
> quite a lot
>
>
> >
> > Thanks,
> > Ximo.
> >
> > ________________________________
> >
> > Este mensaje se dirige exclusivamente a su destinatario. Puede consultar
> nuestra pol?tica de env?o y recepci?n de correo electr?nico en el enlace
> situado m?s abajo.
> > This message is intended exclusively for its addressee. We only send and
> receive email on the basis of the terms set out at:
> > http://www.tid.es/ES/PAGINAS/disclaimer.aspx
>
> > ;'use strict';
> >
> > var Q = require('q');
> > var http = require('http');
> > var PATH_PREFIX = '/api/v1/';
> >
> > var performRequest = function (request_options) {
> > var result = Q.defer();
> > var body = request_options.body;
> > delete request_options.body;
> > var req = http.request(request_options, function(res) {
> > var body = '';
> > res.setEncoding('utf8');
> > res.on('data', function(chunk) {
> > body += chunk;
> > });
> > res.on('end', function() {
> > if (body) {
> > res.body = body;
> > try {
> > var json = JSON.parse(body);
> > res.body = json;
> > } catch(ignored) { }
> > }
> > if (Math.floor(res.statusCode / 100) === 2)
> result.resolve(res);
> > else result.reject(res);
> > });
> > });
> > if (body) req.write(JSON.stringify(body));
> > req.end();
> > return result.promise;
> > };
> >
> > var isString = function(obj) {
> > return Object.prototype.toString.call(obj) == '[object String]';
> > };
> >
> > var Service = function(cluster, serviceInfo) {
> > var service = this;
> > this.cluster = cluster;
> > this.name = serviceInfo.ServiceInfo.service_name;
> > this.state = serviceInfo.ServiceInfo.state;
> > this.componentNames =
> serviceInfo.components.map(function(componentInfo) {
> > return componentInfo.ServiceComponentInfo.component_name;
> > });
> > this.desiredConfigs = serviceInfo.ServiceInfo.desired_configs;
> > this.addComponent = function(componentName) {
> > var request_options = service.cluster.server.getBaseRequest();
> > request_options.method = 'POST';
> > request_options.path = PATH_PREFIX + 'clusters/' +
> service.cluster.name + '/services/' + service.name + '/components/' +
> componentName;
> > return performRequest(request_options).then(function(res) {
> > return componentName;
> > });
> > };
> > this.updateCluster = function() { throw new Error('Not
> implemented'); };
> > this.removeComponent = function() { throw new Error('Not
> implemented'); };
> > this.applyConfiguration = function(type, tag) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'PUT';
> > request_options.path = PATH_PREFIX + 'clusters/' +
> service.cluster.name + '/services/' + service.name;
> > request_options.body = {
> > config: {}
> > };
> > request_options.body.config[type] = tag;
> > return performRequest(request_options);
> > };
> > this.install = function() {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'PUT';
> > request_options.path = PATH_PREFIX + 'clusters/' +
> service.cluster.name + '/services/' + service.name;
> > request_options.body = {
> > ServiceInfo: {
> > state: 'INSTALLED'
> > }
> > };
> > return performRequest(request_options);
> > };
> > this.start = function() {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'PUT';
> > request_options.path = PATH_PREFIX + 'clusters/' +
> service.cluster.name + '/services/' + service.name;
> > request_options.body = {
> > ServiceInfo: {
> > state: 'STARTED'
> > }
> > };
> > return performRequest(request_options);
> > };
> > return this;
> > };
> >
> > var Host = function(cluster, hostInfo) {
> > var host = this;
> > this.cluster = cluster;
> > this.name = hostInfo.Hosts.host_name;
> > this.ip = hostInfo.Hosts.ip;
> > this.status = hostInfo.Hosts.host_status;
> > this.addComponents = function(components) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'POST';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/hosts?Hosts/host_name=' +
> host.name;
> > request_options.body = {
> > "host_components": components.map(function(component) {
> > return { HostRoles: {
> > "component_name": component
> > }};
> > })
> > };
> > return performRequest(request_options);
> > };
> > this.addComponent = function(component) {
> > return host.addComponents([component]);
> > };
> > };
> >
> > var Cluster = function(server, clusterInfo) {
> > var cluster = this;
> > this.server = server;
> > this.name = clusterInfo.Clusters.cluster_name;
> > this.serviceNames = clusterInfo.services.map(function(service) {
> > return service.ServiceInfo.service_name;
> > });
> > this.hostNames = clusterInfo.hosts.map(function(host) {
> > return host.Hosts.host_name;
> > });
> > this.configurations =
> clusterInfo.configurations.map(function(config) {
> > return {
> > type: config.type,
> > tag: config.tag
> > };
> > });
> > this.getService = function(serviceName) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'GET';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/services/' + serviceName;
> > return performRequest(request_options).then(function(res) {
> > return new Service(cluster, res.body);
> > });
> > };
> > this.addService = function(serviceName) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'POST';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/services';
> > request_options.body = {
> > ServiceInfo: { service_name: serviceName }
> > };
> > return performRequest(request_options).then(function(res) {
> > return cluster.getService(serviceName);
> > });
> > };
> > this.removeService = function(service) {
> > if (!isString(service)) {
> > service = service.name;
> > }
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'DELETE';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/services/' + service;
> > return performRequest(request_options);
> > };
> > this.getConfiguration = function() { throw new Error('Not
> implemented'); };
> > this.addConfiguration = function(type, tag, config) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'POST';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/configurations';
> > request_options.body = {
> > type: type,
> > tag: tag,
> > properties: config,
> > };
> > return performRequest(request_options);
> > };
> > this.removeConfiguration = function() { throw new Error('Not
> implemented'); };
> > this.updateConfiguration = function() { throw new Error('Not
> implemented'); };
> > this.getHost = function(hostname) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'GET';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/hosts/' + hostname;
> > return performRequest(request_options).then(function(res) {
> > return new Host(cluster, res.body);
> > });
> > };
> > this.addHost = function(hostname) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'POST';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/hosts';
> > request_options.body = { Hosts: { host_name: hostname} };
> > return performRequest(request_options).then(function(res) {
> > return cluster.getHost(hostname);
> > });
> > };
> > this.removeHost = function(hostname) {
> > var request_options = cluster.server.getBaseRequest();
> > request_options.method = 'DELETE';
> > request_options.path = PATH_PREFIX + 'clusters/' + cluster.name+ '/hosts';
> > return performRequest(request_options);
> > };
> > this.updateHost = function() { throw new Error('Not implemented'); };
> > return this;
> > };
> >
> > exports.Server = function(host, port, user, pass) {
> > var server = this;
> > this.host = host;
> > this.port = port;
> > this.user = user;
> > this.pass = pass;
> > this.getBaseRequest = function() {
> > return {
> > host: host,
> > port: port.toString(),
> > auth: user + ':' + pass
> > };
> > };
> > this.listClusterNames = function() {
> > var request = server.getBaseRequest();
> > request.path = PATH_PREFIX + 'clusters';
> > request.method = 'GET';
> > return performRequest(request).then(function(res) {
> > return res.body.items.map(function(clusterInfo) {
> > return clusterInfo.Clusters.cluster_name;
> > });
> > });
> > };
> > this.getCluster = function(clusterName) {
> > var request = server.getBaseRequest();
> > request.path = PATH_PREFIX + 'clusters/' + clusterName;
> > request.method = 'GET';
> > return performRequest(request).then(function(res) {
> > return new Cluster(server, res.body);
> > });
> > };
> > this.createCluster = function(clusterName, version) {
> > var request = server.getBaseRequest();
> > request.path = PATH_PREFIX + 'clusters/' + clusterName;
> > request.method = 'POST';
> > request.body = { "Clusters": { "version": version }};
> > return performRequest(request).then(function(res) {
> > return server.getCluster(clusterName);
> > });
> > };
> > this.updateCluster = function() { throw new Error('Not
> implemented'); };
> > this.removeCluster = function(clusterName) {
> > var request = server.getBaseRequest();
> > request.path = PATH_PREFIX + 'clusters/' + clusterName;
> > request.method = 'DELETE';
> > return performRequest(request);
> > };
> > return this;
> > };
>
> > ;'use strict';
> >
> > process.on('uncaughtException', function(err) {
> > console.log(err);
> > console.log(err.stack);
> > });
> >
> > var Q = require('q');
> > var ambari = require('./ambari');
> > var server = new ambari.Server('localhost', 8080, 'admin', 'admin');
> >
> > server.createCluster('test', 'HDP-1.2.0').then(function(cluster) {
> > var serviceD = Q.defer();
> > var servicePromise = serviceD.promise;
> > Q.all([
> > cluster.addService('HDFS'),
> > cluster.addConfiguration('core-site', '1', {
> > 'fs.default.name' : 'local',
> > 'fs.checkpoint.dir': '/hdfs/checkpoint',
> > 'hadoop.security.authentication': 'simple',
> > 'hadoop.security.authorization': 'false'
> > }),
> > cluster.addConfiguration('hdfs-site', '1', {
> > 'dfs.name.dir': '/hdfs/name',
> > 'dfs.support.append': 'true',
> > 'dfs.webhdfs.enabled': 'false',
> > 'dfs.datanode.failed.volume.tolerated': '0',
> > 'dfs.block.local-path-access.user': 'root',
> > 'dfs.data.dir': '/hdfs/data',
> > 'dfs.replication': '1',
> > 'dfs.datanode.address': '0.0.0.0:50010',
> > 'dfs.datanode.http.address': '0.0.0.0:50075',
> > 'dfs.http.address': '0.0.0.0:50070',
> >
> > })
> > ]).spread(function(service) {
> > serviceD.resolve(service);
> > });
> > return servicePromise.then(function(service) {
> > return Q.all([
> > service.addComponent('NAMENODE'),
> > service.addComponent('DATANODE'),
> > service.addComponent('HDFS_CLIENT')
> > ]).then(function() { return service });
> > }).then(function(service) {
> > return Q.all([
> >
> service.cluster.addHost('1ddf42d1-0539-45f7-807a-a846a0cab9d8').then(function(host)
> {
> > return host.addComponents(['NAMENODE', 'DATANODE',
> 'HDFS_CLIENT']);
> > }),
> > service.applyConfiguration('core-site', '1'),
> > service.applyConfiguration('hdfs-site', '1')
> > ]).then(function() { return service; });
> > }).then(function(service) {
> > return service.install().then(function() { return service; });
> > });
> > }).done();
>
>
> ________________________________
>
> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar
> nuestra política de envío y recepción de correo electrónico en el enlace
> situado más abajo.
> This message is intended exclusively for its addressee. We only send and
> receive email on the basis of the terms set out at:
> http://www.tid.es/ES/PAGINAS/disclaimer.aspx
>
RE: INSTALL_FAILED status but no logs?
Posted by JOAQUIN GUANTER GONZALBEZ <xi...@tid.es>.
Hi Sylvain,
Thanks for the tips. Unfortunately, /var/log/ambari-* has no useful information and /var/log/hadoop doesn't even exist :(
Ximo.
-----Mensaje original-----
De: Sylvain Trias [mailto:bozo@ovh.net]
Enviado el: miércoles, 03 de abril de 2013 16:53
Para: ambari-user@incubator.apache.org
Asunto: Re: INSTALL_FAILED status but no logs?
On Wed, Apr 03, 2013 at 02:49:28PM +0000, JOAQUIN GUANTER GONZALBEZ wrote:
> Hello,
>
> I am trying to create a cluster with HDFS programmatically by targeting the REST APIs in the Ambari Server directly instead of using the Web frontend. I set the HDFS service state to "INSTALLED" and I can see my host components entering the state "INSTALLING", but after a while, they transition to "INSTALL_FAILED". The problem is that neither the ambari-server.log/out nor the ambari-agent.log/out have any error messages.
>
> I'm not sure if there are any other sources of information I should be looking at for errors, or if I'm doing anything wrong when calling the REST APIs (all the API calls return successfully, though). My test is a NodeJS program which I have attached in case anyone wants to take a look at it. The ambari.js code is a supporting object model that enables easy calls to the REST API.
>
> Does anyone know how can investigate why the state of my host components is ending up in INSTALL_FAILED?
Hi, I'm doing the same thing (and manage to install my first cluster today) When I have that kind of failure I try to log in to the host and manually check logs in /var/log/hadoop... or /var/log/ambari-* it helped me quite a lot
>
> Thanks,
> Ximo.
>
> ________________________________
>
> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra pol?tica de env?o y recepci?n de correo electr?nico en el enlace situado m?s abajo.
> This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
> http://www.tid.es/ES/PAGINAS/disclaimer.aspx
> ;'use strict';
>
> var Q = require('q');
> var http = require('http');
> var PATH_PREFIX = '/api/v1/';
>
> var performRequest = function (request_options) {
> var result = Q.defer();
> var body = request_options.body;
> delete request_options.body;
> var req = http.request(request_options, function(res) {
> var body = '';
> res.setEncoding('utf8');
> res.on('data', function(chunk) {
> body += chunk;
> });
> res.on('end', function() {
> if (body) {
> res.body = body;
> try {
> var json = JSON.parse(body);
> res.body = json;
> } catch(ignored) { }
> }
> if (Math.floor(res.statusCode / 100) === 2) result.resolve(res);
> else result.reject(res);
> });
> });
> if (body) req.write(JSON.stringify(body));
> req.end();
> return result.promise;
> };
>
> var isString = function(obj) {
> return Object.prototype.toString.call(obj) == '[object String]';
> };
>
> var Service = function(cluster, serviceInfo) {
> var service = this;
> this.cluster = cluster;
> this.name = serviceInfo.ServiceInfo.service_name;
> this.state = serviceInfo.ServiceInfo.state;
> this.componentNames = serviceInfo.components.map(function(componentInfo) {
> return componentInfo.ServiceComponentInfo.component_name;
> });
> this.desiredConfigs = serviceInfo.ServiceInfo.desired_configs;
> this.addComponent = function(componentName) {
> var request_options = service.cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name + '/services/' + service.name + '/components/' + componentName;
> return performRequest(request_options).then(function(res) {
> return componentName;
> });
> };
> this.updateCluster = function() { throw new Error('Not implemented'); };
> this.removeComponent = function() { throw new Error('Not implemented'); };
> this.applyConfiguration = function(type, tag) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'PUT';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name + '/services/' + service.name;
> request_options.body = {
> config: {}
> };
> request_options.body.config[type] = tag;
> return performRequest(request_options);
> };
> this.install = function() {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'PUT';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name + '/services/' + service.name;
> request_options.body = {
> ServiceInfo: {
> state: 'INSTALLED'
> }
> };
> return performRequest(request_options);
> };
> this.start = function() {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'PUT';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name + '/services/' + service.name;
> request_options.body = {
> ServiceInfo: {
> state: 'STARTED'
> }
> };
> return performRequest(request_options);
> };
> return this;
> };
>
> var Host = function(cluster, hostInfo) {
> var host = this;
> this.cluster = cluster;
> this.name = hostInfo.Hosts.host_name;
> this.ip = hostInfo.Hosts.ip;
> this.status = hostInfo.Hosts.host_status;
> this.addComponents = function(components) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/hosts?Hosts/host_name=' + host.name;
> request_options.body = {
> "host_components": components.map(function(component) {
> return { HostRoles: {
> "component_name": component
> }};
> })
> };
> return performRequest(request_options);
> };
> this.addComponent = function(component) {
> return host.addComponents([component]);
> };
> };
>
> var Cluster = function(server, clusterInfo) {
> var cluster = this;
> this.server = server;
> this.name = clusterInfo.Clusters.cluster_name;
> this.serviceNames = clusterInfo.services.map(function(service) {
> return service.ServiceInfo.service_name;
> });
> this.hostNames = clusterInfo.hosts.map(function(host) {
> return host.Hosts.host_name;
> });
> this.configurations = clusterInfo.configurations.map(function(config) {
> return {
> type: config.type,
> tag: config.tag
> };
> });
> this.getService = function(serviceName) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'GET';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/services/' + serviceName;
> return performRequest(request_options).then(function(res) {
> return new Service(cluster, res.body);
> });
> };
> this.addService = function(serviceName) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/services';
> request_options.body = {
> ServiceInfo: { service_name: serviceName }
> };
> return performRequest(request_options).then(function(res) {
> return cluster.getService(serviceName);
> });
> };
> this.removeService = function(service) {
> if (!isString(service)) {
> service = service.name;
> }
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'DELETE';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/services/' + service;
> return performRequest(request_options);
> };
> this.getConfiguration = function() { throw new Error('Not implemented'); };
> this.addConfiguration = function(type, tag, config) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/configurations';
> request_options.body = {
> type: type,
> tag: tag,
> properties: config,
> };
> return performRequest(request_options);
> };
> this.removeConfiguration = function() { throw new Error('Not implemented'); };
> this.updateConfiguration = function() { throw new Error('Not implemented'); };
> this.getHost = function(hostname) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'GET';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/hosts/' + hostname;
> return performRequest(request_options).then(function(res) {
> return new Host(cluster, res.body);
> });
> };
> this.addHost = function(hostname) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/hosts';
> request_options.body = { Hosts: { host_name: hostname} };
> return performRequest(request_options).then(function(res) {
> return cluster.getHost(hostname);
> });
> };
> this.removeHost = function(hostname) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'DELETE';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/hosts';
> return performRequest(request_options);
> };
> this.updateHost = function() { throw new Error('Not implemented'); };
> return this;
> };
>
> exports.Server = function(host, port, user, pass) {
> var server = this;
> this.host = host;
> this.port = port;
> this.user = user;
> this.pass = pass;
> this.getBaseRequest = function() {
> return {
> host: host,
> port: port.toString(),
> auth: user + ':' + pass
> };
> };
> this.listClusterNames = function() {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters';
> request.method = 'GET';
> return performRequest(request).then(function(res) {
> return res.body.items.map(function(clusterInfo) {
> return clusterInfo.Clusters.cluster_name;
> });
> });
> };
> this.getCluster = function(clusterName) {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters/' + clusterName;
> request.method = 'GET';
> return performRequest(request).then(function(res) {
> return new Cluster(server, res.body);
> });
> };
> this.createCluster = function(clusterName, version) {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters/' + clusterName;
> request.method = 'POST';
> request.body = { "Clusters": { "version": version }};
> return performRequest(request).then(function(res) {
> return server.getCluster(clusterName);
> });
> };
> this.updateCluster = function() { throw new Error('Not implemented'); };
> this.removeCluster = function(clusterName) {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters/' + clusterName;
> request.method = 'DELETE';
> return performRequest(request);
> };
> return this;
> };
> ;'use strict';
>
> process.on('uncaughtException', function(err) {
> console.log(err);
> console.log(err.stack);
> });
>
> var Q = require('q');
> var ambari = require('./ambari');
> var server = new ambari.Server('localhost', 8080, 'admin', 'admin');
>
> server.createCluster('test', 'HDP-1.2.0').then(function(cluster) {
> var serviceD = Q.defer();
> var servicePromise = serviceD.promise;
> Q.all([
> cluster.addService('HDFS'),
> cluster.addConfiguration('core-site', '1', {
> 'fs.default.name' : 'local',
> 'fs.checkpoint.dir': '/hdfs/checkpoint',
> 'hadoop.security.authentication': 'simple',
> 'hadoop.security.authorization': 'false'
> }),
> cluster.addConfiguration('hdfs-site', '1', {
> 'dfs.name.dir': '/hdfs/name',
> 'dfs.support.append': 'true',
> 'dfs.webhdfs.enabled': 'false',
> 'dfs.datanode.failed.volume.tolerated': '0',
> 'dfs.block.local-path-access.user': 'root',
> 'dfs.data.dir': '/hdfs/data',
> 'dfs.replication': '1',
> 'dfs.datanode.address': '0.0.0.0:50010',
> 'dfs.datanode.http.address': '0.0.0.0:50075',
> 'dfs.http.address': '0.0.0.0:50070',
>
> })
> ]).spread(function(service) {
> serviceD.resolve(service);
> });
> return servicePromise.then(function(service) {
> return Q.all([
> service.addComponent('NAMENODE'),
> service.addComponent('DATANODE'),
> service.addComponent('HDFS_CLIENT')
> ]).then(function() { return service });
> }).then(function(service) {
> return Q.all([
> service.cluster.addHost('1ddf42d1-0539-45f7-807a-a846a0cab9d8').then(function(host) {
> return host.addComponents(['NAMENODE', 'DATANODE', 'HDFS_CLIENT']);
> }),
> service.applyConfiguration('core-site', '1'),
> service.applyConfiguration('hdfs-site', '1')
> ]).then(function() { return service; });
> }).then(function(service) {
> return service.install().then(function() { return service; });
> });
> }).done();
________________________________
Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
http://www.tid.es/ES/PAGINAS/disclaimer.aspx
Re: INSTALL_FAILED status but no logs?
Posted by Sylvain Trias <bo...@ovh.net>.
On Wed, Apr 03, 2013 at 02:49:28PM +0000, JOAQUIN GUANTER GONZALBEZ wrote:
> Hello,
>
> I am trying to create a cluster with HDFS programmatically by targeting the REST APIs in the Ambari Server directly instead of using the Web frontend. I set the HDFS service state to "INSTALLED" and I can see my host components entering the state "INSTALLING", but after a while, they transition to "INSTALL_FAILED". The problem is that neither the ambari-server.log/out nor the ambari-agent.log/out have any error messages.
>
> I'm not sure if there are any other sources of information I should be looking at for errors, or if I'm doing anything wrong when calling the REST APIs (all the API calls return successfully, though). My test is a NodeJS program which I have attached in case anyone wants to take a look at it. The ambari.js code is a supporting object model that enables easy calls to the REST API.
>
> Does anyone know how can investigate why the state of my host components is ending up in INSTALL_FAILED?
Hi, I'm doing the same thing (and manage to install my first cluster today)
When I have that kind of failure I try to log in to the host and manually check logs in /var/log/hadoop... or /var/log/ambari-*
it helped me quite a lot
>
> Thanks,
> Ximo.
>
> ________________________________
>
> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra pol?tica de env?o y recepci?n de correo electr?nico en el enlace situado m?s abajo.
> This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
> http://www.tid.es/ES/PAGINAS/disclaimer.aspx
> ;'use strict';
>
> var Q = require('q');
> var http = require('http');
> var PATH_PREFIX = '/api/v1/';
>
> var performRequest = function (request_options) {
> var result = Q.defer();
> var body = request_options.body;
> delete request_options.body;
> var req = http.request(request_options, function(res) {
> var body = '';
> res.setEncoding('utf8');
> res.on('data', function(chunk) {
> body += chunk;
> });
> res.on('end', function() {
> if (body) {
> res.body = body;
> try {
> var json = JSON.parse(body);
> res.body = json;
> } catch(ignored) { }
> }
> if (Math.floor(res.statusCode / 100) === 2) result.resolve(res);
> else result.reject(res);
> });
> });
> if (body) req.write(JSON.stringify(body));
> req.end();
> return result.promise;
> };
>
> var isString = function(obj) {
> return Object.prototype.toString.call(obj) == '[object String]';
> };
>
> var Service = function(cluster, serviceInfo) {
> var service = this;
> this.cluster = cluster;
> this.name = serviceInfo.ServiceInfo.service_name;
> this.state = serviceInfo.ServiceInfo.state;
> this.componentNames = serviceInfo.components.map(function(componentInfo) {
> return componentInfo.ServiceComponentInfo.component_name;
> });
> this.desiredConfigs = serviceInfo.ServiceInfo.desired_configs;
> this.addComponent = function(componentName) {
> var request_options = service.cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name + '/services/' + service.name + '/components/' + componentName;
> return performRequest(request_options).then(function(res) {
> return componentName;
> });
> };
> this.updateCluster = function() { throw new Error('Not implemented'); };
> this.removeComponent = function() { throw new Error('Not implemented'); };
> this.applyConfiguration = function(type, tag) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'PUT';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name + '/services/' + service.name;
> request_options.body = {
> config: {}
> };
> request_options.body.config[type] = tag;
> return performRequest(request_options);
> };
> this.install = function() {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'PUT';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name + '/services/' + service.name;
> request_options.body = {
> ServiceInfo: {
> state: 'INSTALLED'
> }
> };
> return performRequest(request_options);
> };
> this.start = function() {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'PUT';
> request_options.path = PATH_PREFIX + 'clusters/' + service.cluster.name + '/services/' + service.name;
> request_options.body = {
> ServiceInfo: {
> state: 'STARTED'
> }
> };
> return performRequest(request_options);
> };
> return this;
> };
>
> var Host = function(cluster, hostInfo) {
> var host = this;
> this.cluster = cluster;
> this.name = hostInfo.Hosts.host_name;
> this.ip = hostInfo.Hosts.ip;
> this.status = hostInfo.Hosts.host_status;
> this.addComponents = function(components) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/hosts?Hosts/host_name=' + host.name;
> request_options.body = {
> "host_components": components.map(function(component) {
> return { HostRoles: {
> "component_name": component
> }};
> })
> };
> return performRequest(request_options);
> };
> this.addComponent = function(component) {
> return host.addComponents([component]);
> };
> };
>
> var Cluster = function(server, clusterInfo) {
> var cluster = this;
> this.server = server;
> this.name = clusterInfo.Clusters.cluster_name;
> this.serviceNames = clusterInfo.services.map(function(service) {
> return service.ServiceInfo.service_name;
> });
> this.hostNames = clusterInfo.hosts.map(function(host) {
> return host.Hosts.host_name;
> });
> this.configurations = clusterInfo.configurations.map(function(config) {
> return {
> type: config.type,
> tag: config.tag
> };
> });
> this.getService = function(serviceName) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'GET';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/services/' + serviceName;
> return performRequest(request_options).then(function(res) {
> return new Service(cluster, res.body);
> });
> };
> this.addService = function(serviceName) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/services';
> request_options.body = {
> ServiceInfo: { service_name: serviceName }
> };
> return performRequest(request_options).then(function(res) {
> return cluster.getService(serviceName);
> });
> };
> this.removeService = function(service) {
> if (!isString(service)) {
> service = service.name;
> }
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'DELETE';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/services/' + service;
> return performRequest(request_options);
> };
> this.getConfiguration = function() { throw new Error('Not implemented'); };
> this.addConfiguration = function(type, tag, config) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/configurations';
> request_options.body = {
> type: type,
> tag: tag,
> properties: config,
> };
> return performRequest(request_options);
> };
> this.removeConfiguration = function() { throw new Error('Not implemented'); };
> this.updateConfiguration = function() { throw new Error('Not implemented'); };
> this.getHost = function(hostname) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'GET';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/hosts/' + hostname;
> return performRequest(request_options).then(function(res) {
> return new Host(cluster, res.body);
> });
> };
> this.addHost = function(hostname) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'POST';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/hosts';
> request_options.body = { Hosts: { host_name: hostname} };
> return performRequest(request_options).then(function(res) {
> return cluster.getHost(hostname);
> });
> };
> this.removeHost = function(hostname) {
> var request_options = cluster.server.getBaseRequest();
> request_options.method = 'DELETE';
> request_options.path = PATH_PREFIX + 'clusters/' + cluster.name + '/hosts';
> return performRequest(request_options);
> };
> this.updateHost = function() { throw new Error('Not implemented'); };
> return this;
> };
>
> exports.Server = function(host, port, user, pass) {
> var server = this;
> this.host = host;
> this.port = port;
> this.user = user;
> this.pass = pass;
> this.getBaseRequest = function() {
> return {
> host: host,
> port: port.toString(),
> auth: user + ':' + pass
> };
> };
> this.listClusterNames = function() {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters';
> request.method = 'GET';
> return performRequest(request).then(function(res) {
> return res.body.items.map(function(clusterInfo) {
> return clusterInfo.Clusters.cluster_name;
> });
> });
> };
> this.getCluster = function(clusterName) {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters/' + clusterName;
> request.method = 'GET';
> return performRequest(request).then(function(res) {
> return new Cluster(server, res.body);
> });
> };
> this.createCluster = function(clusterName, version) {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters/' + clusterName;
> request.method = 'POST';
> request.body = { "Clusters": { "version": version }};
> return performRequest(request).then(function(res) {
> return server.getCluster(clusterName);
> });
> };
> this.updateCluster = function() { throw new Error('Not implemented'); };
> this.removeCluster = function(clusterName) {
> var request = server.getBaseRequest();
> request.path = PATH_PREFIX + 'clusters/' + clusterName;
> request.method = 'DELETE';
> return performRequest(request);
> };
> return this;
> };
> ;'use strict';
>
> process.on('uncaughtException', function(err) {
> console.log(err);
> console.log(err.stack);
> });
>
> var Q = require('q');
> var ambari = require('./ambari');
> var server = new ambari.Server('localhost', 8080, 'admin', 'admin');
>
> server.createCluster('test', 'HDP-1.2.0').then(function(cluster) {
> var serviceD = Q.defer();
> var servicePromise = serviceD.promise;
> Q.all([
> cluster.addService('HDFS'),
> cluster.addConfiguration('core-site', '1', {
> 'fs.default.name' : 'local',
> 'fs.checkpoint.dir': '/hdfs/checkpoint',
> 'hadoop.security.authentication': 'simple',
> 'hadoop.security.authorization': 'false'
> }),
> cluster.addConfiguration('hdfs-site', '1', {
> 'dfs.name.dir': '/hdfs/name',
> 'dfs.support.append': 'true',
> 'dfs.webhdfs.enabled': 'false',
> 'dfs.datanode.failed.volume.tolerated': '0',
> 'dfs.block.local-path-access.user': 'root',
> 'dfs.data.dir': '/hdfs/data',
> 'dfs.replication': '1',
> 'dfs.datanode.address': '0.0.0.0:50010',
> 'dfs.datanode.http.address': '0.0.0.0:50075',
> 'dfs.http.address': '0.0.0.0:50070',
>
> })
> ]).spread(function(service) {
> serviceD.resolve(service);
> });
> return servicePromise.then(function(service) {
> return Q.all([
> service.addComponent('NAMENODE'),
> service.addComponent('DATANODE'),
> service.addComponent('HDFS_CLIENT')
> ]).then(function() { return service });
> }).then(function(service) {
> return Q.all([
> service.cluster.addHost('1ddf42d1-0539-45f7-807a-a846a0cab9d8').then(function(host) {
> return host.addComponents(['NAMENODE', 'DATANODE', 'HDFS_CLIENT']);
> }),
> service.applyConfiguration('core-site', '1'),
> service.applyConfiguration('hdfs-site', '1')
> ]).then(function() { return service; });
> }).then(function(service) {
> return service.install().then(function() { return service; });
> });
> }).done();