You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Martin Emrich <ma...@empolis.com> on 2015/07/27 16:35:57 UTC

Virtual Router disks run full: How to fix?

Hi!

We are affected by https://issues.apache.org/jira/browse/CLOUDSTACK-6885  (running ACS 4.4.3).

Our VR disks are running full regularly. As upgrading to 4.5.1 is currently not an option: can I somehow fix this (semi)manually? If I had SSH access to the VRs, I could fix logrotate from a script... Or can I "edit" the template somehow? Editing dozens of machines via the Console Proxy window would be tedious and error-prone...

Thanks,

Martin Emrich
Senior IT Administrator

Empolis Information Management GmbH | Europaallee 10 | 67657 Kaiserslautern | Germany
Phone +49 631 68037-71 | Fax +49 631 68037-77
martin.emrich@empolis.com<mailto:martin.emrich@empolis.com%0d>

www.empolis.com<http://www.empolis.com/>
Sitz Kaiserslautern  | Amtsgericht Kaiserslautern HRB 31317
Geschäftsführer: Dr. Stefan Wess, Stefan Volland, Dr. Christian Schulmeyer, Dr. Peter Tepassé

SMART INFORMATION MANAGEMENT
Empolis-Lösungen befähigen Unternehmen und Organisationen, die exponentiell wachsende Menge
strukturierter und unstrukturierter Daten zu analysieren, zu interpretieren und automatisiert zu verarbeiten.
Sie nutzen damit ihr Wissenskapital, um unternehmenskritische Geschäftsprozesse zu optimieren.
Entscheider, Mitarbeiter und Kunden erhalten so stets situations- und aufgabengerecht genau die
Information, die für sie relevant ist.
Abonnieren Sie unseren Newsletter<http://newsletter.empolis.com/art_resource.php?sid=si4n.23ctc3r> | Folgen Sie uns auf Facebook<http://www.facebook.com/EmpolisSoftware> | Besuchen Sie uns auf YouTube<http://www.youtube.com/EmpolisSoftware>
[Signatur.Experton.DE]<http://www.empolis.com/en/news-events/archiv/pressemitteilung-experton.html>

Re: Virtual Router disks run full: How to fix?

Posted by Martin Emrich <ma...@empolis.com>.
Hi!

This is my script to fix it (Run it on your cloudstack server; You'll 
need SSH keys to the Xen servers or you'll have to type the root 
password often):

----------------
#!/usr/bin/perl -w

use strict;
use Data::Dumper;
use DBI;
use Getopt::Long;
use JSON;
use IO::Socket;
use IO::Socket::INET;

my %dbargs = (dsn    => 'DBI:mysql:dbname=cloud;host=localhost',
               dbuser => "cloud",
               dbpass => "cloud");

sub fetch_server_list {
     my $dbh = DBI->connect( $dbargs{dsn}, $dbargs{dbuser}, 
$dbargs{dbpass} ) || die DBI->errstr;
     my $query =
         "SELECT vm_instance.name as routername, vm_instance.host_id as 
host_id, vm_instance.private_ip_address as routerip, 
host.private_ip_address as hostip from vm_instance, host where host.id = 
vm_instance.host_id AND vm_instance.name LIKE 'r-%' and 
vm_instance.state = 'Running';";

     my $result = $dbh->selectall_arrayref( $query, { "Slice" => {} } ) 
|| die $dbh->errstr;

     return $result;
}

sub run_on_router($$)
{
     my $target = shift || die;
     my $remotecommand = shift || die;

     my $command = 'ssh root@'.$target->{hostip}.' \'ssh -i 
/root/.ssh/id_rsa.cloud -l root -p 3922 '.$target->{routerip}.' 
"'.$remotecommand.'"\'';

     print STDERR "executing:\n  $command";
     system($command);
}

sub fix_router($)
{
     my $target = shift || die;

     run_on_router($target, 'sed -i -e \"s/rsyslog reload/rsyslog 
rotate/\" -e \"s%/nvoke%/invoke%;\" /etc/logrotate.d/rsyslog');
     run_on_router($target, '/usr/sbin/invoke-rc.d rsyslog rotate');
}

sub status_router($)
{
     my $target = shift || die;

     run_on_router($target, 'df -k /var');
}

####################################### main

{
     my @targets = @{ fetch_server_list() };

     for my $target (@targets)
     {
         print STDOUT "VR ".$target->{routername}.":\n";

         if ($ARGV[0] eq "info")
         {
             status_router($target);
         }
         else
         {
             fix_router($target);
         }
     }
}

----------------


Ciao

Martin

Am 28.07.2015 um 09:00 schrieb Martin Emrich:
> Thanks, that's how I can access the VRs. So the procedure would be:
>
> - get Host IP, VR LL IP from MySQL DB
> - Log on to VR via its Host
> - Fix logrotate config
>
> I'll try that.
>
> Thanks
>
> Martin
>

AW: Virtual Router disks run full: How to fix?

Posted by Martin Emrich <ma...@empolis.com>.
Thanks, that's how I can access the VRs. So the procedure would be:

- get Host IP, VR LL IP from MySQL DB
- Log on to VR via its Host
- Fix logrotate config

I'll try that.

Thanks

Martin

-----Ursprüngliche Nachricht-----
Von: Somesh Naidu [mailto:Somesh.Naidu@citrix.com] 
Gesendet: Dienstag, 28. Juli 2015 00:57
An: users@cloudstack.apache.org
Betreff: RE: Virtual Router disks run full: How to fix?

Daan, I believe Martin has Xenserver as hypervisor (got this from another thread) so your solution might work for him.

Martin - you should be able to SSH to virtual router on all hypervisors (XS/KVM/VMware). The procedure is mentioned at the below link (it is for SSVM but works for all system/router VMs):
https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM%2C+templates%2C+Secondary+storage+troubleshooting

Regards,
Somesh


-----Original Message-----
From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
Sent: Monday, July 27, 2015 6:17 PM
To: users@cloudstack.apache.org
Subject: Re: Virtual Router disks run full: How to fix?

Martin, I know of a solution with ansible running on the hypervisors but this depends on your tye of hypervisor. On xen or kvm this should be possible, not on vmware. Why do you say ,'If I had SSH access to the VRs,...'? Are you running vmware?

2015-07-27 16:35 GMT+02:00 Martin Emrich <ma...@empolis.com>:

>  Hi!
>
>
>
> We are affected by 
> https://issues.apache.org/jira/browse/CLOUDSTACK-6885
>  (running ACS 4.4.3).
>
>
>
> Our VR disks are running full regularly. As upgrading to 4.5.1 is 
> currently not an option: can I somehow fix this (semi)manually? If I 
> had SSH access to the VRs, I could fix logrotate from a script… Or can I “edit”
> the template somehow? Editing dozens of machines via the Console Proxy 
> window would be tedious and error-prone…
>
>
>
> Thanks,
>
>
>
> Martin Emrich
>
> Senior IT Administrator
>
>
>
> Empolis Information Management GmbH | Europaallee 10 | 67657 
> Kaiserslautern | Germany
>
> Phone +49 631 68037-71 | Fax +49 631 68037-77
>
> *martin.emrich@empolis.com <martin.emrich@empolis.com%0d>*
>
>
>
> *www.empolis.com <http://www.empolis.com/>*
>
> Sitz Kaiserslautern  | Amtsgericht Kaiserslautern HRB 31317
>
> Geschäftsführer: Dr. Stefan Wess, Stefan Volland, Dr. Christian 
> Schulmeyer, Dr. Peter Tepassé
>
>
>
> *SMART INFORMATION MANAGEMENT*
> Empolis-Lösungen befähigen Unternehmen und Organisationen, die 
> exponentiell wachsende Menge strukturierter und unstrukturierter Daten 
> zu analysieren, zu interpretieren und automatisiert zu verarbeiten.
>
> Sie nutzen damit ihr Wissenskapital, um unternehmenskritische 
> Geschäftsprozesse zu optimieren.
>
> Entscheider, Mitarbeiter und Kunden erhalten so stets situations- und 
> aufgabengerecht genau die
>
> Information, die für sie relevant ist.
>
> Abonnieren Sie unseren Newsletter
> <http://newsletter.empolis.com/art_resource.php?sid=si4n.23ctc3r> | 
> Folgen Sie uns auf Facebook <http://www.facebook.com/EmpolisSoftware> 
> | Besuchen Sie uns auf YouTube 
> <http://www.youtube.com/EmpolisSoftware>
>
> [image: Signatur.Experton.DE]
> <http://www.empolis.com/en/news-events/archiv/pressemitteilung-experto
> n.html>
>
>
>



--
Daan

RE: Virtual Router disks run full: How to fix?

Posted by Somesh Naidu <So...@citrix.com>.
Daan, I believe Martin has Xenserver as hypervisor (got this from another thread) so your solution might work for him.

Martin - you should be able to SSH to virtual router on all hypervisors (XS/KVM/VMware). The procedure is mentioned at the below link (it is for SSVM but works for all system/router VMs):
https://cwiki.apache.org/confluence/display/CLOUDSTACK/SSVM%2C+templates%2C+Secondary+storage+troubleshooting

Regards,
Somesh


-----Original Message-----
From: Daan Hoogland [mailto:daan.hoogland@gmail.com] 
Sent: Monday, July 27, 2015 6:17 PM
To: users@cloudstack.apache.org
Subject: Re: Virtual Router disks run full: How to fix?

Martin, I know of a solution with ansible running on the hypervisors but
this depends on your tye of hypervisor. On xen or kvm this should be
possible, not on vmware. Why do you say ,'If I had SSH access to the
VRs,...'? Are you running vmware?

2015-07-27 16:35 GMT+02:00 Martin Emrich <ma...@empolis.com>:

>  Hi!
>
>
>
> We are affected by https://issues.apache.org/jira/browse/CLOUDSTACK-6885
>  (running ACS 4.4.3).
>
>
>
> Our VR disks are running full regularly. As upgrading to 4.5.1 is
> currently not an option: can I somehow fix this (semi)manually? If I had
> SSH access to the VRs, I could fix logrotate from a script… Or can I “edit”
> the template somehow? Editing dozens of machines via the Console Proxy
> window would be tedious and error-prone…
>
>
>
> Thanks,
>
>
>
> Martin Emrich
>
> Senior IT Administrator
>
>
>
> Empolis Information Management GmbH | Europaallee 10 | 67657
> Kaiserslautern | Germany
>
> Phone +49 631 68037-71 | Fax +49 631 68037-77
>
> *martin.emrich@empolis.com <martin.emrich@empolis.com%0d>*
>
>
>
> *www.empolis.com <http://www.empolis.com/>*
>
> Sitz Kaiserslautern  | Amtsgericht Kaiserslautern HRB 31317
>
> Geschäftsführer: Dr. Stefan Wess, Stefan Volland, Dr. Christian
> Schulmeyer, Dr. Peter Tepassé
>
>
>
> *SMART INFORMATION MANAGEMENT*
> Empolis-Lösungen befähigen Unternehmen und Organisationen, die
> exponentiell wachsende Menge
> strukturierter und unstrukturierter Daten zu analysieren, zu
> interpretieren und automatisiert zu verarbeiten.
>
> Sie nutzen damit ihr Wissenskapital, um unternehmenskritische
> Geschäftsprozesse zu optimieren.
>
> Entscheider, Mitarbeiter und Kunden erhalten so stets situations- und
> aufgabengerecht genau die
>
> Information, die für sie relevant ist.
>
> Abonnieren Sie unseren Newsletter
> <http://newsletter.empolis.com/art_resource.php?sid=si4n.23ctc3r> | Folgen
> Sie uns auf Facebook <http://www.facebook.com/EmpolisSoftware> | Besuchen
> Sie uns auf YouTube <http://www.youtube.com/EmpolisSoftware>
>
> [image: Signatur.Experton.DE]
> <http://www.empolis.com/en/news-events/archiv/pressemitteilung-experton.html>
>
>
>



-- 
Daan

Re: Virtual Router disks run full: How to fix?

Posted by Daan Hoogland <da...@gmail.com>.
Martin, I know of a solution with ansible running on the hypervisors but
this depends on your tye of hypervisor. On xen or kvm this should be
possible, not on vmware. Why do you say ,'If I had SSH access to the
VRs,...'? Are you running vmware?

2015-07-27 16:35 GMT+02:00 Martin Emrich <ma...@empolis.com>:

>  Hi!
>
>
>
> We are affected by https://issues.apache.org/jira/browse/CLOUDSTACK-6885
>  (running ACS 4.4.3).
>
>
>
> Our VR disks are running full regularly. As upgrading to 4.5.1 is
> currently not an option: can I somehow fix this (semi)manually? If I had
> SSH access to the VRs, I could fix logrotate from a script… Or can I “edit”
> the template somehow? Editing dozens of machines via the Console Proxy
> window would be tedious and error-prone…
>
>
>
> Thanks,
>
>
>
> Martin Emrich
>
> Senior IT Administrator
>
>
>
> Empolis Information Management GmbH | Europaallee 10 | 67657
> Kaiserslautern | Germany
>
> Phone +49 631 68037-71 | Fax +49 631 68037-77
>
> *martin.emrich@empolis.com <martin.emrich@empolis.com%0d>*
>
>
>
> *www.empolis.com <http://www.empolis.com/>*
>
> Sitz Kaiserslautern  | Amtsgericht Kaiserslautern HRB 31317
>
> Geschäftsführer: Dr. Stefan Wess, Stefan Volland, Dr. Christian
> Schulmeyer, Dr. Peter Tepassé
>
>
>
> *SMART INFORMATION MANAGEMENT*
> Empolis-Lösungen befähigen Unternehmen und Organisationen, die
> exponentiell wachsende Menge
> strukturierter und unstrukturierter Daten zu analysieren, zu
> interpretieren und automatisiert zu verarbeiten.
>
> Sie nutzen damit ihr Wissenskapital, um unternehmenskritische
> Geschäftsprozesse zu optimieren.
>
> Entscheider, Mitarbeiter und Kunden erhalten so stets situations- und
> aufgabengerecht genau die
>
> Information, die für sie relevant ist.
>
> Abonnieren Sie unseren Newsletter
> <http://newsletter.empolis.com/art_resource.php?sid=si4n.23ctc3r> | Folgen
> Sie uns auf Facebook <http://www.facebook.com/EmpolisSoftware> | Besuchen
> Sie uns auf YouTube <http://www.youtube.com/EmpolisSoftware>
>
> [image: Signatur.Experton.DE]
> <http://www.empolis.com/en/news-events/archiv/pressemitteilung-experton.html>
>
>
>



-- 
Daan