You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@vcl.apache.org by Ben Smith <pr...@gmail.com> on 2010/09/10 02:41:21 UTC
Failing image loads. Log attached.
Hello, whenever I try to load an image onto a VM using the "Manage
Computers" tab in the VCL web administration interface, vcld.log has
the following, and the image fails to load.
I think it could have something to do with the state of the VM; it may
be in RO disk mode... If so, I would like to know how I can reboot it
(I can't seem to raise it on SSH, even with root pw) so that I can
undo the RO disk issue.
****** vcld.log *******
|6983|170:149|reload| (-6) vcld, make_new_child (line: 600)
|6983|170:149|reload| ---- WARNING ----
|6983|170:149|reload| 2010-09-09
20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6792)|failed to
run SSH command after 4 attempts, command: /usr/bin/ssh -i
/etc/vcl/bladelinuxkey_id_rsa -l root -p 24 -x vm5 'uname -s 2>&1'
2>&1, exit status: 255, output:
|6983|170:149|reload| ssh output (uname -s): ssh: connect to host vm5
port 24: Connection refused
|6983|170:149|reload| ( 0) utils.pm, notify (line: 691)
|6983|170:149|reload| (-1) utils.pm, run_ssh_command (line: 6792)
|6983|170:149|reload| (-2) utils.pm, _sshd_status (line: 2837)
|6983|170:149|reload| (-3) esx.pm, node_status (line: 698)
|6983|170:149|reload| (-4) new.pm, reload_image (line: 517)
|6983|170:149|reload| (-5) new.pm, process (line: 266)
|6983|170:149|reload| (-6) vcld, make_new_child (line: 600)
2010-09-09 20:04:26|6983|170:149|reload|esx.pm:node_status(704)|SSH
good, trying to query image name
2010-09-09 20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6695)|executing
SSH command on vm5:
|6983|170:149|reload| cat currentimage.txt
|6983|170:149|reload| ---- WARNING ----
|6983|170:149|reload| 2010-09-09
20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6766)|attempt
1/3: failed to execute SSH command on vm5: cat currentimage.txt, exit
status: 255, SSH exits with the exit status of the remote command or
with 255 if an error occurred, output:
|6983|170:149|reload| ssh output (cat curren...):
ssh_exchange_identification: Connection closed by remote host
|6983|170:149|reload| ( 0) utils.pm, notify (line: 691)
|6983|170:149|reload| (-1) utils.pm, run_ssh_command (line: 6766)
|6983|170:149|reload| (-2) esx.pm, node_status (line: 708)
|6983|170:149|reload| (-3) new.pm, reload_image (line: 517)
|6983|170:149|reload| (-4) new.pm, process (line: 266)
|6983|170:149|reload| (-5) vcld, make_new_child (line: 600)
|6983|170:149|reload| (-6) vcld, main (line: 347)
2010-09-09 20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6699)|attempt
2/3: executing SSH command on vm5:
|6983|170:149|reload| /usr/bin/ssh -i /etc/vcl/bladelinuxkey_id_rsa
-l root -p 22 -x vm5 'cat currentimage.txt' 2>&1
|6983|170:149|reload| ---- WARNING ----
|6983|170:149|reload| 2010-09-09
20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6766)|attempt
2/3: failed to execute SSH command on vm5: cat currentimage.txt, exit
status: 255, SSH exits with the exit status of the remote command or
with 255 if an error occurred, output:
|6983|170:149|reload| ssh output (cat curren...):
ssh_exchange_identification: Connection closed by remote host
|6983|170:149|reload| ( 0) utils.pm, notify (line: 691)
|6983|170:149|reload| (-1) utils.pm, run_ssh_command (line: 6766)
|6983|170:149|reload| (-2) esx.pm, node_status (line: 708)
|6983|170:149|reload| (-3) new.pm, reload_image (line: 517)
|6983|170:149|reload| (-4) new.pm, process (line: 266)
|6983|170:149|reload| (-5) vcld, make_new_child (line: 600)
|6983|170:149|reload| (-6) vcld, main (line: 347)
2010-09-09 20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6699)|attempt
3/3: executing SSH command on vm5:
|6983|170:149|reload| /usr/bin/ssh -i /etc/vcl/bladelinuxkey_id_rsa
-l root -p 22 -x vm5 'cat currentimage.txt' 2>&1
2010-09-09 20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6762)|making
1 more attempt using port 24
|6983|170:149|reload| ---- WARNING ----
|6983|170:149|reload| 2010-09-09
20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6766)|attempt
3/4: failed to execute SSH command on vm5: cat currentimage.txt, exit
status: 255, SSH exits with the exit status of the remote command or
with 255 if an error occurred, output:
|6983|170:149|reload| ssh output (cat curren...):
ssh_exchange_identification: Connection closed by remote host
|6983|170:149|reload| ( 0) utils.pm, notify (line: 691)
|6983|170:149|reload| (-1) utils.pm, run_ssh_command (line: 6766)
|6983|170:149|reload| (-2) esx.pm, node_status (line: 708)
|6983|170:149|reload| (-3) new.pm, reload_image (line: 517)
|6983|170:149|reload| (-4) new.pm, process (line: 266)
|6983|170:149|reload| (-5) vcld, make_new_child (line: 600)
|6983|170:149|reload| (-6) vcld, main (line: 347)
2010-09-09 20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6699)|attempt
4/4: executing SSH command on vm5:
|6983|170:149|reload| /usr/bin/ssh -i /etc/vcl/bladelinuxkey_id_rsa
-l root -p 24 -x vm5 'cat currentimage.txt 2>&1' 2>&1
|6983|170:149|reload| ---- WARNING ----
|6983|170:149|reload| 2010-09-09
20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6766)|attempt
4/4: failed to execute SSH command on vm5: cat currentimage.txt, exit
status: 255, SSH exits with the exit status of the remote command or
with 255 if an error occurred, output:
|6983|170:149|reload| ssh output (cat curren...): ssh: connect to host
vm5 port 24: Connection refused
|6983|170:149|reload| ( 0) utils.pm, notify (line: 691)
|6983|170:149|reload| (-1) utils.pm, run_ssh_command (line: 6766)
|6983|170:149|reload| (-2) esx.pm, node_status (line: 708)
|6983|170:149|reload| (-3) new.pm, reload_image (line: 517)
|6983|170:149|reload| (-4) new.pm, process (line: 266)
|6983|170:149|reload| (-5) vcld, make_new_child (line: 600)
|6983|170:149|reload| (-6) vcld, main (line: 347)
|6983|170:149|reload| ---- WARNING ----
|6983|170:149|reload| 2010-09-09
20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6792)|failed to
run SSH command after 4 attempts, command: /usr/bin/ssh -i
/etc/vcl/bladelinuxkey_id_rsa -l root -p 24 -x vm5 'cat
currentimage.txt 2>&1' 2>&1, exit status: 255, output:
|6983|170:149|reload| ssh output (cat curren...): ssh: connect to host
vm5 port 24: Connection refused
|6983|170:149|reload| ( 0) utils.pm, notify (line: 691)
|6983|170:149|reload| (-1) utils.pm, run_ssh_command (line: 6792)
|6983|170:149|reload| (-2) esx.pm, node_status (line: 708)
|6983|170:149|reload| (-3) new.pm, reload_image (line: 517)
|6983|170:149|reload| (-4) new.pm, process (line: 266)
|6983|170:149|reload| (-5) vcld, make_new_child (line: 600)
|6983|170:149|reload| (-6) vcld, main (line: 347)
Use of uninitialized value $status{"currentimage"} in concatenation (.) or
string at /opt/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm
line 711 (#1)
(W uninitialized) An undefined value was used as if it were already
defined. It was interpreted as a "" or a 0, but maybe it was a mistake.
To suppress this warning assign a defined value to your variables.
To help you figure out what was undefined, perl will try to tell you the
name of the variable (if any) that was undefined. In some cases it cannot
do this, so it also tells you what operation you used the undefined value
in. Note, however, that perl optimizes your program and the operation
displayed in the warning may not necessarily appear literally in your
program. For example, "that $foo" is usually optimized into "that "
. $foo, and the warning will refer to the concatenation (.) operator,
even though there is no . in your program.
|6983|170:149|reload| ---- WARNING ----
|6983|170:149|reload| 2010-09-09
20:04:26|6983|170:149|reload|vcld:warning_handler(642)|Use of
uninitialized value $status{"currentimage"} in concatenation (.) or
string at /opt/vcl/bin/../lib/VCL/Module/Provisioning/esx.pm line 711.
|6983|170:149|reload| ( 0) utils.pm, notify (line: 691)
|6983|170:149|reload| (-1) vcld, warning_handler (line: 642)
|6983|170:149|reload| (-2) esx.pm, node_status (line: 711)
|6983|170:149|reload| (-3) new.pm, reload_image (line: 517)
|6983|170:149|reload| (-4) new.pm, process (line: 266)
|6983|170:149|reload| (-5) vcld, make_new_child (line: 600)
|6983|170:149|reload| (-6) vcld, main (line: 347)
2010-09-09 20:04:26|6983|170:149|reload|esx.pm:node_status(711)|Image name:
2010-09-09 20:04:26|6983|170:149|reload|esx.pm:node_status(733)|status
set to RELOAD
2010-09-09 20:04:26|6983|170:149|reload|esx.pm:node_status(741)|returning
node status hash reference ($node_status->{status}=RELOAD)
2010-09-09 20:04:26|6983|170:149|reload|new.pm:reload_image(528)|node_status
returned a hash reference
2010-09-09 20:04:26|6983|170:149|reload|new.pm:reload_image(533)|node_status
hash reference contains key {status}=RELOAD
2010-09-09 20:04:26|6983|170:149|reload|new.pm:reload_image(601)|node
status is RELOAD, vm5 will be reloaded
2010-09-09 20:04:26|6983|170:149|reload|utils.pm:insertloadlog(5324)|inserted
computer=31, loadimageblade, vm5 must be reloaded with
fc9image-EHRSystemSuite214-v9
2010-09-09 20:04:26|6983|170:149|reload|new.pm:reload_image(615)|calling
VCL::Module::Provisioning::esx->does_image_exist()
2010-09-09 20:04:26|6983|170:149|reload|utils.pm:run_ssh_command(6695)|executing
SSH command on 10.13.1.1:
|6983|170:149|reload| ls -1 /nfsroot/golden 2>&1
2010-09-09 20:04:26|7011|169:148|reload|utils.pm:run_ssh_command(6776)|run_ssh_command
output:
|7011|169:148|reload| esx3-fc11-64b-v0
|7011|169:148|reload| esx3-fc12-64b-v0
|7011|169:148|reload| esx-lamp-3-x86_64-v0
|7011|169:148|reload| fc9image-EHRSystemSuite13-v0
|7011|169:148|reload| fc9image-EHRSystemSuite214-v0
|7011|169:148|reload| fc9image-EHRSystemSuite214-v1
|7011|169:148|reload| fc9image-EHRSystemSuite214-v2
|7011|169:148|reload| fc9image-EHRSystemSuite214-v3
|7011|169:148|reload| fc9image-EHRSystemSuite214-v4
|7011|169:148|reload| fc9image-EHRSystemSuite214-v5
|7011|169:148|reload| fc9image-EHRSystemSuite214-v6
|7011|169:148|reload| fc9image-EHRSystemSuite214-v7
|7011|169:148|reload| fc9image-EHRSystemSuite214-v8
|7011|169:148|reload| fc9image-EHRSystemSuite214-v9
|7011|169:148|reload| fc9image-OpenMRS15-v0
2010-09-09 20:04:26|7011|169:148|reload|utils.pm:run_ssh_command(6784)|SSH
command executed on 10.13.1.1, returning (0, "esx3-fc11-64b-v0
esx3-fc12-64b...")
2010-09-09 20:04:26|7011|169:148|reload|esx.pm:does_image_exist(775)|image
fc9image-EHRSystemSuite214-v9 exists
2010-09-09 20:04:26|7011|169:148|reload|new.pm:reload_image(618)|fc9image-EHRSystemSuite214-v9
exists on this management node
2010-09-09 20:04:26|7011|169:148|reload|utils.pm:insertloadlog(5324)|inserted
computer=28, doesimageexists, confirmed image exists
2010-09-09 20:04:26|7011|169:148|reload|utils.pm:update_computer_state(2325)|computer
28 state updated to: reloading
2010-09-09 20:04:26|7011|169:148|reload|new.pm:reload_image(651)|computer
vm2 state set to reloading
2010-09-09 20:04:26|7011|169:148|reload|utils.pm:insertloadlog(5324)|inserted
computer=28, info, computer state updated to reloading
2010-09-09 20:04:26|7011|169:148|reload|new.pm:reload_image(662)|calling
VCL::Module::Provisioning::esx->load() subroutine
2010-09-09 20:04:26|7011|169:148|reload|utils.pm:insertloadlog(5324)|inserted
computer=28, info, calling VCL::Module::Provisioning::esx->load()
subroutine
2010-09-09 20:04:26|7011|169:148|reload|esx.pm:load(142)|****************************************************
|7011|169:148|reload| ---- WARNING ----
|7011|169:148|reload| 2010-09-09
20:04:26|7011|169:148|reload|DataStructure.pm:_automethod(610)|corresponding
data has not been initialized for get_computer_eth0_mac_address:
$self->request_data->{reservation}{148}{computer}{eth0macaddress}
|7011|169:148|reload| ( 0) utils.pm, notify (line: 691)
|7011|169:148|reload| (-1) DataStructure.pm, _automethod (line: 610)
|7011|169:148|reload| (-2) Autoload.pm, __ANON__ (line: 80)
|7011|169:148|reload| (-3) esx.pm, load (line: 158)
|7011|169:148|reload| (-4) new.pm, reload_image (line: 664)
|7011|169:148|reload| (-5) new.pm, process (line: 266)
|7011|169:148|reload| (-6) vcld, make_new_child (line: 600)
|7011|169:148|reload| ---- WARNING ----
|7011|169:148|reload| 2010-09-09
20:04:26|7011|169:148|reload|DataStructure.pm:_automethod(610)|corresponding
data has not been initialized for get_computer_eth1_mac_address:
$self->request_data->{reservation}{148}{computer}{eth1macaddress}
|7011|169:148|reload| ( 0) utils.pm, notify (line: 691)
|7011|169:148|reload| (-1) DataStructure.pm, _automethod (line: 610)
|7011|169:148|reload| (-2) Autoload.pm, __ANON__ (line: 80)
|7011|169:148|reload| (-3) esx.pm, load (line: 159)
|7011|169:148|reload| (-4) new.pm, reload_image (line: 664)
|7011|169:148|reload| (-5) new.pm, process (line: 266)
|7011|169:148|reload| (-6) vcld, make_new_child (line: 600)
2010-09-09 20:24:02|6983|170:149|reload|utils.pm:mail(1301)|SUCCESS --
Sending mail To: , PROBLEM -- esx.pm
|6983|170:149|reload| ---- CRITICAL ----
|6983|170:149|reload| 2010-09-09
20:24:01|6983|170:149|reload|esx.pm:load(394)|waited acceptable amount
of time for dhcp, please check vm5 on blade6-1.oscar.ncsu.edu
|6983|170:149|reload| ( 0) utils.pm, notify (line: 691)
|6983|170:149|reload| (-1) esx.pm, load (line: 394)
|6983|170:149|reload| (-2) new.pm, reload_image (line: 664)
|6983|170:149|reload| (-3) new.pm, process (line: 266)
|6983|170:149|reload| (-4) vcld, make_new_child (line: 600)
|6983|170:149|reload| (-5) vcld, main (line: 347)
|6983|170:149|reload| ---- WARNING ----
|6983|170:149|reload| 2010-09-09
20:24:02|6983|170:149|reload|new.pm:reload_image(669)|fc9image-EHRSystemSuite214-v9
failed to load on vm5, returning
|6983|170:149|reload| ( 0) utils.pm, notify (line: 691)
|6983|170:149|reload| (-1) new.pm, reload_image (line: 669)
|6983|170:149|reload| (-2) new.pm, process (line: 266)
|6983|170:149|reload| (-3) vcld, make_new_child (line: 600)
|6983|170:149|reload| (-4) vcld, main (line: 347)
2010-09-09 20:24:02|6983|170:149|reload|utils.pm:insertloadlog(5324)|inserted
computer=31, loadimagefailed, fc9image-EHRSystemSuite214-v9 failed to
load on vm5
|6983|170:149|reload| ---- WARNING ----
|6983|170:149|reload| 2010-09-09
20:24:02|6983|170:149|reload|new.pm:process(313)|failed to load vm5
with fc9image-EHRSystemSuite214-v9
|6983|170:149|reload| ( 0) utils.pm, notify (line: 691)
|6983|170:149|reload| (-1) new.pm, process (line: 313)
|6983|170:149|reload| (-2) vcld, make_new_child (line: 600)
|6983|170:149|reload| (-3) vcld, main (line: 347)